* [tarantool-patches] [PATCH v5 0/6] box: run checks on insertions in LUA spaces
@ 2019-05-23 10:19 Kirill Shcherbatov
2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 1/6] sql: introduce a new method to bind a pointer Kirill Shcherbatov
` (7 more replies)
0 siblings, 8 replies; 31+ messages in thread
From: Kirill Shcherbatov @ 2019-05-23 10:19 UTC (permalink / raw)
To: tarantool-patches, v.shpilevoy; +Cc: Kirill Shcherbatov
Fire CK constraints for LUA spaces.
To achieve this goal, we reworked data dictionary, to store ck
constraints in separate space _ck_constraints and updated data
migration script to migrate existent data there. This also would
be useful in future to implement ALTER SPACE ADD CONSTRAINT
operation. Now we do not support CK constraint creation on
non-empty space.
Each CK has own precompiled VDBE machine that performs this
check with tuple fields mapped to it's memory with sql_bind() api.
In case of ck constraint conflict detected by this VM we abort
the transaction and return error to user.
Finally, we introduced a LUA-wrapper that provide a user-friendly
way to manage space ck constraints.
Changes in version 5:
- a new tuple_fetcher class to access tuple fields fast
- refactored _ck_constraint space format and indexes
- one on_replace trigger for all ck constraints
- trim_space_snprintf fixes: do not trim substring in ' or "
- do not wrap ck_constraint methods
- many minor fixes
v4: https://www.freelists.org/post/tarantool-patches/PATCH-v4-04-box-run-checks-on-insertions-in-LUA-spaces
Branch: http://github.com/tarantool/tarantool/tree/kshch/gh-3691-checks-on-server-side
Issue: https://github.com/tarantool/tarantool/issues/3691
Kirill Shcherbatov (6):
sql: introduce a new method to bind a pointer
sql: refactor OP_Column vdbe instruction
sql: introduce tuple_fetcher class
schema: add new system space for CHECK constraints
box: run check constraint tests on space alter
box: user-friendly interface to manage ck constraints
src/box/CMakeLists.txt | 1 +
src/box/alter.cc | 342 +++++++++++++-
src/box/alter.h | 1 +
src/box/bootstrap.snap | Bin 4379 -> 4430 bytes
src/box/ck_constraint.c | 238 ++++++++++
src/box/ck_constraint.h | 196 ++++++++
src/box/errcode.h | 4 +-
src/box/lua/schema.lua | 35 +-
src/box/lua/space.cc | 65 +++
src/box/lua/upgrade.lua | 40 ++
src/box/schema.cc | 8 +
src/box/schema_def.h | 11 +
src/box/space.c | 6 +
src/box/space.h | 8 +
src/box/space_def.c | 98 +---
src/box/space_def.h | 4 -
src/box/sql.c | 103 +----
src/box/sql.h | 82 ++--
src/box/sql/build.c | 220 +++++++--
src/box/sql/expr.c | 25 +-
src/box/sql/insert.c | 115 ++---
src/box/sql/parse.y | 2 +-
src/box/sql/parse_def.h | 24 +
src/box/sql/select.c | 11 +-
src/box/sql/sqlInt.h | 50 ++-
src/box/sql/tarantoolInt.h | 5 +-
src/box/sql/tokenize.c | 1 -
src/box/sql/vdbe.c | 308 +++++++------
src/box/sql/vdbe.h | 1 -
src/box/sql/vdbeInt.h | 20 +-
src/box/sql/vdbeapi.c | 20 +-
src/box/sql/vdbemem.c | 8 +
test/app-tap/tarantoolctl.test.lua | 4 +-
test/box-py/bootstrap.result | 4 +
test/box/access.result | 3 +
test/box/access.test.lua | 1 +
test/box/access_misc.result | 3 +
test/box/access_sysview.result | 6 +-
test/box/alter.result | 5 +-
test/box/misc.result | 2 +
test/sql-tap/check.test.lua | 42 +-
test/sql-tap/fkey2.test.lua | 4 +-
test/sql-tap/sql-errors.test.lua | 2 +-
test/sql-tap/table.test.lua | 12 +-
test/sql/checks.result | 620 ++++++++++++++++++++++++--
test/sql/checks.test.lua | 228 ++++++++--
test/sql/errinj.result | 140 ++++++
test/sql/errinj.test.lua | 45 ++
test/sql/gh-2981-check-autoinc.result | 12 +-
test/sql/types.result | 3 +-
test/sql/upgrade.result | 19 +
test/sql/upgrade.test.lua | 5 +
test/wal_off/alter.result | 2 +-
53 files changed, 2561 insertions(+), 653 deletions(-)
create mode 100644 src/box/ck_constraint.c
create mode 100644 src/box/ck_constraint.h
--
2.21.0
^ permalink raw reply [flat|nested] 31+ messages in thread* [tarantool-patches] [PATCH v5 1/6] sql: introduce a new method to bind a pointer 2019-05-23 10:19 [tarantool-patches] [PATCH v5 0/6] box: run checks on insertions in LUA spaces Kirill Shcherbatov @ 2019-05-23 10:19 ` Kirill Shcherbatov 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 2/6] sql: refactor OP_Column vdbe instruction Kirill Shcherbatov ` (6 subsequent siblings) 7 siblings, 0 replies; 31+ messages in thread From: Kirill Shcherbatov @ 2019-05-23 10:19 UTC (permalink / raw) To: tarantool-patches, v.shpilevoy; +Cc: Kirill Shcherbatov A new sql_bind_ptr routine allows to bing a generic pointer to VDBE variable. This change is required to pass a tuple_fetcher representing a new tuple to the check constraint VDBE. Needed for #3961 --- src/box/sql/sqlInt.h | 12 ++++++++++++ src/box/sql/vdbeInt.h | 8 ++++++++ src/box/sql/vdbeapi.c | 12 ++++++++++++ src/box/sql/vdbemem.c | 8 ++++++++ 4 files changed, 40 insertions(+) diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h index e33d7d95b..7212553f6 100644 --- a/src/box/sql/sqlInt.h +++ b/src/box/sql/sqlInt.h @@ -923,6 +923,18 @@ int sql_bind_zeroblob64(sql_stmt *, int, sql_uint64); +/** + * Perform pointer parameter binding for the prepared sql + * statement. + * @param stmt Prepared statement. + * @param i Index of the variable to be binded. + * @param ptr Pointer value to use. + * @retval 0 On Success. + * @retval Not 0 otherwise. + */ +int +sql_bind_ptr(struct sql_stmt *stmt, int i, void *ptr); + int sql_stmt_busy(sql_stmt *); diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h index 86fd92da1..240283927 100644 --- a/src/box/sql/vdbeInt.h +++ b/src/box/sql/vdbeInt.h @@ -501,6 +501,14 @@ void sqlVdbeMemSetInt64(Mem *, i64); void mem_set_bool(struct Mem *mem, bool value); +/** + * Set VDBE memory register with given pointer as a data. + * @param mem VDBE memory register to update. + * @param ptr Pointer to use. + */ +void +mem_set_ptr(struct Mem *mem, void *ptr); + void sqlVdbeMemSetDouble(Mem *, double); void sqlVdbeMemInit(Mem *, sql *, u32); void sqlVdbeMemSetNull(Mem *); diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c index e480ae720..198c2a9d7 100644 --- a/src/box/sql/vdbeapi.c +++ b/src/box/sql/vdbeapi.c @@ -1392,6 +1392,18 @@ sql_bind_null(sql_stmt * pStmt, int i) return rc; } +int +sql_bind_ptr(struct sql_stmt *stmt, int i, void *ptr) +{ + struct Vdbe *p = (struct Vdbe *) stmt; + int rc = vdbeUnbind(p, i); + if (rc == SQL_OK) { + rc = sql_bind_type(p, i, "BLOB"); + mem_set_ptr(&p->aVar[i - 1], ptr); + } + return rc; +} + int sql_bind_text(sql_stmt * pStmt, int i, const char *zData, int nData, void (*xDel) (void *) diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c index f73ea0a71..382d8ae56 100644 --- a/src/box/sql/vdbemem.c +++ b/src/box/sql/vdbemem.c @@ -749,6 +749,14 @@ sqlValueSetNull(sql_value * p) sqlVdbeMemSetNull((Mem *) p); } +void +mem_set_ptr(struct Mem *mem, void *ptr) +{ + sqlVdbeMemRelease(mem); + mem->flags = MEM_Ptr; + mem->u.p = ptr; +} + /* * Delete any previous value and set the value to be a BLOB of length * n containing all zeros. -- 2.21.0 ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] [PATCH v5 2/6] sql: refactor OP_Column vdbe instruction 2019-05-23 10:19 [tarantool-patches] [PATCH v5 0/6] box: run checks on insertions in LUA spaces Kirill Shcherbatov 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 1/6] sql: introduce a new method to bind a pointer Kirill Shcherbatov @ 2019-05-23 10:19 ` Kirill Shcherbatov 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 3/6] sql: introduce tuple_fetcher class Kirill Shcherbatov ` (5 subsequent siblings) 7 siblings, 0 replies; 31+ messages in thread From: Kirill Shcherbatov @ 2019-05-23 10:19 UTC (permalink / raw) To: tarantool-patches, v.shpilevoy; +Cc: Kirill Shcherbatov This preparatory refactoring is necessary to simplify the process of introducing a new OP_Fetch statement in the next patch. - got rid of useless sMem local variable - got rid of useless payloadSize in VdbeCursor structure Needed for #3691 --- src/box/sql/vdbe.c | 35 +++++++---------------------------- src/box/sql/vdbeInt.h | 1 - 2 files changed, 7 insertions(+), 29 deletions(-) diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c index 11f5685f3..5d37f63fb 100644 --- a/src/box/sql/vdbe.c +++ b/src/box/sql/vdbe.c @@ -2600,7 +2600,6 @@ case OP_Column: { u32 *aOffset; /* aOffset[i] is offset to start of data for i-th column */ int i; /* Loop counter */ Mem *pDest; /* Where to write the extracted value */ - Mem sMem; /* For storing the record being decoded */ const u8 *zData; /* Part of the record being decoded */ const u8 MAYBE_UNUSED *zEnd; /* Data end */ const u8 *zParse; /* Next unparsed byte of the row */ @@ -2626,7 +2625,7 @@ case OP_Column: { pReg = &aMem[pC->uc.pseudoTableReg]; assert(pReg->flags & MEM_Blob); assert(memIsValid(pReg)); - pC->payloadSize = pC->szRow = pReg->n; + pC->szRow = pReg->n; pC->aRow = (u8*)pReg->z; } else { sqlVdbeMemSetNull(pDest); @@ -2639,9 +2638,7 @@ case OP_Column: { assert(sqlCursorIsValid(pCrsr)); assert(pCrsr->curFlags & BTCF_TaCursor || pCrsr->curFlags & BTCF_TEphemCursor); - pC->aRow = tarantoolsqlPayloadFetch(pCrsr, - &pC->payloadSize); - pC->szRow = pC->payloadSize; + pC->aRow = tarantoolsqlPayloadFetch(pCrsr, &pC->szRow); } pC->cacheStatus = p->cacheCtr; @@ -2659,22 +2656,8 @@ case OP_Column: { } goto op_column_out; } - - /* Sometimes the data is too large and overflow pages come into play. - * In the later case allocate a buffer and reassamble the row. - * Stock sql utilized several clever techniques to optimize here. - * The techniques we ripped out to simplify the code. - */ - if (pC->szRow==pC->payloadSize) { - zData = pC->aRow; - zEnd = zData + pC->payloadSize; - } else { - memset(&sMem, 0, sizeof(sMem)); - rc = sqlVdbeMemFromBtree(pC->uc.pCursor, 0, pC->payloadSize, &sMem); - if (rc!=SQL_OK) goto abort_due_to_error; - zData = (u8*)sMem.z; - zEnd = zData + pC->payloadSize; - } + zData = pC->aRow; + zEnd = zData + pC->szRow; /* * Make sure at least the first p2+1 entries of the header @@ -2767,7 +2750,8 @@ case OP_Column: { if ((pDest->flags & (MEM_Ephem | MEM_Str)) == (MEM_Ephem | MEM_Str)) { int len = pDest->n; if (pDest->szMalloc<len+1) { - if (sqlVdbeMemGrow(pDest, len+1, 1)) goto op_column_error; + if (sqlVdbeMemGrow(pDest, len+1, 1)) + goto abort_due_to_error; } else { pDest->z = memcpy(pDest->zMalloc, pDest->z, len); pDest->flags &= ~MEM_Ephem; @@ -2776,15 +2760,10 @@ case OP_Column: { pDest->flags |= MEM_Term; } - if (zData!=pC->aRow) sqlVdbeMemRelease(&sMem); - op_column_out: +op_column_out: UPDATE_MAX_BLOBSIZE(pDest); REGISTER_TRACE(p, pOp->p3, pDest); break; - - op_column_error: - if (zData!=pC->aRow) sqlVdbeMemRelease(&sMem); - goto abort_due_to_error; } /* Opcode: ApplyType P1 P2 * P4 * diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h index 240283927..d58e3df7b 100644 --- a/src/box/sql/vdbeInt.h +++ b/src/box/sql/vdbeInt.h @@ -114,7 +114,6 @@ struct VdbeCursor { i16 nField; /* Number of fields in the header */ u16 nHdrParsed; /* Number of header fields parsed so far */ const u8 *aRow; /* Data for the current row, if all on one page */ - u32 payloadSize; /* Total number of bytes in the record */ u32 szRow; /* Byte available in aRow */ #ifdef SQL_ENABLE_COLUMN_USED_MASK u64 maskUsed; /* Mask of columns used by this cursor */ -- 2.21.0 ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] [PATCH v5 3/6] sql: introduce tuple_fetcher class 2019-05-23 10:19 [tarantool-patches] [PATCH v5 0/6] box: run checks on insertions in LUA spaces Kirill Shcherbatov 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 1/6] sql: introduce a new method to bind a pointer Kirill Shcherbatov 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 2/6] sql: refactor OP_Column vdbe instruction Kirill Shcherbatov @ 2019-05-23 10:19 ` Kirill Shcherbatov 2019-05-26 12:05 ` [tarantool-patches] " Vladislav Shpilevoy 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 4/6] schema: add new system space for CHECK constraints Kirill Shcherbatov ` (4 subsequent siblings) 7 siblings, 1 reply; 31+ messages in thread From: Kirill Shcherbatov @ 2019-05-23 10:19 UTC (permalink / raw) To: tarantool-patches, v.shpilevoy; +Cc: Kirill Shcherbatov Refactored OP_Column instruction with a new tuple_fetcher class. The tuple_fetcher is a reusable object that speed-up field access for given tuple. Introduced OP_Fetch opcode that uses tuple_fetcher given as a first argument. This opcode makes possible to perform binding of a new tuple to an existent VDBE without decoding it's fields. Needed for #3691 --- src/box/sql.c | 25 +++- src/box/sql.h | 46 ++++++ src/box/sql/tarantoolInt.h | 5 +- src/box/sql/vdbe.c | 287 ++++++++++++++++++++++--------------- src/box/sql/vdbeInt.h | 11 +- 5 files changed, 239 insertions(+), 135 deletions(-) diff --git a/src/box/sql.c b/src/box/sql.c index 385676055..7f581aad8 100644 --- a/src/box/sql.c +++ b/src/box/sql.c @@ -182,18 +182,15 @@ const void *tarantoolsqlPayloadFetch(BtCursor *pCur, u32 *pAmt) } const void * -tarantoolsqlTupleColumnFast(BtCursor *pCur, u32 fieldno, u32 *field_size) +tarantool_tuple_field_fast(struct tuple *tuple, uint32_t fieldno, + uint32_t *field_size) { - assert(pCur->curFlags & BTCF_TaCursor || - pCur->curFlags & BTCF_TEphemCursor); - assert(pCur->last_tuple != NULL); - - struct tuple_format *format = tuple_format(pCur->last_tuple); + struct tuple_format *format = tuple_format(tuple); if (fieldno >= tuple_format_field_count(format) || tuple_format_field(format, fieldno)->offset_slot == TUPLE_OFFSET_SLOT_NIL) return NULL; - const char *field = tuple_field(pCur->last_tuple, fieldno); + const char *field = tuple_field(tuple, fieldno); const char *end = field; mp_next(&end); *field_size = end - field; @@ -1362,3 +1359,17 @@ sql_checks_resolve_space_def_reference(ExprList *expr_list, sql_parser_destroy(&parser); return rc; } + +void +tuple_fetcher_create(struct tuple_fetcher *fetcher, struct tuple *tuple, + const char *data, uint32_t data_sz) +{ + fetcher->tuple = tuple; + fetcher->data = data; + fetcher->data_sz = data_sz; + + const char *field0 = data; + fetcher->field_count = mp_decode_array((const char **) &field0); + fetcher->slots[0] = (uint32_t)(field0 - data); + fetcher->rightmost_slot = 0; +} diff --git a/src/box/sql.h b/src/box/sql.h index 15ef74b19..3aaeb2274 100644 --- a/src/box/sql.h +++ b/src/box/sql.h @@ -386,6 +386,52 @@ sql_src_list_entry_name(const struct SrcList *list, int i); void sqlSrcListDelete(struct sql *db, struct SrcList *list); +/** + * Auxilary VDBE structure to speed-up tuple data field access. + * A memory allocation that manage this structure must have + * trailing unused bytes that extends the last 'slots' array. + * The amount of reserved memory should correspond to the problem + * to be solved and is usually equal to the greatest number of + * fields in the tuple. + * + * +------------------------+ + * | struct tuple_fetcher | + * +------------------------+ + * | RESERVED MEMORY | + * +------------------------+ + */ +struct tuple_fetcher { + /** Tuple pointer or NULL when undefined. */ + struct tuple *tuple; + /** Tuple data pointer. */ + const char *data; + /** Tuple data size. */ + uint32_t data_sz; + /** Count of fields in tuple. */ + uint32_t field_count; + /** + * Index of the rightmost initialized slot in slots + * array. + */ + uint32_t rightmost_slot; + /** + * Array of offsets of tuple fields. + * Only values <= rightmost_slot are valid. + */ + uint32_t slots[1]; +}; + +/** + * Initialize a new tuple_fetcher instance with given tuple + * data. + * @param fetcher The tuple_fetcher instance to initialize. + * @param tuple The tuple object pointer or NULL when undefined. + * @param data The tuple data (is always defined). + * @param data_sz The size of tuple data (is always defined). + */ +void +tuple_fetcher_create(struct tuple_fetcher *fetcher, struct tuple *tuple, + const char *data, uint32_t data_sz); #if defined(__cplusplus) } /* extern "C" { */ diff --git a/src/box/sql/tarantoolInt.h b/src/box/sql/tarantoolInt.h index 2b04d961e..bb0f8f1a4 100644 --- a/src/box/sql/tarantoolInt.h +++ b/src/box/sql/tarantoolInt.h @@ -20,7 +20,7 @@ const void *tarantoolsqlPayloadFetch(BtCursor * pCur, u32 * pAmt); /** * Try to get a current tuple field using its field map. - * @param pCur Btree cursor holding a tuple. + * @param tuple Tuple to process. * @param fieldno Number of a field to get. * @param[out] field_size Result field size. * @retval not NULL MessagePack field. @@ -28,7 +28,8 @@ const void *tarantoolsqlPayloadFetch(BtCursor * pCur, u32 * pAmt); * offset to @a fieldno. */ const void * -tarantoolsqlTupleColumnFast(BtCursor *pCur, u32 fieldno, u32 *field_size); +tarantool_tuple_field_fast(struct tuple *tuple, uint32_t fieldno, + uint32_t *field_size); int tarantoolsqlFirst(BtCursor * pCur, int *pRes); int tarantoolsqlLast(BtCursor * pCur, int *pRes); diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c index 5d37f63fb..33c5458cd 100644 --- a/src/box/sql/vdbe.c +++ b/src/box/sql/vdbe.c @@ -612,6 +612,119 @@ mem_type_to_str(const struct Mem *p) } } +/** + * Fetch field by field_idx using tuple_fetcher and store result + * in dest_mem. + * @param fetcher The initialized tuple_fetcher instance to use. + * @param field_idx The id of the field to fetch. + * @param field_type The destination memory field type is used + * when fetch result type is undefined. + * @param default_val_mem The value to return when fetcher's data + * lacks field by given @a field_idx. + * @param[out] dest_mem The memory variable to store result. + * @retval SQL_OK Status code in case of success. + * @retval sql_ret_code Error code otherwise. + */ +static int +tuple_fetcher_fetch(struct tuple_fetcher *fetcher, uint32_t field_idx, + enum field_type field_type, struct Mem *default_val_mem, + struct Mem *dest_mem) +{ + sqlVdbeMemSetNull(dest_mem); + uint32_t *slots = fetcher->slots; + + if (field_idx >= fetcher->field_count) { + if (default_val_mem != NULL) { + sqlVdbeMemShallowCopy(dest_mem, default_val_mem, + MEM_Static); + } + return SQL_OK; + } + + /* + * Make sure at least the first field_idx + 1 entries + * of the header have been parsed and valid information + * is in field_map[]. + * If there is more header available for parsing in the + * record, try to extract additional fields up through the + * field_map+1-th field. + */ + const char *data; + if (fetcher->rightmost_slot <= field_idx) { + uint32_t field_sz; + if (fetcher->tuple != NULL && + (data = tarantool_tuple_field_fast(fetcher->tuple, + field_idx, + &field_sz)) != NULL) { + /* + * Special case for tarantool spaces: for + * indexed fields a tuple field map can be + * used. Else there is no sense in + * tuple_field usage, because it makes + * foreach field { mp_next(); } to find + * a field. In such a case sql is + * better - it saves offsets to all fields + * visited in mp_next() cycle. + */ + uint32_t offset = (uint32_t)(data - fetcher->data); + slots[field_idx] = offset; + slots[field_idx + 1] = offset + field_sz; + } else { + uint32_t i = fetcher->rightmost_slot; + data = fetcher->data + slots[i]; + do { + mp_next(&data); + slots[++i] = (uint32_t)(data - fetcher->data); + } while (i <= field_idx); + fetcher->rightmost_slot = i; + } + } + + /* + * Extract the content for the p2+1-th column. Control + * can only reach this point if field_map[field_idx], + * field_map[field_idx+1] are valid. + */ + assert(sqlVdbeCheckMemInvariants(dest_mem) != 0); + uint32_t dummy; + data = fetcher->data + slots[field_idx]; + if (vdbe_decode_msgpack_into_mem(data, dest_mem, &dummy) != 0) + return SQL_TARANTOOL_ERROR; + + /* + * MsgPack map, array or extension (unsupported in sql). + * Wrap it in a blob verbatim. + */ + if (dest_mem->flags == 0) { + dest_mem->n = slots[field_idx + 1] - slots[field_idx]; + dest_mem->z = (char *) data; + dest_mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype; + dest_mem->subtype = SQL_SUBTYPE_MSGPACK; + } + if ((dest_mem->flags & MEM_Int) != 0) { + if (field_type == FIELD_TYPE_NUMBER) + sqlVdbeMemSetDouble(dest_mem, dest_mem->u.i); + } + /* + * Add 0 termination (at most for strings) + * Not sure why do we check MEM_Ephem + */ + if ((dest_mem->flags & (MEM_Ephem | MEM_Str)) == (MEM_Ephem | MEM_Str)) { + int len = dest_mem->n; + if (dest_mem->szMalloc < len + 1) { + if (sqlVdbeMemGrow(dest_mem, len + 1, 1) != 0) + return SQL_NOMEM; + } else { + dest_mem->z = + memcpy(dest_mem->zMalloc, dest_mem->z, len); + dest_mem->flags &= ~MEM_Ephem; + } + dest_mem->z[len] = 0; + dest_mem->flags |= MEM_Term; + } + return SQL_OK; +} + /* * Execute as much of a VDBE program as we can. * This is the core of sql_step(). @@ -2597,12 +2710,7 @@ case OP_Column: { int p2; /* column number to retrieve */ VdbeCursor *pC; /* The VDBE cursor */ BtCursor *pCrsr = NULL; /* The BTree cursor */ - u32 *aOffset; /* aOffset[i] is offset to start of data for i-th column */ - int i; /* Loop counter */ Mem *pDest; /* Where to write the extracted value */ - const u8 *zData; /* Part of the record being decoded */ - const u8 MAYBE_UNUSED *zEnd; /* Data end */ - const u8 *zParse; /* Next unparsed byte of the row */ Mem *pReg; /* PseudoTable input register */ pC = p->apCsr[pOp->p1]; @@ -2614,19 +2722,21 @@ case OP_Column: { assert(pOp->p1>=0 && pOp->p1<p->nCursor); assert(pC!=0); assert(p2<pC->nField); - aOffset = pC->aOffset; assert(pC->eCurType!=CURTYPE_PSEUDO || pC->nullRow); assert(pC->eCurType!=CURTYPE_SORTER); if (pC->cacheStatus!=p->cacheCtr) { /*OPTIMIZATION-IF-FALSE*/ + struct tuple *tuple = NULL; + uint32_t data_sz; + const char *data; if (pC->nullRow) { if (pC->eCurType==CURTYPE_PSEUDO) { assert(pC->uc.pseudoTableReg>0); pReg = &aMem[pC->uc.pseudoTableReg]; assert(pReg->flags & MEM_Blob); assert(memIsValid(pReg)); - pC->szRow = pReg->n; - pC->aRow = (u8*)pReg->z; + data_sz = pReg->n; + data = pReg->z; } else { sqlVdbeMemSetNull(pDest); goto op_column_out; @@ -2638,134 +2748,73 @@ case OP_Column: { assert(sqlCursorIsValid(pCrsr)); assert(pCrsr->curFlags & BTCF_TaCursor || pCrsr->curFlags & BTCF_TEphemCursor); - pC->aRow = tarantoolsqlPayloadFetch(pCrsr, &pC->szRow); - + tuple = pCrsr->last_tuple; + data = tarantoolsqlPayloadFetch(pCrsr, &data_sz); } pC->cacheStatus = p->cacheCtr; - zParse = pC->aRow; - pC->nRowField = mp_decode_array((const char **)&zParse); /* # of fields */ - aOffset[0] = (u32)(zParse - pC->aRow); - pC->nHdrParsed = 0; - } - - if ( (u32)p2>=pC->nRowField) { - if (pOp->p4type==P4_MEM) { - sqlVdbeMemShallowCopy(pDest, pOp->p4.pMem, MEM_Static); - } else { - sqlVdbeMemSetNull(pDest); - } - goto op_column_out; + tuple_fetcher_create(&pC->fetcher, tuple, data, data_sz); } - zData = pC->aRow; - zEnd = zData + pC->szRow; - - /* - * Make sure at least the first p2+1 entries of the header - * have been parsed and valid information is in aOffset[]. - * If there is more header available for parsing in the - * record, try to extract additional fields up through the - * p2+1-th field. - */ - if (pC->nHdrParsed <= p2) { - u32 size; - if (pC->eCurType == CURTYPE_TARANTOOL && - pCrsr != NULL && ((pCrsr->curFlags & BTCF_TaCursor) != 0 || - (pCrsr->curFlags & BTCF_TEphemCursor)) && - (zParse = tarantoolsqlTupleColumnFast(pCrsr, p2, - &size)) != NULL) { - /* - * Special case for tarantool spaces: for - * indexed fields a tuple field map can be - * used. Else there is no sense in - * tuple_field usage, because it makes - * foreach field { mp_next(); } to find - * a field. In such a case sql is - * better - it saves offsets to all fields - * visited in mp_next() cycle. - */ - aOffset[p2] = zParse - zData; - aOffset[p2 + 1] = aOffset[p2] + size; - } else { - i = pC->nHdrParsed; - zParse = zData+aOffset[i]; - - /* - * Fill in aOffset[i] values through the - * p2-th field. - */ - do{ - mp_next((const char **) &zParse); - aOffset[++i] = (u32)(zParse-zData); - }while( i<=p2); - assert((u32)p2 != pC->nRowField || zParse == zEnd); - pC->nHdrParsed = i; - } - } - - /* Extract the content for the p2+1-th column. Control can only - * reach this point if aOffset[p2], aOffset[p2+1] are - * all valid. - */ - assert(rc==SQL_OK); - assert(sqlVdbeCheckMemInvariants(pDest)); - if (VdbeMemDynamic(pDest)) { - sqlVdbeMemSetNull(pDest); - } - uint32_t unused; - if (vdbe_decode_msgpack_into_mem((const char *)(zData + aOffset[p2]), - pDest, &unused) != 0) { - rc = SQL_TARANTOOL_ERROR; - goto abort_due_to_error; - } - /* MsgPack map, array or extension (unsupported in sql). - * Wrap it in a blob verbatim. - */ - - if (pDest->flags == 0) { - pDest->n = aOffset[p2+1]-aOffset[p2]; - pDest->z = (char *)zData+aOffset[p2]; - pDest->flags = MEM_Blob|MEM_Ephem|MEM_Subtype; - pDest->subtype = SQL_SUBTYPE_MSGPACK; - } - if ((pDest->flags & MEM_Int) != 0 && - pC->eCurType == CURTYPE_TARANTOOL) { - enum field_type f = FIELD_TYPE_ANY; + enum field_type field_type = FIELD_TYPE_ANY; + if (pC->eCurType == CURTYPE_TARANTOOL) { /* * Ephemeral spaces feature only one index * covering all fields, but ephemeral spaces * lack format. So, we can fetch type from * key parts. */ - if (pC->uc.pCursor->curFlags & BTCF_TEphemCursor) - f = pC->uc.pCursor->index->def->key_def->parts[p2].type; - else if (pC->uc.pCursor->curFlags & BTCF_TaCursor) - f = pC->uc.pCursor->space->def->fields[p2].type; - if (f == FIELD_TYPE_NUMBER) - sqlVdbeMemSetDouble(pDest, pDest->u.i); - } - /* - * Add 0 termination (at most for strings) - * Not sure why do we check MEM_Ephem - */ - if ((pDest->flags & (MEM_Ephem | MEM_Str)) == (MEM_Ephem | MEM_Str)) { - int len = pDest->n; - if (pDest->szMalloc<len+1) { - if (sqlVdbeMemGrow(pDest, len+1, 1)) - goto abort_due_to_error; - } else { - pDest->z = memcpy(pDest->zMalloc, pDest->z, len); - pDest->flags &= ~MEM_Ephem; + if (pC->uc.pCursor->curFlags & BTCF_TEphemCursor) { + field_type = pC->uc.pCursor->index->def-> + key_def->parts[p2].type; + } else if (pC->uc.pCursor->curFlags & BTCF_TaCursor) { + field_type = pC->uc.pCursor->space->def-> + fields[p2].type; } - pDest->z[len] = 0; - pDest->flags |= MEM_Term; } - + struct Mem *default_val_mem = + pOp->p4type == P4_MEM ? pOp->p4.pMem : NULL; + rc = tuple_fetcher_fetch(&pC->fetcher, p2, field_type, + default_val_mem, pDest); + if (rc != SQL_OK) + goto abort_due_to_error; op_column_out: UPDATE_MAX_BLOBSIZE(pDest); REGISTER_TRACE(p, pOp->p3, pDest); break; } +/* Opcode: Fetch P1 P2 P3 P4 P5 + * Synopsis: r[P3]=PX + * + * Interpret the data that P1 points as an initialized + * tuple_fetcher object. + * + * Fetch the P2-th column from their tuple. The value extracted + * is stored in register P3. + * + * If the column contains fewer than P2 fields, then extract + * a NULL. Or, if the P4 argument is a P4_MEM use the value of + * the P4 argument as the result. + * + * Value of P5 register is an 'expected' destination value type. + */ +case OP_Fetch: { + struct tuple_fetcher *fetcher = + (struct tuple_fetcher *) p->aMem[pOp->p1].u.p; + uint32_t field_idx = pOp->p2; + struct Mem *dest_mem = &aMem[pOp->p3]; + struct Mem *default_val_mem = + pOp->p4type == P4_MEM ? pOp->p4.pMem : NULL; + enum field_type field_type = pOp->p5; + memAboutToChange(p, dest_mem); + rc = tuple_fetcher_fetch(fetcher, field_idx, field_type, + default_val_mem, dest_mem); + if (rc != SQL_OK) + goto abort_due_to_error; + UPDATE_MAX_BLOBSIZE(dest_mem); + REGISTER_TRACE(p, pOp->p3, dest_mem); + break; +} + /* Opcode: ApplyType P1 P2 * P4 * * Synopsis: type(r[P1@P2]) * diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h index d58e3df7b..cb51446a8 100644 --- a/src/box/sql/vdbeInt.h +++ b/src/box/sql/vdbeInt.h @@ -112,17 +112,14 @@ struct VdbeCursor { /** Info about keys needed by index cursors. */ struct key_def *key_def; i16 nField; /* Number of fields in the header */ - u16 nHdrParsed; /* Number of header fields parsed so far */ - const u8 *aRow; /* Data for the current row, if all on one page */ - u32 szRow; /* Byte available in aRow */ #ifdef SQL_ENABLE_COLUMN_USED_MASK u64 maskUsed; /* Mask of columns used by this cursor */ #endif - u32 nRowField; /* Number of fields in the current row */ - /* Offsets for all fields in the record [nField+1] - * Order of fields is the same as it was passes to create table statement + /** + * Auxilary VDBE structure to speed-up tuple data + * field access. */ - u32 aOffset[1]; + struct tuple_fetcher fetcher; }; /* -- 2.21.0 ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 3/6] sql: introduce tuple_fetcher class 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 3/6] sql: introduce tuple_fetcher class Kirill Shcherbatov @ 2019-05-26 12:05 ` Vladislav Shpilevoy 2019-05-31 13:45 ` Kirill Shcherbatov 0 siblings, 1 reply; 31+ messages in thread From: Vladislav Shpilevoy @ 2019-05-26 12:05 UTC (permalink / raw) To: Kirill Shcherbatov, tarantool-patches Hi! Thanks for the fixes! See 10 comments below. > diff --git a/src/box/sql.h b/src/box/sql.h > index 15ef74b19..3aaeb2274 100644 > --- a/src/box/sql.h > +++ b/src/box/sql.h > @@ -386,6 +386,52 @@ sql_src_list_entry_name(const struct SrcList *list, int i); > void > sqlSrcListDelete(struct sql *db, struct SrcList *list); > > +/** > + * Auxilary VDBE structure to speed-up tuple data field access. 1. 'Auxilary' -> 'Auxiliary'. Looks like you still did not install spell checker, as I asked in some previous patchsets. Please, do it. > + * A memory allocation that manage this structure must have > + * trailing unused bytes that extends the last 'slots' array. > + * The amount of reserved memory should correspond to the problem > + * to be solved and is usually equal to the greatest number of > + * fields in the tuple. > + * > + * +------------------------+ > + * | struct tuple_fetcher | > + * +------------------------+ > + * | RESERVED MEMORY | > + * +------------------------+ > + */ > +struct tuple_fetcher { > + /** Tuple pointer or NULL when undefined. */ > + struct tuple *tuple; > + /** Tuple data pointer. */ > + const char *data; > + /** Tuple data size. */ > + uint32_t data_sz; > + /** Count of fields in tuple. */ > + uint32_t field_count; > + /** > + * Index of the rightmost initialized slot in slots > + * array. > + */ > + uint32_t rightmost_slot; > + /** > + * Array of offsets of tuple fields. > + * Only values <= rightmost_slot are valid. > + */ > + uint32_t slots[1]; > +}; > + > +/** > + * Initialize a new tuple_fetcher instance with given tuple > + * data. > + * @param fetcher The tuple_fetcher instance to initialize. > + * @param tuple The tuple object pointer or NULL when undefined. > + * @param data The tuple data (is always defined). > + * @param data_sz The size of tuple data (is always defined). > + */ > +void > +tuple_fetcher_create(struct tuple_fetcher *fetcher, struct tuple *tuple, > + const char *data, uint32_t data_sz); 2. Great, I like this function, especially that it does not have any single 'if' - perfect case for processor's conveyor parallelizing. But the API is ambiguous - what should a user pass here? Always both tuple and data, or tuple and different data, or tuple and its data, or tuple + NULL vs NULL + data? Please, expose two public functions: tuple_fetcher_prepare_data and tuple_fetcher_prepare_tuple. Each takes either const char *data or struct tuple *tuple. Internally they call tuple_fetcher_create with both tuple and data. Once you did it, you can stop calling tarantoolsqlPayloadFetch in OP_Column and do either tuple_fetcher_prepare_tuple(pCrsr->last_tuple) in case it is a normal cursor, or do tuple_fetcher_prepare_data(pReg->z, pReg->n) in case of a pseudo cursor. > > #if defined(__cplusplus) > } /* extern "C" { */ > diff --git a/src/box/sql/tarantoolInt.h b/src/box/sql/tarantoolInt.h > index 2b04d961e..bb0f8f1a4 100644 > --- a/src/box/sql/tarantoolInt.h > +++ b/src/box/sql/tarantoolInt.h > @@ -28,7 +28,8 @@ const void *tarantoolsqlPayloadFetch(BtCursor * pCur, u32 * pAmt); > * offset to @a fieldno. > */ > const void * > -tarantoolsqlTupleColumnFast(BtCursor *pCur, u32 fieldno, u32 *field_size); > +tarantool_tuple_field_fast(struct tuple *tuple, uint32_t fieldno, > + uint32_t *field_size); 3. I suggest you to make this function method of the fetcher and static inline right above tuple_fetcher_fetch(). It is not needed in any other place. > > int tarantoolsqlFirst(BtCursor * pCur, int *pRes); > int tarantoolsqlLast(BtCursor * pCur, int *pRes); > diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c > index 5d37f63fb..33c5458cd 100644 > --- a/src/box/sql/vdbe.c > +++ b/src/box/sql/vdbe.c > @@ -612,6 +612,119 @@ mem_type_to_str(const struct Mem *p) > } > } > > +/** > + * Fetch field by field_idx using tuple_fetcher and store result > + * in dest_mem. > + * @param fetcher The initialized tuple_fetcher instance to use. > + * @param field_idx The id of the field to fetch. > + * @param field_type The destination memory field type is used > + * when fetch result type is undefined. 4. The only reason to make tuple_fetcher_fetch() a separate function was to get rid of this parameter and related code, keeping it in OP_Column. Please, do it. Remove the parameter and keep it in OP_Column. > + * @param default_val_mem The value to return when fetcher's data > + * lacks field by given @a field_idx. 5. Why would ever need that parameter for OP_Fetch? It is not a task of a fetcher to return default fields IMO. It should only fetch a field, not replace it with a default value. > + * @param[out] dest_mem The memory variable to store result. > + * @retval SQL_OK Status code in case of success. > + * @retval sql_ret_code Error code otherwise. > + */ > +static int > +tuple_fetcher_fetch(struct tuple_fetcher *fetcher, uint32_t field_idx, > + enum field_type field_type, struct Mem *default_val_mem, > + struct Mem *dest_mem) > @@ -2614,19 +2722,21 @@ case OP_Column: {> > +/* Opcode: Fetch P1 P2 P3 P4 P5 > + * Synopsis: r[P3]=PX > + * > + * Interpret the data that P1 points as an initialized 6. "Interpret data P1 points at as an initialized ...". > + * tuple_fetcher object. > + * > + * Fetch the P2-th column from their tuple. The value extracted 7. 'Their' is for plural. Use 'its'. > + * is stored in register P3. > + * > + * If the column contains fewer than P2 fields, then extract > + * a NULL. Or, if the P4 argument is a P4_MEM use the value of > + * the P4 argument as the result. 8. You never generate OP_Fetch with a default value. Please, drop P4 argument. As well as P5. They are never used, even in the last patch. > + * > + * Value of P5 register is an 'expected' destination value type. 9. It is not 'expected'. I can't pass here FIELD_TYPE_STRING and get a string value despite an actually stored type. It is rather a flag, that you need to transform 'int' into 'number'. Anyway, it should be dropped. > + */ > +case OP_Fetch: { > + struct tuple_fetcher *fetcher = > + (struct tuple_fetcher *) p->aMem[pOp->p1].u.p; > + uint32_t field_idx = pOp->p2; > + struct Mem *dest_mem = &aMem[pOp->p3]; > + struct Mem *default_val_mem = > + pOp->p4type == P4_MEM ? pOp->p4.pMem : NULL; > + enum field_type field_type = pOp->p5; > + memAboutToChange(p, dest_mem); > + rc = tuple_fetcher_fetch(fetcher, field_idx, field_type, > + default_val_mem, dest_mem); > + if (rc != SQL_OK) > + goto abort_due_to_error; > + UPDATE_MAX_BLOBSIZE(dest_mem); 10. Why is not this command a part of tuple_fetcher_fetch()? > + REGISTER_TRACE(p, pOp->p3, dest_mem); > + break; > +} ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 3/6] sql: introduce tuple_fetcher class 2019-05-26 12:05 ` [tarantool-patches] " Vladislav Shpilevoy @ 2019-05-31 13:45 ` Kirill Shcherbatov 2019-05-31 19:45 ` Konstantin Osipov 0 siblings, 1 reply; 31+ messages in thread From: Kirill Shcherbatov @ 2019-05-31 13:45 UTC (permalink / raw) To: tarantool-patches, Vladislav Shpilevoy Hi! Thank you for review. On 26.05.2019 15:05, Vladislav Shpilevoy wrote: > Hi! Thanks for the fixes! See 10 comments below. > >> diff --git a/src/box/sql.h b/src/box/sql.h >> index 15ef74b19..3aaeb2274 100644 >> --- a/src/box/sql.h >> +++ b/src/box/sql.h >> @@ -386,6 +386,52 @@ sql_src_list_entry_name(const struct SrcList *list, int i); >> void >> sqlSrcListDelete(struct sql *db, struct SrcList *list); >> >> +/** >> + * Auxilary VDBE structure to speed-up tuple data field access. > > 1. 'Auxilary' -> 'Auxiliary'. Looks like you still did not install > spell checker, as I asked in some previous patchsets. Please, do it. Done. I use spell checker. It is unobtrusive. Moreover, this word exists in this meaning. Doesn't matter. Fixed. >> +void >> +tuple_fetcher_create(struct tuple_fetcher *fetcher, struct tuple *tuple, >> + const char *data, uint32_t data_sz); > > 2.>... > Please, expose two public functions: tuple_fetcher_prepare_data and > tuple_fetcher_prepare_tuple. Each takes either const char *data or > struct tuple *tuple. Internally they call tuple_fetcher_create with > both tuple and data. >... Done > >> >> #if defined(__cplusplus) >> } /* extern "C" { */ >> diff --git a/src/box/sql/tarantoolInt.h b/src/box/sql/tarantoolInt.h >> index 2b04d961e..bb0f8f1a4 100644 >> --- a/src/box/sql/tarantoolInt.h >> +++ b/src/box/sql/tarantoolInt.h >> @@ -28,7 +28,8 @@ const void *tarantoolsqlPayloadFetch(BtCursor * pCur, u32 * pAmt); >> * offset to @a fieldno. >> */ >> const void * >> -tarantoolsqlTupleColumnFast(BtCursor *pCur, u32 fieldno, u32 *field_size); >> +tarantool_tuple_field_fast(struct tuple *tuple, uint32_t fieldno, >> + uint32_t *field_size); > > 3. I suggest you to make this function method of the fetcher and static inline > right above tuple_fetcher_fetch(). It is not needed in any other place. Done. > 4. The only reason to make tuple_fetcher_fetch() a separate function > was to get rid of this parameter and related code, keeping it in > OP_Column. Please, do it. Remove the parameter and keep it in > OP_Column. > 5. Why would ever need that parameter for OP_Fetch? It is not a > task of a fetcher to return default fields IMO. It should only fetch > a field, not replace it with a default value. static int tuple_fetcher_fetch(struct tuple_fetcher *fetcher, uint32_t field_idx, struct Mem *dest_mem) rc = tuple_fetcher_fetch(&pC->fetcher, p2, pDest); if (rc != SQL_OK) goto abort_due_to_error; if ((pDest->flags & MEM_Null) && (uint32_t) p2 >= pC->fetcher.field_count && default_val_mem != NULL) { sqlVdbeMemShallowCopy(pDest, default_val_mem, MEM_Static); } if ((pDest->flags & MEM_Int) != 0) { if (field_type == FIELD_TYPE_NUMBER) sqlVdbeMemSetDouble(pDest, pDest->u.i); } >> + * Interpret the data that P1 points as an initialized > > 6. "Interpret data P1 points at as an initialized ...". Fixed. >> + * Fetch the P2-th column from their tuple. The value extracted > > 7. 'Their' is for plural. Use 'its'. Fixed. > 8. You never generate OP_Fetch with a default value. Please, > drop P4 argument. As well as P5. They are never used, even in the > last patch. > >> + * >> + * Value of P5 register is an 'expected' destination value type. > > 9. It is not 'expected'. I can't pass here FIELD_TYPE_STRING and get > a string value despite an actually stored type. It is rather a flag, > that you need to transform 'int' into 'number'. Anyway, it should be > dropped. Done. > 10. Why is not this command a part of tuple_fetcher_fetch()? Don't mind. Done. ===================================================== Refactored OP_Column instruction with a new tuple_fetcher class. The tuple_fetcher is a reusable object that speed-up field access for given tuple. Introduced OP_Fetch opcode that uses tuple_fetcher given as a first argument. This opcode makes possible to perform binding of a new tuple to an existent VDBE without decoding it's fields. Needed for #3691 --- src/box/sql.c | 54 ++++--- src/box/sql.h | 54 +++++++ src/box/sql/tarantoolInt.h | 12 -- src/box/sql/vdbe.c | 296 ++++++++++++++++++++++--------------- src/box/sql/vdbeInt.h | 11 +- 5 files changed, 270 insertions(+), 157 deletions(-) diff --git a/src/box/sql.c b/src/box/sql.c index 385676055..07ceec2d2 100644 --- a/src/box/sql.c +++ b/src/box/sql.c @@ -181,25 +181,6 @@ const void *tarantoolsqlPayloadFetch(BtCursor *pCur, u32 *pAmt) return tuple_data(pCur->last_tuple); } -const void * -tarantoolsqlTupleColumnFast(BtCursor *pCur, u32 fieldno, u32 *field_size) -{ - assert(pCur->curFlags & BTCF_TaCursor || - pCur->curFlags & BTCF_TEphemCursor); - assert(pCur->last_tuple != NULL); - - struct tuple_format *format = tuple_format(pCur->last_tuple); - if (fieldno >= tuple_format_field_count(format) || - tuple_format_field(format, fieldno)->offset_slot == - TUPLE_OFFSET_SLOT_NIL) - return NULL; - const char *field = tuple_field(pCur->last_tuple, fieldno); - const char *end = field; - mp_next(&end); - *field_size = end - field; - return field; -} - /* * Set cursor to the first tuple in given space. * It is a simple wrapper around cursor_seek(). @@ -1362,3 +1343,38 @@ sql_checks_resolve_space_def_reference(ExprList *expr_list, sql_parser_destroy(&parser); return rc; } + +/** + * Initialize a new tuple_fetcher instance with given tuple + * data. + * @param fetcher The tuple_fetcher instance to initialize. + * @param tuple The tuple object pointer or NULL when undefined. + * @param data The tuple data (is always defined). + * @param data_sz The size of tuple data (is always defined). + */ +static void +tuple_fetcher_create(struct tuple_fetcher *fetcher, struct tuple *tuple, + const char *data, uint32_t data_sz) +{ + fetcher->tuple = tuple; + fetcher->data = data; + fetcher->data_sz = data_sz; + + const char *field0 = data; + fetcher->field_count = mp_decode_array((const char **) &field0); + fetcher->slots[0] = (uint32_t)(field0 - data); + fetcher->rightmost_slot = 0; +} + +void +tuple_fetcher_prepare_data(struct tuple_fetcher *fetcher, const char *data, + uint32_t data_sz) +{ + tuple_fetcher_create(fetcher, NULL, data, data_sz); +} + +void +tuple_fetcher_prepare_tuple(struct tuple_fetcher *fetcher, struct tuple *tuple) +{ + tuple_fetcher_create(fetcher, tuple, tuple_data(tuple), tuple->bsize); +} diff --git a/src/box/sql.h b/src/box/sql.h index 15ef74b19..a13f78ae8 100644 --- a/src/box/sql.h +++ b/src/box/sql.h @@ -386,6 +386,60 @@ sql_src_list_entry_name(const struct SrcList *list, int i); void sqlSrcListDelete(struct sql *db, struct SrcList *list); +/** + * Auxiliary VDBE structure to speed-up tuple data field access. + * A memory allocation that manage this structure must have + * trailing unused bytes that extends the last 'slots' array. + * The amount of reserved memory should correspond to the problem + * to be solved and is usually equal to the greatest number of + * fields in the tuple. + * + * +------------------------+ + * | struct tuple_fetcher | + * +------------------------+ + * | RESERVED MEMORY | + * +------------------------+ + */ +struct tuple_fetcher { + /** Tuple pointer or NULL when undefined. */ + struct tuple *tuple; + /** Tuple data pointer. */ + const char *data; + /** Tuple data size. */ + uint32_t data_sz; + /** Count of fields in tuple. */ + uint32_t field_count; + /** + * Index of the rightmost initialized slot in slots + * array. + */ + uint32_t rightmost_slot; + /** + * Array of offsets of tuple fields. + * Only values <= rightmost_slot are valid. + */ + uint32_t slots[1]; +}; + +/** + * Initialize a new tuple_fetcher instance with given tuple + * data. + * @param fetcher The tuple_fetcher instance to initialize. + * @param data The tuple data. + * @param data_sz The size of tuple data. + */ +void +tuple_fetcher_prepare_data(struct tuple_fetcher *fetcher, const char *data, + uint32_t data_sz); + +/** + * Initialize a new tuple_fetcher instance with given tuple + * data. + * @param fetcher The tuple_fetcher instance to initialize. + * @param tuple The tuple object pointer. + */ +void +tuple_fetcher_prepare_tuple(struct tuple_fetcher *fetcher, struct tuple *tuple); #if defined(__cplusplus) } /* extern "C" { */ diff --git a/src/box/sql/tarantoolInt.h b/src/box/sql/tarantoolInt.h index 2b04d961e..2dd7eef20 100644 --- a/src/box/sql/tarantoolInt.h +++ b/src/box/sql/tarantoolInt.h @@ -18,18 +18,6 @@ int is_tarantool_error(int rc); /* Storage interface. */ const void *tarantoolsqlPayloadFetch(BtCursor * pCur, u32 * pAmt); -/** - * Try to get a current tuple field using its field map. - * @param pCur Btree cursor holding a tuple. - * @param fieldno Number of a field to get. - * @param[out] field_size Result field size. - * @retval not NULL MessagePack field. - * @retval NULL Can not use field_map - it does not contain - * offset to @a fieldno. - */ -const void * -tarantoolsqlTupleColumnFast(BtCursor *pCur, u32 fieldno, u32 *field_size); - int tarantoolsqlFirst(BtCursor * pCur, int *pRes); int tarantoolsqlLast(BtCursor * pCur, int *pRes); int tarantoolsqlNext(BtCursor * pCur, int *pRes); diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c index 91830813d..250646c9b 100644 --- a/src/box/sql/vdbe.c +++ b/src/box/sql/vdbe.c @@ -43,6 +43,7 @@ #include "box/error.h" #include "box/fk_constraint.h" #include "box/txn.h" +#include "box/tuple.h" #include "sqlInt.h" #include "vdbeInt.h" #include "tarantoolInt.h" @@ -612,6 +613,133 @@ mem_type_to_str(const struct Mem *p) } } +/** + * Try to get a current tuple field using its field map. + * @param tuple Tuple to process. + * @param fieldno Number of a field to get. + * @param[out] field_size Result field size. + * @retval not NULL MessagePack field. + * @retval NULL Can not use field_map - it does not contain + * offset to @a fieldno. + */ +static const void * +tuple_fetcher_tuple_field_fast(struct tuple_fetcher *fetcher, uint32_t fieldno, + uint32_t *field_size) +{ + if (fetcher->tuple == NULL) + return NULL; + struct tuple_format *format = tuple_format(fetcher->tuple); + if (fieldno >= tuple_format_field_count(format) || + tuple_format_field(format, fieldno)->offset_slot == + TUPLE_OFFSET_SLOT_NIL) + return NULL; + const char *field = tuple_field(fetcher->tuple, fieldno); + const char *end = field; + mp_next(&end); + *field_size = end - field; + return field; +} + +/** + * Fetch field by field_idx using tuple_fetcher and store result + * in dest_mem. + * @param fetcher The initialized tuple_fetcher instance to use. + * @param field_idx The id of the field to fetch. + * @param[out] dest_mem The memory variable to store result. + * @retval SQL_OK Status code in case of success. + * @retval sql_ret_code Error code otherwise. + */ +static int +tuple_fetcher_fetch(struct tuple_fetcher *fetcher, uint32_t field_idx, + struct Mem *dest_mem) +{ + sqlVdbeMemSetNull(dest_mem); + uint32_t *slots = fetcher->slots; + if (field_idx >= fetcher->field_count) { + UPDATE_MAX_BLOBSIZE(dest_mem); + return SQL_OK; + } + + /* + * Make sure at least the first field_idx + 1 entries + * of the header have been parsed and valid information + * is in field_map[]. + * If there is more header available for parsing in the + * record, try to extract additional fields up through the + * field_map+1-th field. + */ + const char *data; + if (fetcher->rightmost_slot <= field_idx) { + uint32_t field_sz; + data = tuple_fetcher_tuple_field_fast(fetcher, field_idx, + &field_sz); + if (data != NULL) { + /* + * Special case for tarantool spaces: for + * indexed fields a tuple field map can be + * used. Else there is no sense in + * tuple_field usage, because it makes + * foreach field { mp_next(); } to find + * a field. In such a case sql is + * better - it saves offsets to all fields + * visited in mp_next() cycle. + */ + uint32_t offset = (uint32_t)(data - fetcher->data); + slots[field_idx] = offset; + slots[field_idx + 1] = offset + field_sz; + } else { + uint32_t i = fetcher->rightmost_slot; + data = fetcher->data + slots[i]; + do { + mp_next(&data); + slots[++i] = (uint32_t)(data - fetcher->data); + } while (i <= field_idx); + fetcher->rightmost_slot = i; + } + } + + /* + * Extract the content for the p2+1-th column. Control + * can only reach this point if field_map[field_idx], + * field_map[field_idx+1] are valid. + */ + assert(sqlVdbeCheckMemInvariants(dest_mem) != 0); + uint32_t dummy; + data = fetcher->data + slots[field_idx]; + if (vdbe_decode_msgpack_into_mem(data, dest_mem, &dummy) != 0) + return SQL_TARANTOOL_ERROR; + + /* + * MsgPack map, array or extension (unsupported in sql). + * Wrap it in a blob verbatim. + */ + if (dest_mem->flags == 0) { + dest_mem->n = slots[field_idx + 1] - slots[field_idx]; + dest_mem->z = (char *) data; + dest_mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype; + dest_mem->subtype = SQL_SUBTYPE_MSGPACK; + } + /* + * Add 0 termination (at most for strings) + * Not sure why do we check MEM_Ephem + */ + if ((dest_mem->flags & (MEM_Ephem | MEM_Str)) == (MEM_Ephem | MEM_Str)) { + int len = dest_mem->n; + if (dest_mem->szMalloc < len + 1) { + if (sqlVdbeMemGrow(dest_mem, len + 1, 1) != 0) + return SQL_NOMEM; + } else { + dest_mem->z = + memcpy(dest_mem->zMalloc, dest_mem->z, len); + dest_mem->flags &= ~MEM_Ephem; + } + dest_mem->z[len] = 0; + dest_mem->flags |= MEM_Term; + } + UPDATE_MAX_BLOBSIZE(dest_mem); + return SQL_OK; +} + /* * Execute as much of a VDBE program as we can. * This is the core of sql_step(). @@ -2597,12 +2725,7 @@ case OP_Column: { int p2; /* column number to retrieve */ VdbeCursor *pC; /* The VDBE cursor */ BtCursor *pCrsr = NULL; /* The BTree cursor */ - u32 *aOffset; /* aOffset[i] is offset to start of data for i-th column */ - int i; /* Loop counter */ Mem *pDest; /* Where to write the extracted value */ - const u8 *zData; /* Part of the record being decoded */ - const u8 MAYBE_UNUSED *zEnd; /* Data end */ - const u8 *zParse; /* Next unparsed byte of the row */ Mem *pReg; /* PseudoTable input register */ pC = p->apCsr[pOp->p1]; @@ -2614,7 +2737,6 @@ case OP_Column: { assert(pOp->p1>=0 && pOp->p1<p->nCursor); assert(pC!=0); assert(p2<pC->nField); - aOffset = pC->aOffset; assert(pC->eCurType!=CURTYPE_PSEUDO || pC->nullRow); assert(pC->eCurType!=CURTYPE_SORTER); @@ -2625,8 +2747,8 @@ case OP_Column: { pReg = &aMem[pC->uc.pseudoTableReg]; assert(pReg->flags & MEM_Blob); assert(memIsValid(pReg)); - pC->szRow = pReg->n; - pC->aRow = (u8*)pReg->z; + tuple_fetcher_prepare_data(&pC->fetcher, + pReg->z, pReg->n); } else { sqlVdbeMemSetNull(pDest); goto op_column_out; @@ -2638,134 +2760,70 @@ case OP_Column: { assert(sqlCursorIsValid(pCrsr)); assert(pCrsr->curFlags & BTCF_TaCursor || pCrsr->curFlags & BTCF_TEphemCursor); - pC->aRow = tarantoolsqlPayloadFetch(pCrsr, &pC->szRow); - + tuple_fetcher_prepare_tuple(&pC->fetcher, + pCrsr->last_tuple); } pC->cacheStatus = p->cacheCtr; - zParse = pC->aRow; - pC->nRowField = mp_decode_array((const char **)&zParse); /* # of fields */ - aOffset[0] = (u32)(zParse - pC->aRow); - pC->nHdrParsed = 0; - } - - if ( (u32)p2>=pC->nRowField) { - if (pOp->p4type==P4_MEM) { - sqlVdbeMemShallowCopy(pDest, pOp->p4.pMem, MEM_Static); - } else { - sqlVdbeMemSetNull(pDest); - } - goto op_column_out; } - zData = pC->aRow; - zEnd = zData + pC->szRow; - - /* - * Make sure at least the first p2+1 entries of the header - * have been parsed and valid information is in aOffset[]. - * If there is more header available for parsing in the - * record, try to extract additional fields up through the - * p2+1-th field. - */ - if (pC->nHdrParsed <= p2) { - u32 size; - if (pC->eCurType == CURTYPE_TARANTOOL && - pCrsr != NULL && ((pCrsr->curFlags & BTCF_TaCursor) != 0 || - (pCrsr->curFlags & BTCF_TEphemCursor)) && - (zParse = tarantoolsqlTupleColumnFast(pCrsr, p2, - &size)) != NULL) { - /* - * Special case for tarantool spaces: for - * indexed fields a tuple field map can be - * used. Else there is no sense in - * tuple_field usage, because it makes - * foreach field { mp_next(); } to find - * a field. In such a case sql is - * better - it saves offsets to all fields - * visited in mp_next() cycle. - */ - aOffset[p2] = zParse - zData; - aOffset[p2 + 1] = aOffset[p2] + size; - } else { - i = pC->nHdrParsed; - zParse = zData+aOffset[i]; - - /* - * Fill in aOffset[i] values through the - * p2-th field. - */ - do{ - mp_next((const char **) &zParse); - aOffset[++i] = (u32)(zParse-zData); - }while( i<=p2); - assert((u32)p2 != pC->nRowField || zParse == zEnd); - pC->nHdrParsed = i; - } - } - - /* Extract the content for the p2+1-th column. Control can only - * reach this point if aOffset[p2], aOffset[p2+1] are - * all valid. - */ - assert(rc==SQL_OK); - assert(sqlVdbeCheckMemInvariants(pDest)); - if (VdbeMemDynamic(pDest)) { - sqlVdbeMemSetNull(pDest); - } - uint32_t unused; - if (vdbe_decode_msgpack_into_mem((const char *)(zData + aOffset[p2]), - pDest, &unused) != 0) { - rc = SQL_TARANTOOL_ERROR; - goto abort_due_to_error; - } - /* MsgPack map, array or extension (unsupported in sql). - * Wrap it in a blob verbatim. - */ - - if (pDest->flags == 0) { - pDest->n = aOffset[p2+1]-aOffset[p2]; - pDest->z = (char *)zData+aOffset[p2]; - pDest->flags = MEM_Blob|MEM_Ephem|MEM_Subtype; - pDest->subtype = SQL_SUBTYPE_MSGPACK; - } - if ((pDest->flags & MEM_Int) != 0 && - pC->eCurType == CURTYPE_TARANTOOL) { - enum field_type f = FIELD_TYPE_ANY; + enum field_type field_type = FIELD_TYPE_ANY; + if (pC->eCurType == CURTYPE_TARANTOOL) { /* * Ephemeral spaces feature only one index * covering all fields, but ephemeral spaces * lack format. So, we can fetch type from * key parts. */ - if (pC->uc.pCursor->curFlags & BTCF_TEphemCursor) - f = pC->uc.pCursor->index->def->key_def->parts[p2].type; - else if (pC->uc.pCursor->curFlags & BTCF_TaCursor) - f = pC->uc.pCursor->space->def->fields[p2].type; - if (f == FIELD_TYPE_NUMBER) - sqlVdbeMemSetDouble(pDest, pDest->u.i); - } - /* - * Add 0 termination (at most for strings) - * Not sure why do we check MEM_Ephem - */ - if ((pDest->flags & (MEM_Ephem | MEM_Str)) == (MEM_Ephem | MEM_Str)) { - int len = pDest->n; - if (pDest->szMalloc<len+1) { - if (sqlVdbeMemGrow(pDest, len+1, 1)) - goto abort_due_to_error; - } else { - pDest->z = memcpy(pDest->zMalloc, pDest->z, len); - pDest->flags &= ~MEM_Ephem; + if (pC->uc.pCursor->curFlags & BTCF_TEphemCursor) { + field_type = pC->uc.pCursor->index->def-> + key_def->parts[p2].type; + } else if (pC->uc.pCursor->curFlags & BTCF_TaCursor) { + field_type = pC->uc.pCursor->space->def-> + fields[p2].type; } - pDest->z[len] = 0; - pDest->flags |= MEM_Term; } + struct Mem *default_val_mem = + pOp->p4type == P4_MEM ? pOp->p4.pMem : NULL; + rc = tuple_fetcher_fetch(&pC->fetcher, p2, pDest); + if (rc != SQL_OK) + goto abort_due_to_error; + if ((pDest->flags & MEM_Null) && + (uint32_t) p2 >= pC->fetcher.field_count && + default_val_mem != NULL) { + sqlVdbeMemShallowCopy(pDest, default_val_mem, MEM_Static); + } + if ((pDest->flags & MEM_Int) != 0) { + if (field_type == FIELD_TYPE_NUMBER) + sqlVdbeMemSetDouble(pDest, pDest->u.i); + } op_column_out: - UPDATE_MAX_BLOBSIZE(pDest); REGISTER_TRACE(p, pOp->p3, pDest); break; } +/* Opcode: Fetch P1 P2 P3 P4 P5 + * Synopsis: r[P3]=PX + * + * Interpret data P1 points at as an initialized tuple_fetcher + * object. + * + * Fetch the P2-th column from its tuple. The value extracted + * is stored in register P3. If the column contains fewer than + * P2 fields, then extract a NULL. + */ +case OP_Fetch: { + struct tuple_fetcher *fetcher = + (struct tuple_fetcher *) p->aMem[pOp->p1].u.p; + uint32_t field_idx = pOp->p2; + struct Mem *dest_mem = &aMem[pOp->p3]; + memAboutToChange(p, dest_mem); + rc = tuple_fetcher_fetch(fetcher, field_idx, dest_mem); + if (rc != SQL_OK) + goto abort_due_to_error; + REGISTER_TRACE(p, pOp->p3, dest_mem); + break; +} + /* Opcode: ApplyType P1 P2 * P4 * * Synopsis: type(r[P1@P2]) * diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h index d58e3df7b..1a428afa9 100644 --- a/src/box/sql/vdbeInt.h +++ b/src/box/sql/vdbeInt.h @@ -112,17 +112,14 @@ struct VdbeCursor { /** Info about keys needed by index cursors. */ struct key_def *key_def; i16 nField; /* Number of fields in the header */ - u16 nHdrParsed; /* Number of header fields parsed so far */ - const u8 *aRow; /* Data for the current row, if all on one page */ - u32 szRow; /* Byte available in aRow */ #ifdef SQL_ENABLE_COLUMN_USED_MASK u64 maskUsed; /* Mask of columns used by this cursor */ #endif - u32 nRowField; /* Number of fields in the current row */ - /* Offsets for all fields in the record [nField+1] - * Order of fields is the same as it was passes to create table statement + /** + * Auxiliary VDBE structure to speed-up tuple data + * field access. */ - u32 aOffset[1]; + struct tuple_fetcher fetcher; }; /* -- 2.21.0 ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 3/6] sql: introduce tuple_fetcher class 2019-05-31 13:45 ` Kirill Shcherbatov @ 2019-05-31 19:45 ` Konstantin Osipov 2019-05-31 19:50 ` Kirill Shcherbatov ` (2 more replies) 0 siblings, 3 replies; 31+ messages in thread From: Konstantin Osipov @ 2019-05-31 19:45 UTC (permalink / raw) To: tarantool-patches; +Cc: Vladislav Shpilevoy * Kirill Shcherbatov <kshcherbatov@tarantool.org> [19/05/31 20:34]: > Hi! Thank you for review. We have a yet enother -er class. It seems not to be a coincidence that all these -er classes are coming from the same contributor. Kirill, please rethink the design, I kind of hit the limit of active classes. -- Konstantin Osipov, Moscow, Russia ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 3/6] sql: introduce tuple_fetcher class 2019-05-31 19:45 ` Konstantin Osipov @ 2019-05-31 19:50 ` Kirill Shcherbatov 2019-05-31 22:36 ` Vladislav Shpilevoy 2019-06-02 18:50 ` Kirill Shcherbatov 2 siblings, 0 replies; 31+ messages in thread From: Kirill Shcherbatov @ 2019-05-31 19:50 UTC (permalink / raw) To: tarantool-patches, Konstantin Osipov; +Cc: Vladislav Shpilevoy > We have a yet enother -er class. It seems not to be a coincidence > that all these -er classes are coming from the same contributor. > > Kirill, please rethink the design, I kind of hit the limit of > active classes. Please, motivate your requirement. ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 3/6] sql: introduce tuple_fetcher class 2019-05-31 19:45 ` Konstantin Osipov 2019-05-31 19:50 ` Kirill Shcherbatov @ 2019-05-31 22:36 ` Vladislav Shpilevoy 2019-06-01 5:45 ` Konstantin Osipov 2019-06-02 18:50 ` Kirill Shcherbatov 2 siblings, 1 reply; 31+ messages in thread From: Vladislav Shpilevoy @ 2019-05-31 22:36 UTC (permalink / raw) To: Konstantin Osipov, tarantool-patches, Kirill Shcherbatov On 31/05/2019 22:45, Konstantin Osipov wrote: > * Kirill Shcherbatov <kshcherbatov@tarantool.org> [19/05/31 20:34]: >> Hi! Thank you for review. > > We have a yet enother -er class. It seems not to be a coincidence > that all these -er classes are coming from the same contributor. > > Kirill, please rethink the design, I kind of hit the limit of > active classes. > I would agree with Kirill here. 1) Number of classes does not tell anything how good code is. If a task can be solved with a new class better and simpler than with modification of an existing one, then it is ok. Classes genocide does not solve any problem. 2) I do not understand this dig on Kirill. He works on SQL, introduces brand new features, which have never been presented in Tarantool before, in any kind. You could with the same manner dig on yourself, when you added MemtxSpace, MemtxEngine, MemtxTreeIndex, MemtxHashIndex etc Memtx* + Vinyl* classes. Or when we were adding new classes for vinyl internals like vy_run_iterator, vy_write_iterator, vy_cache_iterator, vy_any_other_shit_iterator etc. Or when *you* asked Nikita to add these numerous useless classes like "create_table_def", "create_constraint_def", "alter_def" (see parse_def.h for the others), which just store some values earlier passed as function parameters. A class is ok, if it solves one atomic simple task, IMO. In this concrete patch tuple_fetcher solves the task of not copying tuple data into Vdbe cells, but still fetching fields. 3) The last point by order, but not by importance - the whole 'fetch' theme was your idea. And I good one, to be honest. You saw a proposed solution weeks ago. I explained it in details. Even if you do not like it now for an unknown reason, it is still good and should be implemented. It makes checks *copy-free*. We do not copy any single byte from a tuple. We only keep offsets to its parts. And even more - multiple checks share the same fetcher now, and if a first check filled the offset table, the next checks will reuse it. Sorry, if you feel that this email insults you, that was not on purpose. Just my thoughts about your actions. ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 3/6] sql: introduce tuple_fetcher class 2019-05-31 22:36 ` Vladislav Shpilevoy @ 2019-06-01 5:45 ` Konstantin Osipov 0 siblings, 0 replies; 31+ messages in thread From: Konstantin Osipov @ 2019-06-01 5:45 UTC (permalink / raw) To: Vladislav Shpilevoy; +Cc: tarantool-patches, Kirill Shcherbatov * Vladislav Shpilevoy <v.shpilevoy@tarantool.org> [19/06/01 06:03]: I had very little time to react to the patch, so this was a bit of a rushed response. I will take time to rethink the class layout with Kirill to avoid yet-another-er. I have already suggested in the channel that instead of a fetcher, one should use a proxy-like class, like vdbe_field_proxy, or vdbe_tuple_proxy. Perhaps many of the fetcher methods would be the same, but the object-oriented approach makes you reason about responsibility layout differently. E.g. vdbe_tuple_proxy could have just the same api as the tuple itself, and contain an internal cache for the already fetched fields. But generally this doesn't diminish the fact that this design choice was made and passed the review, even though abuse of "active" classes is a well known anti-pattern. I encourage you to read up on this, so that I don't have to "lecture" anyone. > > On 31/05/2019 22:45, Konstantin Osipov wrote: > > * Kirill Shcherbatov <kshcherbatov@tarantool.org> [19/05/31 20:34]: > >> Hi! Thank you for review. > > > > We have a yet enother -er class. It seems not to be a coincidence > > that all these -er classes are coming from the same contributor. > > > > Kirill, please rethink the design, I kind of hit the limit of > > active classes. > > > > I would agree with Kirill here. > > 1) Number of classes does not tell anything how good code is. If a > task can be solved with a new class better and simpler than with > modification of an existing one, then it is ok. Classes genocide > does not solve any problem. > > 2) I do not understand this dig on Kirill. He works on SQL, > introduces brand new features, which have never been presented > in Tarantool before, in any kind. You could with the same manner > dig on yourself, when you added MemtxSpace, MemtxEngine, MemtxTreeIndex, > MemtxHashIndex etc Memtx* + Vinyl* classes. Or when we were adding > new classes for vinyl internals like vy_run_iterator, vy_write_iterator, > vy_cache_iterator, vy_any_other_shit_iterator etc. Or when *you* asked > Nikita to add these numerous useless classes like "create_table_def", > "create_constraint_def", "alter_def" (see parse_def.h for the others), > which just store some values earlier passed as function parameters. > > A class is ok, if it solves one atomic simple task, IMO. In this > concrete patch tuple_fetcher solves the task of not copying tuple > data into Vdbe cells, but still fetching fields. > > 3) The last point by order, but not by importance - the whole 'fetch' > theme was your idea. And I good one, to be honest. You saw a proposed > solution weeks ago. I explained it in details. Even if you do not > like it now for an unknown reason, it is still good and should be > implemented. It makes checks *copy-free*. We do not copy any single > byte from a tuple. We only keep offsets to its parts. And even more - > multiple checks share the same fetcher now, and if a first check > filled the offset table, the next checks will reuse it. > > Sorry, if you feel that this email insults you, that was not on > purpose. Just my thoughts about your actions. -- Konstantin Osipov, Moscow, Russia ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 3/6] sql: introduce tuple_fetcher class 2019-05-31 19:45 ` Konstantin Osipov 2019-05-31 19:50 ` Kirill Shcherbatov 2019-05-31 22:36 ` Vladislav Shpilevoy @ 2019-06-02 18:50 ` Kirill Shcherbatov 2019-06-03 21:15 ` Vladislav Shpilevoy 2019-06-05 6:47 ` Konstantin Osipov 2 siblings, 2 replies; 31+ messages in thread From: Kirill Shcherbatov @ 2019-06-02 18:50 UTC (permalink / raw) To: tarantool-patches, Konstantin Osipov; +Cc: Vladislav Shpilevoy Hi! I've renamed tuple_fetcher to vdbe_field_ref as you've proposed. Here and everywhere. The corresponding diff below: ================================================ diff --git a/src/box/sql.c b/src/box/sql.c index 07ceec2d2..b4702fd78 100644 --- a/src/box/sql.c +++ b/src/box/sql.c @@ -1345,36 +1345,37 @@ sql_checks_resolve_space_def_reference(ExprList *expr_list, } /** - * Initialize a new tuple_fetcher instance with given tuple + * Initialize a new vdbe_field_ref instance with given tuple * data. - * @param fetcher The tuple_fetcher instance to initialize. + * @param field_ref The vdbe_field_ref instance to initialize. * @param tuple The tuple object pointer or NULL when undefined. * @param data The tuple data (is always defined). * @param data_sz The size of tuple data (is always defined). */ static void -tuple_fetcher_create(struct tuple_fetcher *fetcher, struct tuple *tuple, - const char *data, uint32_t data_sz) +vdbe_field_ref_create(struct vdbe_field_ref *field_ref, struct tuple *tuple, + const char *data, uint32_t data_sz) { - fetcher->tuple = tuple; - fetcher->data = data; - fetcher->data_sz = data_sz; + field_ref->tuple = tuple; + field_ref->data = data; + field_ref->data_sz = data_sz; const char *field0 = data; - fetcher->field_count = mp_decode_array((const char **) &field0); - fetcher->slots[0] = (uint32_t)(field0 - data); - fetcher->rightmost_slot = 0; + field_ref->field_count = mp_decode_array((const char **) &field0); + field_ref->slots[0] = (uint32_t)(field0 - data); + field_ref->rightmost_slot = 0; } void -tuple_fetcher_prepare_data(struct tuple_fetcher *fetcher, const char *data, +vdbe_field_ref_prepare_data(struct vdbe_field_ref *field_ref, const char *data, uint32_t data_sz) { - tuple_fetcher_create(fetcher, NULL, data, data_sz); + vdbe_field_ref_create(field_ref, NULL, data, data_sz); } void -tuple_fetcher_prepare_tuple(struct tuple_fetcher *fetcher, struct tuple *tuple) +vdbe_field_ref_prepare_tuple(struct vdbe_field_ref *field_ref, + struct tuple *tuple) { - tuple_fetcher_create(fetcher, tuple, tuple_data(tuple), tuple->bsize); + vdbe_field_ref_create(field_ref, tuple, tuple_data(tuple), tuple->bsize); } diff --git a/src/box/sql.h b/src/box/sql.h index a13f78ae8..066a0d322 100644 --- a/src/box/sql.h +++ b/src/box/sql.h @@ -394,13 +394,13 @@ sqlSrcListDelete(struct sql *db, struct SrcList *list); * to be solved and is usually equal to the greatest number of * fields in the tuple. * - * +------------------------+ - * | struct tuple_fetcher | - * +------------------------+ - * | RESERVED MEMORY | - * +------------------------+ + * +-------------------------+ + * | struct vdbe_field_ref | + * +-------------------------+ + * | RESERVED MEMORY | + * +-------------------------+ */ -struct tuple_fetcher { +struct vdbe_field_ref { /** Tuple pointer or NULL when undefined. */ struct tuple *tuple; /** Tuple data pointer. */ @@ -422,24 +422,25 @@ struct tuple_fetcher { }; /** - * Initialize a new tuple_fetcher instance with given tuple + * Initialize a new vdbe_field_ref instance with given tuple * data. - * @param fetcher The tuple_fetcher instance to initialize. + * @param field_ref The vdbe_field_ref instance to initialize. * @param data The tuple data. * @param data_sz The size of tuple data. */ void -tuple_fetcher_prepare_data(struct tuple_fetcher *fetcher, const char *data, +vdbe_field_ref_prepare_data(struct vdbe_field_ref *field_ref, const char *data, uint32_t data_sz); /** - * Initialize a new tuple_fetcher instance with given tuple + * Initialize a new vdbe_field_ref instance with given tuple * data. - * @param fetcher The tuple_fetcher instance to initialize. + * @param field_ref The vdbe_field_ref instance to initialize. * @param tuple The tuple object pointer. */ void -tuple_fetcher_prepare_tuple(struct tuple_fetcher *fetcher, struct tuple *tuple); +vdbe_field_ref_prepare_tuple(struct vdbe_field_ref *field_ref, + struct tuple *tuple); #if defined(__cplusplus) } /* extern "C" { */ diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c index 250646c9b..256ee77a0 100644 --- a/src/box/sql/vdbe.c +++ b/src/box/sql/vdbe.c @@ -615,7 +615,7 @@ mem_type_to_str(const struct Mem *p) /** * Try to get a current tuple field using its field map. - * @param tuple Tuple to process. + * @param field_ref The vdbe_field_ref instance to use. * @param fieldno Number of a field to get. * @param[out] field_size Result field size. * @retval not NULL MessagePack field. @@ -623,17 +623,17 @@ mem_type_to_str(const struct Mem *p) * offset to @a fieldno. */ static const void * -tuple_fetcher_tuple_field_fast(struct tuple_fetcher *fetcher, uint32_t fieldno, - uint32_t *field_size) +vdbe_field_ref_tuple_field_fast(struct vdbe_field_ref *field_ref, + uint32_t fieldno, uint32_t *field_size) { - if (fetcher->tuple == NULL) + if (field_ref->tuple == NULL) return NULL; - struct tuple_format *format = tuple_format(fetcher->tuple); + struct tuple_format *format = tuple_format(field_ref->tuple); if (fieldno >= tuple_format_field_count(format) || tuple_format_field(format, fieldno)->offset_slot == TUPLE_OFFSET_SLOT_NIL) return NULL; - const char *field = tuple_field(fetcher->tuple, fieldno); + const char *field = tuple_field(field_ref->tuple, fieldno); const char *end = field; mp_next(&end); *field_size = end - field; @@ -641,21 +641,21 @@ tuple_fetcher_tuple_field_fast(struct tuple_fetcher *fetcher, uint32_t fieldno, } /** - * Fetch field by field_idx using tuple_fetcher and store result + * Fetch field by field_idx using vdbe_field_ref and store result * in dest_mem. - * @param fetcher The initialized tuple_fetcher instance to use. + * @param field_ref The initialized vdbe_field_ref instance to use. * @param field_idx The id of the field to fetch. * @param[out] dest_mem The memory variable to store result. * @retval SQL_OK Status code in case of success. * @retval sql_ret_code Error code otherwise. */ static int -tuple_fetcher_fetch(struct tuple_fetcher *fetcher, uint32_t field_idx, +vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t field_idx, struct Mem *dest_mem) { sqlVdbeMemSetNull(dest_mem); - uint32_t *slots = fetcher->slots; - if (field_idx >= fetcher->field_count) { + uint32_t *slots = field_ref->slots; + if (field_idx >= field_ref->field_count) { UPDATE_MAX_BLOBSIZE(dest_mem); return SQL_OK; } @@ -669,10 +669,10 @@ tuple_fetcher_fetch(struct tuple_fetcher *fetcher, uint32_t field_idx, * field_map+1-th field. */ const char *data; - if (fetcher->rightmost_slot <= field_idx) { + if (field_ref->rightmost_slot <= field_idx) { uint32_t field_sz; - data = tuple_fetcher_tuple_field_fast(fetcher, field_idx, - &field_sz); + data = vdbe_field_ref_tuple_field_fast(field_ref, field_idx, + &field_sz); if (data != NULL) { /* * Special case for tarantool spaces: for @@ -684,17 +684,17 @@ tuple_fetcher_fetch(struct tuple_fetcher *fetcher, uint32_t field_idx, * better - it saves offsets to all fields * visited in mp_next() cycle. */ - uint32_t offset = (uint32_t)(data - fetcher->data); + uint32_t offset = (uint32_t)(data - field_ref->data); slots[field_idx] = offset; slots[field_idx + 1] = offset + field_sz; } else { - uint32_t i = fetcher->rightmost_slot; - data = fetcher->data + slots[i]; + uint32_t i = field_ref->rightmost_slot; + data = field_ref->data + slots[i]; do { mp_next(&data); - slots[++i] = (uint32_t)(data - fetcher->data); + slots[++i] = (uint32_t)(data - field_ref->data); } while (i <= field_idx); - fetcher->rightmost_slot = i; + field_ref->rightmost_slot = i; } } @@ -705,7 +705,7 @@ tuple_fetcher_fetch(struct tuple_fetcher *fetcher, uint32_t field_idx, */ assert(sqlVdbeCheckMemInvariants(dest_mem) != 0); uint32_t dummy; - data = fetcher->data + slots[field_idx]; + data = field_ref->data + slots[field_idx]; if (vdbe_decode_msgpack_into_mem(data, dest_mem, &dummy) != 0) return SQL_TARANTOOL_ERROR; @@ -2747,8 +2747,8 @@ case OP_Column: { pReg = &aMem[pC->uc.pseudoTableReg]; assert(pReg->flags & MEM_Blob); assert(memIsValid(pReg)); - tuple_fetcher_prepare_data(&pC->fetcher, - pReg->z, pReg->n); + vdbe_field_ref_prepare_data(&pC->field_ref, + pReg->z, pReg->n); } else { sqlVdbeMemSetNull(pDest); goto op_column_out; @@ -2760,8 +2760,8 @@ case OP_Column: { assert(sqlCursorIsValid(pCrsr)); assert(pCrsr->curFlags & BTCF_TaCursor || pCrsr->curFlags & BTCF_TEphemCursor); - tuple_fetcher_prepare_tuple(&pC->fetcher, - pCrsr->last_tuple); + vdbe_field_ref_prepare_tuple(&pC->field_ref, + pCrsr->last_tuple); } pC->cacheStatus = p->cacheCtr; } @@ -2783,12 +2783,12 @@ case OP_Column: { } struct Mem *default_val_mem = pOp->p4type == P4_MEM ? pOp->p4.pMem : NULL; - rc = tuple_fetcher_fetch(&pC->fetcher, p2, pDest); + rc = vdbe_field_ref_fetch(&pC->field_ref, p2, pDest); if (rc != SQL_OK) goto abort_due_to_error; if ((pDest->flags & MEM_Null) && - (uint32_t) p2 >= pC->fetcher.field_count && + (uint32_t) p2 >= pC->field_ref.field_count && default_val_mem != NULL) { sqlVdbeMemShallowCopy(pDest, default_val_mem, MEM_Static); } @@ -2804,7 +2804,7 @@ op_column_out: /* Opcode: Fetch P1 P2 P3 P4 P5 * Synopsis: r[P3]=PX * - * Interpret data P1 points at as an initialized tuple_fetcher + * Interpret data P1 points at as an initialized vdbe_field_ref * object. * * Fetch the P2-th column from its tuple. The value extracted @@ -2812,12 +2812,12 @@ op_column_out: * P2 fields, then extract a NULL. */ case OP_Fetch: { - struct tuple_fetcher *fetcher = - (struct tuple_fetcher *) p->aMem[pOp->p1].u.p; + struct vdbe_field_ref *field_ref = + (struct vdbe_field_ref *) p->aMem[pOp->p1].u.p; uint32_t field_idx = pOp->p2; struct Mem *dest_mem = &aMem[pOp->p3]; memAboutToChange(p, dest_mem); - rc = tuple_fetcher_fetch(fetcher, field_idx, dest_mem); + rc = vdbe_field_ref_fetch(field_ref, field_idx, dest_mem); if (rc != SQL_OK) goto abort_due_to_error; REGISTER_TRACE(p, pOp->p3, dest_mem); diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h index 1a428afa9..68e867fbe 100644 --- a/src/box/sql/vdbeInt.h +++ b/src/box/sql/vdbeInt.h @@ -119,7 +119,7 @@ struct VdbeCursor { * Auxiliary VDBE structure to speed-up tuple data * field access. */ - struct tuple_fetcher fetcher; + struct vdbe_field_ref field_ref; }; /* ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 3/6] sql: introduce tuple_fetcher class 2019-06-02 18:50 ` Kirill Shcherbatov @ 2019-06-03 21:15 ` Vladislav Shpilevoy 2019-06-05 6:47 ` Konstantin Osipov 1 sibling, 0 replies; 31+ messages in thread From: Vladislav Shpilevoy @ 2019-06-03 21:15 UTC (permalink / raw) To: tarantool-patches, Kirill Shcherbatov, Konstantin Osipov Hi! Thanks for the fixes! See 2 comments below, review fixes on the branch, and at the end of the email. > commit 80251ea01b0eeca9cf8bcb50f4b97d97bbcb41ca > Author: Kirill Shcherbatov <kshcherbatov@tarantool.org> > Date: Mon May 20 19:28:42 2019 +0300 > > sql: introduce vdbe_field_ref class > > Refactored OP_Column instruction with a new vdbe_field_ref class. > The vdbe_field_ref is a reusable object that speed-up field 1. 'speed' -> 'speeds'. > access for given tuple or tuple data. > Introduced OP_Fetch opcode that uses tuple_fetcher given as a 2. There are no 'tuple_fetcher' anymore. > first argument. This opcode makes possible to perform binding > of a new tuple to an existent VDBE without decoding it's fields. > > Needed for #3691 > I've pushed my review fixes in a separate commit. I didn't fix the commit message, because I can't do it without amendment. Below are my explanations, and diff. 1) I fixed indentation and width > 80. 2) I renamed vdbe_field_ref_tuple_field_fast to vdbe_field_ref_fast_fetch. I didn't like tautology with word 'field' used twice. 3) I dropped a couple of very outdated comments from fetch() function. 4) I fixed a comment for OP_Fetch with declared but unused P4 and P5. 5) I renames field_idx to fieldno in fetch() to be consistent with fast_fetch(). On the whole, the patch is great, even without CHECKs stuff. It strongly improves tuples integration into Vdbe, which beforehand treated them as mere arrays. ================================================================== diff --git a/src/box/sql.c b/src/box/sql.c index b4702fd78..f7b90ab5c 100644 --- a/src/box/sql.c +++ b/src/box/sql.c @@ -1368,7 +1368,7 @@ vdbe_field_ref_create(struct vdbe_field_ref *field_ref, struct tuple *tuple, void vdbe_field_ref_prepare_data(struct vdbe_field_ref *field_ref, const char *data, - uint32_t data_sz) + uint32_t data_sz) { vdbe_field_ref_create(field_ref, NULL, data, data_sz); } @@ -1377,5 +1377,6 @@ void vdbe_field_ref_prepare_tuple(struct vdbe_field_ref *field_ref, struct tuple *tuple) { - vdbe_field_ref_create(field_ref, tuple, tuple_data(tuple), tuple->bsize); + vdbe_field_ref_create(field_ref, tuple, tuple_data(tuple), + tuple->bsize); } diff --git a/src/box/sql.h b/src/box/sql.h index 066a0d322..3056d8f7e 100644 --- a/src/box/sql.h +++ b/src/box/sql.h @@ -430,7 +430,7 @@ struct vdbe_field_ref { */ void vdbe_field_ref_prepare_data(struct vdbe_field_ref *field_ref, const char *data, - uint32_t data_sz); + uint32_t data_sz); /** * Initialize a new vdbe_field_ref instance with given tuple diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c index 256ee77a0..3f0b856b3 100644 --- a/src/box/sql/vdbe.c +++ b/src/box/sql/vdbe.c @@ -623,8 +623,8 @@ mem_type_to_str(const struct Mem *p) * offset to @a fieldno. */ static const void * -vdbe_field_ref_tuple_field_fast(struct vdbe_field_ref *field_ref, - uint32_t fieldno, uint32_t *field_size) +vdbe_field_ref_fast_fetch(struct vdbe_field_ref *field_ref, uint32_t fieldno, + uint32_t *field_size) { if (field_ref->tuple == NULL) return NULL; @@ -641,38 +641,30 @@ vdbe_field_ref_tuple_field_fast(struct vdbe_field_ref *field_ref, } /** - * Fetch field by field_idx using vdbe_field_ref and store result + * Fetch field by fieldno using vdbe_field_ref and store result * in dest_mem. * @param field_ref The initialized vdbe_field_ref instance to use. - * @param field_idx The id of the field to fetch. + * @param fieldno The id of the field to fetch. * @param[out] dest_mem The memory variable to store result. * @retval SQL_OK Status code in case of success. * @retval sql_ret_code Error code otherwise. */ static int -vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t field_idx, - struct Mem *dest_mem) +vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t fieldno, + struct Mem *dest_mem) { sqlVdbeMemSetNull(dest_mem); uint32_t *slots = field_ref->slots; - if (field_idx >= field_ref->field_count) { + if (fieldno >= field_ref->field_count) { UPDATE_MAX_BLOBSIZE(dest_mem); return SQL_OK; } - /* - * Make sure at least the first field_idx + 1 entries - * of the header have been parsed and valid information - * is in field_map[]. - * If there is more header available for parsing in the - * record, try to extract additional fields up through the - * field_map+1-th field. - */ const char *data; - if (field_ref->rightmost_slot <= field_idx) { + if (field_ref->rightmost_slot <= fieldno) { uint32_t field_sz; - data = vdbe_field_ref_tuple_field_fast(field_ref, field_idx, - &field_sz); + data = vdbe_field_ref_fast_fetch(field_ref, fieldno, + &field_sz); if (data != NULL) { /* * Special case for tarantool spaces: for @@ -685,27 +677,22 @@ vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t field_idx, * visited in mp_next() cycle. */ uint32_t offset = (uint32_t)(data - field_ref->data); - slots[field_idx] = offset; - slots[field_idx + 1] = offset + field_sz; + slots[fieldno] = offset; + slots[fieldno + 1] = offset + field_sz; } else { uint32_t i = field_ref->rightmost_slot; data = field_ref->data + slots[i]; do { mp_next(&data); slots[++i] = (uint32_t)(data - field_ref->data); - } while (i <= field_idx); + } while (i <= fieldno); field_ref->rightmost_slot = i; } } - /* - * Extract the content for the p2+1-th column. Control - * can only reach this point if field_map[field_idx], - * field_map[field_idx+1] are valid. - */ assert(sqlVdbeCheckMemInvariants(dest_mem) != 0); uint32_t dummy; - data = field_ref->data + slots[field_idx]; + data = field_ref->data + slots[fieldno]; if (vdbe_decode_msgpack_into_mem(data, dest_mem, &dummy) != 0) return SQL_TARANTOOL_ERROR; @@ -714,7 +701,7 @@ vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t field_idx, * Wrap it in a blob verbatim. */ if (dest_mem->flags == 0) { - dest_mem->n = slots[field_idx + 1] - slots[field_idx]; + dest_mem->n = slots[fieldno + 1] - slots[fieldno]; dest_mem->z = (char *) data; dest_mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype; dest_mem->subtype = SQL_SUBTYPE_MSGPACK; @@ -723,7 +710,8 @@ vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t field_idx, * Add 0 termination (at most for strings) * Not sure why do we check MEM_Ephem */ - if ((dest_mem->flags & (MEM_Ephem | MEM_Str)) == (MEM_Ephem | MEM_Str)) { + if ((dest_mem->flags & (MEM_Ephem | MEM_Str)) == + (MEM_Ephem | MEM_Str)) { int len = dest_mem->n; if (dest_mem->szMalloc < len + 1) { if (sqlVdbeMemGrow(dest_mem, len + 1, 1) != 0) @@ -2801,7 +2789,7 @@ op_column_out: break; } -/* Opcode: Fetch P1 P2 P3 P4 P5 +/* Opcode: Fetch P1 P2 P3 * * * Synopsis: r[P3]=PX * * Interpret data P1 points at as an initialized vdbe_field_ref ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 3/6] sql: introduce tuple_fetcher class 2019-06-02 18:50 ` Kirill Shcherbatov 2019-06-03 21:15 ` Vladislav Shpilevoy @ 2019-06-05 6:47 ` Konstantin Osipov 2019-06-05 6:48 ` Konstantin Osipov 1 sibling, 1 reply; 31+ messages in thread From: Konstantin Osipov @ 2019-06-05 6:47 UTC (permalink / raw) To: tarantool-patches; +Cc: Vladislav Shpilevoy * Kirill Shcherbatov <kshcherbatov@tarantool.org> [19/06/03 14:34]: > Hi! I've renamed tuple_fetcher to vdbe_field_ref as you've proposed. > Here and everywhere. The corresponding diff below: > A rename is only the first step. It's also about managing the responsibility. As far as I can see, the new class is a proxy for an entire tuple, not a single field. This doesn't seem convenient going forward, but since there is a shared state (the slots array), let it be. But then the name is wrong, it's vdbe_tuple_ref, not vdbe_field_ref. Why did you blindly rename without objecting to a another stupid name? The next thing is managing the responsibility for the tuple itself. If we begin to think that this is a tuple ref from vdbe, then this ref should own the tuple, and vdbe should always access the tuple using this ref. So vdbe_field_ref_create should ref the tuple, vdbe_field_ref_destroy() should free it. Again, please object if it doesn't make sense - or it may require some refactoring (likely). Finally, the choice of the representation for the slots array and the implementation itself. When initializing the fetcher class, you do not consult with vdbe what tuple fields are used. Yet you do know it from the parsing - and only select fields (one or two) are used in most cases. For example: CHECK age >=0 and age <= 170 -- sane age So you don't need to allocate or store a lot of slots in most cases. Moreover, perhaps it's wise to only use an optimization of there are few slots in use, and cache Mem objects, not offsets to fields, and if there are too many slots in use, do nothing? Please keep in mind that the same object could be used for WHERE, HAVING or other clauses in the future. -- Konstantin Osipov, Moscow, Russia ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 3/6] sql: introduce tuple_fetcher class 2019-06-05 6:47 ` Konstantin Osipov @ 2019-06-05 6:48 ` Konstantin Osipov 0 siblings, 0 replies; 31+ messages in thread From: Konstantin Osipov @ 2019-06-05 6:48 UTC (permalink / raw) To: tarantool-patches; +Cc: Vladislav Shpilevoy * Konstantin Osipov <kostja@tarantool.org> [19/06/05 09:47]: All in all the job turns out to be big and complex enough so that you please feel free to extract it into a separate patch. I do not insist, I just don't want to hold up the push. On the other hand, I do think it's critical enough to not postpone it any further. -- Konstantin Osipov, Moscow, Russia ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] [PATCH v5 4/6] schema: add new system space for CHECK constraints 2019-05-23 10:19 [tarantool-patches] [PATCH v5 0/6] box: run checks on insertions in LUA spaces Kirill Shcherbatov ` (2 preceding siblings ...) 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 3/6] sql: introduce tuple_fetcher class Kirill Shcherbatov @ 2019-05-23 10:19 ` Kirill Shcherbatov 2019-05-26 12:06 ` [tarantool-patches] " Vladislav Shpilevoy 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 5/6] box: run check constraint tests on space alter Kirill Shcherbatov ` (3 subsequent siblings) 7 siblings, 1 reply; 31+ messages in thread From: Kirill Shcherbatov @ 2019-05-23 10:19 UTC (permalink / raw) To: tarantool-patches, v.shpilevoy; +Cc: Kirill Shcherbatov This patch introduces a new system space to persist check constraints. The format of the new system space is _ck_constraint (space id = 357) [<space id> UINT, <constraint name> STR, <is_deferred>BOOL, <language>STR], <code>STR A CK constraint is local for a space, so every pair <space id, CK name> is unique (it is also the PK in the _ck_constraint space). After insertion into this space, a new instance describing check constraint is created. Check constraint holds an exspression AST. While space features some check constraints, it isn't allowed to be dropped. The :drop() space method firstly deletes all check constraints and then removes an entry from the _space. Because the space alter, the index alter and the space truncate operations cause space recreation process, a new RebuildCkConstrains object is introduced. This alter object compiles a new ck constraint object, replaces and removes an existent instances atomically (but if the assembly of some ck constraint object fails, nothing is changed). In fact, in scope of this patch we don't really need to recreate a ck_constraint object in such situations (it is enough to patch space_def pointer in AST tree like we did it before, but we are going to recompile a VDBE that represents ck constraint in further patches, and that operation is not safe). The main motivation for these changes is an ability to support ADD CHECK CONSTRAINT operation in the future. CK constraints are easier to manage as self-sustained objects: such change is managed with atomic insertion(unlike the current architecture). Finally, the xfer optimization is disabled now if some space have ck constraints. In following patches this xfer optimisation becomes impossible, so there is no reason to rewrite this code now. Needed for #3691 --- src/box/CMakeLists.txt | 1 + src/box/alter.cc | 267 ++++++++++++++++++++++++-- src/box/alter.h | 1 + src/box/bootstrap.snap | Bin 4379 -> 4430 bytes src/box/ck_constraint.c | 117 +++++++++++ src/box/ck_constraint.h | 177 +++++++++++++++++ src/box/errcode.h | 3 +- src/box/lua/schema.lua | 4 + src/box/lua/space.cc | 2 + src/box/lua/upgrade.lua | 40 ++++ src/box/schema.cc | 8 + src/box/schema_def.h | 11 ++ src/box/space.c | 2 + src/box/space.h | 5 + src/box/space_def.c | 98 +--------- src/box/space_def.h | 4 - src/box/sql.c | 86 +-------- src/box/sql.h | 36 ---- src/box/sql/build.c | 220 +++++++++++++++++---- src/box/sql/insert.c | 55 +++--- src/box/sql/parse.y | 2 +- src/box/sql/parse_def.h | 24 +++ src/box/sql/select.c | 11 +- src/box/sql/sqlInt.h | 2 +- src/box/sql/tokenize.c | 1 - test/app-tap/tarantoolctl.test.lua | 4 +- test/box-py/bootstrap.result | 4 + test/box/access.result | 3 + test/box/access.test.lua | 1 + test/box/access_misc.result | 3 + test/box/access_sysview.result | 6 +- test/box/alter.result | 5 +- test/box/misc.result | 1 + test/sql-tap/check.test.lua | 32 +-- test/sql-tap/fkey2.test.lua | 4 +- test/sql-tap/sql-errors.test.lua | 2 +- test/sql-tap/table.test.lua | 4 +- test/sql/checks.result | 224 +++++++++++++++++---- test/sql/checks.test.lua | 104 +++++++--- test/sql/errinj.result | 134 +++++++++++++ test/sql/errinj.test.lua | 45 +++++ test/sql/gh-2981-check-autoinc.result | 8 +- test/sql/types.result | 2 +- test/sql/upgrade.result | 19 ++ test/sql/upgrade.test.lua | 5 + test/wal_off/alter.result | 2 +- 46 files changed, 1376 insertions(+), 413 deletions(-) create mode 100644 src/box/ck_constraint.c create mode 100644 src/box/ck_constraint.h diff --git a/src/box/CMakeLists.txt b/src/box/CMakeLists.txt index 0864c3433..481842a39 100644 --- a/src/box/CMakeLists.txt +++ b/src/box/CMakeLists.txt @@ -97,6 +97,7 @@ add_library(box STATIC space.c space_def.c sequence.c + ck_constraint.c fk_constraint.c func.c func_def.c diff --git a/src/box/alter.cc b/src/box/alter.cc index 965945af6..463357c67 100644 --- a/src/box/alter.cc +++ b/src/box/alter.cc @@ -29,6 +29,7 @@ * SUCH DAMAGE. */ #include "alter.h" +#include "ck_constraint.h" #include "column_mask.h" #include "schema.h" #include "user.h" @@ -535,17 +536,6 @@ space_def_new_from_tuple(struct tuple *tuple, uint32_t errcode, engine_name, engine_name_len, &opts, fields, field_count); auto def_guard = make_scoped_guard([=] { space_def_delete(def); }); - if (def->opts.checks != NULL && - sql_checks_resolve_space_def_reference(def->opts.checks, - def) != 0) { - box_error_t *err = box_error_last(); - if (box_error_code(err) != ENOMEM) { - tnt_raise(ClientError, errcode, def->name, - box_error_message(err)); - } else { - diag_raise(); - } - } struct engine *engine = engine_find_xc(def->engine_name); engine_check_space_def_xc(engine, def); def_guard.is_active = false; @@ -1387,6 +1377,79 @@ UpdateSchemaVersion::alter(struct alter_space *alter) ++schema_version; } +/** + * As ck_constraint object depends on space_def we must rebuild + * all ck constraints on space alter. + * + * To perform it transactionally, we create a list of new ck + * constraints objects in ::prepare method that is fault-tolerant. + * Finally in ::alter or ::rollback methods we only swap thouse + * lists securely. + */ +class RebuildCkConstraints: public AlterSpaceOp +{ + void space_swap_ck_constraint(struct space *old_space, + struct space *new_space); +public: + RebuildCkConstraints(struct alter_space *alter) : AlterSpaceOp(alter), + ck_constraint(RLIST_HEAD_INITIALIZER(ck_constraint)) {} + struct rlist ck_constraint; + virtual void prepare(struct alter_space *alter); + virtual void alter(struct alter_space *alter); + virtual void rollback(struct alter_space *alter); + virtual ~RebuildCkConstraints(); +}; + +void +RebuildCkConstraints::prepare(struct alter_space *alter) +{ + struct ck_constraint *old_ck_constraint; + rlist_foreach_entry(old_ck_constraint, &alter->old_space->ck_constraint, + link) { + struct ck_constraint *new_ck_constraint = + ck_constraint_new(old_ck_constraint->def, + alter->new_space->def); + if (new_ck_constraint == NULL) + diag_raise(); + rlist_add_entry(&ck_constraint, new_ck_constraint, link); + } +} + +void +RebuildCkConstraints::space_swap_ck_constraint(struct space *old_space, + struct space *new_space) +{ + rlist_swap(&new_space->ck_constraint, &ck_constraint); + rlist_swap(&ck_constraint, &old_space->ck_constraint); +} + +void +RebuildCkConstraints::alter(struct alter_space *alter) +{ + space_swap_ck_constraint(alter->old_space, alter->new_space); +} + +void +RebuildCkConstraints::rollback(struct alter_space *alter) +{ + space_swap_ck_constraint(alter->new_space, alter->old_space); +} + +RebuildCkConstraints::~RebuildCkConstraints() +{ + struct ck_constraint *old_ck_constraint, *tmp; + rlist_foreach_entry_safe(old_ck_constraint, &ck_constraint, link, tmp) { + /** + * Ck constraint definition is now managed by + * other Ck constraint object. Prevent it's + * destruction as a part of ck_constraint_delete + * call. + */ + old_ck_constraint->def = NULL; + ck_constraint_delete(old_ck_constraint); + } +} + /* }}} */ /** @@ -1752,6 +1815,11 @@ on_replace_dd_space(struct trigger * /* trigger */, void *event) space_name(old_space), "the space has foreign key constraints"); } + if (!rlist_empty(&old_space->ck_constraint)) { + tnt_raise(ClientError, ER_DROP_SPACE, + space_name(old_space), + "the space has check constraints"); + } /** * The space must be deleted from the space * cache right away to achieve linearisable @@ -1849,6 +1917,7 @@ on_replace_dd_space(struct trigger * /* trigger */, void *event) def->field_count); (void) new CheckSpaceFormat(alter); (void) new ModifySpace(alter, def); + (void) new RebuildCkConstraints(alter); def_guard.is_active = false; /* Create MoveIndex ops for all space indexes. */ alter_space_move_indexes(alter, 0, old_space->index_id_max + 1); @@ -2091,6 +2160,7 @@ on_replace_dd_index(struct trigger * /* trigger */, void *event) * old space. */ alter_space_move_indexes(alter, iid + 1, old_space->index_id_max + 1); + (void) new RebuildCkConstraints(alter); /* Add an op to update schema_version on commit. */ (void) new UpdateSchemaVersion(alter); alter_space_do(txn, alter); @@ -2159,6 +2229,7 @@ on_replace_dd_truncate(struct trigger * /* trigger */, void *event) (void) new TruncateIndex(alter, old_index->def->iid); } + (void) new RebuildCkConstraints(alter); alter_space_do(txn, alter); scoped_guard.is_active = false; } @@ -4040,6 +4111,176 @@ on_replace_dd_fk_constraint(struct trigger * /* trigger*/, void *event) } } +/** Create an instance of check constraint definition by tuple. */ +static struct ck_constraint_def * +ck_constraint_def_new_from_tuple(struct tuple *tuple) +{ + uint32_t name_len; + const char *name = + tuple_field_str_xc(tuple, BOX_CK_CONSTRAINT_FIELD_NAME, + &name_len); + if (name_len > BOX_NAME_MAX) { + tnt_raise(ClientError, ER_CREATE_CK_CONSTRAINT, + tt_cstr(name, BOX_INVALID_NAME_MAX), + "check constraint name is too long"); + } + identifier_check_xc(name, name_len); + const char *language_str = + tuple_field_cstr_xc(tuple, BOX_CK_CONSTRAINT_FIELD_LANGUAGE); + enum ck_constraint_language language = + STR2ENUM(ck_constraint_language, language_str); + if (language == ck_constraint_language_MAX) { + tnt_raise(ClientError, ER_FUNCTION_LANGUAGE, language_str, + tt_cstr(name, name_len)); + } + uint32_t expr_str_len; + const char *expr_str = + tuple_field_str_xc(tuple, BOX_CK_CONSTRAINT_FIELD_CODE, + &expr_str_len); + uint32_t name_offset, expr_str_offset; + uint32_t ck_def_sz = + ck_constraint_def_sizeof(name_len, expr_str_len, &name_offset, + &expr_str_offset); + struct ck_constraint_def *ck_def = + (struct ck_constraint_def *)malloc(ck_def_sz); + if (ck_def == NULL) + tnt_raise(OutOfMemory, ck_def_sz, "malloc", "ck_def"); + + ck_def->name = (char *)ck_def + name_offset; + ck_def->expr_str = (char *)ck_def + expr_str_offset; + ck_def->language = language; + memcpy(ck_def->expr_str, expr_str, expr_str_len); + ck_def->expr_str[expr_str_len] = '\0'; + memcpy(ck_def->name, name, name_len); + ck_def->name[name_len] = '\0'; + + return ck_def; +} + +/** Trigger invoked on rollback in the _ck_constraint space. */ +static void +on_replace_ck_constraint_rollback(struct trigger *trigger, void *event) +{ + struct txn_stmt *stmt = txn_last_stmt((struct txn *) event); + struct ck_constraint *ck = (struct ck_constraint *)trigger->data; + struct space *space = NULL; + if (ck != NULL) + space = space_by_id(ck->space_id); + if (stmt->old_tuple != NULL && stmt->new_tuple == NULL) { + /* Rollback DELETE check constraint. */ + assert(ck != NULL); + assert(space != NULL); + assert(space_ck_constraint_by_name(space, + ck->def->name, strlen(ck->def->name)) == NULL); + rlist_add_entry(&space->ck_constraint, ck, link); + } else if (stmt->new_tuple != NULL && stmt->old_tuple == NULL) { + /* Rollback INSERT check constraint. */ + assert(space != NULL); + assert(space_ck_constraint_by_name(space, + ck->def->name, strlen(ck->def->name)) != NULL); + rlist_del_entry(ck, link); + ck_constraint_delete(ck); + } else { + /* Rollback REPLACE check constraint. */ + assert(space != NULL); + const char *name = ck->def->name; + struct ck_constraint *new_ck = + space_ck_constraint_by_name(space, name, strlen(name)); + assert(new_ck != NULL); + rlist_del_entry(new_ck, link); + rlist_add_entry(&space->ck_constraint, ck, link); + ck_constraint_delete(new_ck); + } +} + +/** + * Trigger invoked on commit in the _ck_constraint space. + * Drop useless old check constraint object if exists. + */ +static void +on_replace_ck_constraint_commit(struct trigger *trigger, void *event) +{ + struct txn_stmt *stmt = txn_last_stmt((struct txn *) event); + struct ck_constraint *ck = (struct ck_constraint *)trigger->data; + if (stmt->old_tuple != NULL) + ck_constraint_delete(ck); +} + +/** A trigger invoked on replace in the _ck_constraint space. */ +static void +on_replace_dd_ck_constraint(struct trigger * /* trigger*/, void *event) +{ + struct txn *txn = (struct txn *) event; + txn_check_singlestatement_xc(txn, "Space _ck_constraint"); + struct txn_stmt *stmt = txn_current_stmt(txn); + struct tuple *old_tuple = stmt->old_tuple; + struct tuple *new_tuple = stmt->new_tuple; + uint32_t space_id = + tuple_field_u32_xc(old_tuple != NULL ? old_tuple : new_tuple, + BOX_CK_CONSTRAINT_FIELD_SPACE_ID); + struct space *space = space_cache_find_xc(space_id); + struct trigger *on_rollback = + txn_alter_trigger_new(on_replace_ck_constraint_rollback, NULL); + struct trigger *on_commit = + txn_alter_trigger_new(on_replace_ck_constraint_commit, NULL); + + if (new_tuple != NULL) { + bool is_deferred = + tuple_field_bool_xc(new_tuple, + BOX_CK_CONSTRAINT_FIELD_DEFERRED); + if (is_deferred) { + tnt_raise(ClientError, ER_UNSUPPORTED, "Tarantool", + "deferred ck constraints"); + } + /* Create or replace check constraint. */ + struct ck_constraint_def *ck_def = + ck_constraint_def_new_from_tuple(new_tuple); + auto ck_guard = make_scoped_guard([=] { free(ck_def); }); + /* + * FIXME: Ck constraint creation on non-empty + * space must be implemented as preparatory + * step for ALTER SPACE ADD CONSTRAINT feature. + */ + struct index *pk = space_index(space, 0); + if (pk != NULL && index_size(pk) > 0) { + tnt_raise(ClientError, ER_CREATE_CK_CONSTRAINT, + ck_def->name, + "referencing space must be empty"); + } + struct ck_constraint *new_ck_constraint = + ck_constraint_new(ck_def, space->def); + if (new_ck_constraint == NULL) + diag_raise(); + ck_guard.is_active = false; + const char *name = new_ck_constraint->def->name; + struct ck_constraint *old_ck_constraint = + space_ck_constraint_by_name(space, name, strlen(name)); + if (old_ck_constraint != NULL) + rlist_del_entry(old_ck_constraint, link); + rlist_add_entry(&space->ck_constraint, new_ck_constraint, link); + on_commit->data = old_tuple == NULL ? new_ck_constraint : + old_ck_constraint; + on_rollback->data = on_commit->data; + } else { + assert(new_tuple == NULL && old_tuple != NULL); + /* Drop check constraint. */ + uint32_t name_len; + const char *name = + tuple_field_str_xc(old_tuple, + BOX_CK_CONSTRAINT_FIELD_NAME, + &name_len); + struct ck_constraint *old_ck_constraint = + space_ck_constraint_by_name(space, name, name_len); + assert(old_ck_constraint != NULL); + rlist_del_entry(old_ck_constraint, link); + on_commit->data = old_ck_constraint; + on_rollback->data = old_ck_constraint; + } + + txn_on_rollback(txn, on_rollback); + txn_on_commit(txn, on_commit); +} + struct trigger alter_space_on_replace_space = { RLIST_LINK_INITIALIZER, on_replace_dd_space, NULL, NULL }; @@ -4108,4 +4349,8 @@ struct trigger on_replace_fk_constraint = { RLIST_LINK_INITIALIZER, on_replace_dd_fk_constraint, NULL, NULL }; +struct trigger on_replace_ck_constraint = { + RLIST_LINK_INITIALIZER, on_replace_dd_ck_constraint, NULL, NULL +}; + /* vim: set foldmethod=marker */ diff --git a/src/box/alter.h b/src/box/alter.h index 4108fa47c..b9ba7b846 100644 --- a/src/box/alter.h +++ b/src/box/alter.h @@ -46,6 +46,7 @@ extern struct trigger on_replace_sequence_data; extern struct trigger on_replace_space_sequence; extern struct trigger on_replace_trigger; extern struct trigger on_replace_fk_constraint; +extern struct trigger on_replace_ck_constraint; extern struct trigger on_stmt_begin_space; extern struct trigger on_stmt_begin_index; extern struct trigger on_stmt_begin_truncate; diff --git a/src/box/bootstrap.snap b/src/box/bootstrap.snap index 8c3b10e8417494edba07e6eba78377226ee5a31a..9a285781e83fbf3f3d4150a72b152e27064f124f 100644 diff --git a/src/box/ck_constraint.c b/src/box/ck_constraint.c new file mode 100644 index 000000000..c74687b31 --- /dev/null +++ b/src/box/ck_constraint.c @@ -0,0 +1,117 @@ +/* + * Copyright 2010-2019, Tarantool AUTHORS, please see AUTHORS file. + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * 1. Redistributions of source code must retain the above + * copyright notice, this list of conditions and the + * following disclaimer. + * + * 2. Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL + * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF + * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ +#include "box/session.h" +#include "ck_constraint.h" +#include "errcode.h" +#include "sql.h" +#include "sql/sqlInt.h" + +const char *ck_constraint_language_strs[] = {"SQL"}; + +/** + * Resolve space_def references for check constraint via AST + * tree traversal. + * @param expr Check constraint AST object to update. + * @param space_def Space definition to use. + * @retval 0 On success. + * @retval -1 On error. + */ +static int +ck_constraint_resolve_field_names(struct Expr *expr, + struct space_def *space_def) +{ + struct Parse parser; + sql_parser_create(&parser, sql_get(), default_flags); + parser.parse_only = true; + sql_resolve_self_reference(&parser, space_def, NC_IsCheck, expr, NULL); + int rc = parser.is_aborted ? -1 : 0; + sql_parser_destroy(&parser); + return rc; +} + +struct ck_constraint * +ck_constraint_new(struct ck_constraint_def *ck_constraint_def, + struct space_def *space_def) +{ + if (space_def->field_count == 0) { + diag_set(ClientError, ER_UNSUPPORTED, "Tarantool", + "CK constraint for space without format"); + return NULL; + } + struct ck_constraint *ck_constraint = malloc(sizeof(*ck_constraint)); + if (ck_constraint == NULL) { + diag_set(OutOfMemory, sizeof(*ck_constraint), "malloc", + "ck_constraint"); + return NULL; + } + ck_constraint->def = NULL; + ck_constraint->space_id = space_def->id; + rlist_create(&ck_constraint->link); + ck_constraint->expr = + sql_expr_compile(sql_get(), ck_constraint_def->expr_str, + strlen(ck_constraint_def->expr_str)); + if (ck_constraint->expr == NULL || + ck_constraint_resolve_field_names(ck_constraint->expr, + space_def) != 0) { + diag_set(ClientError, ER_CREATE_CK_CONSTRAINT, + ck_constraint_def->name, + box_error_message(box_error_last())); + goto error; + } + + ck_constraint->def = ck_constraint_def; + return ck_constraint; +error: + ck_constraint_delete(ck_constraint); + return NULL; +} + +void +ck_constraint_delete(struct ck_constraint *ck_constraint) +{ + sql_expr_delete(sql_get(), ck_constraint->expr, false); + free(ck_constraint->def); + TRASH(ck_constraint); + free(ck_constraint); +} + +struct ck_constraint * +space_ck_constraint_by_name(struct space *space, const char *name, + uint32_t name_len) +{ + struct ck_constraint *ck_constraint = NULL; + rlist_foreach_entry(ck_constraint, &space->ck_constraint, link) { + if (strlen(ck_constraint->def->name) == name_len && + memcmp(ck_constraint->def->name, name, name_len) == 0) + return ck_constraint; + } + return NULL; +} diff --git a/src/box/ck_constraint.h b/src/box/ck_constraint.h new file mode 100644 index 000000000..e20203bb5 --- /dev/null +++ b/src/box/ck_constraint.h @@ -0,0 +1,177 @@ +#ifndef INCLUDES_BOX_CK_CONSTRAINT_H +#define INCLUDES_BOX_CK_CONSTRAINT_H +/* + * Copyright 2010-2019, Tarantool AUTHORS, please see AUTHORS file. + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * 1. Redistributions of source code must retain the above + * copyright notice, this list of conditions and the + * following disclaimer. + * + * 2. Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL + * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF + * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <stdint.h> +#include "small/rlist.h" + +#if defined(__cplusplus) +extern "C" { +#endif + +struct space; +struct space_def; +struct Expr; + +/** Supported languages of ck constraint. */ +enum ck_constraint_language { + CK_CONSTRAINT_LANGUAGE_SQL, + ck_constraint_language_MAX, +}; + +/** The supported languages strings. */ +extern const char *ck_constraint_language_strs[]; + +/** + * Check constraint definition. + * See ck_constraint_def_sizeof() definition for implementation + * details and memory layout. + */ +struct ck_constraint_def { + /** + * The 0-terminated string, a name of the check + * constraint. Must be unique for a given space. + */ + char *name; + /** + * The 0-terminated string that defines check constraint + * expression. + * + * For instance: "field1 + field2 > 2 * 3". + */ + char *expr_str; + /** The language of ck constraint. */ + enum ck_constraint_language language; +}; + +/** + * Structure representing ck constraint. + * See ck_constraint_new() definition. + */ +struct ck_constraint { + /** The check constraint definition. */ + struct ck_constraint_def *def; + /** + * The check constraint expression AST is built for + * ck_constraint::def::expr_str with sql_expr_compile + * and resolved with sql_resolve_self_reference for + * space with space[ck_constraint::space_id] definition. + */ + struct Expr *expr; + /** + * The id of the space this check constraint is + * built for. + */ + uint32_t space_id; + /** + * Organize check constraint structs into linked list + * with space::ck_constraint. + */ + struct rlist link; +}; + +/** + * Calculate check constraint definition memory size and fields + * offsets for given arguments. + * + * Alongside with struct ck_constraint_def itself, we reserve + * memory for string containing its name and expression string. + * + * Memory layout: + * +-----------------------------+ <- Allocated memory starts here + * | struct ck_constraint_def | + * |-----------------------------| + * | name + \0 | + * |-----------------------------| + * | expr_str + \0 | + * +-----------------------------+ + * + * @param name_len The length of the name. + * @param expr_str_len The length of the expr_str. + * @param[out] name_offset The offset of the name string. + * @param[out] expr_str_offset The offset of the expr_str string. + * @retval The size of the ck constraint definition object for + * given parameters. + */ +static inline uint32_t +ck_constraint_def_sizeof(uint32_t name_len, uint32_t expr_str_len, + uint32_t *name_offset, uint32_t *expr_str_offset) +{ + *name_offset = sizeof(struct ck_constraint_def); + *expr_str_offset = *name_offset + name_len + 1; + return *expr_str_offset + expr_str_len + 1; +} + +/** + * Create a new check constraint object by given check constraint + * definition and definition of the space this constraint is + * related to. + * + * @param ck_constraint_def The check constraint definition object + * to use. Expected to be allocated with + * malloc. Ck constraint object manages + * this allocation in case of successful + * creation. + * @param space_def The space definition of the space this check + * constraint must be constructed for. + * @retval not NULL Check constraint object pointer on success. + * @retval NULL Otherwise. The diag message is set. +*/ +struct ck_constraint * +ck_constraint_new(struct ck_constraint_def *ck_constraint_def, + struct space_def *space_def); + +/** + * Destroy check constraint memory, release acquired resources. + * @param ck_constraint The check constraint object to destroy. + */ +void +ck_constraint_delete(struct ck_constraint *ck_constraint); + +/** + * Find check constraint object in space by given name and + * name_len. + * @param space The space to lookup check constraint. + * @param name The check constraint name. + * @param name_len The length of the name. + * @retval Not NULL Ck constraint pointer if exists. + * @retval NULL Otherwise. + */ +struct ck_constraint * +space_ck_constraint_by_name(struct space *space, const char *name, + uint32_t name_len); + +#if defined(__cplusplus) +} /* extern "C" { */ +#endif + +#endif /* INCLUDES_BOX_CK_CONSTRAINT_H */ diff --git a/src/box/errcode.h b/src/box/errcode.h index 9c15f3322..1f7c81693 100644 --- a/src/box/errcode.h +++ b/src/box/errcode.h @@ -245,8 +245,9 @@ struct errcode_record { /*190 */_(ER_INT_LITERAL_MAX, "Integer literal %s%s exceeds the supported range %lld - %lld") \ /*191 */_(ER_SQL_PARSER_LIMIT, "%s %d exceeds the limit (%d)") \ /*192 */_(ER_INDEX_DEF_UNSUPPORTED, "%s are prohibited in an index definition") \ - /*193 */_(ER_CK_DEF_UNSUPPORTED, "%s are prohibited in a CHECK constraint definition") \ + /*193 */_(ER_CK_DEF_UNSUPPORTED, "%s are prohibited in a ck constraint definition") \ /*194 */_(ER_MULTIKEY_INDEX_MISMATCH, "Field %s is used as multikey in one index and as single key in another") \ + /*195 */_(ER_CREATE_CK_CONSTRAINT, "Failed to create check constraint '%s': %s") \ /* * !IMPORTANT! Please follow instructions at start of the file diff --git a/src/box/lua/schema.lua b/src/box/lua/schema.lua index dcaa13659..f2d76d0af 100644 --- a/src/box/lua/schema.lua +++ b/src/box/lua/schema.lua @@ -514,6 +514,7 @@ box.schema.space.drop = function(space_id, space_name, opts) local _truncate = box.space[box.schema.TRUNCATE_ID] local _space_sequence = box.space[box.schema.SPACE_SEQUENCE_ID] local _fk_constraint = box.space[box.schema.FK_CONSTRAINT_ID] + local _ck_constraint = box.space[box.schema.CK_CONSTRAINT_ID] local sequence_tuple = _space_sequence:delete{space_id} if sequence_tuple ~= nil and sequence_tuple.is_generated == true then -- Delete automatically generated sequence. @@ -525,6 +526,9 @@ box.schema.space.drop = function(space_id, space_name, opts) for _, t in _fk_constraint.index.child_id:pairs({space_id}) do _fk_constraint:delete({t.name, space_id}) end + for _, t in _ck_constraint.index.primary:pairs({space_id}) do + _ck_constraint:delete({space_id, t.name}) + end local keys = _vindex:select(space_id) for i = #keys, 1, -1 do local v = keys[i] diff --git a/src/box/lua/space.cc b/src/box/lua/space.cc index e342bfcca..3e8f3b2e5 100644 --- a/src/box/lua/space.cc +++ b/src/box/lua/space.cc @@ -544,6 +544,8 @@ box_lua_space_init(struct lua_State *L) lua_setfield(L, -2, "TRIGGER_ID"); lua_pushnumber(L, BOX_FK_CONSTRAINT_ID); lua_setfield(L, -2, "FK_CONSTRAINT_ID"); + lua_pushnumber(L, BOX_CK_CONSTRAINT_ID); + lua_setfield(L, -2, "CK_CONSTRAINT_ID"); lua_pushnumber(L, BOX_TRUNCATE_ID); lua_setfield(L, -2, "TRUNCATE_ID"); lua_pushnumber(L, BOX_SEQUENCE_ID); diff --git a/src/box/lua/upgrade.lua b/src/box/lua/upgrade.lua index 23f4df019..b4e4b2b5b 100644 --- a/src/box/lua/upgrade.lua +++ b/src/box/lua/upgrade.lua @@ -74,6 +74,7 @@ local function set_system_triggers(val) box.space._schema:run_triggers(val) box.space._cluster:run_triggers(val) box.space._fk_constraint:run_triggers(val) + box.space._ck_constraint:run_triggers(val) end -------------------------------------------------------------------------------- @@ -94,6 +95,7 @@ local function erase() truncate(box.space._schema) truncate(box.space._cluster) truncate(box.space._fk_constraint) + truncate(box.space._ck_constraint) end local function create_sysview(source_id, target_id) @@ -763,8 +765,46 @@ local function upgrade_sequence_to_2_2_1() _space_sequence:format(format) end +local function upgrade_ck_constraint_to_2_2_1() + -- In previous Tarantool releases check constraints were + -- stored in space opts. Now we use separate space + -- _ck_constraint for this purpose. Perform legacy data + -- migration. + local MAP = setmap({}) + local _space = box.space._space + local _index = box.space._index + local _ck_constraint = box.space._ck_constraint + log.info("create space _ck_constraint") + local format = {{name='space_id', type='unsigned'}, + {name='name', type='string'}, + {name='is_deferred', type='boolean'}, + {name='language', type='str'}, {name='code', type='str'}} + _space:insert{_ck_constraint.id, ADMIN, '_ck_constraint', 'memtx', 0, MAP, format} + + log.info("create index primary on _ck_constraint") + _index:insert{_ck_constraint.id, 0, 'primary', 'tree', + {unique = true}, {{0, 'unsigned'}, {1, 'string'}}} + + for _, space in _space:pairs() do + local flags = space.flags + if flags.checks then + for i, check in pairs(flags.checks) do + local expr_str = check.expr + local check_name = check.name or + "CK_CONSTRAINT_"..i.."_"..space.name + _ck_constraint:insert({space.id, check_name, false, + 'SQL', expr_str}) + end + flags.checks = nil + _space:replace({space.id, space.owner, space.name, space.engine, + space.field_count, flags, space.format}) + end + end +end + local function upgrade_to_2_2_1() upgrade_sequence_to_2_2_1() + upgrade_ck_constraint_to_2_2_1() end -------------------------------------------------------------------------------- diff --git a/src/box/schema.cc b/src/box/schema.cc index 9a55c2f14..6d3153815 100644 --- a/src/box/schema.cc +++ b/src/box/schema.cc @@ -460,6 +460,14 @@ schema_init() sc_space_new(BOX_FK_CONSTRAINT_ID, "_fk_constraint", key_parts, 2, &on_replace_fk_constraint, NULL); + /* _ck_сonstraint - check constraints. */ + key_parts[0].fieldno = 0; /* space id */ + key_parts[0].type = FIELD_TYPE_UNSIGNED; + key_parts[1].fieldno = 1; /* constraint name */ + key_parts[1].type = FIELD_TYPE_STRING; + sc_space_new(BOX_CK_CONSTRAINT_ID, "_ck_constraint", key_parts, 2, + &on_replace_ck_constraint, NULL); + /* * _vinyl_deferred_delete - blackhole that is needed * for writing deferred DELETE statements generated by diff --git a/src/box/schema_def.h b/src/box/schema_def.h index dea3fad19..317e6b51f 100644 --- a/src/box/schema_def.h +++ b/src/box/schema_def.h @@ -108,6 +108,8 @@ enum { BOX_SPACE_SEQUENCE_ID = 340, /** Space id of _fk_constraint. */ BOX_FK_CONSTRAINT_ID = 356, + /** Space id of _ck_contraint. */ + BOX_CK_CONSTRAINT_ID = 357, /** End of the reserved range of system spaces. */ BOX_SYSTEM_ID_MAX = 511, BOX_ID_NIL = 2147483647 @@ -239,6 +241,15 @@ enum { BOX_FK_CONSTRAINT_FIELD_PARENT_COLS = 8, }; +/** _ck_constraint fields. */ +enum { + BOX_CK_CONSTRAINT_FIELD_SPACE_ID = 0, + BOX_CK_CONSTRAINT_FIELD_NAME = 1, + BOX_CK_CONSTRAINT_FIELD_DEFERRED = 2, + BOX_CK_CONSTRAINT_FIELD_LANGUAGE = 3, + BOX_CK_CONSTRAINT_FIELD_CODE = 4, +}; + /* * Different objects which can be subject to access * control. diff --git a/src/box/space.c b/src/box/space.c index 243e7da2f..a42b3a64b 100644 --- a/src/box/space.c +++ b/src/box/space.c @@ -165,6 +165,7 @@ space_create(struct space *space, struct engine *engine, space_fill_index_map(space); rlist_create(&space->parent_fk_constraint); rlist_create(&space->child_fk_constraint); + rlist_create(&space->ck_constraint); return 0; fail_free_indexes: @@ -225,6 +226,7 @@ space_delete(struct space *space) assert(space->sql_triggers == NULL); assert(rlist_empty(&space->parent_fk_constraint)); assert(rlist_empty(&space->child_fk_constraint)); + assert(rlist_empty(&space->ck_constraint)); space->vtab->destroy(space); } diff --git a/src/box/space.h b/src/box/space.h index c3eef71c6..823f291ce 100644 --- a/src/box/space.h +++ b/src/box/space.h @@ -198,6 +198,11 @@ struct space { * of index id. */ struct index **index; + /** + * List of check constraints linked with + * ck_constraint::link. + */ + struct rlist ck_constraint; /** * Lists of foreign key constraints. In SQL terms child * space is the "from" table i.e. the table that contains diff --git a/src/box/space_def.c b/src/box/space_def.c index d825def75..661131295 100644 --- a/src/box/space_def.c +++ b/src/box/space_def.c @@ -36,28 +36,12 @@ #include "msgpuck.h" #include "tt_static.h" -/** - * Make checks from msgpack. - * @param str pointer to array of maps - * e.g. [{"expr": "x < y", "name": "ONE"}, ..]. - * @param len array items count. - * @param[out] opt pointer to store parsing result. - * @param errcode Code of error to set if something is wrong. - * @param field_no Field number of an option in a parent element. - * @retval 0 on success. - * @retval not 0 on error. Also set diag message. - */ -static int -checks_array_decode(const char **str, uint32_t len, char *opt, uint32_t errcode, - uint32_t field_no); - const struct space_opts space_opts_default = { /* .group_id = */ 0, /* .is_temporary = */ false, /* .is_ephemeral = */ false, /* .view = */ false, /* .sql = */ NULL, - /* .checks = */ NULL, }; const struct opt_def space_opts_reg[] = { @@ -65,8 +49,7 @@ const struct opt_def space_opts_reg[] = { OPT_DEF("temporary", OPT_BOOL, struct space_opts, is_temporary), OPT_DEF("view", OPT_BOOL, struct space_opts, is_view), OPT_DEF("sql", OPT_STRPTR, struct space_opts, sql), - OPT_DEF_ARRAY("checks", struct space_opts, checks, - checks_array_decode), + OPT_DEF_LEGACY("checks"), OPT_END, }; @@ -114,16 +97,6 @@ space_def_dup_opts(struct space_def *def, const struct space_opts *opts) return -1; } } - if (opts->checks != NULL) { - def->opts.checks = sql_expr_list_dup(sql_get(), opts->checks, 0); - if (def->opts.checks == NULL) { - free(def->opts.sql); - diag_set(OutOfMemory, 0, "sql_expr_list_dup", - "def->opts.checks"); - return -1; - } - sql_checks_update_space_def_reference(def->opts.checks, def); - } return 0; } @@ -301,74 +274,5 @@ void space_opts_destroy(struct space_opts *opts) { free(opts->sql); - sql_expr_list_delete(sql_get(), opts->checks); TRASH(opts); } - -static int -checks_array_decode(const char **str, uint32_t len, char *opt, uint32_t errcode, - uint32_t field_no) -{ - char *errmsg = tt_static_buf(); - struct ExprList *checks = NULL; - const char **map = str; - struct sql *db = sql_get(); - for (uint32_t i = 0; i < len; i++) { - checks = sql_expr_list_append(db, checks, NULL); - if (checks == NULL) { - diag_set(OutOfMemory, 0, "sql_expr_list_append", - "checks"); - goto error; - } - const char *expr_name = NULL; - const char *expr_str = NULL; - uint32_t expr_name_len = 0; - uint32_t expr_str_len = 0; - uint32_t map_size = mp_decode_map(map); - for (uint32_t j = 0; j < map_size; j++) { - if (mp_typeof(**map) != MP_STR) { - diag_set(ClientError, errcode, field_no, - "key must be a string"); - goto error; - } - uint32_t key_len; - const char *key = mp_decode_str(map, &key_len); - if (mp_typeof(**map) != MP_STR) { - snprintf(errmsg, TT_STATIC_BUF_LEN, - "invalid MsgPack map field '%.*s' type", - key_len, key); - diag_set(ClientError, errcode, field_no, errmsg); - goto error; - } - if (key_len == 4 && memcmp(key, "expr", key_len) == 0) { - expr_str = mp_decode_str(map, &expr_str_len); - } else if (key_len == 4 && - memcmp(key, "name", key_len) == 0) { - expr_name = mp_decode_str(map, &expr_name_len); - } else { - snprintf(errmsg, TT_STATIC_BUF_LEN, - "invalid MsgPack map field '%.*s'", - key_len, key); - diag_set(ClientError, errcode, field_no, errmsg); - goto error; - } - } - if (sql_check_list_item_init(checks, i, expr_name, expr_name_len, - expr_str, expr_str_len) != 0) { - box_error_t *err = box_error_last(); - if (box_error_code(err) != ENOMEM) { - snprintf(errmsg, TT_STATIC_BUF_LEN, - "invalid expression specified (%s)", - box_error_message(err)); - diag_set(ClientError, errcode, field_no, - errmsg); - } - goto error; - } - } - *(struct ExprList **)opt = checks; - return 0; -error: - sql_expr_list_delete(db, checks); - return -1; -} diff --git a/src/box/space_def.h b/src/box/space_def.h index c6639a8d8..ac6d226c5 100644 --- a/src/box/space_def.h +++ b/src/box/space_def.h @@ -40,8 +40,6 @@ extern "C" { #endif /* defined(__cplusplus) */ -struct ExprList; - /** Space options */ struct space_opts { /** @@ -71,8 +69,6 @@ struct space_opts { bool is_view; /** SQL statement that produced this space. */ char *sql; - /** SQL Checks expressions list. */ - struct ExprList *checks; }; extern const struct space_opts space_opts_default; diff --git a/src/box/sql.c b/src/box/sql.c index 7f581aad8..6c9464921 100644 --- a/src/box/sql.c +++ b/src/box/sql.c @@ -1032,15 +1032,8 @@ sql_encode_table_opts(struct region *region, struct space_def *def, bool is_error = false; mpstream_init(&stream, region, region_reserve_cb, region_alloc_cb, set_encode_error, &is_error); - int checks_cnt = 0; - struct ExprList_item *a; bool is_view = def->opts.is_view; - struct ExprList *checks = def->opts.checks; - if (checks != NULL) { - checks_cnt = checks->nExpr; - a = checks->a; - } - mpstream_encode_map(&stream, 2 * is_view + (checks_cnt > 0)); + mpstream_encode_map(&stream, 2 * is_view); if (is_view) { assert(def->opts.sql != NULL); @@ -1049,23 +1042,6 @@ sql_encode_table_opts(struct region *region, struct space_def *def, mpstream_encode_str(&stream, "view"); mpstream_encode_bool(&stream, true); } - if (checks_cnt > 0) { - mpstream_encode_str(&stream, "checks"); - mpstream_encode_array(&stream, checks_cnt); - } - for (int i = 0; i < checks_cnt && !is_error; ++i, ++a) { - int items = (a->pExpr != NULL) + (a->zName != NULL); - mpstream_encode_map(&stream, items); - assert(a->pExpr != NULL); - struct Expr *pExpr = a->pExpr; - assert(pExpr->u.zToken != NULL); - mpstream_encode_str(&stream, "expr"); - mpstream_encode_str(&stream, pExpr->u.zToken); - if (a->zName != NULL) { - mpstream_encode_str(&stream, "name"); - mpstream_encode_str(&stream, a->zName); - } - } mpstream_flush(&stream); if (is_error) { diag_set(OutOfMemory, stream.pos - stream.buf, @@ -1300,66 +1276,6 @@ sql_ephemeral_space_new(Parse *parser, const char *name) return space; } -int -sql_check_list_item_init(struct ExprList *expr_list, int column, - const char *expr_name, uint32_t expr_name_len, - const char *expr_str, uint32_t expr_str_len) -{ - assert(column < expr_list->nExpr); - struct ExprList_item *item = &expr_list->a[column]; - memset(item, 0, sizeof(*item)); - if (expr_name != NULL) { - item->zName = sqlDbStrNDup(db, expr_name, expr_name_len); - if (item->zName == NULL) { - diag_set(OutOfMemory, expr_name_len, "sqlDbStrNDup", - "item->zName"); - return -1; - } - } - if (expr_str != NULL) { - item->pExpr = sql_expr_compile(db, expr_str, expr_str_len); - /* The item->zName would be released later. */ - if (item->pExpr == NULL) - return -1; - } - return 0; -} - -static int -update_space_def_callback(Walker *walker, Expr *expr) -{ - if (expr->op == TK_COLUMN && ExprHasProperty(expr, EP_Resolved)) - expr->space_def = walker->u.space_def; - return WRC_Continue; -} - -void -sql_checks_update_space_def_reference(ExprList *expr_list, - struct space_def *def) -{ - assert(expr_list != NULL); - Walker w; - memset(&w, 0, sizeof(w)); - w.xExprCallback = update_space_def_callback; - w.u.space_def = def; - for (int i = 0; i < expr_list->nExpr; i++) - sqlWalkExpr(&w, expr_list->a[i].pExpr); -} - -int -sql_checks_resolve_space_def_reference(ExprList *expr_list, - struct space_def *def) -{ - Parse parser; - sql_parser_create(&parser, sql_get(), default_flags); - parser.parse_only = true; - - sql_resolve_self_reference(&parser, def, NC_IsCheck, NULL, expr_list); - int rc = parser.is_aborted ? -1 : 0; - sql_parser_destroy(&parser); - return rc; -} - void tuple_fetcher_create(struct tuple_fetcher *fetcher, struct tuple *tuple, const char *data, uint32_t data_sz) diff --git a/src/box/sql.h b/src/box/sql.h index 3aaeb2274..b6db356e2 100644 --- a/src/box/sql.h +++ b/src/box/sql.h @@ -285,42 +285,6 @@ sql_resolve_self_reference(struct Parse *parser, struct space_def *def, int type, struct Expr *expr, struct ExprList *expr_list); -/** - * Initialize check_list_item. - * @param expr_list ExprList with item. - * @param column index. - * @param expr_name expression name (optional). - * @param expr_name_len expresson name length (optional). - * @param expr_str expression to build string. - * @param expr_str_len expression to build string length. - * @retval 0 on success. - * @retval -1 on error. - */ -int -sql_check_list_item_init(struct ExprList *expr_list, int column, - const char *expr_name, uint32_t expr_name_len, - const char *expr_str, uint32_t expr_str_len); - -/** - * Resolve space_def references checks for expr_list. - * @param expr_list to modify. - * @param def to refer to. - * @retval 0 on success. - * @retval -1 on error. - */ -int -sql_checks_resolve_space_def_reference(struct ExprList *expr_list, - struct space_def *def); - -/** - * Update space_def references for expr_list. - * @param expr_list to modify. - * @param def to refer to. - */ -void -sql_checks_update_space_def_reference(struct ExprList *expr_list, - struct space_def *def); - /** * Initialize a new parser object. * A number of service allocations are performed on the region, diff --git a/src/box/sql/build.c b/src/box/sql/build.c index 91b977de2..a3b11531f 100644 --- a/src/box/sql/build.c +++ b/src/box/sql/build.c @@ -43,10 +43,12 @@ * COMMIT * ROLLBACK */ +#include <ctype.h> #include "sqlInt.h" #include "vdbeInt.h" #include "tarantoolInt.h" #include "box/box.h" +#include "box/ck_constraint.h" #include "box/fk_constraint.h" #include "box/sequence.h" #include "box/session.h" @@ -638,40 +640,116 @@ primary_key_exit: return; } +/** + * Prepare a 0-terminated string in the wptr memory buffer that + * does not contain a sequence of more than one whatespace + * character. Routine enforces ' ' (space) as whitespace + * delimiter. When character ' or " was met, the sting is copied + * without any changes until the next (corres) ' or " . + * The wptr buffer is expected to have str_len + 1 bytes + * (this is the expected scenario where no extra whitespace + * characters preset in the source string). + * @param wptr The destination memory buffer of size + * @a str_len + 1. + * @param str The source string to be copied. + * @param str_len The source string @a str length. + */ +static void +trim_space_snprintf(char *wptr, const char *str, uint32_t str_len) +{ + const char *str_end = str + str_len; + char quote_type = '\0'; + bool is_prev_chr_space = false; + while (str < str_end) { + if (quote_type == '\0') { + if (*str == '\'' || *str == '\"') { + quote_type = *str; + } else if (isspace((unsigned char)*str)) { + if (!is_prev_chr_space) + *wptr++ = ' '; + is_prev_chr_space = true; + str++; + continue; + } + } else if (*str == quote_type) { + quote_type = '\0'; + } + is_prev_chr_space = false; + *wptr++ = *str++; + } + *wptr = '\0'; +} + void -sql_add_check_constraint(struct Parse *parser) +sql_create_check_contraint(struct Parse *parser) { - struct create_ck_def *ck_def = &parser->create_ck_def; + struct create_ck_def *create_ck_def = &parser->create_ck_def; + struct ExprSpan *expr_span = create_ck_def->expr; + sql_expr_delete(parser->db, expr_span->pExpr, false); + struct alter_entity_def *alter_def = - (struct alter_entity_def *) &parser->create_ck_def; + (struct alter_entity_def *) create_ck_def; assert(alter_def->entity_type == ENTITY_TYPE_CK); (void) alter_def; - struct Expr *expr = ck_def->expr->pExpr; struct space *space = parser->create_table_def.new_space; - if (space != NULL) { - expr->u.zToken = - sqlDbStrNDup(parser->db, - (char *) ck_def->expr->zStart, - (int) (ck_def->expr->zEnd - - ck_def->expr->zStart)); - if (expr->u.zToken == NULL) - goto release_expr; - space->def->opts.checks = - sql_expr_list_append(parser->db, - space->def->opts.checks, expr); - if (space->def->opts.checks == NULL) { - sqlDbFree(parser->db, expr->u.zToken); - goto release_expr; - } - struct create_entity_def *entity_def = &ck_def->base.base; - if (entity_def->name.n > 0) { - sqlExprListSetName(parser, space->def->opts.checks, - &entity_def->name, 1); + assert(space != NULL); + + /* Prepare payload for ck constraint definition. */ + struct region *region = &parser->region; + struct Token *name_token = &create_ck_def->base.base.name; + const char *name; + if (name_token->n > 0) { + name = sql_normalized_name_region_new(region, name_token->z, + name_token->n); + if (name == NULL) { + parser->is_aborted = true; + return; } } else { -release_expr: - sql_expr_delete(parser->db, expr, false); + uint32_t ck_idx = ++parser->create_table_def.check_count; + name = tt_sprintf("CK_CONSTRAINT_%d_%s", ck_idx, + space->def->name); + } + size_t name_len = strlen(name); + + uint32_t expr_str_len = (uint32_t)(expr_span->zEnd - expr_span->zStart); + const char *expr_str = expr_span->zStart; + + /* + * Allocate memory for ck constraint parse structure and + * ck constraint definition as a single memory chunk on + * region: + * + * [ck_parse][ck_def[name][expr_str]] + * |_____^ |___^ ^ + * |_________| + */ + uint32_t name_offset, expr_str_offset; + uint32_t ck_def_sz = + ck_constraint_def_sizeof(name_len, expr_str_len, &name_offset, + &expr_str_offset); + struct ck_constraint_parse *ck_parse = + region_alloc(region, sizeof(*ck_parse) + ck_def_sz); + if (ck_parse == NULL) { + diag_set(OutOfMemory, sizeof(*ck_parse) + ck_def_sz, "region", + "ck_parse"); + parser->is_aborted = true; + return; } + struct ck_constraint_def *ck_def = + (struct ck_constraint_def *)((char *)ck_parse + + sizeof(*ck_parse)); + ck_parse->ck_def = ck_def; + rlist_create(&ck_parse->link); + + ck_def->name = (char *)ck_def + name_offset; + ck_def->expr_str = (char *)ck_def + expr_str_offset; + ck_def->language = CK_CONSTRAINT_LANGUAGE_SQL; + trim_space_snprintf(ck_def->expr_str, expr_str, expr_str_len); + memcpy(ck_def->name, name, name_len); + ck_def->name[name_len] = '\0'; + + rlist_add_entry(&parser->create_table_def.new_check, ck_parse, link); } /* @@ -939,6 +1017,40 @@ emitNewSysSequenceRecord(Parse *pParse, int reg_seq_id, const char *seq_name) return first_col; } +/** + * Generate opcodes to serialize check constraint definition into + * MsgPack and insert produced tuple into _ck_constraint space. + * @param parser Parsing context. + * @param ck_def Check constraint definition to be serialized. + * @param reg_space_id The VDBE register containing space id. +*/ +static void +vdbe_emit_ck_constraint_create(struct Parse *parser, + const struct ck_constraint_def *ck_def, + uint32_t reg_space_id) +{ + struct sql *db = parser->db; + struct Vdbe *v = sqlGetVdbe(parser); + assert(v != NULL); + int ck_constraint_reg = sqlGetTempRange(parser, 6); + sqlVdbeAddOp2(v, OP_SCopy, reg_space_id, ck_constraint_reg); + sqlVdbeAddOp4(v, OP_String8, 0, ck_constraint_reg + 1, 0, + sqlDbStrDup(db, ck_def->name), P4_DYNAMIC); + sqlVdbeAddOp2(v, OP_Bool, false, ck_constraint_reg + 2); + sqlVdbeAddOp4(v, OP_String8, 0, ck_constraint_reg + 3, 0, + ck_constraint_language_strs[ck_def->language], P4_STATIC); + sqlVdbeAddOp4(v, OP_String8, 0, ck_constraint_reg + 4, 0, + sqlDbStrDup(db, ck_def->expr_str), P4_DYNAMIC); + sqlVdbeAddOp3(v, OP_MakeRecord, ck_constraint_reg, 5, + ck_constraint_reg + 5); + sqlVdbeAddOp3(v, OP_SInsert, BOX_CK_CONSTRAINT_ID, 0, + ck_constraint_reg + 5); + save_record(parser, BOX_CK_CONSTRAINT_ID, ck_constraint_reg, 2, + v->nOp - 1); + VdbeComment((v, "Create CK constraint %s", ck_def->name)); + sqlReleaseTempRange(parser, ck_constraint_reg, 5); +} + int emitNewSysSpaceSequenceRecord(Parse *pParse, int space_id, const char reg_seq_id) { @@ -1113,20 +1225,18 @@ resolve_link(struct Parse *parse_context, const struct space_def *def, void sqlEndTable(struct Parse *pParse) { - sql *db = pParse->db; /* The database connection */ - - assert(!db->mallocFailed); + assert(!pParse->db->mallocFailed); struct space *new_space = pParse->create_table_def.new_space; if (new_space == NULL) return; - assert(!db->init.busy); + assert(!pParse->db->init.busy); assert(!new_space->def->opts.is_view); if (sql_space_primary_key(new_space) == NULL) { diag_set(ClientError, ER_CREATE_SPACE, new_space->def->name, "PRIMARY KEY missing"); pParse->is_aborted = true; - goto cleanup; + return; } /* @@ -1216,9 +1326,12 @@ sqlEndTable(struct Parse *pParse) fk_def->child_id = reg_space_id; vdbe_emit_fk_constraint_create(pParse, fk_def); } -cleanup: - sql_expr_list_delete(db, new_space->def->opts.checks); - new_space->def->opts.checks = NULL; + struct ck_constraint_parse *ck_parse; + rlist_foreach_entry(ck_parse, &pParse->create_table_def.new_check, + link) { + vdbe_emit_ck_constraint_create(pParse, ck_parse->ck_def, + reg_space_id); + } } void @@ -1416,6 +1529,37 @@ vdbe_emit_fk_constraint_drop(struct Parse *parse_context, char *constraint_name, sqlReleaseTempRange(parse_context, key_reg, 3); } +/** + * Generate VDBE program to remove entry from _ck_constraint space. + * + * @param parser Parsing context. + * @param ck_name Name of CK constraint to be dropped. + * @param child_id Id of table which constraint belongs to. + */ +static void +vdbe_emit_ck_constraint_drop(struct Parse *parser, const char *ck_name, + uint32_t space_id) +{ + struct Vdbe *v = sqlGetVdbe(parser); + struct sql *db = v->db; + assert(v != NULL); + int key_reg = sqlGetTempRange(parser, 3); + sqlVdbeAddOp2(v, OP_Integer, space_id, key_reg); + sqlVdbeAddOp4(v, OP_String8, 0, key_reg + 1, 0, + sqlDbStrDup(db, ck_name), P4_DYNAMIC); + const char *error_msg = + tt_sprintf(tnt_errcode_desc(ER_NO_SUCH_CONSTRAINT), ck_name); + if (vdbe_emit_halt_with_presence_test(parser, BOX_CK_CONSTRAINT_ID, 0, + key_reg, 2, ER_NO_SUCH_CONSTRAINT, + error_msg, false, + OP_Found) != 0) + return; + sqlVdbeAddOp3(v, OP_MakeRecord, key_reg, 2, key_reg + 2); + sqlVdbeAddOp2(v, OP_SDelete, BOX_CK_CONSTRAINT_ID, key_reg + 2); + VdbeComment((v, "Delete CK constraint %s", ck_name)); + sqlReleaseTempRange(parser, key_reg, 3); +} + /** * Generate code to drop a table. * This routine includes dropping triggers, sequences, @@ -1489,6 +1633,13 @@ sql_code_drop_table(struct Parse *parse_context, struct space *space, return; vdbe_emit_fk_constraint_drop(parse_context, fk_name_dup, space_id); } + /* Delete all CK constraints. */ + struct ck_constraint *ck_constraint; + rlist_foreach_entry(ck_constraint, &space->ck_constraint, link) { + vdbe_emit_ck_constraint_drop(parse_context, + ck_constraint->def->name, + space_id); + } /* * Drop all _space and _index entries that refer to the * table. @@ -2775,8 +2926,7 @@ sqlSrcListDelete(sql * db, SrcList * pList) */ assert(pItem->space == NULL || !pItem->space->def->opts.is_temporary || - (pItem->space->index == NULL && - pItem->space->def->opts.checks == NULL)); + pItem->space->index == NULL); sql_select_delete(db, pItem->pSelect); sql_expr_delete(db, pItem->pOn, false); sqlIdListDelete(db, pItem->pUsing); diff --git a/src/box/sql/insert.c b/src/box/sql/insert.c index 45bfd519a..b28fe5760 100644 --- a/src/box/sql/insert.c +++ b/src/box/sql/insert.c @@ -36,9 +36,10 @@ #include "sqlInt.h" #include "tarantoolInt.h" #include "vdbeInt.h" -#include "box/schema.h" +#include "box/ck_constraint.h" #include "bit/bit.h" #include "box/box.h" +#include "box/schema.h" enum field_type * sql_index_type_str(struct sql *db, const struct index_def *idx_def) @@ -932,34 +933,27 @@ vdbe_emit_constraint_checks(struct Parse *parse_context, struct space *space, if (on_conflict == ON_CONFLICT_ACTION_DEFAULT) on_conflict = ON_CONFLICT_ACTION_ABORT; /* Test all CHECK constraints. */ - struct ExprList *checks = def->opts.checks; enum on_conflict_action on_conflict_check = on_conflict; if (on_conflict == ON_CONFLICT_ACTION_REPLACE) on_conflict_check = ON_CONFLICT_ACTION_ABORT; - if (checks != NULL) { + if (!rlist_empty(&space->ck_constraint)) parse_context->ckBase = new_tuple_reg; - for (int i = 0; i < checks->nExpr; i++) { - struct Expr *expr = checks->a[i].pExpr; - if (is_update && - checkConstraintUnchanged(expr, upd_cols)) - continue; - int all_ok = sqlVdbeMakeLabel(v); - sqlExprIfTrue(parse_context, expr, all_ok, - SQL_JUMPIFNULL); - if (on_conflict == ON_CONFLICT_ACTION_IGNORE) { - sqlVdbeGoto(v, ignore_label); - } else { - char *name = checks->a[i].zName; - if (name == NULL) - name = def->name; - sqlHaltConstraint(parse_context, - SQL_CONSTRAINT_CHECK, - on_conflict_check, name, - P4_TRANSIENT, - P5_ConstraintCheck); - } - sqlVdbeResolveLabel(v, all_ok); + struct ck_constraint *ck_constraint; + rlist_foreach_entry(ck_constraint, &space->ck_constraint, link) { + struct Expr *expr = ck_constraint->expr; + if (is_update && checkConstraintUnchanged(expr, upd_cols) != 0) + continue; + int all_ok = sqlVdbeMakeLabel(v); + sqlExprIfTrue(parse_context, expr, all_ok, SQL_JUMPIFNULL); + if (on_conflict == ON_CONFLICT_ACTION_IGNORE) { + sqlVdbeGoto(v, ignore_label); + } else { + char *name = ck_constraint->def->name; + sqlHaltConstraint(parse_context, SQL_CONSTRAINT_CHECK, + on_conflict_check, name, P4_TRANSIENT, + P5_ConstraintCheck); } + sqlVdbeResolveLabel(v, all_ok); } sql_emit_table_types(v, space->def, new_tuple_reg); /* @@ -1244,14 +1238,13 @@ xferOptimization(Parse * pParse, /* Parser context */ if (pSrcIdx == NULL) return 0; } - /* Get server checks. */ - ExprList *pCheck_src = src->def->opts.checks; - ExprList *pCheck_dest = dest->def->opts.checks; - if (pCheck_dest != NULL && - sqlExprListCompare(pCheck_src, pCheck_dest, -1) != 0) { - /* Tables have different CHECK constraints. Ticket #2252 */ + /* + * Dissallow the transfer optimization if the are check + * constraints. + */ + if (!rlist_empty(&dest->ck_constraint) || + !rlist_empty(&src->ck_constraint)) return 0; - } /* Disallow the transfer optimization if the destination table constains * any foreign key constraints. This is more restrictive than necessary. * So the extra complication to make this rule less restrictive is probably diff --git a/src/box/sql/parse.y b/src/box/sql/parse.y index f241b8d52..e93dfe751 100644 --- a/src/box/sql/parse.y +++ b/src/box/sql/parse.y @@ -297,7 +297,7 @@ ccons ::= check_constraint_def . check_constraint_def ::= cconsname(N) CHECK LP expr(X) RP. { create_ck_def_init(&pParse->create_ck_def, &N, &X); - sql_add_check_constraint(pParse); + sql_create_check_contraint(pParse); } ccons ::= cconsname(N) REFERENCES nm(T) eidlist_opt(TA) matcharg(M) refargs(R). { diff --git a/src/box/sql/parse_def.h b/src/box/sql/parse_def.h index 5899a7e4e..6c1b6fddd 100644 --- a/src/box/sql/parse_def.h +++ b/src/box/sql/parse_def.h @@ -123,6 +123,22 @@ struct fk_constraint_parse { struct rlist link; }; +/** + * Structure representing check constraint appeared within + * CREATE TABLE statement. Used only during parsing. + * All allocations are performed on region, so no cleanups are + * required. + */ +struct ck_constraint_parse { + /** + * Check constraint declared in <CREATE TABLE ...> + * statement. Must be coded after space creation. + */ + struct ck_constraint_def *ck_def; + /** Organize these structs into linked list. */ + struct rlist link; +}; + /** * Possible SQL index types. Note that PK and UNIQUE constraints * are implemented as indexes and have their own types: @@ -189,6 +205,13 @@ struct create_table_def { * Foreign key constraint appeared in CREATE TABLE stmt. */ struct rlist new_fkey; + /** + * Number of CK constraints declared within + * CREATE TABLE statement. + */ + uint32_t check_count; + /** Check constraint appeared in CREATE TABLE stmt. */ + struct rlist new_check; /** True, if table to be created has AUTOINCREMENT PK. */ bool has_autoinc; }; @@ -437,6 +460,7 @@ create_table_def_init(struct create_table_def *table_def, struct Token *name, create_entity_def_init(&table_def->base, ENTITY_TYPE_TABLE, NULL, name, if_not_exists); rlist_create(&table_def->new_fkey); + rlist_create(&table_def->new_check); } static inline void diff --git a/src/box/sql/select.c b/src/box/sql/select.c index d7376ae11..22aa62830 100644 --- a/src/box/sql/select.c +++ b/src/box/sql/select.c @@ -6440,7 +6440,12 @@ sql_expr_extract_select(struct Parse *parser, struct Select *select) struct ExprList *expr_list = select->pEList; assert(expr_list->nExpr == 1); parser->parsed_ast_type = AST_TYPE_EXPR; - parser->parsed_ast.expr = sqlExprDup(parser->db, - expr_list->a->pExpr, - EXPRDUP_REDUCE); + /* + * Extract a copy of parsed expression. + * We cannot use EXPRDUP_REDUCE flag in sqlExprDup call + * because some compiled Expr (like Checks expressions) + * may require further resolve with sqlResolveExprNames. + */ + parser->parsed_ast.expr = + sqlExprDup(parser->db, expr_list->a->pExpr, 0); } diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h index 7212553f6..abb7b9c2a 100644 --- a/src/box/sql/sqlInt.h +++ b/src/box/sql/sqlInt.h @@ -3359,7 +3359,7 @@ sqlAddPrimaryKey(struct Parse *parse); * @param parser Parsing context. */ void -sql_add_check_constraint(Parse *parser); +sql_create_check_contraint(Parse *parser); void sqlAddDefaultValue(Parse *, ExprSpan *); void sqlAddCollateType(Parse *, Token *); diff --git a/src/box/sql/tokenize.c b/src/box/sql/tokenize.c index 11ef7b97e..dbaebfefc 100644 --- a/src/box/sql/tokenize.c +++ b/src/box/sql/tokenize.c @@ -435,7 +435,6 @@ parser_space_delete(struct sql *db, struct space *space) assert(space->def->opts.is_temporary); for (uint32_t i = 0; i < space->index_count; ++i) index_def_delete(space->index[i]->def); - sql_expr_list_delete(db, space->def->opts.checks); } /** diff --git a/test/app-tap/tarantoolctl.test.lua b/test/app-tap/tarantoolctl.test.lua index c1e1490ca..4777d8a6f 100755 --- a/test/app-tap/tarantoolctl.test.lua +++ b/test/app-tap/tarantoolctl.test.lua @@ -405,8 +405,8 @@ do check_ctlcat_xlog(test_i, dir, "--from=3 --to=6 --format=json --show-system --replica 1", "\n", 3) check_ctlcat_xlog(test_i, dir, "--from=3 --to=6 --format=json --show-system --replica 1 --replica 2", "\n", 3) check_ctlcat_xlog(test_i, dir, "--from=3 --to=6 --format=json --show-system --replica 2", "\n", 0) - check_ctlcat_snap(test_i, dir, "--space=280", "---\n", 21) - check_ctlcat_snap(test_i, dir, "--space=288", "---\n", 47) + check_ctlcat_snap(test_i, dir, "--space=280", "---\n", 22) + check_ctlcat_snap(test_i, dir, "--space=288", "---\n", 48) end) end) diff --git a/test/box-py/bootstrap.result b/test/box-py/bootstrap.result index de90beee6..dc0c145ae 100644 --- a/test/box-py/bootstrap.result +++ b/test/box-py/bootstrap.result @@ -79,6 +79,9 @@ box.space._space:select{} {'name': 'is_deferred', 'type': 'boolean'}, {'name': 'match', 'type': 'string'}, {'name': 'on_delete', 'type': 'string'}, {'name': 'on_update', 'type': 'string'}, {'name': 'child_cols', 'type': 'array'}, {'name': 'parent_cols', 'type': 'array'}]] + - [357, 1, '_ck_constraint', 'memtx', 0, {}, [{'name': 'space_id', 'type': 'unsigned'}, + {'name': 'name', 'type': 'string'}, {'name': 'is_deferred', 'type': 'boolean'}, + {'name': 'language', 'type': 'str'}, {'name': 'code', 'type': 'str'}]] ... box.space._index:select{} --- @@ -131,6 +134,7 @@ box.space._index:select{} - [340, 1, 'sequence', 'tree', {'unique': false}, [[1, 'unsigned']]] - [356, 0, 'primary', 'tree', {'unique': true}, [[0, 'string'], [1, 'unsigned']]] - [356, 1, 'child_id', 'tree', {'unique': false}, [[1, 'unsigned']]] + - [357, 0, 'primary', 'tree', {'unique': true}, [[0, 'unsigned'], [1, 'string']]] ... box.space._user:select{} --- diff --git a/test/box/access.result b/test/box/access.result index 9c190240f..801112799 100644 --- a/test/box/access.result +++ b/test/box/access.result @@ -1487,6 +1487,9 @@ box.schema.user.grant('tester', 'read', 'space', '_trigger') box.schema.user.grant('tester', 'read', 'space', '_fk_constraint') --- ... +box.schema.user.grant('tester', 'read', 'space', '_ck_constraint') +--- +... box.session.su("tester") --- ... diff --git a/test/box/access.test.lua b/test/box/access.test.lua index 4baeb2ef6..2f62d6f53 100644 --- a/test/box/access.test.lua +++ b/test/box/access.test.lua @@ -554,6 +554,7 @@ box.schema.user.grant('tester', 'create' , 'sequence') box.schema.user.grant('tester', 'read', 'space', '_sequence') box.schema.user.grant('tester', 'read', 'space', '_trigger') box.schema.user.grant('tester', 'read', 'space', '_fk_constraint') +box.schema.user.grant('tester', 'read', 'space', '_ck_constraint') box.session.su("tester") -- successful create s1 = box.schema.space.create("test_space") diff --git a/test/box/access_misc.result b/test/box/access_misc.result index 877a9b533..f47c8e3cc 100644 --- a/test/box/access_misc.result +++ b/test/box/access_misc.result @@ -819,6 +819,9 @@ box.space._space:select() {'name': 'is_deferred', 'type': 'boolean'}, {'name': 'match', 'type': 'string'}, {'name': 'on_delete', 'type': 'string'}, {'name': 'on_update', 'type': 'string'}, {'name': 'child_cols', 'type': 'array'}, {'name': 'parent_cols', 'type': 'array'}]] + - [357, 1, '_ck_constraint', 'memtx', 0, {}, [{'name': 'space_id', 'type': 'unsigned'}, + {'name': 'name', 'type': 'string'}, {'name': 'is_deferred', 'type': 'boolean'}, + {'name': 'language', 'type': 'str'}, {'name': 'code', 'type': 'str'}]] ... box.space._func:select() --- diff --git a/test/box/access_sysview.result b/test/box/access_sysview.result index c6a2b22ed..65a9da8a5 100644 --- a/test/box/access_sysview.result +++ b/test/box/access_sysview.result @@ -230,11 +230,11 @@ box.session.su('guest') ... #box.space._vspace:select{} --- -- 22 +- 23 ... #box.space._vindex:select{} --- -- 48 +- 49 ... #box.space._vuser:select{} --- @@ -262,7 +262,7 @@ box.session.su('guest') ... #box.space._vindex:select{} --- -- 48 +- 49 ... #box.space._vuser:select{} --- diff --git a/test/box/alter.result b/test/box/alter.result index 75d6dae2a..bb39d3824 100644 --- a/test/box/alter.result +++ b/test/box/alter.result @@ -92,7 +92,7 @@ space = box.space[t[1]] ... space.id --- -- 357 +- 358 ... space.field_count --- @@ -137,7 +137,7 @@ space_deleted ... space:replace{0} --- -- error: Space '357' does not exist +- error: Space '358' does not exist ... _index:insert{_space.id, 0, 'primary', 'tree', {unique=true}, {{0, 'unsigned'}}} --- @@ -215,6 +215,7 @@ _index:select{} - [340, 1, 'sequence', 'tree', {'unique': false}, [[1, 'unsigned']]] - [356, 0, 'primary', 'tree', {'unique': true}, [[0, 'string'], [1, 'unsigned']]] - [356, 1, 'child_id', 'tree', {'unique': false}, [[1, 'unsigned']]] + - [357, 0, 'primary', 'tree', {'unique': true}, [[0, 'unsigned'], [1, 'string']]] ... -- modify indexes of a system space _index:delete{_index.id, 0} diff --git a/test/box/misc.result b/test/box/misc.result index 4fcd13a78..5bf419d4f 100644 --- a/test/box/misc.result +++ b/test/box/misc.result @@ -523,6 +523,7 @@ t; 192: box.error.INDEX_DEF_UNSUPPORTED 193: box.error.CK_DEF_UNSUPPORTED 194: box.error.MULTIKEY_INDEX_MISMATCH + 195: box.error.CREATE_CK_CONSTRAINT ... test_run:cmd("setopt delimiter ''"); --- diff --git a/test/sql-tap/check.test.lua b/test/sql-tap/check.test.lua index b01afca7c..ede77c630 100755 --- a/test/sql-tap/check.test.lua +++ b/test/sql-tap/check.test.lua @@ -55,7 +55,7 @@ test:do_catchsql_test( INSERT INTO t1 VALUES(6,7, 2); ]], { -- <check-1.3> - 1, "Failed to execute SQL statement: CHECK constraint failed: T1" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" -- </check-1.3> }) @@ -75,7 +75,7 @@ test:do_catchsql_test( INSERT INTO t1 VALUES(4,3, 2); ]], { -- <check-1.5> - 1, "Failed to execute SQL statement: CHECK constraint failed: T1" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_2_T1" -- </check-1.5> }) @@ -147,7 +147,7 @@ test:do_catchsql_test( UPDATE t1 SET x=7 WHERE x==2 ]], { -- <check-1.12> - 1, "Failed to execute SQL statement: CHECK constraint failed: T1" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" -- </check-1.12> }) @@ -167,7 +167,7 @@ test:do_catchsql_test( UPDATE t1 SET x=5 WHERE x==2 ]], { -- <check-1.14> - 1, "Failed to execute SQL statement: CHECK constraint failed: T1" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" -- </check-1.14> }) @@ -319,7 +319,7 @@ test:do_catchsql_test( ); ]], { -- <check-3.1> - 1, "Failed to create space 'T3': Subqueries are prohibited in a CHECK constraint definition" + 1, "Failed to create check constraint 'CK_CONSTRAINT_1_T3': Subqueries are prohibited in a ck constraint definition" -- </check-3.1> }) @@ -344,7 +344,7 @@ test:do_catchsql_test( ); ]], { -- <check-3.3> - 1, "Failed to create space 'T3': Can’t resolve field 'Q'" + 1, "Failed to create check constraint 'CK_CONSTRAINT_1_T3': Can’t resolve field 'Q'" -- </check-3.3> }) @@ -368,7 +368,7 @@ test:do_catchsql_test( ); ]], { -- <check-3.5> - 1, "Failed to create space 'T3': Field 'X' was not found in the space 'T2' format" + 1, "Failed to create check constraint 'CK_CONSTRAINT_1_T3': Field 'X' was not found in the space 'T2' format" -- </check-3.5> }) @@ -413,7 +413,7 @@ test:do_catchsql_test( INSERT INTO t3 VALUES(111,222,333); ]], { -- <check-3.9> - 1, "Failed to execute SQL statement: CHECK constraint failed: T3" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T3" -- </check-3.9> }) @@ -484,7 +484,7 @@ test:do_catchsql_test( UPDATE t4 SET x=0, y=1; ]], { -- <check-4.6> - 1, "Failed to execute SQL statement: CHECK constraint failed: T4" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T4" -- </check-4.6> }) @@ -504,7 +504,7 @@ test:do_catchsql_test( UPDATE t4 SET x=0, y=2; ]], { -- <check-4.9> - 1, "Failed to execute SQL statement: CHECK constraint failed: T4" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T4" -- </check-4.9> }) @@ -516,7 +516,7 @@ test:do_catchsql_test( ); ]], { -- <check-5.1> - 1, "Wrong space options (field 5): invalid expression specified (bindings are not allowed in DDL)" + 1, "Failed to create check constraint 'CK_CONSTRAINT_1_T5': bindings are not allowed in DDL" -- </check-5.1> }) @@ -528,7 +528,7 @@ test:do_catchsql_test( ); ]], { -- <check-5.2> - 1, "Wrong space options (field 5): invalid expression specified (bindings are not allowed in DDL)" + 1, "Failed to create check constraint 'CK_CONSTRAINT_1_T5': bindings are not allowed in DDL" -- </check-5.2> }) @@ -581,7 +581,7 @@ test:do_catchsql_test( UPDATE OR FAIL t1 SET x=7-x, y=y+1; ]], { -- <check-6.5> - 1, "Failed to execute SQL statement: CHECK constraint failed: T1" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" -- </check-6.5> }) @@ -603,7 +603,7 @@ test:do_catchsql_test( INSERT OR ROLLBACK INTO t1 VALUES(8,40.0, 10); ]], { -- <check-6.7> - 1, "Failed to execute SQL statement: CHECK constraint failed: T1" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" -- </check-6.7> }) @@ -636,7 +636,7 @@ test:do_catchsql_test( REPLACE INTO t1 VALUES(6,7, 11); ]], { -- <check-6.12> - 1, "Failed to execute SQL statement: CHECK constraint failed: T1" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" -- </check-6.12> }) @@ -700,7 +700,7 @@ test:do_catchsql_test( 7.3, " INSERT INTO t6 VALUES(11) ", { -- <7.3> - 1, "Failed to execute SQL statement: CHECK constraint failed: T6" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T6" -- </7.3> }) diff --git a/test/sql-tap/fkey2.test.lua b/test/sql-tap/fkey2.test.lua index 54e5059b3..695a379a6 100755 --- a/test/sql-tap/fkey2.test.lua +++ b/test/sql-tap/fkey2.test.lua @@ -362,7 +362,7 @@ test:do_catchsql_test( UPDATE ab SET a = 5; ]], { -- <fkey2-3.2> - 1, "Failed to execute SQL statement: CHECK constraint failed: EF" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_EF" -- </fkey2-3.2> }) @@ -382,7 +382,7 @@ test:do_catchsql_test( UPDATE ab SET a = 5; ]], { -- <fkey2-3.4> - 1, "Failed to execute SQL statement: CHECK constraint failed: EF" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_EF" -- </fkey2-3.4> }) diff --git a/test/sql-tap/sql-errors.test.lua b/test/sql-tap/sql-errors.test.lua index 9357c406b..a8d39472d 100755 --- a/test/sql-tap/sql-errors.test.lua +++ b/test/sql-tap/sql-errors.test.lua @@ -313,7 +313,7 @@ test:do_catchsql_test( CREATE TABLE t27 (i INT PRIMARY KEY, CHECK(i < (SELECT * FROM t0))); ]], { -- <sql-errors-1.27> - 1,"Failed to create space 'T27': Subqueries are prohibited in a CHECK constraint definition" + 1,"Failed to create check constraint 'CK_CONSTRAINT_1_T27': Subqueries are prohibited in a ck constraint definition" -- </sql-errors-1.27> }) diff --git a/test/sql-tap/table.test.lua b/test/sql-tap/table.test.lua index 5b793c0fc..066662f33 100755 --- a/test/sql-tap/table.test.lua +++ b/test/sql-tap/table.test.lua @@ -1221,7 +1221,7 @@ test:do_catchsql_test( INSERT INTO T21 VALUES(1, -1, 1); ]], { -- <table-21.3> - 1, "Failed to execute SQL statement: CHECK constraint failed: T21" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T21" -- </table-21.3> }) @@ -1231,7 +1231,7 @@ test:do_catchsql_test( INSERT INTO T21 VALUES(1, 1, -1); ]], { -- <table-21.4> - 1, "Failed to execute SQL statement: CHECK constraint failed: T21" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_2_T21" -- </table-21.4> }) diff --git a/test/sql/checks.result b/test/sql/checks.result index f7cddec43..5f0098e23 100644 --- a/test/sql/checks.result +++ b/test/sql/checks.result @@ -18,8 +18,10 @@ box.execute('pragma sql_default_engine=\''..engine..'\'') -- -- gh-3272: Move SQL CHECK into server -- --- invalid expression -opts = {checks = {{expr = 'X><5'}}} +-- Until Tarantool version 2.2 check constraints were stored in +-- space opts. +-- Make sure that now this legacy option is ignored. +opts = {checks = {{expr = 'X>5'}}} --- ... format = {{name = 'X', type = 'unsigned'}} @@ -30,89 +32,216 @@ t = {513, 1, 'test', 'memtx', 0, opts, format} ... s = box.space._space:insert(t) --- -- error: 'Wrong space options (field 5): invalid expression specified (Syntax error - near ''<'')' ... -opts = {checks = {{expr = 'X>5'}}} +_ = box.space.test:create_index('pk') --- ... -format = {{name = 'X', type = 'unsigned'}} +-- Invalid expression test. +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', false, 'SQL', 'X><5'}) --- +- error: 'Failed to create check constraint ''CK_CONSTRAINT_01'': Syntax error near + ''<''' ... -t = {513, 1, 'test', 'memtx', 0, opts, format} +-- Unexistent space test. +box.space._ck_constraint:insert({550, 'CK_CONSTRAINT_01', false, 'SQL', 'X<5'}) --- +- error: Space '550' does not exist ... -s = box.space._space:insert(t) +-- Pass integer instead of expression. +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', false, 'SQL', 666}) +--- +- error: 'Tuple field 5 type does not match one required by operation: expected string' +... +-- Defered CK constraints are not supported. +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', true, 'SQL', 'X<5'}) +--- +- error: Tarantool does not support deferred ck constraints +... +-- The only supperted language is SQL. +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', false, 'LUA', 'X<5'}) +--- +- error: Unsupported language 'LUA' specified for function 'CK_CONSTRAINT_01' +... +-- Check constraints LUA creation test. +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', false, 'SQL', 'X<5'}) +--- +- [513, 'CK_CONSTRAINT_01', false, 'SQL', 'X<5'] +... +box.space._ck_constraint:count({}) +--- +- 1 +... +box.execute("INSERT INTO \"test\" VALUES(5);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_01' +... +box.space._ck_constraint:replace({513, 'CK_CONSTRAINT_01', false, 'SQL', 'X<=5'}) +--- +- [513, 'CK_CONSTRAINT_01', false, 'SQL', 'X<=5'] +... +box.execute("INSERT INTO \"test\" VALUES(5);") +--- +- row_count: 1 +... +box.execute("INSERT INTO \"test\" VALUES(6);") --- +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_01' ... -box.space._space:delete(513) +-- Can't drop table with check constraints. +box.space.test:delete({5}) +--- +- [5] +... +box.space.test.index.pk:drop() +--- +... +box.space._space:delete({513}) +--- +- error: 'Can''t drop space ''test'': the space has check constraints' +... +box.space._ck_constraint:delete({513, 'CK_CONSTRAINT_01'}) +--- +- [513, 'CK_CONSTRAINT_01', false, 'SQL', 'X<=5'] +... +box.space._space:delete({513}) --- - [513, 1, 'test', 'memtx', 0, {'checks': [{'expr': 'X>5'}]}, [{'name': 'X', 'type': 'unsigned'}]] ... -opts = {checks = {{expr = 'X>5', name = 'ONE'}}} +-- Create table with checks in sql. +box.execute("CREATE TABLE t1(x INTEGER CONSTRAINT ONE CHECK( x<5 ), y REAL CONSTRAINT TWO CHECK( y>x ), z INTEGER PRIMARY KEY);") --- +- row_count: 1 ... -format = {{name = 'X', type = 'unsigned'}} +box.space._ck_constraint:count() --- +- 2 ... -t = {513, 1, 'test', 'memtx', 0, opts, format} +box.execute("INSERT INTO t1 VALUES (7, 1, 1)") --- +- error: 'Failed to execute SQL statement: CHECK constraint failed: ONE' ... -s = box.space._space:insert(t) +box.execute("INSERT INTO t1 VALUES (2, 1, 1)") --- +- error: 'Failed to execute SQL statement: CHECK constraint failed: TWO' ... -box.space._space:delete(513) +box.execute("INSERT INTO t1 VALUES (2, 4, 1)") --- -- [513, 1, 'test', 'memtx', 0, {'checks': [{'name': 'ONE', 'expr': 'X>5'}]}, [{'name': 'X', - 'type': 'unsigned'}]] +- row_count: 1 ... --- extra invlalid field name -opts = {checks = {{expr = 'X>5', name = 'ONE', extra = 'TWO'}}} +box.execute("DROP TABLE t1") --- +- row_count: 1 ... -format = {{name = 'X', type = 'unsigned'}} +-- Test space creation rollback on spell error in ck constraint. +box.execute("CREATE TABLE first (id FLOAT PRIMARY KEY CHECK(id < 5), a INT CONSTRAINT ONE CHECK(a >< 5));") --- +- error: Syntax error near '<' ... -t = {513, 1, 'test', 'memtx', 0, opts, format} +box.space.FIRST == nil --- +- true ... -s = box.space._space:insert(t) +box.space._ck_constraint:count() == 0 --- -- error: 'Wrong space options (field 5): invalid MsgPack map field ''extra''' +- true ... -opts = {checks = {{expr_invalid_label = 'X>5'}}} +-- Ck constraints are disallowed for spaces having no format. +s = box.schema.create_space('test', {engine = engine}) --- ... -format = {{name = 'X', type = 'unsigned'}} +_ = s:create_index('pk') --- ... -t = {513, 1, 'test', 'memtx', 0, opts, format} +_ = box.space._ck_constraint:insert({s.id, 'physics', false, 'SQL', 'X<Y'}) --- +- error: Tarantool does not support CK constraint for space without format ... -s = box.space._space:insert(t) +s:format({{name='X', type='integer'}, {name='Y', type='integer'}}) --- -- error: 'Wrong space options (field 5): invalid MsgPack map field ''expr_invalid_label''' ... --- invalid field type -opts = {checks = {{name = 123}}} +_ = box.space._ck_constraint:insert({s.id, 'physics', false, 'SQL', 'X<Y'}) --- ... -format = {{name = 'X', type = 'unsigned'}} +box.execute("INSERT INTO \"test\" VALUES(2, 1);") --- +- error: 'Failed to execute SQL statement: CHECK constraint failed: physics' ... -t = {513, 1, 'test', 'memtx', 0, opts, format} +s:format({{name='Y', type='integer'}, {name='X', type='integer'}}) --- ... -s = box.space._space:insert(t) +box.execute("INSERT INTO \"test\" VALUES(1, 2);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: physics' +... +box.execute("INSERT INTO \"test\" VALUES(2, 1);") +--- +- row_count: 1 +... +s:truncate() --- -- error: 'Wrong space options (field 5): invalid MsgPack map field ''name'' type' +... +box.execute("INSERT INTO \"test\" VALUES(1, 2);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: physics' +... +s:format({}) +--- +- error: Tarantool does not support CK constraint for space without format +... +s:format() +--- +- [{'name': 'Y', 'type': 'integer'}, {'name': 'X', 'type': 'integer'}] +... +s:format({{name='Y1', type='integer'}, {name='X1', type='integer'}}) +--- +- error: 'Failed to create check constraint ''physics'': Can’t resolve field ''X''' +... +-- Ck constraint creation is forbidden for non-empty space +s:insert({2, 1}) +--- +- [2, 1] +... +_ = box.space._ck_constraint:insert({s.id, 'conflict', false, 'SQL', 'X>10'}) +--- +- error: 'Failed to create check constraint ''conflict'': referencing space must be + empty' +... +s:truncate() +--- +... +_ = box.space._ck_constraint:insert({s.id, 'conflict', false, 'SQL', 'X>10'}) +--- +... +box.execute("INSERT INTO \"test\" VALUES(1, 2);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: conflict' +... +box.execute("INSERT INTO \"test\" VALUES(11, 11);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: physics' +... +box.execute("INSERT INTO \"test\" VALUES(12, 11);") +--- +- row_count: 1 +... +s:drop() +--- +... +box.execute("CREATE TABLE T2(ID INT PRIMARY KEY, CONSTRAINT CK1 CHECK(ID > 0), CONSTRAINT CK1 CHECK(ID < 0))") +--- +- error: Duplicate key exists in unique index 'primary' in space '_ck_constraint' +... +box.space._ck_constraint:select() +--- +- [] ... -- -- gh-3611: Segfault on table creation with check referencing this table -- box.execute("CREATE TABLE w2 (s1 INT PRIMARY KEY, CHECK ((SELECT COUNT(*) FROM w2) = 0));") --- -- error: 'Failed to create space ''W2'': Space ''W2'' does not exist' +- error: 'Failed to create check constraint ''CK_CONSTRAINT_1_W2'': Subqueries are + prohibited in a ck constraint definition' ... box.execute("DROP TABLE w2;") --- @@ -123,22 +252,33 @@ box.execute("DROP TABLE w2;") -- box.execute("CREATE TABLE t5(x INT PRIMARY KEY, y INT, CHECK( x*y < ? ));") --- -- error: 'Wrong space options (field 5): invalid expression specified (bindings are - not allowed in DDL)' +- error: 'Failed to create check constraint ''CK_CONSTRAINT_1_T5'': bindings are not + allowed in DDL' ... -opts = {checks = {{expr = '?>5', name = 'ONE'}}} +-- Test trim CK constraint code correctness. +box.execute("CREATE TABLE t1(x TEXT PRIMARY KEY CHECK(x LIKE '1 a'))") --- +- row_count: 1 ... -format = {{name = 'X', type = 'unsigned'}} +box.space._ck_constraint:select()[1].code --- +- x LIKE '1 a' ... -t = {513, 1, 'test', 'memtx', 0, opts, format} +box.execute("INSERT INTO t1 VALUES('1 a')") --- +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1' ... -s = box.space._space:insert(t) +box.execute("INSERT INTO t1 VALUES('1 a')") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1' +... +box.execute("INSERT INTO t1 VALUES('1 a')") +--- +- row_count: 1 +... +box.execute("DROP TABLE t1") --- -- error: 'Wrong space options (field 5): invalid expression specified (bindings are - not allowed in DDL)' +- row_count: 1 ... test_run:cmd("clear filter") --- diff --git a/test/sql/checks.test.lua b/test/sql/checks.test.lua index 5bfcf12f8..3892138fd 100644 --- a/test/sql/checks.test.lua +++ b/test/sql/checks.test.lua @@ -8,41 +8,81 @@ box.execute('pragma sql_default_engine=\''..engine..'\'') -- gh-3272: Move SQL CHECK into server -- --- invalid expression -opts = {checks = {{expr = 'X><5'}}} -format = {{name = 'X', type = 'unsigned'}} -t = {513, 1, 'test', 'memtx', 0, opts, format} -s = box.space._space:insert(t) - +-- Until Tarantool version 2.2 check constraints were stored in +-- space opts. +-- Make sure that now this legacy option is ignored. opts = {checks = {{expr = 'X>5'}}} format = {{name = 'X', type = 'unsigned'}} t = {513, 1, 'test', 'memtx', 0, opts, format} s = box.space._space:insert(t) -box.space._space:delete(513) +_ = box.space.test:create_index('pk') -opts = {checks = {{expr = 'X>5', name = 'ONE'}}} -format = {{name = 'X', type = 'unsigned'}} -t = {513, 1, 'test', 'memtx', 0, opts, format} -s = box.space._space:insert(t) -box.space._space:delete(513) +-- Invalid expression test. +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', false, 'SQL', 'X><5'}) +-- Unexistent space test. +box.space._ck_constraint:insert({550, 'CK_CONSTRAINT_01', false, 'SQL', 'X<5'}) +-- Pass integer instead of expression. +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', false, 'SQL', 666}) +-- Defered CK constraints are not supported. +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', true, 'SQL', 'X<5'}) +-- The only supperted language is SQL. +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', false, 'LUA', 'X<5'}) --- extra invlalid field name -opts = {checks = {{expr = 'X>5', name = 'ONE', extra = 'TWO'}}} -format = {{name = 'X', type = 'unsigned'}} -t = {513, 1, 'test', 'memtx', 0, opts, format} -s = box.space._space:insert(t) +-- Check constraints LUA creation test. +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', false, 'SQL', 'X<5'}) +box.space._ck_constraint:count({}) -opts = {checks = {{expr_invalid_label = 'X>5'}}} -format = {{name = 'X', type = 'unsigned'}} -t = {513, 1, 'test', 'memtx', 0, opts, format} -s = box.space._space:insert(t) +box.execute("INSERT INTO \"test\" VALUES(5);") +box.space._ck_constraint:replace({513, 'CK_CONSTRAINT_01', false, 'SQL', 'X<=5'}) +box.execute("INSERT INTO \"test\" VALUES(5);") +box.execute("INSERT INTO \"test\" VALUES(6);") +-- Can't drop table with check constraints. +box.space.test:delete({5}) +box.space.test.index.pk:drop() +box.space._space:delete({513}) +box.space._ck_constraint:delete({513, 'CK_CONSTRAINT_01'}) +box.space._space:delete({513}) --- invalid field type -opts = {checks = {{name = 123}}} -format = {{name = 'X', type = 'unsigned'}} -t = {513, 1, 'test', 'memtx', 0, opts, format} -s = box.space._space:insert(t) +-- Create table with checks in sql. +box.execute("CREATE TABLE t1(x INTEGER CONSTRAINT ONE CHECK( x<5 ), y REAL CONSTRAINT TWO CHECK( y>x ), z INTEGER PRIMARY KEY);") +box.space._ck_constraint:count() +box.execute("INSERT INTO t1 VALUES (7, 1, 1)") +box.execute("INSERT INTO t1 VALUES (2, 1, 1)") +box.execute("INSERT INTO t1 VALUES (2, 4, 1)") +box.execute("DROP TABLE t1") + +-- Test space creation rollback on spell error in ck constraint. +box.execute("CREATE TABLE first (id FLOAT PRIMARY KEY CHECK(id < 5), a INT CONSTRAINT ONE CHECK(a >< 5));") +box.space.FIRST == nil +box.space._ck_constraint:count() == 0 +-- Ck constraints are disallowed for spaces having no format. +s = box.schema.create_space('test', {engine = engine}) +_ = s:create_index('pk') +_ = box.space._ck_constraint:insert({s.id, 'physics', false, 'SQL', 'X<Y'}) +s:format({{name='X', type='integer'}, {name='Y', type='integer'}}) +_ = box.space._ck_constraint:insert({s.id, 'physics', false, 'SQL', 'X<Y'}) +box.execute("INSERT INTO \"test\" VALUES(2, 1);") +s:format({{name='Y', type='integer'}, {name='X', type='integer'}}) +box.execute("INSERT INTO \"test\" VALUES(1, 2);") +box.execute("INSERT INTO \"test\" VALUES(2, 1);") +s:truncate() +box.execute("INSERT INTO \"test\" VALUES(1, 2);") +s:format({}) +s:format() +s:format({{name='Y1', type='integer'}, {name='X1', type='integer'}}) +-- Ck constraint creation is forbidden for non-empty space +s:insert({2, 1}) +_ = box.space._ck_constraint:insert({s.id, 'conflict', false, 'SQL', 'X>10'}) +s:truncate() +_ = box.space._ck_constraint:insert({s.id, 'conflict', false, 'SQL', 'X>10'}) +box.execute("INSERT INTO \"test\" VALUES(1, 2);") +box.execute("INSERT INTO \"test\" VALUES(11, 11);") +box.execute("INSERT INTO \"test\" VALUES(12, 11);") +s:drop() + +box.execute("CREATE TABLE T2(ID INT PRIMARY KEY, CONSTRAINT CK1 CHECK(ID > 0), CONSTRAINT CK1 CHECK(ID < 0))") +box.space._ck_constraint:select() -- -- gh-3611: Segfault on table creation with check referencing this table @@ -55,10 +95,12 @@ box.execute("DROP TABLE w2;") -- box.execute("CREATE TABLE t5(x INT PRIMARY KEY, y INT, CHECK( x*y < ? ));") -opts = {checks = {{expr = '?>5', name = 'ONE'}}} -format = {{name = 'X', type = 'unsigned'}} -t = {513, 1, 'test', 'memtx', 0, opts, format} -s = box.space._space:insert(t) - +-- Test trim CK constraint code correctness. +box.execute("CREATE TABLE t1(x TEXT PRIMARY KEY CHECK(x LIKE '1 a'))") +box.space._ck_constraint:select()[1].code +box.execute("INSERT INTO t1 VALUES('1 a')") +box.execute("INSERT INTO t1 VALUES('1 a')") +box.execute("INSERT INTO t1 VALUES('1 a')") +box.execute("DROP TABLE t1") test_run:cmd("clear filter") diff --git a/test/sql/errinj.result b/test/sql/errinj.result index ee36a387b..414e3c476 100644 --- a/test/sql/errinj.result +++ b/test/sql/errinj.result @@ -463,3 +463,137 @@ errinj.set("ERRINJ_SQL_NAME_NORMALIZATION", false) --- - ok ... +-- +-- Tests which are aimed at verifying work of commit/rollback +-- triggers on _ck_constraint space. +-- +s = box.schema.space.create('test', {format = {{name = 'X', type = 'unsigned'}}}) +--- +... +pk = box.space.test:create_index('pk') +--- +... +errinj.set("ERRINJ_WAL_IO", true) +--- +- ok +... +_ = box.space._ck_constraint:insert({s.id, 'CK_CONSTRAINT_01', false, 'SQL', 'X<5'}) +--- +- error: Failed to write to disk +... +errinj.set("ERRINJ_WAL_IO", false) +--- +- ok +... +_ = box.space._ck_constraint:insert({s.id, 'CK_CONSTRAINT_01', false, 'SQL', 'X<5'}) +--- +... +box.execute("INSERT INTO \"test\" VALUES(5);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_01' +... +errinj.set("ERRINJ_WAL_IO", true) +--- +- ok +... +_ = box.space._ck_constraint:replace({s.id, 'CK_CONSTRAINT_01', false, 'SQL', 'X<=5'}) +--- +- error: Failed to write to disk +... +errinj.set("ERRINJ_WAL_IO", false) +--- +- ok +... +_ = box.space._ck_constraint:replace({s.id, 'CK_CONSTRAINT_01', false, 'SQL', 'X<=5'}) +--- +... +box.execute("INSERT INTO \"test\" VALUES(5);") +--- +- row_count: 1 +... +errinj.set("ERRINJ_WAL_IO", true) +--- +- ok +... +_ = box.space._ck_constraint:delete({s.id, 'CK_CONSTRAINT_01'}) +--- +- error: Failed to write to disk +... +box.execute("INSERT INTO \"test\" VALUES(6);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_01' +... +errinj.set("ERRINJ_WAL_IO", false) +--- +- ok +... +_ = box.space._ck_constraint:delete({s.id, 'CK_CONSTRAINT_01'}) +--- +... +box.execute("INSERT INTO \"test\" VALUES(6);") +--- +- row_count: 1 +... +s:drop() +--- +... +-- +-- Test that failed space alter doesn't harm ck constraints +-- +s = box.schema.create_space('test') +--- +... +_ = s:create_index('pk') +--- +... +s:format({{name='X', type='integer'}, {name='Y', type='integer'}}) +--- +... +_ = box.space._ck_constraint:insert({s.id, 'XlessY', false, 'SQL', 'X < Y'}) +--- +... +_ = box.space._ck_constraint:insert({s.id, 'Xgreater10', false, 'SQL', 'X > 10'}) +--- +... +box.execute("INSERT INTO \"test\" VALUES(1, 2);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: Xgreater10' +... +box.execute("INSERT INTO \"test\" VALUES(20, 10);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: XlessY' +... +box.execute("INSERT INTO \"test\" VALUES(20, 100);") +--- +- row_count: 1 +... +s:truncate() +--- +... +errinj.set("ERRINJ_WAL_IO", true) +--- +- ok +... +s:format({{name='Y', type='integer'}, {name='X', type='integer'}}) +--- +- error: Failed to write to disk +... +errinj.set("ERRINJ_WAL_IO", false) +--- +- ok +... +box.execute("INSERT INTO \"test\" VALUES(1, 2);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: Xgreater10' +... +box.execute("INSERT INTO \"test\" VALUES(20, 10);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: XlessY' +... +box.execute("INSERT INTO \"test\" VALUES(20, 100);") +--- +- row_count: 1 +... +s:drop() +--- +... diff --git a/test/sql/errinj.test.lua b/test/sql/errinj.test.lua index 1aff6d77e..48b80a443 100644 --- a/test/sql/errinj.test.lua +++ b/test/sql/errinj.test.lua @@ -149,3 +149,48 @@ box.execute("CREATE TABLE hello (id INT primary key,x INT,y INT);") dummy_f = function(int) return 1 end box.internal.sql_create_function("counter1", "INT", dummy_f, -1, false) errinj.set("ERRINJ_SQL_NAME_NORMALIZATION", false) + +-- +-- Tests which are aimed at verifying work of commit/rollback +-- triggers on _ck_constraint space. +-- +s = box.schema.space.create('test', {format = {{name = 'X', type = 'unsigned'}}}) +pk = box.space.test:create_index('pk') + +errinj.set("ERRINJ_WAL_IO", true) +_ = box.space._ck_constraint:insert({s.id, 'CK_CONSTRAINT_01', false, 'SQL', 'X<5'}) +errinj.set("ERRINJ_WAL_IO", false) +_ = box.space._ck_constraint:insert({s.id, 'CK_CONSTRAINT_01', false, 'SQL', 'X<5'}) +box.execute("INSERT INTO \"test\" VALUES(5);") +errinj.set("ERRINJ_WAL_IO", true) +_ = box.space._ck_constraint:replace({s.id, 'CK_CONSTRAINT_01', false, 'SQL', 'X<=5'}) +errinj.set("ERRINJ_WAL_IO", false) +_ = box.space._ck_constraint:replace({s.id, 'CK_CONSTRAINT_01', false, 'SQL', 'X<=5'}) +box.execute("INSERT INTO \"test\" VALUES(5);") +errinj.set("ERRINJ_WAL_IO", true) +_ = box.space._ck_constraint:delete({s.id, 'CK_CONSTRAINT_01'}) +box.execute("INSERT INTO \"test\" VALUES(6);") +errinj.set("ERRINJ_WAL_IO", false) +_ = box.space._ck_constraint:delete({s.id, 'CK_CONSTRAINT_01'}) +box.execute("INSERT INTO \"test\" VALUES(6);") +s:drop() + +-- +-- Test that failed space alter doesn't harm ck constraints +-- +s = box.schema.create_space('test') +_ = s:create_index('pk') +s:format({{name='X', type='integer'}, {name='Y', type='integer'}}) +_ = box.space._ck_constraint:insert({s.id, 'XlessY', false, 'SQL', 'X < Y'}) +_ = box.space._ck_constraint:insert({s.id, 'Xgreater10', false, 'SQL', 'X > 10'}) +box.execute("INSERT INTO \"test\" VALUES(1, 2);") +box.execute("INSERT INTO \"test\" VALUES(20, 10);") +box.execute("INSERT INTO \"test\" VALUES(20, 100);") +s:truncate() +errinj.set("ERRINJ_WAL_IO", true) +s:format({{name='Y', type='integer'}, {name='X', type='integer'}}) +errinj.set("ERRINJ_WAL_IO", false) +box.execute("INSERT INTO \"test\" VALUES(1, 2);") +box.execute("INSERT INTO \"test\" VALUES(20, 10);") +box.execute("INSERT INTO \"test\" VALUES(20, 100);") +s:drop() diff --git a/test/sql/gh-2981-check-autoinc.result b/test/sql/gh-2981-check-autoinc.result index 9e347ca6b..7384c81e8 100644 --- a/test/sql/gh-2981-check-autoinc.result +++ b/test/sql/gh-2981-check-autoinc.result @@ -29,7 +29,7 @@ box.execute("insert into t1 values (18, null);") ... box.execute("insert into t1(s2) values (null);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: T1' +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1' ... box.execute("insert into t2 values (18, null);") --- @@ -37,7 +37,7 @@ box.execute("insert into t2 values (18, null);") ... box.execute("insert into t2(s2) values (null);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: T2' +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T2' ... box.execute("insert into t2 values (24, null);") --- @@ -45,7 +45,7 @@ box.execute("insert into t2 values (24, null);") ... box.execute("insert into t2(s2) values (null);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: T2' +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T2' ... box.execute("insert into t3 values (9, null)") --- @@ -53,7 +53,7 @@ box.execute("insert into t3 values (9, null)") ... box.execute("insert into t3(s2) values (null)") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: T3' +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T3' ... box.execute("DROP TABLE t1") --- diff --git a/test/sql/types.result b/test/sql/types.result index bc4518c01..582785413 100644 --- a/test/sql/types.result +++ b/test/sql/types.result @@ -709,7 +709,7 @@ box.execute("CREATE TABLE t1 (id INT PRIMARY KEY, a BOOLEAN CHECK (a = true));") ... box.execute("INSERT INTO t1 VALUES (1, false);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: T1' +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1' ... box.execute("INSERT INTO t1 VALUES (1, true);") --- diff --git a/test/sql/upgrade.result b/test/sql/upgrade.result index 3a55f7c53..f0997e17f 100644 --- a/test/sql/upgrade.result +++ b/test/sql/upgrade.result @@ -188,6 +188,25 @@ i[1].opts.sql == nil --- - true ... +box.space._space:get(s.id).flags.checks == nil +--- +- true +... +check = box.space._ck_constraint:select()[1] +--- +... +check ~= nil +--- +- true +... +check.name +--- +- CK_CONSTRAINT_1_T5 +... +check.code +--- +- x < 2 +... s:drop() --- ... diff --git a/test/sql/upgrade.test.lua b/test/sql/upgrade.test.lua index b76a8f373..37425ae21 100644 --- a/test/sql/upgrade.test.lua +++ b/test/sql/upgrade.test.lua @@ -62,6 +62,11 @@ s ~= nil i = box.space._index:select(s.id) i ~= nil i[1].opts.sql == nil +box.space._space:get(s.id).flags.checks == nil +check = box.space._ck_constraint:select()[1] +check ~= nil +check.name +check.code s:drop() test_run:switch('default') diff --git a/test/wal_off/alter.result b/test/wal_off/alter.result index ee280fcbb..8040efa1a 100644 --- a/test/wal_off/alter.result +++ b/test/wal_off/alter.result @@ -28,7 +28,7 @@ end; ... #spaces; --- -- 65505 +- 65504 ... -- cleanup for k, v in pairs(spaces) do -- 2.21.0 ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 4/6] schema: add new system space for CHECK constraints 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 4/6] schema: add new system space for CHECK constraints Kirill Shcherbatov @ 2019-05-26 12:06 ` Vladislav Shpilevoy 2019-05-26 13:31 ` n.pettik 2019-05-31 13:45 ` Kirill Shcherbatov 0 siblings, 2 replies; 31+ messages in thread From: Vladislav Shpilevoy @ 2019-05-26 12:06 UTC (permalink / raw) To: tarantool-patches, Kirill Shcherbatov Thanks for the patch! See 12 comments below. > @@ -1387,6 +1377,79 @@ UpdateSchemaVersion::alter(struct alter_space *alter) > ++schema_version; > } > > +/** > + * As ck_constraint object depends on space_def we must rebuild > + * all ck constraints on space alter. > + * > + * To perform it transactionally, we create a list of new ck > + * constraints objects in ::prepare method that is fault-tolerant. > + * Finally in ::alter or ::rollback methods we only swap thouse 1. 'thouse' word does not exist, it is clearly visible with a spell checker, which usually highlights unknown words. > + * lists securely. > + */ > +class RebuildCkConstraints: public AlterSpaceOp > +{ > + void space_swap_ck_constraint(struct space *old_space, > + struct space *new_space); > +public: > + RebuildCkConstraints(struct alter_space *alter) : AlterSpaceOp(alter), > + ck_constraint(RLIST_HEAD_INITIALIZER(ck_constraint)) {} > + struct rlist ck_constraint; > + virtual void prepare(struct alter_space *alter); > + virtual void alter(struct alter_space *alter); > + virtual void rollback(struct alter_space *alter); > + virtual ~RebuildCkConstraints(); > +}; > @@ -4040,6 +4111,176 @@ on_replace_dd_fk_constraint(struct trigger * /* trigger*/, void *event) > + > +/** A trigger invoked on replace in the _ck_constraint space. */ > +static void > +on_replace_dd_ck_constraint(struct trigger * /* trigger*/, void *event) > +{ > + struct txn *txn = (struct txn *) event; > + txn_check_singlestatement_xc(txn, "Space _ck_constraint"); > + struct txn_stmt *stmt = txn_current_stmt(txn); > + struct tuple *old_tuple = stmt->old_tuple; > + struct tuple *new_tuple = stmt->new_tuple; > + uint32_t space_id = > + tuple_field_u32_xc(old_tuple != NULL ? old_tuple : new_tuple, > + BOX_CK_CONSTRAINT_FIELD_SPACE_ID); > + struct space *space = space_cache_find_xc(space_id); > + struct trigger *on_rollback = > + txn_alter_trigger_new(on_replace_ck_constraint_rollback, NULL); > + struct trigger *on_commit = > + txn_alter_trigger_new(on_replace_ck_constraint_commit, NULL); > + > + if (new_tuple != NULL) { > + bool is_deferred = > + tuple_field_bool_xc(new_tuple, > + BOX_CK_CONSTRAINT_FIELD_DEFERRED); > + if (is_deferred) { > + tnt_raise(ClientError, ER_UNSUPPORTED, "Tarantool", > + "deferred ck constraints"); > + } > + /* Create or replace check constraint. */ > + struct ck_constraint_def *ck_def = > + ck_constraint_def_new_from_tuple(new_tuple); > + auto ck_guard = make_scoped_guard([=] { free(ck_def); }); 2. Please, implement and use ck_constraint_def_delete. Free() will leak, when ck_constraint will become more complex. > + /* > + * FIXME: Ck constraint creation on non-empty > + * space must be implemented as preparatory > + * step for ALTER SPACE ADD CONSTRAINT feature. > + */ > + struct index *pk = space_index(space, 0); > + if (pk != NULL && index_size(pk) > 0) { > + tnt_raise(ClientError, ER_CREATE_CK_CONSTRAINT, > + ck_def->name, > + "referencing space must be empty"); > + } > + struct ck_constraint *new_ck_constraint = > + ck_constraint_new(ck_def, space->def); > + if (new_ck_constraint == NULL) > + diag_raise(); > + ck_guard.is_active = false; > + const char *name = new_ck_constraint->def->name; > + struct ck_constraint *old_ck_constraint = > + space_ck_constraint_by_name(space, name, strlen(name)); > + if (old_ck_constraint != NULL) > + rlist_del_entry(old_ck_constraint, link); > + rlist_add_entry(&space->ck_constraint, new_ck_constraint, link); > + on_commit->data = old_tuple == NULL ? new_ck_constraint : > + old_ck_constraint; > + on_rollback->data = on_commit->data; > diff --git a/src/box/ck_constraint.c b/src/box/ck_constraint.c > new file mode 100644 > index 000000000..c74687b31 > --- /dev/null > +++ b/src/box/ck_constraint.c > + > +void > +ck_constraint_delete(struct ck_constraint *ck_constraint) > +{ > + sql_expr_delete(sql_get(), ck_constraint->expr, false); > + free(ck_constraint->def); 3. The same. Use ck_constraint_def_delete. > + TRASH(ck_constraint); > + free(ck_constraint); > +} > diff --git a/src/box/ck_constraint.h b/src/box/ck_constraint.h > new file mode 100644 > index 000000000..e20203bb5 > --- /dev/null > +++ b/src/box/ck_constraint.h > + > +/** > + * Calculate check constraint definition memory size and fields > + * offsets for given arguments. > + * > + * Alongside with struct ck_constraint_def itself, we reserve > + * memory for string containing its name and expression string. > + * > + * Memory layout: > + * +-----------------------------+ <- Allocated memory starts here > + * | struct ck_constraint_def | > + * |-----------------------------| > + * | name + \0 | > + * |-----------------------------| > + * | expr_str + \0 | > + * +-----------------------------+ > + * > + * @param name_len The length of the name. > + * @param expr_str_len The length of the expr_str. > + * @param[out] name_offset The offset of the name string. > + * @param[out] expr_str_offset The offset of the expr_str string. > + * @retval The size of the ck constraint definition object for > + * given parameters. 4. I said it probably 2-3 times, but you still ignore. @retval takes one parameter, @return take 0 parameters. Here you wrote, that returned value is 'The'. > + */ > +static inline uint32_t > +ck_constraint_def_sizeof(uint32_t name_len, uint32_t expr_str_len, > + uint32_t *name_offset, uint32_t *expr_str_offset) > +{ > + *name_offset = sizeof(struct ck_constraint_def); > + *expr_str_offset = *name_offset + name_len + 1; > + return *expr_str_offset + expr_str_len + 1; > +} > diff --git a/src/box/schema_def.h b/src/box/schema_def.h > index dea3fad19..317e6b51f 100644 > --- a/src/box/schema_def.h > +++ b/src/box/schema_def.h > @@ -108,6 +108,8 @@ enum { > BOX_SPACE_SEQUENCE_ID = 340, > /** Space id of _fk_constraint. */ > BOX_FK_CONSTRAINT_ID = 356, > + /** Space id of _ck_contraint. */ > + BOX_CK_CONSTRAINT_ID = 357, 5. Please, leave a gap after BOX_FK_CONSTRAINT_ID. We keep gaps to be able in future to add a view space, and to choose recovery order. Gap is typically 7 unused IDs. The new ID should be 364. > /** End of the reserved range of system spaces. */ > BOX_SYSTEM_ID_MAX = 511, > BOX_ID_NIL = 2147483647 > diff --git a/src/box/sql/build.c b/src/box/sql/build.c > index 91b977de2..a3b11531f 100644 > --- a/src/box/sql/build.c > +++ b/src/box/sql/build.c > @@ -638,40 +640,116 @@ primary_key_exit: > return; > } > > +/** > + * Prepare a 0-terminated string in the wptr memory buffer that > + * does not contain a sequence of more than one whatespace > + * character. Routine enforces ' ' (space) as whitespace > + * delimiter. When character ' or " was met, the sting is copied > + * without any changes until the next (corres) ' or " . 6. What is 'corres'? > + * The wptr buffer is expected to have str_len + 1 bytes > + * (this is the expected scenario where no extra whitespace > + * characters preset in the source string). 7. What is 'characters preset'? > + * @param wptr The destination memory buffer of size > + * @a str_len + 1. > + * @param str The source string to be copied. > + * @param str_len The source string @a str length. > + */ > +static void > +trim_space_snprintf(char *wptr, const char *str, uint32_t str_len) > @@ -1416,6 +1529,37 @@ vdbe_emit_fk_constraint_drop(struct Parse *parse_context, char *constraint_name, > sqlReleaseTempRange(parse_context, key_reg, 3); > } > > +/** > + * Generate VDBE program to remove entry from _ck_constraint space. > + * > + * @param parser Parsing context. > + * @param ck_name Name of CK constraint to be dropped. > + * @param child_id Id of table which constraint belongs to. 8. There is no parameter 'child_id'. > + */ > +static void > +vdbe_emit_ck_constraint_drop(struct Parse *parser, const char *ck_name, > + uint32_t space_id) > diff --git a/src/box/sql/parse.y b/src/box/sql/parse.y > index f241b8d52..e93dfe751 100644 > --- a/src/box/sql/parse.y > +++ b/src/box/sql/parse.y > @@ -297,7 +297,7 @@ ccons ::= check_constraint_def . > > check_constraint_def ::= cconsname(N) CHECK LP expr(X) RP. { > create_ck_def_init(&pParse->create_ck_def, &N, &X); > - sql_add_check_constraint(pParse); > + sql_create_check_contraint(pParse); 9. What was a motivation of this rename? As I understand, we can add many CHECK constraints in one CREATE TABLE. We do not 'create' one single CHECK, we add multiple ones. Please, keep the old name. > } > > ccons ::= cconsname(N) REFERENCES nm(T) eidlist_opt(TA) matcharg(M) refargs(R). { > diff --git a/test/sql/checks.result b/test/sql/checks.result > index f7cddec43..5f0098e23 100644 > --- a/test/sql/checks.result > +++ b/test/sql/checks.result > @@ -30,89 +32,216 @@ t = {513, 1, 'test', 'memtx', 0, opts, format} > -s = box.space._space:insert(t) > +-- Pass integer instead of expression. > +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', false, 'SQL', 666}) > +--- > +- error: 'Tuple field 5 type does not match one required by operation: expected string' > +... > +-- Defered CK constraints are not supported. 10. 'Defered' word does not exist, use 'deferred'. 'Unexistent' word does not exist as well. Non-existent is better. > +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', true, 'SQL', 'X<5'}) > +--- > +- error: Tarantool does not support deferred ck constraints > +... > +-- The only supperted language is SQL. 11. 'Supperted' word does not exist. > +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', false, 'LUA', 'X<5'}) > +--- > +- error: Unsupported language 'LUA' specified for function 'CK_CONSTRAINT_01' > +... > ... > +--- > +... > +box.execute("CREATE TABLE T2(ID INT PRIMARY KEY, CONSTRAINT CK1 CHECK(ID > 0), CONSTRAINT CK1 CHECK(ID < 0))") > +--- > +- error: Duplicate key exists in unique index 'primary' in space '_ck_constraint' 12. The error is not ok. It should not expose any system details like _ck_constraint space. Please, check duplicate names before an insertion. > +... > +box.space._ck_constraint:select() > +--- > +- [] > ... ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 4/6] schema: add new system space for CHECK constraints 2019-05-26 12:06 ` [tarantool-patches] " Vladislav Shpilevoy @ 2019-05-26 13:31 ` n.pettik 2019-05-26 13:32 ` Vladislav Shpilevoy 2019-05-31 13:45 ` Kirill Shcherbatov 1 sibling, 1 reply; 31+ messages in thread From: n.pettik @ 2019-05-26 13:31 UTC (permalink / raw) To: tarantool-patches; +Cc: Vladislav Shpilevoy, Kirill Shcherbatov >> diff --git a/src/box/sql/parse.y b/src/box/sql/parse.y >> index f241b8d52..e93dfe751 100644 >> --- a/src/box/sql/parse.y >> +++ b/src/box/sql/parse.y >> @@ -297,7 +297,7 @@ ccons ::= check_constraint_def . >> >> check_constraint_def ::= cconsname(N) CHECK LP expr(X) RP. { >> create_ck_def_init(&pParse->create_ck_def, &N, &X); >> - sql_add_check_constraint(pParse); >> + sql_create_check_contraint(pParse); > > 9. What was a motivation of this rename? As I understand, > we can add many CHECK constraints in one CREATE TABLE. We > do not 'create' one single CHECK, we add multiple ones. Please, > keep the old name. It was me who asked for this refactoring. We already have sql_create_foreign_key() and sql_create_index(), so my intention was to keep naming consistent. ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 4/6] schema: add new system space for CHECK constraints 2019-05-26 13:31 ` n.pettik @ 2019-05-26 13:32 ` Vladislav Shpilevoy 0 siblings, 0 replies; 31+ messages in thread From: Vladislav Shpilevoy @ 2019-05-26 13:32 UTC (permalink / raw) To: n.pettik, tarantool-patches; +Cc: Kirill Shcherbatov On 26/05/2019 16:31, n.pettik wrote: > >>> diff --git a/src/box/sql/parse.y b/src/box/sql/parse.y >>> index f241b8d52..e93dfe751 100644 >>> --- a/src/box/sql/parse.y >>> +++ b/src/box/sql/parse.y >>> @@ -297,7 +297,7 @@ ccons ::= check_constraint_def . >>> >>> check_constraint_def ::= cconsname(N) CHECK LP expr(X) RP. { >>> create_ck_def_init(&pParse->create_ck_def, &N, &X); >>> - sql_add_check_constraint(pParse); >>> + sql_create_check_contraint(pParse); >> >> 9. What was a motivation of this rename? As I understand, >> we can add many CHECK constraints in one CREATE TABLE. We >> do not 'create' one single CHECK, we add multiple ones. Please, >> keep the old name. > > It was me who asked for this refactoring. We already have > sql_create_foreign_key() and sql_create_index(), so my > intention was to keep naming consistent. > Thanks for the clarification. Then the new name is ok. ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 4/6] schema: add new system space for CHECK constraints 2019-05-26 12:06 ` [tarantool-patches] " Vladislav Shpilevoy 2019-05-26 13:31 ` n.pettik @ 2019-05-31 13:45 ` Kirill Shcherbatov 2019-06-03 21:15 ` Vladislav Shpilevoy 1 sibling, 1 reply; 31+ messages in thread From: Kirill Shcherbatov @ 2019-05-31 13:45 UTC (permalink / raw) To: tarantool-patches, Vladislav Shpilevoy > 1. 'thouse' word does not exist, it is clearly visible with a > spell checker, which usually highlights unknown words. Fixed. >> + /* Create or replace check constraint. */ >> + struct ck_constraint_def *ck_def = >> + ck_constraint_def_new_from_tuple(new_tuple); >> + auto ck_guard = make_scoped_guard([=] { free(ck_def); }); > > 2. Please, implement and use ck_constraint_def_delete. Free() will leak, when > ck_constraint will become more complex. > 3. The same. Use ck_constraint_def_delete. Done. struct ck_constraint_def * ck_constraint_def_new(const char *name, uint32_t name_len, const char *expr, uint32_t expr_str_len, uint32_t space_id, enum ck_constraint_language language); void ck_constraint_def_delete(struct ck_constraint_def *ck_def); >> + * @retval The size of the ck constraint definition object for >> + * given parameters. > > 4. I said it probably 2-3 times, but you still ignore. @retval takes > one parameter, @return take 0 parameters. Here you wrote, that > returned value is 'The'. Hm, I finally understood the difference between them. >> + BOX_CK_CONSTRAINT_ID = 357, > > 5. Please, leave a gap after BOX_FK_CONSTRAINT_ID. We keep > gaps to be able in future to add a view space, and to choose > recovery order. Gap is typically 7 unused IDs. The new ID > should be 364. Done. >> +/** >> + * Prepare a 0-terminated string in the wptr memory buffer that >> + * does not contain a sequence of more than one whatespace >> + * character. Routine enforces ' ' (space) as whitespace >> + * delimiter. When character ' or " was met, the sting is copied >> + * without any changes until the next (corres) ' or " . > >> + * The wptr buffer is expected to have str_len + 1 bytes >> + * (this is the expected scenario where no extra whitespace >> + * characters preset in the source string). > 6. What is 'corres'?> 7. What is 'characters preset'? /** * Prepare a 0-terminated string in the wptr memory buffer that * does not contain a sequence of more than one whatespace * character. Routine enforces ' ' (space) as whitespace * delimiter. When character ' or " was met, the sting is copied * without any changes until the next ' or " sign. * The wptr buffer is expected to have str_len + 1 bytes * (this is the expected scenario where no extra whitespace * characters in the source string). * @param wptr The destination memory buffer of size * @a str_len + 1. * @param str The source string to be copied. * @param str_len The source string @a str length. */ static void trim_space_snprintf(char *wptr, const char *str, uint32_t str_len) >> +/** >> + * Generate VDBE program to remove entry from _ck_constraint space. >> + * >> + * @param parser Parsing context. >> + * @param ck_name Name of CK constraint to be dropped. >> + * @param child_id Id of table which constraint belongs to. > > 8. There is no parameter 'child_id'. space_id. fixed. >> + sql_create_check_contraint(pParse); > > 9. What was a motivation of this rename? As I understand, > we can add many CHECK constraints in one CREATE TABLE. We > do not 'create' one single CHECK, we add multiple ones. Please, > keep the old name. Nikita asked me to do this. >> +-- Defered CK constraints are not supported. > > 10. 'Defered' word does not exist, use 'deferred'. > 'Unexistent' word does not exist as well. Non-existent is better. > 11. 'Supperted' word does not exist. Fixed. >> +--- >> +... >> +box.execute("CREATE TABLE T2(ID INT PRIMARY KEY, CONSTRAINT CK1 CHECK(ID > 0), CONSTRAINT CK1 CHECK(ID < 0))") >> +--- >> +- error: Duplicate key exists in unique index 'primary' in space '_ck_constraint' > > 12. The error is not ok. It should not expose any system details like > _ck_constraint space. Please, check duplicate names before an insertion. Ok, done. box.execute("CREATE TABLE T2(ID INT PRIMARY KEY, CONSTRAINT CK1 CHECK(ID > 0), CONSTRAINT CK1 CHECK(ID < 0))") --- - error: Constraint CK1 already exists ... ============================================================= This patch introduces a new system space to persist check constraints. The format of the new system space is _ck_constraint (space id = 364) [<space id> UINT, <constraint name> STR, <is_deferred>BOOL, <language>STR], <code>STR A CK constraint is local for a space, so every pair <space id, CK name> is unique (it is also the PK in the _ck_constraint space). After insertion into this space, a new instance describing check constraint is created. Check constraint holds an exspression AST. While space features some check constraints, it isn't allowed to be dropped. The :drop() space method firstly deletes all check constraints and then removes an entry from the _space. Because the space alter, the index alter and the space truncate operations cause space recreation process, a new RebuildCkConstrains object is introduced. This alter object compiles a new ck constraint object, replaces and removes an existent instances atomically (but if the assembly of some ck constraint object fails, nothing is changed). In fact, in scope of this patch we don't really need to recreate a ck_constraint object in such situations (it is enough to patch space_def pointer in AST tree like we did it before, but we are going to recompile a VDBE that represents ck constraint in further patches, and that operation is not safe). The main motivation for these changes is an ability to support ADD CHECK CONSTRAINT operation in the future. CK constraints are easier to manage as self-sustained objects: such change is managed with atomic insertion(unlike the current architecture). Finally, the xfer optimization is disabled now if some space have ck constraints. In following patches this xfer optimisation becomes impossible, so there is no reason to rewrite this code now. Needed for #3691 --- src/box/CMakeLists.txt | 1 + src/box/alter.cc | 256 ++++++++++++++++++++++++-- src/box/alter.h | 1 + src/box/bootstrap.snap | Bin 4393 -> 4444 bytes src/box/ck_constraint.c | 148 +++++++++++++++ src/box/ck_constraint.h | 205 +++++++++++++++++++++ src/box/errcode.h | 3 +- src/box/lua/schema.lua | 4 + src/box/lua/space.cc | 2 + src/box/lua/upgrade.lua | 40 ++++ src/box/schema.cc | 8 + src/box/schema_def.h | 11 ++ src/box/space.c | 2 + src/box/space.h | 5 + src/box/space_def.c | 98 +--------- src/box/space_def.h | 4 - src/box/sql.c | 86 +-------- src/box/sql.h | 40 +--- src/box/sql/build.c | 232 +++++++++++++++++++---- src/box/sql/insert.c | 55 +++--- src/box/sql/parse.y | 2 +- src/box/sql/parse_def.h | 24 +++ src/box/sql/resolve.c | 8 +- src/box/sql/select.c | 11 +- src/box/sql/sqlInt.h | 2 +- src/box/sql/tokenize.c | 1 - test/app-tap/tarantoolctl.test.lua | 4 +- test/box-py/bootstrap.result | 4 + test/box/access.result | 3 + test/box/access.test.lua | 1 + test/box/access_misc.result | 3 + test/box/access_sysview.result | 6 +- test/box/alter.result | 5 +- test/box/misc.result | 1 + test/sql-tap/check.test.lua | 32 ++-- test/sql-tap/fkey2.test.lua | 4 +- test/sql-tap/sql-errors.test.lua | 2 +- test/sql-tap/table.test.lua | 4 +- test/sql/checks.result | 227 ++++++++++++++++++----- test/sql/checks.test.lua | 104 +++++++---- test/sql/errinj.result | 134 ++++++++++++++ test/sql/errinj.test.lua | 45 +++++ test/sql/gh-2981-check-autoinc.result | 8 +- test/sql/types.result | 2 +- test/sql/upgrade.result | 19 ++ test/sql/upgrade.test.lua | 5 + test/wal_off/alter.result | 2 +- 47 files changed, 1440 insertions(+), 424 deletions(-) create mode 100644 src/box/ck_constraint.c create mode 100644 src/box/ck_constraint.h diff --git a/src/box/CMakeLists.txt b/src/box/CMakeLists.txt index 0864c3433..481842a39 100644 --- a/src/box/CMakeLists.txt +++ b/src/box/CMakeLists.txt @@ -97,6 +97,7 @@ add_library(box STATIC space.c space_def.c sequence.c + ck_constraint.c fk_constraint.c func.c func_def.c diff --git a/src/box/alter.cc b/src/box/alter.cc index ed9e55907..bedeb71cd 100644 --- a/src/box/alter.cc +++ b/src/box/alter.cc @@ -29,6 +29,7 @@ * SUCH DAMAGE. */ #include "alter.h" +#include "ck_constraint.h" #include "column_mask.h" #include "schema.h" #include "user.h" @@ -551,17 +552,6 @@ space_def_new_from_tuple(struct tuple *tuple, uint32_t errcode, engine_name, engine_name_len, &opts, fields, field_count); auto def_guard = make_scoped_guard([=] { space_def_delete(def); }); - if (def->opts.checks != NULL && - sql_checks_resolve_space_def_reference(def->opts.checks, - def) != 0) { - box_error_t *err = box_error_last(); - if (box_error_code(err) != ENOMEM) { - tnt_raise(ClientError, errcode, def->name, - box_error_message(err)); - } else { - diag_raise(); - } - } struct engine *engine = engine_find_xc(def->engine_name); engine_check_space_def_xc(engine, def); def_guard.is_active = false; @@ -1404,6 +1394,79 @@ UpdateSchemaVersion::alter(struct alter_space *alter) ++schema_version; } +/** + * As ck_constraint object depends on space_def we must rebuild + * all ck constraints on space alter. + * + * To perform it transactionally, we create a list of new ck + * constraint objects in ::prepare method that is fault-tolerant. + * Finally in ::alter or ::rollback methods we only swap those + * lists securely. + */ +class RebuildCkConstraints: public AlterSpaceOp +{ + void space_swap_ck_constraint(struct space *old_space, + struct space *new_space); +public: + RebuildCkConstraints(struct alter_space *alter) : AlterSpaceOp(alter), + ck_constraint(RLIST_HEAD_INITIALIZER(ck_constraint)) {} + struct rlist ck_constraint; + virtual void prepare(struct alter_space *alter); + virtual void alter(struct alter_space *alter); + virtual void rollback(struct alter_space *alter); + virtual ~RebuildCkConstraints(); +}; + +void +RebuildCkConstraints::prepare(struct alter_space *alter) +{ + struct ck_constraint *old_ck_constraint; + rlist_foreach_entry(old_ck_constraint, &alter->old_space->ck_constraint, + link) { + struct ck_constraint *new_ck_constraint = + ck_constraint_new(old_ck_constraint->def, + alter->new_space->def); + if (new_ck_constraint == NULL) + diag_raise(); + rlist_add_entry(&ck_constraint, new_ck_constraint, link); + } +} + +void +RebuildCkConstraints::space_swap_ck_constraint(struct space *old_space, + struct space *new_space) +{ + rlist_swap(&new_space->ck_constraint, &ck_constraint); + rlist_swap(&ck_constraint, &old_space->ck_constraint); +} + +void +RebuildCkConstraints::alter(struct alter_space *alter) +{ + space_swap_ck_constraint(alter->old_space, alter->new_space); +} + +void +RebuildCkConstraints::rollback(struct alter_space *alter) +{ + space_swap_ck_constraint(alter->new_space, alter->old_space); +} + +RebuildCkConstraints::~RebuildCkConstraints() +{ + struct ck_constraint *old_ck_constraint, *tmp; + rlist_foreach_entry_safe(old_ck_constraint, &ck_constraint, link, tmp) { + /** + * Ck constraint definition is now managed by + * other Ck constraint object. Prevent it's + * destruction as a part of ck_constraint_delete + * call. + */ + old_ck_constraint->def = NULL; + ck_constraint_delete(old_ck_constraint); + } +} + /* }}} */ /** @@ -1769,6 +1832,11 @@ on_replace_dd_space(struct trigger * /* trigger */, void *event) space_name(old_space), "the space has foreign key constraints"); } + if (!rlist_empty(&old_space->ck_constraint)) { + tnt_raise(ClientError, ER_DROP_SPACE, + space_name(old_space), + "the space has check constraints"); + } /** * The space must be deleted from the space * cache right away to achieve linearisable @@ -1866,6 +1934,7 @@ on_replace_dd_space(struct trigger * /* trigger */, void *event) def->field_count); (void) new CheckSpaceFormat(alter); (void) new ModifySpace(alter, def); + (void) new RebuildCkConstraints(alter); def_guard.is_active = false; /* Create MoveIndex ops for all space indexes. */ alter_space_move_indexes(alter, 0, old_space->index_id_max + 1); @@ -2108,6 +2177,7 @@ on_replace_dd_index(struct trigger * /* trigger */, void *event) * old space. */ alter_space_move_indexes(alter, iid + 1, old_space->index_id_max + 1); + (void) new RebuildCkConstraints(alter); /* Add an op to update schema_version on commit. */ (void) new UpdateSchemaVersion(alter); alter_space_do(txn, alter); @@ -2176,6 +2246,7 @@ on_replace_dd_truncate(struct trigger * /* trigger */, void *event) (void) new TruncateIndex(alter, old_index->def->iid); } + (void) new RebuildCkConstraints(alter); alter_space_do(txn, alter); scoped_guard.is_active = false; } @@ -4088,6 +4159,165 @@ on_replace_dd_fk_constraint(struct trigger * /* trigger*/, void *event) } } +/** Create an instance of check constraint definition by tuple. */ +static struct ck_constraint_def * +ck_constraint_def_new_from_tuple(struct tuple *tuple) +{ + uint32_t name_len; + const char *name = + tuple_field_str_xc(tuple, BOX_CK_CONSTRAINT_FIELD_NAME, + &name_len); + if (name_len > BOX_NAME_MAX) { + tnt_raise(ClientError, ER_CREATE_CK_CONSTRAINT, + tt_cstr(name, BOX_INVALID_NAME_MAX), + "check constraint name is too long"); + } + identifier_check_xc(name, name_len); + uint32_t space_id = + tuple_field_u32_xc(tuple, BOX_CK_CONSTRAINT_FIELD_SPACE_ID); + const char *language_str = + tuple_field_cstr_xc(tuple, BOX_CK_CONSTRAINT_FIELD_LANGUAGE); + enum ck_constraint_language language = + STR2ENUM(ck_constraint_language, language_str); + if (language == ck_constraint_language_MAX) { + tnt_raise(ClientError, ER_FUNCTION_LANGUAGE, language_str, + tt_cstr(name, name_len)); + } + uint32_t expr_str_len; + const char *expr_str = + tuple_field_str_xc(tuple, BOX_CK_CONSTRAINT_FIELD_CODE, + &expr_str_len); + struct ck_constraint_def *ck_def = + ck_constraint_def_new(name, name_len, expr_str, expr_str_len, + space_id, language); + if (ck_def == NULL) + diag_raise(); + return ck_def; +} + +/** Trigger invoked on rollback in the _ck_constraint space. */ +static void +on_replace_ck_constraint_rollback(struct trigger *trigger, void *event) +{ + struct txn_stmt *stmt = txn_last_stmt((struct txn *) event); + struct ck_constraint *ck = (struct ck_constraint *)trigger->data; + struct space *space = NULL; + if (ck != NULL) + space = space_by_id(ck->def->space_id); + if (stmt->old_tuple != NULL && stmt->new_tuple == NULL) { + /* Rollback DELETE check constraint. */ + assert(ck != NULL); + assert(space != NULL); + assert(space_ck_constraint_by_name(space, + ck->def->name, strlen(ck->def->name)) == NULL); + rlist_add_entry(&space->ck_constraint, ck, link); + } else if (stmt->new_tuple != NULL && stmt->old_tuple == NULL) { + /* Rollback INSERT check constraint. */ + assert(space != NULL); + assert(space_ck_constraint_by_name(space, + ck->def->name, strlen(ck->def->name)) != NULL); + rlist_del_entry(ck, link); + ck_constraint_delete(ck); + } else { + /* Rollback REPLACE check constraint. */ + assert(space != NULL); + const char *name = ck->def->name; + struct ck_constraint *new_ck = + space_ck_constraint_by_name(space, name, strlen(name)); + assert(new_ck != NULL); + rlist_del_entry(new_ck, link); + rlist_add_entry(&space->ck_constraint, ck, link); + ck_constraint_delete(new_ck); + } +} + +/** + * Trigger invoked on commit in the _ck_constraint space. + * Drop useless old check constraint object if exists. + */ +static void +on_replace_ck_constraint_commit(struct trigger *trigger, void *event) +{ + struct txn_stmt *stmt = txn_last_stmt((struct txn *) event); + struct ck_constraint *ck = (struct ck_constraint *)trigger->data; + if (stmt->old_tuple != NULL) + ck_constraint_delete(ck); +} + +/** A trigger invoked on replace in the _ck_constraint space. */ +static void +on_replace_dd_ck_constraint(struct trigger * /* trigger*/, void *event) +{ + struct txn *txn = (struct txn *) event; + txn_check_singlestatement_xc(txn, "Space _ck_constraint"); + struct txn_stmt *stmt = txn_current_stmt(txn); + struct tuple *old_tuple = stmt->old_tuple; + struct tuple *new_tuple = stmt->new_tuple; + uint32_t space_id = + tuple_field_u32_xc(old_tuple != NULL ? old_tuple : new_tuple, + BOX_CK_CONSTRAINT_FIELD_SPACE_ID); + struct space *space = space_cache_find_xc(space_id); + struct trigger *on_rollback = + txn_alter_trigger_new(on_replace_ck_constraint_rollback, NULL); + struct trigger *on_commit = + txn_alter_trigger_new(on_replace_ck_constraint_commit, NULL); + + if (new_tuple != NULL) { + bool is_deferred = + tuple_field_bool_xc(new_tuple, + BOX_CK_CONSTRAINT_FIELD_DEFERRED); + if (is_deferred) { + tnt_raise(ClientError, ER_UNSUPPORTED, "Tarantool", + "deferred ck constraints"); + } + /* Create or replace check constraint. */ + struct ck_constraint_def *ck_def = + ck_constraint_def_new_from_tuple(new_tuple); + auto ck_guard = make_scoped_guard([=] { free(ck_def); }); + /* + * FIXME: Ck constraint creation on non-empty + * space is not implemented yet. + */ + struct index *pk = space_index(space, 0); + if (pk != NULL && index_size(pk) > 0) { + tnt_raise(ClientError, ER_CREATE_CK_CONSTRAINT, + ck_def->name, + "referencing space must be empty"); + } + struct ck_constraint *new_ck_constraint = + ck_constraint_new(ck_def, space->def); + if (new_ck_constraint == NULL) + diag_raise(); + ck_guard.is_active = false; + const char *name = new_ck_constraint->def->name; + struct ck_constraint *old_ck_constraint = + space_ck_constraint_by_name(space, name, strlen(name)); + if (old_ck_constraint != NULL) + rlist_del_entry(old_ck_constraint, link); + rlist_add_entry(&space->ck_constraint, new_ck_constraint, link); + on_commit->data = old_tuple == NULL ? new_ck_constraint : + old_ck_constraint; + on_rollback->data = on_commit->data; + } else { + assert(new_tuple == NULL && old_tuple != NULL); + /* Drop check constraint. */ + uint32_t name_len; + const char *name = + tuple_field_str_xc(old_tuple, + BOX_CK_CONSTRAINT_FIELD_NAME, + &name_len); + struct ck_constraint *old_ck_constraint = + space_ck_constraint_by_name(space, name, name_len); + assert(old_ck_constraint != NULL); + rlist_del_entry(old_ck_constraint, link); + on_commit->data = old_ck_constraint; + on_rollback->data = old_ck_constraint; + } + + txn_on_rollback(txn, on_rollback); + txn_on_commit(txn, on_commit); +} + struct trigger alter_space_on_replace_space = { RLIST_LINK_INITIALIZER, on_replace_dd_space, NULL, NULL }; @@ -4156,4 +4386,8 @@ struct trigger on_replace_fk_constraint = { RLIST_LINK_INITIALIZER, on_replace_dd_fk_constraint, NULL, NULL }; +struct trigger on_replace_ck_constraint = { + RLIST_LINK_INITIALIZER, on_replace_dd_ck_constraint, NULL, NULL +}; + /* vim: set foldmethod=marker */ diff --git a/src/box/alter.h b/src/box/alter.h index 4108fa47c..b9ba7b846 100644 --- a/src/box/alter.h +++ b/src/box/alter.h @@ -46,6 +46,7 @@ extern struct trigger on_replace_sequence_data; extern struct trigger on_replace_space_sequence; extern struct trigger on_replace_trigger; extern struct trigger on_replace_fk_constraint; +extern struct trigger on_replace_ck_constraint; extern struct trigger on_stmt_begin_space; extern struct trigger on_stmt_begin_index; extern struct trigger on_stmt_begin_truncate; diff --git a/src/box/bootstrap.snap b/src/box/bootstrap.snap index bb8fbeba114b1e72a5585548fb7f22796931d90f..fd7c7e16920d0de05799c9265d3cc461462aca2a 100644 GIT binary patch literal 4444 diff --git a/src/box/ck_constraint.c b/src/box/ck_constraint.c new file mode 100644 index 000000000..69b9793ea --- /dev/null +++ b/src/box/ck_constraint.c @@ -0,0 +1,148 @@ +/* + * Copyright 2010-2019, Tarantool AUTHORS, please see AUTHORS file. + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * 1. Redistributions of source code must retain the above + * copyright notice, this list of conditions and the + * following disclaimer. + * + * 2. Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL + * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF + * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ +#include "box/session.h" +#include "ck_constraint.h" +#include "errcode.h" +#include "sql.h" +#include "sql/sqlInt.h" + +const char *ck_constraint_language_strs[] = {"SQL"}; + +struct ck_constraint_def * +ck_constraint_def_new(const char *name, uint32_t name_len, const char *expr_str, + uint32_t expr_str_len, uint32_t space_id, + enum ck_constraint_language language) +{ + uint32_t name_offset, expr_str_offset; + uint32_t ck_def_sz = + ck_constraint_def_sizeof(name_len, expr_str_len, &name_offset, + &expr_str_offset); + struct ck_constraint_def *ck_def = + (struct ck_constraint_def *) malloc(ck_def_sz); + if (ck_def == NULL) { + diag_set(OutOfMemory, ck_def_sz, "malloc", "ck_def"); + return NULL; + } + ck_def->name = (char *)ck_def + name_offset; + ck_def->expr_str = (char *)ck_def + expr_str_offset; + ck_def->language = language; + ck_def->space_id = space_id; + memcpy(ck_def->expr_str, expr_str, expr_str_len); + ck_def->expr_str[expr_str_len] = '\0'; + memcpy(ck_def->name, name, name_len); + ck_def->name[name_len] = '\0'; + return ck_def; +} + +void +ck_constraint_def_delete(struct ck_constraint_def *ck_def) +{ + free(ck_def); +} + +/** + * Resolve space_def references for check constraint via AST + * tree traversal. + * @param expr Check constraint AST object to update. + * @param space_def Space definition to use. + * @retval 0 On success. + * @retval -1 On error. + */ +static int +ck_constraint_resolve_field_names(struct Expr *expr, + struct space_def *space_def) +{ + struct Parse parser; + sql_parser_create(&parser, sql_get(), default_flags); + parser.parse_only = true; + sql_resolve_self_reference(&parser, space_def, NC_IsCheck, expr); + int rc = parser.is_aborted ? -1 : 0; + sql_parser_destroy(&parser); + return rc; +} + +struct ck_constraint * +ck_constraint_new(struct ck_constraint_def *ck_constraint_def, + struct space_def *space_def) +{ + if (space_def->field_count == 0) { + diag_set(ClientError, ER_UNSUPPORTED, "Tarantool", + "CK constraint for space without format"); + return NULL; + } + struct ck_constraint *ck_constraint = malloc(sizeof(*ck_constraint)); + if (ck_constraint == NULL) { + diag_set(OutOfMemory, sizeof(*ck_constraint), "malloc", + "ck_constraint"); + return NULL; + } + ck_constraint->def = NULL; + rlist_create(&ck_constraint->link); + ck_constraint->expr = + sql_expr_compile(sql_get(), ck_constraint_def->expr_str, + strlen(ck_constraint_def->expr_str)); + if (ck_constraint->expr == NULL || + ck_constraint_resolve_field_names(ck_constraint->expr, + space_def) != 0) { + diag_set(ClientError, ER_CREATE_CK_CONSTRAINT, + ck_constraint_def->name, + box_error_message(box_error_last())); + goto error; + } + + ck_constraint->def = ck_constraint_def; + return ck_constraint; +error: + ck_constraint_delete(ck_constraint); + return NULL; +} + +void +ck_constraint_delete(struct ck_constraint *ck_constraint) +{ + sql_expr_delete(sql_get(), ck_constraint->expr, false); + ck_constraint_def_delete(ck_constraint->def); + TRASH(ck_constraint); + free(ck_constraint); +} + +struct ck_constraint * +space_ck_constraint_by_name(struct space *space, const char *name, + uint32_t name_len) +{ + struct ck_constraint *ck_constraint = NULL; + rlist_foreach_entry(ck_constraint, &space->ck_constraint, link) { + if (strlen(ck_constraint->def->name) == name_len && + memcmp(ck_constraint->def->name, name, name_len) == 0) + return ck_constraint; + } + return NULL; +} diff --git a/src/box/ck_constraint.h b/src/box/ck_constraint.h new file mode 100644 index 000000000..3ae3d5c91 --- /dev/null +++ b/src/box/ck_constraint.h @@ -0,0 +1,205 @@ +#ifndef INCLUDES_BOX_CK_CONSTRAINT_H +#define INCLUDES_BOX_CK_CONSTRAINT_H +/* + * Copyright 2010-2019, Tarantool AUTHORS, please see AUTHORS file. + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * 1. Redistributions of source code must retain the above + * copyright notice, this list of conditions and the + * following disclaimer. + * + * 2. Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL + * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF + * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <stdint.h> +#include "small/rlist.h" + +#if defined(__cplusplus) +extern "C" { +#endif + +struct space; +struct space_def; +struct Expr; + +/** Supported languages of ck constraint. */ +enum ck_constraint_language { + CK_CONSTRAINT_LANGUAGE_SQL, + ck_constraint_language_MAX, +}; + +/** The supported languages strings. */ +extern const char *ck_constraint_language_strs[]; + +/** + * Check constraint definition. + * See ck_constraint_def_sizeof() definition for implementation + * details and memory layout. + */ +struct ck_constraint_def { + /** + * The 0-terminated string, a name of the check + * constraint. Must be unique for a given space. + */ + char *name; + /** + * The 0-terminated string that defines check constraint + * expression. + * + * For instance: "field1 + field2 > 2 * 3". + */ + char *expr_str; + /** + * The id of the space this check constraint is + * defined for. + */ + uint32_t space_id; + /** The language of ck constraint. */ + enum ck_constraint_language language; +}; + +/** + * Structure representing ck constraint. + * See ck_constraint_new() definition. + */ +struct ck_constraint { + /** The check constraint definition. */ + struct ck_constraint_def *def; + /** + * The check constraint expression AST is built for + * ck_constraint::def::expr_str with sql_expr_compile + * and resolved with sql_resolve_self_reference for + * space with space[ck_constraint::space_id] definition. + */ + struct Expr *expr; + /** + * Organize check constraint structs into linked list + * with space::ck_constraint. + */ + struct rlist link; +}; + +/** + * Calculate check constraint definition memory size and fields + * offsets for given arguments. + * + * Alongside with struct ck_constraint_def itself, we reserve + * memory for string containing its name and expression string. + * + * Memory layout: + * +-----------------------------+ <- Allocated memory starts here + * | struct ck_constraint_def | + * |-----------------------------| + * | name + \0 | + * |-----------------------------| + * | expr_str + \0 | + * +-----------------------------+ + * + * @param name_len The length of the name. + * @param expr_str_len The length of the expr_str. + * @param[out] name_offset The offset of the name string. + * @param[out] expr_str_offset The offset of the expr_str string. + * @return The size of the ck constraint definition object for + * given parameters. + */ +static inline uint32_t +ck_constraint_def_sizeof(uint32_t name_len, uint32_t expr_str_len, + uint32_t *name_offset, uint32_t *expr_str_offset) +{ + *name_offset = sizeof(struct ck_constraint_def); + *expr_str_offset = *name_offset + name_len + 1; + return *expr_str_offset + expr_str_len + 1; +} + +/** + * Create a new check constraint definition object with given + * fields. + * + * @param name The name string of a new ck constraint definition. + * @param name_len The length of @a name string. + * @param expr The check expression string. + * @param expr_str_len The length of the @a expr string. + * @param space_id The identifier of the target space. + * @param language The language of the @a expr string. + * @retval not NULL Check constraint definition object pointer + * on success. + * @retval NULL Otherwise. The diag message is set. +*/ +struct ck_constraint_def * +ck_constraint_def_new(const char *name, uint32_t name_len, const char *expr, + uint32_t expr_str_len, uint32_t space_id, + enum ck_constraint_language language); + +/** + * Destroy check constraint definition memory, release acquired + * resources. + * @param ck_def The check constraint definition object to + * destroy. + */ +void +ck_constraint_def_delete(struct ck_constraint_def *ck_def); + +/** + * Create a new check constraint object by given check constraint + * definition and definition of the space this constraint is + * related to. + * + * @param ck_constraint_def The check constraint definition object + * to use. Expected to be allocated with + * malloc. Ck constraint object manages + * this allocation in case of successful + * creation. + * @param space_def The space definition of the space this check + * constraint must be constructed for. + * @retval not NULL Check constraint object pointer on success. + * @retval NULL Otherwise. The diag message is set. +*/ +struct ck_constraint * +ck_constraint_new(struct ck_constraint_def *ck_constraint_def, + struct space_def *space_def); + +/** + * Destroy check constraint memory, release acquired resources. + * @param ck_constraint The check constraint object to destroy. + */ +void +ck_constraint_delete(struct ck_constraint *ck_constraint); + +/** + * Find check constraint object in space by given name and + * name_len. + * @param space The space to lookup check constraint. + * @param name The check constraint name. + * @param name_len The length of the name. + * @retval Not NULL Ck constraint pointer if exists. + * @retval NULL Otherwise. + */ +struct ck_constraint * +space_ck_constraint_by_name(struct space *space, const char *name, + uint32_t name_len); + +#if defined(__cplusplus) +} /* extern "C" { */ +#endif + +#endif /* INCLUDES_BOX_CK_CONSTRAINT_H */ diff --git a/src/box/errcode.h b/src/box/errcode.h index 9c15f3322..1f7c81693 100644 --- a/src/box/errcode.h +++ b/src/box/errcode.h @@ -245,8 +245,9 @@ struct errcode_record { /*190 */_(ER_INT_LITERAL_MAX, "Integer literal %s%s exceeds the supported range %lld - %lld") \ /*191 */_(ER_SQL_PARSER_LIMIT, "%s %d exceeds the limit (%d)") \ /*192 */_(ER_INDEX_DEF_UNSUPPORTED, "%s are prohibited in an index definition") \ - /*193 */_(ER_CK_DEF_UNSUPPORTED, "%s are prohibited in a CHECK constraint definition") \ + /*193 */_(ER_CK_DEF_UNSUPPORTED, "%s are prohibited in a ck constraint definition") \ /*194 */_(ER_MULTIKEY_INDEX_MISMATCH, "Field %s is used as multikey in one index and as single key in another") \ + /*195 */_(ER_CREATE_CK_CONSTRAINT, "Failed to create check constraint '%s': %s") \ /* * !IMPORTANT! Please follow instructions at start of the file diff --git a/src/box/lua/schema.lua b/src/box/lua/schema.lua index 39bd8da6d..91fae8378 100644 --- a/src/box/lua/schema.lua +++ b/src/box/lua/schema.lua @@ -514,6 +514,7 @@ box.schema.space.drop = function(space_id, space_name, opts) local _truncate = box.space[box.schema.TRUNCATE_ID] local _space_sequence = box.space[box.schema.SPACE_SEQUENCE_ID] local _fk_constraint = box.space[box.schema.FK_CONSTRAINT_ID] + local _ck_constraint = box.space[box.schema.CK_CONSTRAINT_ID] local sequence_tuple = _space_sequence:delete{space_id} if sequence_tuple ~= nil and sequence_tuple.is_generated == true then -- Delete automatically generated sequence. @@ -525,6 +526,9 @@ box.schema.space.drop = function(space_id, space_name, opts) for _, t in _fk_constraint.index.child_id:pairs({space_id}) do _fk_constraint:delete({t.name, space_id}) end + for _, t in _ck_constraint.index.primary:pairs({space_id}) do + _ck_constraint:delete({space_id, t.name}) + end local keys = _vindex:select(space_id) for i = #keys, 1, -1 do local v = keys[i] diff --git a/src/box/lua/space.cc b/src/box/lua/space.cc index 87adaeb16..509306eae 100644 --- a/src/box/lua/space.cc +++ b/src/box/lua/space.cc @@ -552,6 +552,8 @@ box_lua_space_init(struct lua_State *L) lua_setfield(L, -2, "TRIGGER_ID"); lua_pushnumber(L, BOX_FK_CONSTRAINT_ID); lua_setfield(L, -2, "FK_CONSTRAINT_ID"); + lua_pushnumber(L, BOX_CK_CONSTRAINT_ID); + lua_setfield(L, -2, "CK_CONSTRAINT_ID"); lua_pushnumber(L, BOX_TRUNCATE_ID); lua_setfield(L, -2, "TRUNCATE_ID"); lua_pushnumber(L, BOX_SEQUENCE_ID); diff --git a/src/box/lua/upgrade.lua b/src/box/lua/upgrade.lua index 070662698..4ff2efd3a 100644 --- a/src/box/lua/upgrade.lua +++ b/src/box/lua/upgrade.lua @@ -74,6 +74,7 @@ local function set_system_triggers(val) box.space._schema:run_triggers(val) box.space._cluster:run_triggers(val) box.space._fk_constraint:run_triggers(val) + box.space._ck_constraint:run_triggers(val) end -------------------------------------------------------------------------------- @@ -94,6 +95,7 @@ local function erase() truncate(box.space._schema) truncate(box.space._cluster) truncate(box.space._fk_constraint) + truncate(box.space._ck_constraint) end local function create_sysview(source_id, target_id) @@ -774,8 +776,46 @@ local function upgrade_sequence_to_2_2_1() _space_sequence:format(format) end +local function upgrade_ck_constraint_to_2_2_1() + -- In previous Tarantool releases check constraints were + -- stored in space opts. Now we use separate space + -- _ck_constraint for this purpose. Perform legacy data + -- migration. + local MAP = setmap({}) + local _space = box.space._space + local _index = box.space._index + local _ck_constraint = box.space._ck_constraint + log.info("create space _ck_constraint") + local format = {{name='space_id', type='unsigned'}, + {name='name', type='string'}, + {name='is_deferred', type='boolean'}, + {name='language', type='str'}, {name='code', type='str'}} + _space:insert{_ck_constraint.id, ADMIN, '_ck_constraint', 'memtx', 0, MAP, format} + + log.info("create index primary on _ck_constraint") + _index:insert{_ck_constraint.id, 0, 'primary', 'tree', + {unique = true}, {{0, 'unsigned'}, {1, 'string'}}} + + for _, space in _space:pairs() do + local flags = space.flags + if flags.checks then + for i, check in pairs(flags.checks) do + local expr_str = check.expr + local check_name = check.name or + "CK_CONSTRAINT_"..i.."_"..space.name + _ck_constraint:insert({space.id, check_name, false, + 'SQL', expr_str}) + end + flags.checks = nil + _space:replace({space.id, space.owner, space.name, space.engine, + space.field_count, flags, space.format}) + end + end +end + local function upgrade_to_2_2_1() upgrade_sequence_to_2_2_1() + upgrade_ck_constraint_to_2_2_1() end -------------------------------------------------------------------------------- diff --git a/src/box/schema.cc b/src/box/schema.cc index 9a55c2f14..6d3153815 100644 --- a/src/box/schema.cc +++ b/src/box/schema.cc @@ -460,6 +460,14 @@ schema_init() sc_space_new(BOX_FK_CONSTRAINT_ID, "_fk_constraint", key_parts, 2, &on_replace_fk_constraint, NULL); + /* _ck_сonstraint - check constraints. */ + key_parts[0].fieldno = 0; /* space id */ + key_parts[0].type = FIELD_TYPE_UNSIGNED; + key_parts[1].fieldno = 1; /* constraint name */ + key_parts[1].type = FIELD_TYPE_STRING; + sc_space_new(BOX_CK_CONSTRAINT_ID, "_ck_constraint", key_parts, 2, + &on_replace_ck_constraint, NULL); + /* * _vinyl_deferred_delete - blackhole that is needed * for writing deferred DELETE statements generated by diff --git a/src/box/schema_def.h b/src/box/schema_def.h index b817b49f6..3648639bc 100644 --- a/src/box/schema_def.h +++ b/src/box/schema_def.h @@ -108,6 +108,8 @@ enum { BOX_SPACE_SEQUENCE_ID = 340, /** Space id of _fk_constraint. */ BOX_FK_CONSTRAINT_ID = 356, + /** Space id of _ck_contraint. */ + BOX_CK_CONSTRAINT_ID = 364, /** End of the reserved range of system spaces. */ BOX_SYSTEM_ID_MAX = 511, BOX_ID_NIL = 2147483647 @@ -240,6 +242,15 @@ enum { BOX_FK_CONSTRAINT_FIELD_PARENT_COLS = 8, }; +/** _ck_constraint fields. */ +enum { + BOX_CK_CONSTRAINT_FIELD_SPACE_ID = 0, + BOX_CK_CONSTRAINT_FIELD_NAME = 1, + BOX_CK_CONSTRAINT_FIELD_DEFERRED = 2, + BOX_CK_CONSTRAINT_FIELD_LANGUAGE = 3, + BOX_CK_CONSTRAINT_FIELD_CODE = 4, +}; + /* * Different objects which can be subject to access * control. diff --git a/src/box/space.c b/src/box/space.c index 243e7da2f..a42b3a64b 100644 --- a/src/box/space.c +++ b/src/box/space.c @@ -165,6 +165,7 @@ space_create(struct space *space, struct engine *engine, space_fill_index_map(space); rlist_create(&space->parent_fk_constraint); rlist_create(&space->child_fk_constraint); + rlist_create(&space->ck_constraint); return 0; fail_free_indexes: @@ -225,6 +226,7 @@ space_delete(struct space *space) assert(space->sql_triggers == NULL); assert(rlist_empty(&space->parent_fk_constraint)); assert(rlist_empty(&space->child_fk_constraint)); + assert(rlist_empty(&space->ck_constraint)); space->vtab->destroy(space); } diff --git a/src/box/space.h b/src/box/space.h index be1178b99..7ffe884b3 100644 --- a/src/box/space.h +++ b/src/box/space.h @@ -197,6 +197,11 @@ struct space { * of index id. */ struct index **index; + /** + * List of check constraints linked with + * ck_constraint::link. + */ + struct rlist ck_constraint; /** * Lists of foreign key constraints. In SQL terms child * space is the "from" table i.e. the table that contains diff --git a/src/box/space_def.c b/src/box/space_def.c index d825def75..661131295 100644 --- a/src/box/space_def.c +++ b/src/box/space_def.c @@ -36,28 +36,12 @@ #include "msgpuck.h" #include "tt_static.h" -/** - * Make checks from msgpack. - * @param str pointer to array of maps - * e.g. [{"expr": "x < y", "name": "ONE"}, ..]. - * @param len array items count. - * @param[out] opt pointer to store parsing result. - * @param errcode Code of error to set if something is wrong. - * @param field_no Field number of an option in a parent element. - * @retval 0 on success. - * @retval not 0 on error. Also set diag message. - */ -static int -checks_array_decode(const char **str, uint32_t len, char *opt, uint32_t errcode, - uint32_t field_no); - const struct space_opts space_opts_default = { /* .group_id = */ 0, /* .is_temporary = */ false, /* .is_ephemeral = */ false, /* .view = */ false, /* .sql = */ NULL, - /* .checks = */ NULL, }; const struct opt_def space_opts_reg[] = { @@ -65,8 +49,7 @@ const struct opt_def space_opts_reg[] = { OPT_DEF("temporary", OPT_BOOL, struct space_opts, is_temporary), OPT_DEF("view", OPT_BOOL, struct space_opts, is_view), OPT_DEF("sql", OPT_STRPTR, struct space_opts, sql), - OPT_DEF_ARRAY("checks", struct space_opts, checks, - checks_array_decode), + OPT_DEF_LEGACY("checks"), OPT_END, }; @@ -114,16 +97,6 @@ space_def_dup_opts(struct space_def *def, const struct space_opts *opts) return -1; } } - if (opts->checks != NULL) { - def->opts.checks = sql_expr_list_dup(sql_get(), opts->checks, 0); - if (def->opts.checks == NULL) { - free(def->opts.sql); - diag_set(OutOfMemory, 0, "sql_expr_list_dup", - "def->opts.checks"); - return -1; - } - sql_checks_update_space_def_reference(def->opts.checks, def); - } return 0; } @@ -301,74 +274,5 @@ void space_opts_destroy(struct space_opts *opts) { free(opts->sql); - sql_expr_list_delete(sql_get(), opts->checks); TRASH(opts); } - -static int -checks_array_decode(const char **str, uint32_t len, char *opt, uint32_t errcode, - uint32_t field_no) -{ - char *errmsg = tt_static_buf(); - struct ExprList *checks = NULL; - const char **map = str; - struct sql *db = sql_get(); - for (uint32_t i = 0; i < len; i++) { - checks = sql_expr_list_append(db, checks, NULL); - if (checks == NULL) { - diag_set(OutOfMemory, 0, "sql_expr_list_append", - "checks"); - goto error; - } - const char *expr_name = NULL; - const char *expr_str = NULL; - uint32_t expr_name_len = 0; - uint32_t expr_str_len = 0; - uint32_t map_size = mp_decode_map(map); - for (uint32_t j = 0; j < map_size; j++) { - if (mp_typeof(**map) != MP_STR) { - diag_set(ClientError, errcode, field_no, - "key must be a string"); - goto error; - } - uint32_t key_len; - const char *key = mp_decode_str(map, &key_len); - if (mp_typeof(**map) != MP_STR) { - snprintf(errmsg, TT_STATIC_BUF_LEN, - "invalid MsgPack map field '%.*s' type", - key_len, key); - diag_set(ClientError, errcode, field_no, errmsg); - goto error; - } - if (key_len == 4 && memcmp(key, "expr", key_len) == 0) { - expr_str = mp_decode_str(map, &expr_str_len); - } else if (key_len == 4 && - memcmp(key, "name", key_len) == 0) { - expr_name = mp_decode_str(map, &expr_name_len); - } else { - snprintf(errmsg, TT_STATIC_BUF_LEN, - "invalid MsgPack map field '%.*s'", - key_len, key); - diag_set(ClientError, errcode, field_no, errmsg); - goto error; - } - } - if (sql_check_list_item_init(checks, i, expr_name, expr_name_len, - expr_str, expr_str_len) != 0) { - box_error_t *err = box_error_last(); - if (box_error_code(err) != ENOMEM) { - snprintf(errmsg, TT_STATIC_BUF_LEN, - "invalid expression specified (%s)", - box_error_message(err)); - diag_set(ClientError, errcode, field_no, - errmsg); - } - goto error; - } - } - *(struct ExprList **)opt = checks; - return 0; -error: - sql_expr_list_delete(db, checks); - return -1; -} diff --git a/src/box/space_def.h b/src/box/space_def.h index c6639a8d8..ac6d226c5 100644 --- a/src/box/space_def.h +++ b/src/box/space_def.h @@ -40,8 +40,6 @@ extern "C" { #endif /* defined(__cplusplus) */ -struct ExprList; - /** Space options */ struct space_opts { /** @@ -71,8 +69,6 @@ struct space_opts { bool is_view; /** SQL statement that produced this space. */ char *sql; - /** SQL Checks expressions list. */ - struct ExprList *checks; }; extern const struct space_opts space_opts_default; diff --git a/src/box/sql.c b/src/box/sql.c index 07ceec2d2..b47f7498c 100644 --- a/src/box/sql.c +++ b/src/box/sql.c @@ -1016,15 +1016,8 @@ sql_encode_table_opts(struct region *region, struct space_def *def, bool is_error = false; mpstream_init(&stream, region, region_reserve_cb, region_alloc_cb, set_encode_error, &is_error); - int checks_cnt = 0; - struct ExprList_item *a; bool is_view = def->opts.is_view; - struct ExprList *checks = def->opts.checks; - if (checks != NULL) { - checks_cnt = checks->nExpr; - a = checks->a; - } - mpstream_encode_map(&stream, 2 * is_view + (checks_cnt > 0)); + mpstream_encode_map(&stream, 2 * is_view); if (is_view) { assert(def->opts.sql != NULL); @@ -1033,23 +1026,6 @@ sql_encode_table_opts(struct region *region, struct space_def *def, mpstream_encode_str(&stream, "view"); mpstream_encode_bool(&stream, true); } - if (checks_cnt > 0) { - mpstream_encode_str(&stream, "checks"); - mpstream_encode_array(&stream, checks_cnt); - } - for (int i = 0; i < checks_cnt && !is_error; ++i, ++a) { - int items = (a->pExpr != NULL) + (a->zName != NULL); - mpstream_encode_map(&stream, items); - assert(a->pExpr != NULL); - struct Expr *pExpr = a->pExpr; - assert(pExpr->u.zToken != NULL); - mpstream_encode_str(&stream, "expr"); - mpstream_encode_str(&stream, pExpr->u.zToken); - if (a->zName != NULL) { - mpstream_encode_str(&stream, "name"); - mpstream_encode_str(&stream, a->zName); - } - } mpstream_flush(&stream); if (is_error) { diag_set(OutOfMemory, stream.pos - stream.buf, @@ -1284,66 +1260,6 @@ sql_ephemeral_space_new(Parse *parser, const char *name) return space; } -int -sql_check_list_item_init(struct ExprList *expr_list, int column, - const char *expr_name, uint32_t expr_name_len, - const char *expr_str, uint32_t expr_str_len) -{ - assert(column < expr_list->nExpr); - struct ExprList_item *item = &expr_list->a[column]; - memset(item, 0, sizeof(*item)); - if (expr_name != NULL) { - item->zName = sqlDbStrNDup(db, expr_name, expr_name_len); - if (item->zName == NULL) { - diag_set(OutOfMemory, expr_name_len, "sqlDbStrNDup", - "item->zName"); - return -1; - } - } - if (expr_str != NULL) { - item->pExpr = sql_expr_compile(db, expr_str, expr_str_len); - /* The item->zName would be released later. */ - if (item->pExpr == NULL) - return -1; - } - return 0; -} - -static int -update_space_def_callback(Walker *walker, Expr *expr) -{ - if (expr->op == TK_COLUMN && ExprHasProperty(expr, EP_Resolved)) - expr->space_def = walker->u.space_def; - return WRC_Continue; -} - -void -sql_checks_update_space_def_reference(ExprList *expr_list, - struct space_def *def) -{ - assert(expr_list != NULL); - Walker w; - memset(&w, 0, sizeof(w)); - w.xExprCallback = update_space_def_callback; - w.u.space_def = def; - for (int i = 0; i < expr_list->nExpr; i++) - sqlWalkExpr(&w, expr_list->a[i].pExpr); -} - -int -sql_checks_resolve_space_def_reference(ExprList *expr_list, - struct space_def *def) -{ - Parse parser; - sql_parser_create(&parser, sql_get(), default_flags); - parser.parse_only = true; - - sql_resolve_self_reference(&parser, def, NC_IsCheck, NULL, expr_list); - int rc = parser.is_aborted ? -1 : 0; - sql_parser_destroy(&parser); - return rc; -} - /** * Initialize a new tuple_fetcher instance with given tuple * data. diff --git a/src/box/sql.h b/src/box/sql.h index a13f78ae8..bb90bf838 100644 --- a/src/box/sql.h +++ b/src/box/sql.h @@ -278,48 +278,10 @@ sql_expr_list_append(struct sql *db, struct ExprList *expr_list, * @param def The definition of space being referenced. * @param type NC_IsCheck or NC_IdxExpr. * @param expr Expression to resolve. May be NULL. - * @param expr_list Expression list to resolve. May be NUL. */ void sql_resolve_self_reference(struct Parse *parser, struct space_def *def, - int type, struct Expr *expr, - struct ExprList *expr_list); - -/** - * Initialize check_list_item. - * @param expr_list ExprList with item. - * @param column index. - * @param expr_name expression name (optional). - * @param expr_name_len expresson name length (optional). - * @param expr_str expression to build string. - * @param expr_str_len expression to build string length. - * @retval 0 on success. - * @retval -1 on error. - */ -int -sql_check_list_item_init(struct ExprList *expr_list, int column, - const char *expr_name, uint32_t expr_name_len, - const char *expr_str, uint32_t expr_str_len); - -/** - * Resolve space_def references checks for expr_list. - * @param expr_list to modify. - * @param def to refer to. - * @retval 0 on success. - * @retval -1 on error. - */ -int -sql_checks_resolve_space_def_reference(struct ExprList *expr_list, - struct space_def *def); - -/** - * Update space_def references for expr_list. - * @param expr_list to modify. - * @param def to refer to. - */ -void -sql_checks_update_space_def_reference(struct ExprList *expr_list, - struct space_def *def); + int type, struct Expr *expr); /** * Initialize a new parser object. diff --git a/src/box/sql/build.c b/src/box/sql/build.c index e2353d8cc..09a15cb18 100644 --- a/src/box/sql/build.c +++ b/src/box/sql/build.c @@ -43,10 +43,12 @@ * COMMIT * ROLLBACK */ +#include <ctype.h> #include "sqlInt.h" #include "vdbeInt.h" #include "tarantoolInt.h" #include "box/box.h" +#include "box/ck_constraint.h" #include "box/fk_constraint.h" #include "box/sequence.h" #include "box/session.h" @@ -638,40 +640,117 @@ primary_key_exit: return; } +/** + * Prepare a 0-terminated string in the wptr memory buffer that + * does not contain a sequence of more than one whatespace + * character. Routine enforces ' ' (space) as whitespace + * delimiter. When character ' or " was met, the sting is copied + * without any changes until the next ' or " sign. + * The wptr buffer is expected to have str_len + 1 bytes + * (this is the expected scenario where no extra whitespace + * characters in the source string). + * @param wptr The destination memory buffer of size + * @a str_len + 1. + * @param str The source string to be copied. + * @param str_len The source string @a str length. + */ +static void +trim_space_snprintf(char *wptr, const char *str, uint32_t str_len) +{ + const char *str_end = str + str_len; + char quote_type = '\0'; + bool is_prev_chr_space = false; + while (str < str_end) { + if (quote_type == '\0') { + if (*str == '\'' || *str == '\"') { + quote_type = *str; + } else if (isspace((unsigned char)*str)) { + if (!is_prev_chr_space) + *wptr++ = ' '; + is_prev_chr_space = true; + str++; + continue; + } + } else if (*str == quote_type) { + quote_type = '\0'; + } + is_prev_chr_space = false; + *wptr++ = *str++; + } + *wptr = '\0'; +} + void -sql_add_check_constraint(struct Parse *parser) +sql_create_check_contraint(struct Parse *parser) { - struct create_ck_def *ck_def = &parser->create_ck_def; + struct create_ck_def *create_ck_def = &parser->create_ck_def; + struct ExprSpan *expr_span = create_ck_def->expr; + sql_expr_delete(parser->db, expr_span->pExpr, false); + struct alter_entity_def *alter_def = - (struct alter_entity_def *) &parser->create_ck_def; + (struct alter_entity_def *) create_ck_def; assert(alter_def->entity_type == ENTITY_TYPE_CK); (void) alter_def; - struct Expr *expr = ck_def->expr->pExpr; struct space *space = parser->create_table_def.new_space; - if (space != NULL) { - expr->u.zToken = - sqlDbStrNDup(parser->db, - (char *) ck_def->expr->zStart, - (int) (ck_def->expr->zEnd - - ck_def->expr->zStart)); - if (expr->u.zToken == NULL) - goto release_expr; - space->def->opts.checks = - sql_expr_list_append(parser->db, - space->def->opts.checks, expr); - if (space->def->opts.checks == NULL) { - sqlDbFree(parser->db, expr->u.zToken); - goto release_expr; - } - struct create_entity_def *entity_def = &ck_def->base.base; - if (entity_def->name.n > 0) { - sqlExprListSetName(parser, space->def->opts.checks, - &entity_def->name, 1); + assert(space != NULL); + + /* Prepare payload for ck constraint definition. */ + struct region *region = &parser->region; + struct Token *name_token = &create_ck_def->base.base.name; + const char *name; + if (name_token->n > 0) { + name = sql_normalized_name_region_new(region, name_token->z, + name_token->n); + if (name == NULL) { + parser->is_aborted = true; + return; } } else { -release_expr: - sql_expr_delete(parser->db, expr, false); + uint32_t ck_idx = ++parser->create_table_def.check_count; + name = tt_sprintf("CK_CONSTRAINT_%d_%s", ck_idx, + space->def->name); } + size_t name_len = strlen(name); + + uint32_t expr_str_len = (uint32_t)(expr_span->zEnd - expr_span->zStart); + const char *expr_str = expr_span->zStart; + + /* + * Allocate memory for ck constraint parse structure and + * ck constraint definition as a single memory chunk on + * region: + * + * [ck_parse][ck_def[name][expr_str]] + * |_____^ |___^ ^ + * |_________| + */ + uint32_t name_offset, expr_str_offset; + uint32_t ck_def_sz = + ck_constraint_def_sizeof(name_len, expr_str_len, &name_offset, + &expr_str_offset); + struct ck_constraint_parse *ck_parse = + region_alloc(region, sizeof(*ck_parse) + ck_def_sz); + if (ck_parse == NULL) { + diag_set(OutOfMemory, sizeof(*ck_parse) + ck_def_sz, "region", + "ck_parse"); + parser->is_aborted = true; + return; + } + struct ck_constraint_def *ck_def = + (struct ck_constraint_def *)((char *)ck_parse + + sizeof(*ck_parse)); + ck_parse->ck_def = ck_def; + rlist_create(&ck_parse->link); + + ck_def->name = (char *)ck_def + name_offset; + ck_def->expr_str = (char *)ck_def + expr_str_offset; + ck_def->language = CK_CONSTRAINT_LANGUAGE_SQL; + ck_def->space_id = BOX_ID_NIL; + trim_space_snprintf(ck_def->expr_str, expr_str, expr_str_len); + memcpy(ck_def->name, name, name_len); + ck_def->name[name_len] = '\0'; + + rlist_add_entry(&parser->create_table_def.new_check, ck_parse, link); } /* @@ -978,6 +1057,48 @@ emitNewSysSpaceSequenceRecord(Parse *pParse, int reg_space_id, int reg_seq_id, return first_col; } +/** + * Generate opcodes to serialize check constraint definition into + * MsgPack and insert produced tuple into _ck_constraint space. + * @param parser Parsing context. + * @param ck_def Check constraint definition to be serialized. + * @param reg_space_id The VDBE register containing space id. +*/ +static void +vdbe_emit_ck_constraint_create(struct Parse *parser, + const struct ck_constraint_def *ck_def, + uint32_t reg_space_id) +{ + struct sql *db = parser->db; + struct Vdbe *v = sqlGetVdbe(parser); + assert(v != NULL); + int ck_constraint_reg = sqlGetTempRange(parser, 6); + sqlVdbeAddOp2(v, OP_SCopy, reg_space_id, ck_constraint_reg); + sqlVdbeAddOp4(v, OP_String8, 0, ck_constraint_reg + 1, 0, + sqlDbStrDup(db, ck_def->name), P4_DYNAMIC); + sqlVdbeAddOp2(v, OP_Bool, false, ck_constraint_reg + 2); + sqlVdbeAddOp4(v, OP_String8, 0, ck_constraint_reg + 3, 0, + ck_constraint_language_strs[ck_def->language], P4_STATIC); + sqlVdbeAddOp4(v, OP_String8, 0, ck_constraint_reg + 4, 0, + sqlDbStrDup(db, ck_def->expr_str), P4_DYNAMIC); + sqlVdbeAddOp3(v, OP_MakeRecord, ck_constraint_reg, 5, + ck_constraint_reg + 5); + const char *error_msg = + tt_sprintf(tnt_errcode_desc(ER_CONSTRAINT_EXISTS), + ck_def->name); + if (vdbe_emit_halt_with_presence_test(parser, BOX_CK_CONSTRAINT_ID, 0, + ck_constraint_reg, 2, + ER_CONSTRAINT_EXISTS, error_msg, + false, OP_NoConflict) != 0) + return; + sqlVdbeAddOp3(v, OP_SInsert, BOX_CK_CONSTRAINT_ID, 0, + ck_constraint_reg + 5); + save_record(parser, BOX_CK_CONSTRAINT_ID, ck_constraint_reg, 2, + v->nOp - 1); + VdbeComment((v, "Create CK constraint %s", ck_def->name)); + sqlReleaseTempRange(parser, ck_constraint_reg, 5); +} + /** * Generate opcodes to serialize foreign key into MsgPack and * insert produced tuple into _fk_constraint space. @@ -1129,20 +1250,18 @@ resolve_link(struct Parse *parse_context, const struct space_def *def, void sqlEndTable(struct Parse *pParse) { - sql *db = pParse->db; /* The database connection */ - - assert(!db->mallocFailed); + assert(!pParse->db->mallocFailed); struct space *new_space = pParse->create_table_def.new_space; if (new_space == NULL) return; - assert(!db->init.busy); + assert(!pParse->db->init.busy); assert(!new_space->def->opts.is_view); if (sql_space_primary_key(new_space) == NULL) { diag_set(ClientError, ER_CREATE_SPACE, new_space->def->name, "PRIMARY KEY missing"); pParse->is_aborted = true; - goto cleanup; + return; } /* @@ -1232,9 +1351,12 @@ sqlEndTable(struct Parse *pParse) fk_def->child_id = reg_space_id; vdbe_emit_fk_constraint_create(pParse, fk_def); } -cleanup: - sql_expr_list_delete(db, new_space->def->opts.checks); - new_space->def->opts.checks = NULL; + struct ck_constraint_parse *ck_parse; + rlist_foreach_entry(ck_parse, &pParse->create_table_def.new_check, + link) { + vdbe_emit_ck_constraint_create(pParse, ck_parse->ck_def, + reg_space_id); + } } void @@ -1432,6 +1554,37 @@ vdbe_emit_fk_constraint_drop(struct Parse *parse_context, char *constraint_name, sqlReleaseTempRange(parse_context, key_reg, 3); } +/** + * Generate VDBE program to remove entry from _ck_constraint space. + * + * @param parser Parsing context. + * @param ck_name Name of CK constraint to be dropped. + * @param space_id Id of table which constraint belongs to. + */ +static void +vdbe_emit_ck_constraint_drop(struct Parse *parser, const char *ck_name, + uint32_t space_id) +{ + struct Vdbe *v = sqlGetVdbe(parser); + struct sql *db = v->db; + assert(v != NULL); + int key_reg = sqlGetTempRange(parser, 3); + sqlVdbeAddOp2(v, OP_Integer, space_id, key_reg); + sqlVdbeAddOp4(v, OP_String8, 0, key_reg + 1, 0, + sqlDbStrDup(db, ck_name), P4_DYNAMIC); + const char *error_msg = + tt_sprintf(tnt_errcode_desc(ER_NO_SUCH_CONSTRAINT), ck_name); + if (vdbe_emit_halt_with_presence_test(parser, BOX_CK_CONSTRAINT_ID, 0, + key_reg, 2, ER_NO_SUCH_CONSTRAINT, + error_msg, false, + OP_Found) != 0) + return; + sqlVdbeAddOp3(v, OP_MakeRecord, key_reg, 2, key_reg + 2); + sqlVdbeAddOp2(v, OP_SDelete, BOX_CK_CONSTRAINT_ID, key_reg + 2); + VdbeComment((v, "Delete CK constraint %s", ck_name)); + sqlReleaseTempRange(parser, key_reg, 3); +} + /** * Generate code to drop a table. * This routine includes dropping triggers, sequences, @@ -1505,6 +1658,13 @@ sql_code_drop_table(struct Parse *parse_context, struct space *space, return; vdbe_emit_fk_constraint_drop(parse_context, fk_name_dup, space_id); } + /* Delete all CK constraints. */ + struct ck_constraint *ck_constraint; + rlist_foreach_entry(ck_constraint, &space->ck_constraint, link) { + vdbe_emit_ck_constraint_drop(parse_context, + ck_constraint->def->name, + space_id); + } /* * Drop all _space and _index entries that refer to the * table. @@ -2074,8 +2234,7 @@ index_fill_def(struct Parse *parse, struct index *index, } for (int i = 0; i < expr_list->nExpr; i++) { struct Expr *expr = expr_list->a[i].pExpr; - sql_resolve_self_reference(parse, space_def, NC_IdxExpr, - expr, 0); + sql_resolve_self_reference(parse, space_def, NC_IdxExpr, expr); if (parse->is_aborted) goto cleanup; @@ -2791,8 +2950,7 @@ sqlSrcListDelete(sql * db, SrcList * pList) */ assert(pItem->space == NULL || !pItem->space->def->opts.is_temporary || - (pItem->space->index == NULL && - pItem->space->def->opts.checks == NULL)); + pItem->space->index == NULL); sql_select_delete(db, pItem->pSelect); sql_expr_delete(db, pItem->pOn, false); sqlIdListDelete(db, pItem->pUsing); diff --git a/src/box/sql/insert.c b/src/box/sql/insert.c index 81feca7e0..c4b629c6c 100644 --- a/src/box/sql/insert.c +++ b/src/box/sql/insert.c @@ -36,9 +36,10 @@ #include "sqlInt.h" #include "tarantoolInt.h" #include "vdbeInt.h" -#include "box/schema.h" +#include "box/ck_constraint.h" #include "bit/bit.h" #include "box/box.h" +#include "box/schema.h" enum field_type * sql_index_type_str(struct sql *db, const struct index_def *idx_def) @@ -918,34 +919,27 @@ vdbe_emit_constraint_checks(struct Parse *parse_context, struct space *space, if (on_conflict == ON_CONFLICT_ACTION_DEFAULT) on_conflict = ON_CONFLICT_ACTION_ABORT; /* Test all CHECK constraints. */ - struct ExprList *checks = def->opts.checks; enum on_conflict_action on_conflict_check = on_conflict; if (on_conflict == ON_CONFLICT_ACTION_REPLACE) on_conflict_check = ON_CONFLICT_ACTION_ABORT; - if (checks != NULL) { + if (!rlist_empty(&space->ck_constraint)) parse_context->ckBase = new_tuple_reg; - for (int i = 0; i < checks->nExpr; i++) { - struct Expr *expr = checks->a[i].pExpr; - if (is_update && - checkConstraintUnchanged(expr, upd_cols)) - continue; - int all_ok = sqlVdbeMakeLabel(v); - sqlExprIfTrue(parse_context, expr, all_ok, - SQL_JUMPIFNULL); - if (on_conflict == ON_CONFLICT_ACTION_IGNORE) { - sqlVdbeGoto(v, ignore_label); - } else { - char *name = checks->a[i].zName; - if (name == NULL) - name = def->name; - sqlHaltConstraint(parse_context, - SQL_CONSTRAINT_CHECK, - on_conflict_check, name, - P4_TRANSIENT, - P5_ConstraintCheck); - } - sqlVdbeResolveLabel(v, all_ok); + struct ck_constraint *ck_constraint; + rlist_foreach_entry(ck_constraint, &space->ck_constraint, link) { + struct Expr *expr = ck_constraint->expr; + if (is_update && checkConstraintUnchanged(expr, upd_cols) != 0) + continue; + int all_ok = sqlVdbeMakeLabel(v); + sqlExprIfTrue(parse_context, expr, all_ok, SQL_JUMPIFNULL); + if (on_conflict == ON_CONFLICT_ACTION_IGNORE) { + sqlVdbeGoto(v, ignore_label); + } else { + char *name = ck_constraint->def->name; + sqlHaltConstraint(parse_context, SQL_CONSTRAINT_CHECK, + on_conflict_check, name, P4_TRANSIENT, + P5_ConstraintCheck); } + sqlVdbeResolveLabel(v, all_ok); } sql_emit_table_types(v, space->def, new_tuple_reg); /* @@ -1230,14 +1224,13 @@ xferOptimization(Parse * pParse, /* Parser context */ if (pSrcIdx == NULL) return 0; } - /* Get server checks. */ - ExprList *pCheck_src = src->def->opts.checks; - ExprList *pCheck_dest = dest->def->opts.checks; - if (pCheck_dest != NULL && - sqlExprListCompare(pCheck_src, pCheck_dest, -1) != 0) { - /* Tables have different CHECK constraints. Ticket #2252 */ + /* + * Dissallow the transfer optimization if the are check + * constraints. + */ + if (!rlist_empty(&dest->ck_constraint) || + !rlist_empty(&src->ck_constraint)) return 0; - } /* Disallow the transfer optimization if the destination table constains * any foreign key constraints. This is more restrictive than necessary. * So the extra complication to make this rule less restrictive is probably diff --git a/src/box/sql/parse.y b/src/box/sql/parse.y index f241b8d52..e93dfe751 100644 --- a/src/box/sql/parse.y +++ b/src/box/sql/parse.y @@ -297,7 +297,7 @@ ccons ::= check_constraint_def . check_constraint_def ::= cconsname(N) CHECK LP expr(X) RP. { create_ck_def_init(&pParse->create_ck_def, &N, &X); - sql_add_check_constraint(pParse); + sql_create_check_contraint(pParse); } ccons ::= cconsname(N) REFERENCES nm(T) eidlist_opt(TA) matcharg(M) refargs(R). { diff --git a/src/box/sql/parse_def.h b/src/box/sql/parse_def.h index 5899a7e4e..6c1b6fddd 100644 --- a/src/box/sql/parse_def.h +++ b/src/box/sql/parse_def.h @@ -123,6 +123,22 @@ struct fk_constraint_parse { struct rlist link; }; +/** + * Structure representing check constraint appeared within + * CREATE TABLE statement. Used only during parsing. + * All allocations are performed on region, so no cleanups are + * required. + */ +struct ck_constraint_parse { + /** + * Check constraint declared in <CREATE TABLE ...> + * statement. Must be coded after space creation. + */ + struct ck_constraint_def *ck_def; + /** Organize these structs into linked list. */ + struct rlist link; +}; + /** * Possible SQL index types. Note that PK and UNIQUE constraints * are implemented as indexes and have their own types: @@ -189,6 +205,13 @@ struct create_table_def { * Foreign key constraint appeared in CREATE TABLE stmt. */ struct rlist new_fkey; + /** + * Number of CK constraints declared within + * CREATE TABLE statement. + */ + uint32_t check_count; + /** Check constraint appeared in CREATE TABLE stmt. */ + struct rlist new_check; /** True, if table to be created has AUTOINCREMENT PK. */ bool has_autoinc; }; @@ -437,6 +460,7 @@ create_table_def_init(struct create_table_def *table_def, struct Token *name, create_entity_def_init(&table_def->base, ENTITY_TYPE_TABLE, NULL, name, if_not_exists); rlist_create(&table_def->new_fkey); + rlist_create(&table_def->new_check); } static inline void diff --git a/src/box/sql/resolve.c b/src/box/sql/resolve.c index 504096e6d..ac85b85a9 100644 --- a/src/box/sql/resolve.c +++ b/src/box/sql/resolve.c @@ -1610,8 +1610,7 @@ sqlResolveSelectNames(Parse * pParse, /* The parser context */ void sql_resolve_self_reference(struct Parse *parser, struct space_def *def, - int type, struct Expr *expr, - struct ExprList *expr_list) + int type, struct Expr *expr) { /* Fake SrcList for parser->create_table_def */ SrcList sSrc; @@ -1631,8 +1630,5 @@ sql_resolve_self_reference(struct Parse *parser, struct space_def *def, sNC.pParse = parser; sNC.pSrcList = &sSrc; sNC.ncFlags = type; - if (sqlResolveExprNames(&sNC, expr) != 0) - return; - if (expr_list != NULL) - sqlResolveExprListNames(&sNC, expr_list); + sqlResolveExprNames(&sNC, expr); } diff --git a/src/box/sql/select.c b/src/box/sql/select.c index d7376ae11..22aa62830 100644 --- a/src/box/sql/select.c +++ b/src/box/sql/select.c @@ -6440,7 +6440,12 @@ sql_expr_extract_select(struct Parse *parser, struct Select *select) struct ExprList *expr_list = select->pEList; assert(expr_list->nExpr == 1); parser->parsed_ast_type = AST_TYPE_EXPR; - parser->parsed_ast.expr = sqlExprDup(parser->db, - expr_list->a->pExpr, - EXPRDUP_REDUCE); + /* + * Extract a copy of parsed expression. + * We cannot use EXPRDUP_REDUCE flag in sqlExprDup call + * because some compiled Expr (like Checks expressions) + * may require further resolve with sqlResolveExprNames. + */ + parser->parsed_ast.expr = + sqlExprDup(parser->db, expr_list->a->pExpr, 0); } diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h index 7212553f6..abb7b9c2a 100644 --- a/src/box/sql/sqlInt.h +++ b/src/box/sql/sqlInt.h @@ -3359,7 +3359,7 @@ sqlAddPrimaryKey(struct Parse *parse); * @param parser Parsing context. */ void -sql_add_check_constraint(Parse *parser); +sql_create_check_contraint(Parse *parser); void sqlAddDefaultValue(Parse *, ExprSpan *); void sqlAddCollateType(Parse *, Token *); diff --git a/src/box/sql/tokenize.c b/src/box/sql/tokenize.c index 11ef7b97e..dbaebfefc 100644 --- a/src/box/sql/tokenize.c +++ b/src/box/sql/tokenize.c @@ -435,7 +435,6 @@ parser_space_delete(struct sql *db, struct space *space) assert(space->def->opts.is_temporary); for (uint32_t i = 0; i < space->index_count; ++i) index_def_delete(space->index[i]->def); - sql_expr_list_delete(db, space->def->opts.checks); } /** diff --git a/test/app-tap/tarantoolctl.test.lua b/test/app-tap/tarantoolctl.test.lua index c1e1490ca..4777d8a6f 100755 --- a/test/app-tap/tarantoolctl.test.lua +++ b/test/app-tap/tarantoolctl.test.lua @@ -405,8 +405,8 @@ do check_ctlcat_xlog(test_i, dir, "--from=3 --to=6 --format=json --show-system --replica 1", "\n", 3) check_ctlcat_xlog(test_i, dir, "--from=3 --to=6 --format=json --show-system --replica 1 --replica 2", "\n", 3) check_ctlcat_xlog(test_i, dir, "--from=3 --to=6 --format=json --show-system --replica 2", "\n", 0) - check_ctlcat_snap(test_i, dir, "--space=280", "---\n", 21) - check_ctlcat_snap(test_i, dir, "--space=288", "---\n", 47) + check_ctlcat_snap(test_i, dir, "--space=280", "---\n", 22) + check_ctlcat_snap(test_i, dir, "--space=288", "---\n", 48) end) end) diff --git a/test/box-py/bootstrap.result b/test/box-py/bootstrap.result index 0684914c0..21fcbea7e 100644 --- a/test/box-py/bootstrap.result +++ b/test/box-py/bootstrap.result @@ -79,6 +79,9 @@ box.space._space:select{} {'name': 'is_deferred', 'type': 'boolean'}, {'name': 'match', 'type': 'string'}, {'name': 'on_delete', 'type': 'string'}, {'name': 'on_update', 'type': 'string'}, {'name': 'child_cols', 'type': 'array'}, {'name': 'parent_cols', 'type': 'array'}]] + - [364, 1, '_ck_constraint', 'memtx', 0, {}, [{'name': 'space_id', 'type': 'unsigned'}, + {'name': 'name', 'type': 'string'}, {'name': 'is_deferred', 'type': 'boolean'}, + {'name': 'language', 'type': 'str'}, {'name': 'code', 'type': 'str'}]] ... box.space._index:select{} --- @@ -131,6 +134,7 @@ box.space._index:select{} - [340, 1, 'sequence', 'tree', {'unique': false}, [[1, 'unsigned']]] - [356, 0, 'primary', 'tree', {'unique': true}, [[0, 'string'], [1, 'unsigned']]] - [356, 1, 'child_id', 'tree', {'unique': false}, [[1, 'unsigned']]] + - [364, 0, 'primary', 'tree', {'unique': true}, [[0, 'unsigned'], [1, 'string']]] ... box.space._user:select{} --- diff --git a/test/box/access.result b/test/box/access.result index 44a74e17c..0ebc6a350 100644 --- a/test/box/access.result +++ b/test/box/access.result @@ -1493,6 +1493,9 @@ box.schema.user.grant('tester', 'read', 'space', '_trigger') box.schema.user.grant('tester', 'read', 'space', '_fk_constraint') --- ... +box.schema.user.grant('tester', 'read', 'space', '_ck_constraint') +--- +... box.session.su("tester") --- ... diff --git a/test/box/access.test.lua b/test/box/access.test.lua index ee408f53e..ad63a1016 100644 --- a/test/box/access.test.lua +++ b/test/box/access.test.lua @@ -556,6 +556,7 @@ box.schema.user.grant('tester', 'read', 'space', '_sequence') box.schema.user.grant('tester', 'read', 'space', '_space_sequence') box.schema.user.grant('tester', 'read', 'space', '_trigger') box.schema.user.grant('tester', 'read', 'space', '_fk_constraint') +box.schema.user.grant('tester', 'read', 'space', '_ck_constraint') box.session.su("tester") -- successful create s1 = box.schema.space.create("test_space") diff --git a/test/box/access_misc.result b/test/box/access_misc.result index 24bdd9d63..b8ddcb53b 100644 --- a/test/box/access_misc.result +++ b/test/box/access_misc.result @@ -819,6 +819,9 @@ box.space._space:select() {'name': 'is_deferred', 'type': 'boolean'}, {'name': 'match', 'type': 'string'}, {'name': 'on_delete', 'type': 'string'}, {'name': 'on_update', 'type': 'string'}, {'name': 'child_cols', 'type': 'array'}, {'name': 'parent_cols', 'type': 'array'}]] + - [364, 1, '_ck_constraint', 'memtx', 0, {}, [{'name': 'space_id', 'type': 'unsigned'}, + {'name': 'name', 'type': 'string'}, {'name': 'is_deferred', 'type': 'boolean'}, + {'name': 'language', 'type': 'str'}, {'name': 'code', 'type': 'str'}]] ... box.space._func:select() --- diff --git a/test/box/access_sysview.result b/test/box/access_sysview.result index c6a2b22ed..65a9da8a5 100644 --- a/test/box/access_sysview.result +++ b/test/box/access_sysview.result @@ -230,11 +230,11 @@ box.session.su('guest') ... #box.space._vspace:select{} --- -- 22 +- 23 ... #box.space._vindex:select{} --- -- 48 +- 49 ... #box.space._vuser:select{} --- @@ -262,7 +262,7 @@ box.session.su('guest') ... #box.space._vindex:select{} --- -- 48 +- 49 ... #box.space._vuser:select{} --- diff --git a/test/box/alter.result b/test/box/alter.result index e83c0b7ef..87d7290e1 100644 --- a/test/box/alter.result +++ b/test/box/alter.result @@ -92,7 +92,7 @@ space = box.space[t[1]] ... space.id --- -- 357 +- 358 ... space.field_count --- @@ -137,7 +137,7 @@ space_deleted ... space:replace{0} --- -- error: Space '357' does not exist +- error: Space '358' does not exist ... _index:insert{_space.id, 0, 'primary', 'tree', {unique=true}, {{0, 'unsigned'}}} --- @@ -215,6 +215,7 @@ _index:select{} - [340, 1, 'sequence', 'tree', {'unique': false}, [[1, 'unsigned']]] - [356, 0, 'primary', 'tree', {'unique': true}, [[0, 'string'], [1, 'unsigned']]] - [356, 1, 'child_id', 'tree', {'unique': false}, [[1, 'unsigned']]] + - [357, 0, 'primary', 'tree', {'unique': true}, [[0, 'unsigned'], [1, 'string']]] ... -- modify indexes of a system space _index:delete{_index.id, 0} diff --git a/test/box/misc.result b/test/box/misc.result index 4fcd13a78..5bf419d4f 100644 --- a/test/box/misc.result +++ b/test/box/misc.result @@ -523,6 +523,7 @@ t; 192: box.error.INDEX_DEF_UNSUPPORTED 193: box.error.CK_DEF_UNSUPPORTED 194: box.error.MULTIKEY_INDEX_MISMATCH + 195: box.error.CREATE_CK_CONSTRAINT ... test_run:cmd("setopt delimiter ''"); --- diff --git a/test/sql-tap/check.test.lua b/test/sql-tap/check.test.lua index b01afca7c..ede77c630 100755 --- a/test/sql-tap/check.test.lua +++ b/test/sql-tap/check.test.lua @@ -55,7 +55,7 @@ test:do_catchsql_test( INSERT INTO t1 VALUES(6,7, 2); ]], { -- <check-1.3> - 1, "Failed to execute SQL statement: CHECK constraint failed: T1" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" -- </check-1.3> }) @@ -75,7 +75,7 @@ test:do_catchsql_test( INSERT INTO t1 VALUES(4,3, 2); ]], { -- <check-1.5> - 1, "Failed to execute SQL statement: CHECK constraint failed: T1" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_2_T1" -- </check-1.5> }) @@ -147,7 +147,7 @@ test:do_catchsql_test( UPDATE t1 SET x=7 WHERE x==2 ]], { -- <check-1.12> - 1, "Failed to execute SQL statement: CHECK constraint failed: T1" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" -- </check-1.12> }) @@ -167,7 +167,7 @@ test:do_catchsql_test( UPDATE t1 SET x=5 WHERE x==2 ]], { -- <check-1.14> - 1, "Failed to execute SQL statement: CHECK constraint failed: T1" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" -- </check-1.14> }) @@ -319,7 +319,7 @@ test:do_catchsql_test( ); ]], { -- <check-3.1> - 1, "Failed to create space 'T3': Subqueries are prohibited in a CHECK constraint definition" + 1, "Failed to create check constraint 'CK_CONSTRAINT_1_T3': Subqueries are prohibited in a ck constraint definition" -- </check-3.1> }) @@ -344,7 +344,7 @@ test:do_catchsql_test( ); ]], { -- <check-3.3> - 1, "Failed to create space 'T3': Can’t resolve field 'Q'" + 1, "Failed to create check constraint 'CK_CONSTRAINT_1_T3': Can’t resolve field 'Q'" -- </check-3.3> }) @@ -368,7 +368,7 @@ test:do_catchsql_test( ); ]], { -- <check-3.5> - 1, "Failed to create space 'T3': Field 'X' was not found in the space 'T2' format" + 1, "Failed to create check constraint 'CK_CONSTRAINT_1_T3': Field 'X' was not found in the space 'T2' format" -- </check-3.5> }) @@ -413,7 +413,7 @@ test:do_catchsql_test( INSERT INTO t3 VALUES(111,222,333); ]], { -- <check-3.9> - 1, "Failed to execute SQL statement: CHECK constraint failed: T3" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T3" -- </check-3.9> }) @@ -484,7 +484,7 @@ test:do_catchsql_test( UPDATE t4 SET x=0, y=1; ]], { -- <check-4.6> - 1, "Failed to execute SQL statement: CHECK constraint failed: T4" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T4" -- </check-4.6> }) @@ -504,7 +504,7 @@ test:do_catchsql_test( UPDATE t4 SET x=0, y=2; ]], { -- <check-4.9> - 1, "Failed to execute SQL statement: CHECK constraint failed: T4" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T4" -- </check-4.9> }) @@ -516,7 +516,7 @@ test:do_catchsql_test( ); ]], { -- <check-5.1> - 1, "Wrong space options (field 5): invalid expression specified (bindings are not allowed in DDL)" + 1, "Failed to create check constraint 'CK_CONSTRAINT_1_T5': bindings are not allowed in DDL" -- </check-5.1> }) @@ -528,7 +528,7 @@ test:do_catchsql_test( ); ]], { -- <check-5.2> - 1, "Wrong space options (field 5): invalid expression specified (bindings are not allowed in DDL)" + 1, "Failed to create check constraint 'CK_CONSTRAINT_1_T5': bindings are not allowed in DDL" -- </check-5.2> }) @@ -581,7 +581,7 @@ test:do_catchsql_test( UPDATE OR FAIL t1 SET x=7-x, y=y+1; ]], { -- <check-6.5> - 1, "Failed to execute SQL statement: CHECK constraint failed: T1" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" -- </check-6.5> }) @@ -603,7 +603,7 @@ test:do_catchsql_test( INSERT OR ROLLBACK INTO t1 VALUES(8,40.0, 10); ]], { -- <check-6.7> - 1, "Failed to execute SQL statement: CHECK constraint failed: T1" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" -- </check-6.7> }) @@ -636,7 +636,7 @@ test:do_catchsql_test( REPLACE INTO t1 VALUES(6,7, 11); ]], { -- <check-6.12> - 1, "Failed to execute SQL statement: CHECK constraint failed: T1" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" -- </check-6.12> }) @@ -700,7 +700,7 @@ test:do_catchsql_test( 7.3, " INSERT INTO t6 VALUES(11) ", { -- <7.3> - 1, "Failed to execute SQL statement: CHECK constraint failed: T6" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T6" -- </7.3> }) diff --git a/test/sql-tap/fkey2.test.lua b/test/sql-tap/fkey2.test.lua index 54e5059b3..695a379a6 100755 --- a/test/sql-tap/fkey2.test.lua +++ b/test/sql-tap/fkey2.test.lua @@ -362,7 +362,7 @@ test:do_catchsql_test( UPDATE ab SET a = 5; ]], { -- <fkey2-3.2> - 1, "Failed to execute SQL statement: CHECK constraint failed: EF" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_EF" -- </fkey2-3.2> }) @@ -382,7 +382,7 @@ test:do_catchsql_test( UPDATE ab SET a = 5; ]], { -- <fkey2-3.4> - 1, "Failed to execute SQL statement: CHECK constraint failed: EF" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_EF" -- </fkey2-3.4> }) diff --git a/test/sql-tap/sql-errors.test.lua b/test/sql-tap/sql-errors.test.lua index 9357c406b..a8d39472d 100755 --- a/test/sql-tap/sql-errors.test.lua +++ b/test/sql-tap/sql-errors.test.lua @@ -313,7 +313,7 @@ test:do_catchsql_test( CREATE TABLE t27 (i INT PRIMARY KEY, CHECK(i < (SELECT * FROM t0))); ]], { -- <sql-errors-1.27> - 1,"Failed to create space 'T27': Subqueries are prohibited in a CHECK constraint definition" + 1,"Failed to create check constraint 'CK_CONSTRAINT_1_T27': Subqueries are prohibited in a ck constraint definition" -- </sql-errors-1.27> }) diff --git a/test/sql-tap/table.test.lua b/test/sql-tap/table.test.lua index 5b793c0fc..066662f33 100755 --- a/test/sql-tap/table.test.lua +++ b/test/sql-tap/table.test.lua @@ -1221,7 +1221,7 @@ test:do_catchsql_test( INSERT INTO T21 VALUES(1, -1, 1); ]], { -- <table-21.3> - 1, "Failed to execute SQL statement: CHECK constraint failed: T21" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T21" -- </table-21.3> }) @@ -1231,7 +1231,7 @@ test:do_catchsql_test( INSERT INTO T21 VALUES(1, 1, -1); ]], { -- <table-21.4> - 1, "Failed to execute SQL statement: CHECK constraint failed: T21" + 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_2_T21" -- </table-21.4> }) diff --git a/test/sql/checks.result b/test/sql/checks.result index f7cddec43..efe626428 100644 --- a/test/sql/checks.result +++ b/test/sql/checks.result @@ -18,8 +18,10 @@ box.execute('pragma sql_default_engine=\''..engine..'\'') -- -- gh-3272: Move SQL CHECK into server -- --- invalid expression -opts = {checks = {{expr = 'X><5'}}} +-- Until Tarantool version 2.2 check constraints were stored in +-- space opts. +-- Make sure that now this legacy option is ignored. +opts = {checks = {{expr = 'X>5'}}} --- ... format = {{name = 'X', type = 'unsigned'}} @@ -30,89 +32,219 @@ t = {513, 1, 'test', 'memtx', 0, opts, format} ... s = box.space._space:insert(t) --- -- error: 'Wrong space options (field 5): invalid expression specified (Syntax error - near ''<'')' ... -opts = {checks = {{expr = 'X>5'}}} +_ = box.space.test:create_index('pk') --- ... -format = {{name = 'X', type = 'unsigned'}} +-- Invalid expression test. +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', false, 'SQL', 'X><5'}) --- +- error: 'Failed to create check constraint ''CK_CONSTRAINT_01'': Syntax error near + ''<''' ... -t = {513, 1, 'test', 'memtx', 0, opts, format} +-- Non-existent space test. +box.space._ck_constraint:insert({550, 'CK_CONSTRAINT_01', false, 'SQL', 'X<5'}) --- +- error: Space '550' does not exist ... -s = box.space._space:insert(t) +-- Pass integer instead of expression. +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', false, 'SQL', 666}) +--- +- error: 'Tuple field 5 type does not match one required by operation: expected string' +... +-- Deferred CK constraints are not supported. +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', true, 'SQL', 'X<5'}) +--- +- error: Tarantool does not support deferred ck constraints +... +-- The only supported language is SQL. +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', false, 'LUA', 'X<5'}) +--- +- error: Unsupported language 'LUA' specified for function 'CK_CONSTRAINT_01' +... +-- Check constraints LUA creation test. +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', false, 'SQL', 'X<5'}) +--- +- [513, 'CK_CONSTRAINT_01', false, 'SQL', 'X<5'] +... +box.space._ck_constraint:count({}) +--- +- 1 +... +box.execute("INSERT INTO \"test\" VALUES(5);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_01' +... +box.space._ck_constraint:replace({513, 'CK_CONSTRAINT_01', false, 'SQL', 'X<=5'}) +--- +- [513, 'CK_CONSTRAINT_01', false, 'SQL', 'X<=5'] +... +box.execute("INSERT INTO \"test\" VALUES(5);") +--- +- row_count: 1 +... +box.execute("INSERT INTO \"test\" VALUES(6);") --- +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_01' ... -box.space._space:delete(513) +-- Can't drop table with check constraints. +box.space.test:delete({5}) +--- +- [5] +... +box.space.test.index.pk:drop() +--- +... +box.space._space:delete({513}) +--- +- error: 'Can''t drop space ''test'': the space has check constraints' +... +box.space._ck_constraint:delete({513, 'CK_CONSTRAINT_01'}) +--- +- [513, 'CK_CONSTRAINT_01', false, 'SQL', 'X<=5'] +... +box.space._space:delete({513}) --- - [513, 1, 'test', 'memtx', 0, {'checks': [{'expr': 'X>5'}]}, [{'name': 'X', 'type': 'unsigned'}]] ... -opts = {checks = {{expr = 'X>5', name = 'ONE'}}} +-- Create table with checks in sql. +box.execute("CREATE TABLE t1(x INTEGER CONSTRAINT ONE CHECK( x<5 ), y REAL CONSTRAINT TWO CHECK( y>x ), z INTEGER PRIMARY KEY);") --- +- row_count: 1 ... -format = {{name = 'X', type = 'unsigned'}} +box.space._ck_constraint:count() --- +- 2 ... -t = {513, 1, 'test', 'memtx', 0, opts, format} +box.execute("INSERT INTO t1 VALUES (7, 1, 1)") --- +- error: 'Failed to execute SQL statement: CHECK constraint failed: ONE' ... -s = box.space._space:insert(t) +box.execute("INSERT INTO t1 VALUES (2, 1, 1)") --- +- error: 'Failed to execute SQL statement: CHECK constraint failed: TWO' ... -box.space._space:delete(513) +box.execute("INSERT INTO t1 VALUES (2, 4, 1)") --- -- [513, 1, 'test', 'memtx', 0, {'checks': [{'name': 'ONE', 'expr': 'X>5'}]}, [{'name': 'X', - 'type': 'unsigned'}]] +- row_count: 1 ... --- extra invlalid field name -opts = {checks = {{expr = 'X>5', name = 'ONE', extra = 'TWO'}}} +box.execute("DROP TABLE t1") --- +- row_count: 1 ... -format = {{name = 'X', type = 'unsigned'}} +-- Test space creation rollback on spell error in ck constraint. +box.execute("CREATE TABLE first (id FLOAT PRIMARY KEY CHECK(id < 5), a INT CONSTRAINT ONE CHECK(a >< 5));") --- +- error: Syntax error near '<' ... -t = {513, 1, 'test', 'memtx', 0, opts, format} +box.space.FIRST == nil --- +- true ... -s = box.space._space:insert(t) +box.space._ck_constraint:count() == 0 --- -- error: 'Wrong space options (field 5): invalid MsgPack map field ''extra''' +- true ... -opts = {checks = {{expr_invalid_label = 'X>5'}}} +-- Ck constraints are disallowed for spaces having no format. +s = box.schema.create_space('test', {engine = engine}) --- ... -format = {{name = 'X', type = 'unsigned'}} +_ = s:create_index('pk') --- ... -t = {513, 1, 'test', 'memtx', 0, opts, format} +_ = box.space._ck_constraint:insert({s.id, 'physics', false, 'SQL', 'X<Y'}) --- +- error: Tarantool does not support CK constraint for space without format ... -s = box.space._space:insert(t) +s:format({{name='X', type='integer'}, {name='Y', type='integer'}}) --- -- error: 'Wrong space options (field 5): invalid MsgPack map field ''expr_invalid_label''' ... --- invalid field type -opts = {checks = {{name = 123}}} +_ = box.space._ck_constraint:insert({s.id, 'physics', false, 'SQL', 'X<Y'}) --- ... -format = {{name = 'X', type = 'unsigned'}} +box.execute("INSERT INTO \"test\" VALUES(2, 1);") --- +- error: 'Failed to execute SQL statement: CHECK constraint failed: physics' ... -t = {513, 1, 'test', 'memtx', 0, opts, format} +s:format({{name='Y', type='integer'}, {name='X', type='integer'}}) --- ... -s = box.space._space:insert(t) +box.execute("INSERT INTO \"test\" VALUES(1, 2);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: physics' +... +box.execute("INSERT INTO \"test\" VALUES(2, 1);") +--- +- row_count: 1 +... +s:truncate() +--- +... +box.execute("INSERT INTO \"test\" VALUES(1, 2);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: physics' +... +s:format({}) +--- +- error: Tarantool does not support CK constraint for space without format +... +s:format() +--- +- [{'name': 'Y', 'type': 'integer'}, {'name': 'X', 'type': 'integer'}] +... +s:format({{name='Y1', type='integer'}, {name='X1', type='integer'}}) +--- +- error: 'Failed to create check constraint ''physics'': Can’t resolve field ''X''' +... +-- Ck constraint creation is forbidden for non-empty space +s:insert({2, 1}) +--- +- [2, 1] +... +_ = box.space._ck_constraint:insert({s.id, 'conflict', false, 'SQL', 'X>10'}) +--- +- error: 'Failed to create check constraint ''conflict'': referencing space must be + empty' +... +s:truncate() +--- +... +_ = box.space._ck_constraint:insert({s.id, 'conflict', false, 'SQL', 'X>10'}) --- -- error: 'Wrong space options (field 5): invalid MsgPack map field ''name'' type' +... +box.execute("INSERT INTO \"test\" VALUES(1, 2);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: conflict' +... +box.execute("INSERT INTO \"test\" VALUES(11, 11);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: physics' +... +box.execute("INSERT INTO \"test\" VALUES(12, 11);") +--- +- row_count: 1 +... +s:drop() +--- +... +box.execute("CREATE TABLE T2(ID INT PRIMARY KEY, CONSTRAINT CK1 CHECK(ID > 0), CONSTRAINT CK1 CHECK(ID < 0))") +--- +- error: Constraint CK1 already exists +... +box.space.T2:drop() +--- +... +box.space._ck_constraint:select() +--- +- [] ... -- -- gh-3611: Segfault on table creation with check referencing this table -- box.execute("CREATE TABLE w2 (s1 INT PRIMARY KEY, CHECK ((SELECT COUNT(*) FROM w2) = 0));") --- -- error: 'Failed to create space ''W2'': Space ''W2'' does not exist' +- error: 'Failed to create check constraint ''CK_CONSTRAINT_1_W2'': Subqueries are + prohibited in a ck constraint definition' ... box.execute("DROP TABLE w2;") --- @@ -123,22 +255,33 @@ box.execute("DROP TABLE w2;") -- box.execute("CREATE TABLE t5(x INT PRIMARY KEY, y INT, CHECK( x*y < ? ));") --- -- error: 'Wrong space options (field 5): invalid expression specified (bindings are - not allowed in DDL)' +- error: 'Failed to create check constraint ''CK_CONSTRAINT_1_T5'': bindings are not + allowed in DDL' ... -opts = {checks = {{expr = '?>5', name = 'ONE'}}} +-- Test trim CK constraint code correctness. +box.execute("CREATE TABLE t1(x TEXT PRIMARY KEY CHECK(x LIKE '1 a'))") --- +- row_count: 1 ... -format = {{name = 'X', type = 'unsigned'}} +box.space._ck_constraint:select()[1].code --- +- x LIKE '1 a' ... -t = {513, 1, 'test', 'memtx', 0, opts, format} +box.execute("INSERT INTO t1 VALUES('1 a')") --- +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1' ... -s = box.space._space:insert(t) +box.execute("INSERT INTO t1 VALUES('1 a')") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1' +... +box.execute("INSERT INTO t1 VALUES('1 a')") +--- +- row_count: 1 +... +box.execute("DROP TABLE t1") --- -- error: 'Wrong space options (field 5): invalid expression specified (bindings are - not allowed in DDL)' +- row_count: 1 ... test_run:cmd("clear filter") --- diff --git a/test/sql/checks.test.lua b/test/sql/checks.test.lua index 5bfcf12f8..399caf134 100644 --- a/test/sql/checks.test.lua +++ b/test/sql/checks.test.lua @@ -8,41 +8,81 @@ box.execute('pragma sql_default_engine=\''..engine..'\'') -- gh-3272: Move SQL CHECK into server -- --- invalid expression -opts = {checks = {{expr = 'X><5'}}} -format = {{name = 'X', type = 'unsigned'}} -t = {513, 1, 'test', 'memtx', 0, opts, format} -s = box.space._space:insert(t) - +-- Until Tarantool version 2.2 check constraints were stored in +-- space opts. +-- Make sure that now this legacy option is ignored. opts = {checks = {{expr = 'X>5'}}} format = {{name = 'X', type = 'unsigned'}} t = {513, 1, 'test', 'memtx', 0, opts, format} s = box.space._space:insert(t) -box.space._space:delete(513) +_ = box.space.test:create_index('pk') -opts = {checks = {{expr = 'X>5', name = 'ONE'}}} -format = {{name = 'X', type = 'unsigned'}} -t = {513, 1, 'test', 'memtx', 0, opts, format} -s = box.space._space:insert(t) -box.space._space:delete(513) +-- Invalid expression test. +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', false, 'SQL', 'X><5'}) +-- Non-existent space test. +box.space._ck_constraint:insert({550, 'CK_CONSTRAINT_01', false, 'SQL', 'X<5'}) +-- Pass integer instead of expression. +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', false, 'SQL', 666}) +-- Deferred CK constraints are not supported. +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', true, 'SQL', 'X<5'}) +-- The only supported language is SQL. +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', false, 'LUA', 'X<5'}) --- extra invlalid field name -opts = {checks = {{expr = 'X>5', name = 'ONE', extra = 'TWO'}}} -format = {{name = 'X', type = 'unsigned'}} -t = {513, 1, 'test', 'memtx', 0, opts, format} -s = box.space._space:insert(t) +-- Check constraints LUA creation test. +box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', false, 'SQL', 'X<5'}) +box.space._ck_constraint:count({}) -opts = {checks = {{expr_invalid_label = 'X>5'}}} -format = {{name = 'X', type = 'unsigned'}} -t = {513, 1, 'test', 'memtx', 0, opts, format} -s = box.space._space:insert(t) +box.execute("INSERT INTO \"test\" VALUES(5);") +box.space._ck_constraint:replace({513, 'CK_CONSTRAINT_01', false, 'SQL', 'X<=5'}) +box.execute("INSERT INTO \"test\" VALUES(5);") +box.execute("INSERT INTO \"test\" VALUES(6);") +-- Can't drop table with check constraints. +box.space.test:delete({5}) +box.space.test.index.pk:drop() +box.space._space:delete({513}) +box.space._ck_constraint:delete({513, 'CK_CONSTRAINT_01'}) +box.space._space:delete({513}) --- invalid field type -opts = {checks = {{name = 123}}} -format = {{name = 'X', type = 'unsigned'}} -t = {513, 1, 'test', 'memtx', 0, opts, format} -s = box.space._space:insert(t) +-- Create table with checks in sql. +box.execute("CREATE TABLE t1(x INTEGER CONSTRAINT ONE CHECK( x<5 ), y REAL CONSTRAINT TWO CHECK( y>x ), z INTEGER PRIMARY KEY);") +box.space._ck_constraint:count() +box.execute("INSERT INTO t1 VALUES (7, 1, 1)") +box.execute("INSERT INTO t1 VALUES (2, 1, 1)") +box.execute("INSERT INTO t1 VALUES (2, 4, 1)") +box.execute("DROP TABLE t1") +-- Test space creation rollback on spell error in ck constraint. +box.execute("CREATE TABLE first (id FLOAT PRIMARY KEY CHECK(id < 5), a INT CONSTRAINT ONE CHECK(a >< 5));") +box.space.FIRST == nil +box.space._ck_constraint:count() == 0 + +-- Ck constraints are disallowed for spaces having no format. +s = box.schema.create_space('test', {engine = engine}) +_ = s:create_index('pk') +_ = box.space._ck_constraint:insert({s.id, 'physics', false, 'SQL', 'X<Y'}) +s:format({{name='X', type='integer'}, {name='Y', type='integer'}}) +_ = box.space._ck_constraint:insert({s.id, 'physics', false, 'SQL', 'X<Y'}) +box.execute("INSERT INTO \"test\" VALUES(2, 1);") +s:format({{name='Y', type='integer'}, {name='X', type='integer'}}) +box.execute("INSERT INTO \"test\" VALUES(1, 2);") +box.execute("INSERT INTO \"test\" VALUES(2, 1);") +s:truncate() +box.execute("INSERT INTO \"test\" VALUES(1, 2);") +s:format({}) +s:format() +s:format({{name='Y1', type='integer'}, {name='X1', type='integer'}}) +-- Ck constraint creation is forbidden for non-empty space +s:insert({2, 1}) +_ = box.space._ck_constraint:insert({s.id, 'conflict', false, 'SQL', 'X>10'}) +s:truncate() +_ = box.space._ck_constraint:insert({s.id, 'conflict', false, 'SQL', 'X>10'}) +box.execute("INSERT INTO \"test\" VALUES(1, 2);") +box.execute("INSERT INTO \"test\" VALUES(11, 11);") +box.execute("INSERT INTO \"test\" VALUES(12, 11);") +s:drop() +box.execute("CREATE TABLE T2(ID INT PRIMARY KEY, CONSTRAINT CK1 CHECK(ID > 0), CONSTRAINT CK1 CHECK(ID < 0))") +box.space.T2:drop() +box.space._ck_constraint:select() -- -- gh-3611: Segfault on table creation with check referencing this table @@ -55,10 +95,12 @@ box.execute("DROP TABLE w2;") -- box.execute("CREATE TABLE t5(x INT PRIMARY KEY, y INT, CHECK( x*y < ? ));") -opts = {checks = {{expr = '?>5', name = 'ONE'}}} -format = {{name = 'X', type = 'unsigned'}} -t = {513, 1, 'test', 'memtx', 0, opts, format} -s = box.space._space:insert(t) - +-- Test trim CK constraint code correctness. +box.execute("CREATE TABLE t1(x TEXT PRIMARY KEY CHECK(x LIKE '1 a'))") +box.space._ck_constraint:select()[1].code +box.execute("INSERT INTO t1 VALUES('1 a')") +box.execute("INSERT INTO t1 VALUES('1 a')") +box.execute("INSERT INTO t1 VALUES('1 a')") +box.execute("DROP TABLE t1") test_run:cmd("clear filter") diff --git a/test/sql/errinj.result b/test/sql/errinj.result index ee36a387b..414e3c476 100644 --- a/test/sql/errinj.result +++ b/test/sql/errinj.result @@ -463,3 +463,137 @@ errinj.set("ERRINJ_SQL_NAME_NORMALIZATION", false) --- - ok ... +-- +-- Tests which are aimed at verifying work of commit/rollback +-- triggers on _ck_constraint space. +-- +s = box.schema.space.create('test', {format = {{name = 'X', type = 'unsigned'}}}) +--- +... +pk = box.space.test:create_index('pk') +--- +... +errinj.set("ERRINJ_WAL_IO", true) +--- +- ok +... +_ = box.space._ck_constraint:insert({s.id, 'CK_CONSTRAINT_01', false, 'SQL', 'X<5'}) +--- +- error: Failed to write to disk +... +errinj.set("ERRINJ_WAL_IO", false) +--- +- ok +... +_ = box.space._ck_constraint:insert({s.id, 'CK_CONSTRAINT_01', false, 'SQL', 'X<5'}) +--- +... +box.execute("INSERT INTO \"test\" VALUES(5);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_01' +... +errinj.set("ERRINJ_WAL_IO", true) +--- +- ok +... +_ = box.space._ck_constraint:replace({s.id, 'CK_CONSTRAINT_01', false, 'SQL', 'X<=5'}) +--- +- error: Failed to write to disk +... +errinj.set("ERRINJ_WAL_IO", false) +--- +- ok +... +_ = box.space._ck_constraint:replace({s.id, 'CK_CONSTRAINT_01', false, 'SQL', 'X<=5'}) +--- +... +box.execute("INSERT INTO \"test\" VALUES(5);") +--- +- row_count: 1 +... +errinj.set("ERRINJ_WAL_IO", true) +--- +- ok +... +_ = box.space._ck_constraint:delete({s.id, 'CK_CONSTRAINT_01'}) +--- +- error: Failed to write to disk +... +box.execute("INSERT INTO \"test\" VALUES(6);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_01' +... +errinj.set("ERRINJ_WAL_IO", false) +--- +- ok +... +_ = box.space._ck_constraint:delete({s.id, 'CK_CONSTRAINT_01'}) +--- +... +box.execute("INSERT INTO \"test\" VALUES(6);") +--- +- row_count: 1 +... +s:drop() +--- +... +-- +-- Test that failed space alter doesn't harm ck constraints +-- +s = box.schema.create_space('test') +--- +... +_ = s:create_index('pk') +--- +... +s:format({{name='X', type='integer'}, {name='Y', type='integer'}}) +--- +... +_ = box.space._ck_constraint:insert({s.id, 'XlessY', false, 'SQL', 'X < Y'}) +--- +... +_ = box.space._ck_constraint:insert({s.id, 'Xgreater10', false, 'SQL', 'X > 10'}) +--- +... +box.execute("INSERT INTO \"test\" VALUES(1, 2);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: Xgreater10' +... +box.execute("INSERT INTO \"test\" VALUES(20, 10);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: XlessY' +... +box.execute("INSERT INTO \"test\" VALUES(20, 100);") +--- +- row_count: 1 +... +s:truncate() +--- +... +errinj.set("ERRINJ_WAL_IO", true) +--- +- ok +... +s:format({{name='Y', type='integer'}, {name='X', type='integer'}}) +--- +- error: Failed to write to disk +... +errinj.set("ERRINJ_WAL_IO", false) +--- +- ok +... +box.execute("INSERT INTO \"test\" VALUES(1, 2);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: Xgreater10' +... +box.execute("INSERT INTO \"test\" VALUES(20, 10);") +--- +- error: 'Failed to execute SQL statement: CHECK constraint failed: XlessY' +... +box.execute("INSERT INTO \"test\" VALUES(20, 100);") +--- +- row_count: 1 +... +s:drop() +--- +... diff --git a/test/sql/errinj.test.lua b/test/sql/errinj.test.lua index 1aff6d77e..48b80a443 100644 --- a/test/sql/errinj.test.lua +++ b/test/sql/errinj.test.lua @@ -149,3 +149,48 @@ box.execute("CREATE TABLE hello (id INT primary key,x INT,y INT);") dummy_f = function(int) return 1 end box.internal.sql_create_function("counter1", "INT", dummy_f, -1, false) errinj.set("ERRINJ_SQL_NAME_NORMALIZATION", false) + +-- +-- Tests which are aimed at verifying work of commit/rollback +-- triggers on _ck_constraint space. +-- +s = box.schema.space.create('test', {format = {{name = 'X', type = 'unsigned'}}}) +pk = box.space.test:create_index('pk') + +errinj.set("ERRINJ_WAL_IO", true) +_ = box.space._ck_constraint:insert({s.id, 'CK_CONSTRAINT_01', false, 'SQL', 'X<5'}) +errinj.set("ERRINJ_WAL_IO", false) +_ = box.space._ck_constraint:insert({s.id, 'CK_CONSTRAINT_01', false, 'SQL', 'X<5'}) +box.execute("INSERT INTO \"test\" VALUES(5);") +errinj.set("ERRINJ_WAL_IO", true) +_ = box.space._ck_constraint:replace({s.id, 'CK_CONSTRAINT_01', false, 'SQL', 'X<=5'}) +errinj.set("ERRINJ_WAL_IO", false) +_ = box.space._ck_constraint:replace({s.id, 'CK_CONSTRAINT_01', false, 'SQL', 'X<=5'}) +box.execute("INSERT INTO \"test\" VALUES(5);") +errinj.set("ERRINJ_WAL_IO", true) +_ = box.space._ck_constraint:delete({s.id, 'CK_CONSTRAINT_01'}) +box.execute("INSERT INTO \"test\" VALUES(6);") +errinj.set("ERRINJ_WAL_IO", false) +_ = box.space._ck_constraint:delete({s.id, 'CK_CONSTRAINT_01'}) +box.execute("INSERT INTO \"test\" VALUES(6);") +s:drop() + +-- +-- Test that failed space alter doesn't harm ck constraints +-- +s = box.schema.create_space('test') +_ = s:create_index('pk') +s:format({{name='X', type='integer'}, {name='Y', type='integer'}}) +_ = box.space._ck_constraint:insert({s.id, 'XlessY', false, 'SQL', 'X < Y'}) +_ = box.space._ck_constraint:insert({s.id, 'Xgreater10', false, 'SQL', 'X > 10'}) +box.execute("INSERT INTO \"test\" VALUES(1, 2);") +box.execute("INSERT INTO \"test\" VALUES(20, 10);") +box.execute("INSERT INTO \"test\" VALUES(20, 100);") +s:truncate() +errinj.set("ERRINJ_WAL_IO", true) +s:format({{name='Y', type='integer'}, {name='X', type='integer'}}) +errinj.set("ERRINJ_WAL_IO", false) +box.execute("INSERT INTO \"test\" VALUES(1, 2);") +box.execute("INSERT INTO \"test\" VALUES(20, 10);") +box.execute("INSERT INTO \"test\" VALUES(20, 100);") +s:drop() diff --git a/test/sql/gh-2981-check-autoinc.result b/test/sql/gh-2981-check-autoinc.result index 9e347ca6b..7384c81e8 100644 --- a/test/sql/gh-2981-check-autoinc.result +++ b/test/sql/gh-2981-check-autoinc.result @@ -29,7 +29,7 @@ box.execute("insert into t1 values (18, null);") ... box.execute("insert into t1(s2) values (null);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: T1' +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1' ... box.execute("insert into t2 values (18, null);") --- @@ -37,7 +37,7 @@ box.execute("insert into t2 values (18, null);") ... box.execute("insert into t2(s2) values (null);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: T2' +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T2' ... box.execute("insert into t2 values (24, null);") --- @@ -45,7 +45,7 @@ box.execute("insert into t2 values (24, null);") ... box.execute("insert into t2(s2) values (null);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: T2' +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T2' ... box.execute("insert into t3 values (9, null)") --- @@ -53,7 +53,7 @@ box.execute("insert into t3 values (9, null)") ... box.execute("insert into t3(s2) values (null)") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: T3' +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T3' ... box.execute("DROP TABLE t1") --- diff --git a/test/sql/types.result b/test/sql/types.result index a53d6f7ce..1c9ef5468 100644 --- a/test/sql/types.result +++ b/test/sql/types.result @@ -741,7 +741,7 @@ box.execute("CREATE TABLE t1 (id INT PRIMARY KEY, a BOOLEAN CHECK (a = true));") ... box.execute("INSERT INTO t1 VALUES (1, false);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: T1' +- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1' ... box.execute("INSERT INTO t1 VALUES (1, true);") --- diff --git a/test/sql/upgrade.result b/test/sql/upgrade.result index 3a55f7c53..f0997e17f 100644 --- a/test/sql/upgrade.result +++ b/test/sql/upgrade.result @@ -188,6 +188,25 @@ i[1].opts.sql == nil --- - true ... +box.space._space:get(s.id).flags.checks == nil +--- +- true +... +check = box.space._ck_constraint:select()[1] +--- +... +check ~= nil +--- +- true +... +check.name +--- +- CK_CONSTRAINT_1_T5 +... +check.code +--- +- x < 2 +... s:drop() --- ... diff --git a/test/sql/upgrade.test.lua b/test/sql/upgrade.test.lua index b76a8f373..37425ae21 100644 --- a/test/sql/upgrade.test.lua +++ b/test/sql/upgrade.test.lua @@ -62,6 +62,11 @@ s ~= nil i = box.space._index:select(s.id) i ~= nil i[1].opts.sql == nil +box.space._space:get(s.id).flags.checks == nil +check = box.space._ck_constraint:select()[1] +check ~= nil +check.name +check.code s:drop() test_run:switch('default') diff --git a/test/wal_off/alter.result b/test/wal_off/alter.result index ee280fcbb..8040efa1a 100644 --- a/test/wal_off/alter.result +++ b/test/wal_off/alter.result @@ -28,7 +28,7 @@ end; ... #spaces; --- -- 65505 +- 65504 ... -- cleanup for k, v in pairs(spaces) do -- 2.21.0 ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 4/6] schema: add new system space for CHECK constraints 2019-05-31 13:45 ` Kirill Shcherbatov @ 2019-06-03 21:15 ` Vladislav Shpilevoy 0 siblings, 0 replies; 31+ messages in thread From: Vladislav Shpilevoy @ 2019-06-03 21:15 UTC (permalink / raw) To: tarantool-patches, Kirill Shcherbatov Thanks for the fixes! See 1 comment below, review fixes on the branch, and at the end of the email. > commit cae1ff98e7d190a9fec5c573c84a2ad7c642f1c4 > Author: Kirill Shcherbatov <kshcherbatov@tarantool.org> > Date: Thu Apr 4 14:58:23 2019 +0300 > > schema: add new system space for CHECK constraints > > This patch introduces a new system space to persist check > constraints. The format of the new system space is > > _ck_constraint (space id = 364) > [<space id> UINT, <constraint name> STR, > <is_deferred>BOOL, <language>STR], <code>STR 1. '<code> STR' is out of []. > > A CK constraint is local for a space, so every pair > <space id, CK name> is unique > (it is also the PK in the _ck_constraint space). > > After insertion into this space, a new instance describing check > constraint is created. Check constraint holds an exspression AST. > While space features some check constraints, it isn't allowed to > be dropped. The :drop() space method firstly deletes all check > constraints and then removes an entry from the _space. > > Because the space alter, the index alter and the space truncate > operations cause space recreation process, a new > RebuildCkConstrains object is introduced. This alter object > compiles a new ck constraint object, replaces and removes > an existent instances atomically (but if the assembly of some > ck constraint object fails, nothing is changed). > In fact, in scope of this patch we don't really need to recreate > a ck_constraint object in such situations (it is enough to patch > space_def pointer in AST tree like we did it before, but we are > going to recompile a VDBE that represents ck constraint in > further patches, and that operation is not safe). > > The main motivation for these changes is an ability to support > ADD CHECK CONSTRAINT operation in the future. CK constraints are > easier to manage as self-sustained objects: such change is > managed with atomic insertion(unlike the current architecture). > > Finally, the xfer optimization is disabled now if some space have > ck constraints. In following patches this xfer optimisation > becomes impossible, so there is no reason to rewrite this code > now. > > Needed for #3691 > I've pushed my review fixes in a separate commit. I didn't fix the commit message, because I can't do it without amendment. Below are my explanations, and diff. 1) I replaced free() with ck_constraint_def_delete() in one place. 2) I replaced 'char *ck_constraint_def.name' with 'char ck_constraint_def.name[0]' - it simplified several places, and occupies 8 bytes less memory (yes, I like micro optimizations). 3) I fixed one another typo in the comment on trim_space_snprintf. ================================================================== diff --git a/src/box/alter.cc b/src/box/alter.cc index bedeb71cd..f72a2ed25 100644 --- a/src/box/alter.cc +++ b/src/box/alter.cc @@ -4273,7 +4273,9 @@ on_replace_dd_ck_constraint(struct trigger * /* trigger*/, void *event) /* Create or replace check constraint. */ struct ck_constraint_def *ck_def = ck_constraint_def_new_from_tuple(new_tuple); - auto ck_guard = make_scoped_guard([=] { free(ck_def); }); + auto ck_guard = make_scoped_guard([=] { + ck_constraint_def_delete(ck_def); + }); /* * FIXME: Ck constraint creation on non-empty * space is not implemented yet. diff --git a/src/box/ck_constraint.c b/src/box/ck_constraint.c index 69b9793ea..e25f4506e 100644 --- a/src/box/ck_constraint.c +++ b/src/box/ck_constraint.c @@ -41,17 +41,15 @@ ck_constraint_def_new(const char *name, uint32_t name_len, const char *expr_str, uint32_t expr_str_len, uint32_t space_id, enum ck_constraint_language language) { - uint32_t name_offset, expr_str_offset; - uint32_t ck_def_sz = - ck_constraint_def_sizeof(name_len, expr_str_len, &name_offset, - &expr_str_offset); + uint32_t expr_str_offset; + uint32_t ck_def_sz = ck_constraint_def_sizeof(name_len, expr_str_len, + &expr_str_offset); struct ck_constraint_def *ck_def = (struct ck_constraint_def *) malloc(ck_def_sz); if (ck_def == NULL) { diag_set(OutOfMemory, ck_def_sz, "malloc", "ck_def"); return NULL; } - ck_def->name = (char *)ck_def + name_offset; ck_def->expr_str = (char *)ck_def + expr_str_offset; ck_def->language = language; ck_def->space_id = space_id; diff --git a/src/box/ck_constraint.h b/src/box/ck_constraint.h index 3ae3d5c91..a5b7b2c71 100644 --- a/src/box/ck_constraint.h +++ b/src/box/ck_constraint.h @@ -57,11 +57,6 @@ extern const char *ck_constraint_language_strs[]; * details and memory layout. */ struct ck_constraint_def { - /** - * The 0-terminated string, a name of the check - * constraint. Must be unique for a given space. - */ - char *name; /** * The 0-terminated string that defines check constraint * expression. @@ -76,6 +71,11 @@ struct ck_constraint_def { uint32_t space_id; /** The language of ck constraint. */ enum ck_constraint_language language; + /** + * The 0-terminated string, a name of the check + * constraint. Must be unique for a given space. + */ + char name[0]; }; /** @@ -117,17 +117,15 @@ struct ck_constraint { * * @param name_len The length of the name. * @param expr_str_len The length of the expr_str. - * @param[out] name_offset The offset of the name string. * @param[out] expr_str_offset The offset of the expr_str string. * @return The size of the ck constraint definition object for * given parameters. */ static inline uint32_t ck_constraint_def_sizeof(uint32_t name_len, uint32_t expr_str_len, - uint32_t *name_offset, uint32_t *expr_str_offset) + uint32_t *expr_str_offset) { - *name_offset = sizeof(struct ck_constraint_def); - *expr_str_offset = *name_offset + name_len + 1; + *expr_str_offset = sizeof(struct ck_constraint_def) + name_len + 1; return *expr_str_offset + expr_str_len + 1; } diff --git a/src/box/sql/build.c b/src/box/sql/build.c index 09a15cb18..b28e7b66f 100644 --- a/src/box/sql/build.c +++ b/src/box/sql/build.c @@ -644,7 +644,7 @@ primary_key_exit: * Prepare a 0-terminated string in the wptr memory buffer that * does not contain a sequence of more than one whatespace * character. Routine enforces ' ' (space) as whitespace - * delimiter. When character ' or " was met, the sting is copied + * delimiter. When character ' or " was met, the string is copied * without any changes until the next ' or " sign. * The wptr buffer is expected to have str_len + 1 bytes * (this is the expected scenario where no extra whitespace @@ -721,13 +721,11 @@ sql_create_check_contraint(struct Parse *parser) * region: * * [ck_parse][ck_def[name][expr_str]] - * |_____^ |___^ ^ - * |_________| + * |_____^ |_________^ */ - uint32_t name_offset, expr_str_offset; - uint32_t ck_def_sz = - ck_constraint_def_sizeof(name_len, expr_str_len, &name_offset, - &expr_str_offset); + uint32_t expr_str_offset; + uint32_t ck_def_sz = ck_constraint_def_sizeof(name_len, expr_str_len, + &expr_str_offset); struct ck_constraint_parse *ck_parse = region_alloc(region, sizeof(*ck_parse) + ck_def_sz); if (ck_parse == NULL) { @@ -742,7 +740,6 @@ sql_create_check_contraint(struct Parse *parser) ck_parse->ck_def = ck_def; rlist_create(&ck_parse->link); - ck_def->name = (char *)ck_def + name_offset; ck_def->expr_str = (char *)ck_def + expr_str_offset; ck_def->language = CK_CONSTRAINT_LANGUAGE_SQL; ck_def->space_id = BOX_ID_NIL; ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] [PATCH v5 5/6] box: run check constraint tests on space alter 2019-05-23 10:19 [tarantool-patches] [PATCH v5 0/6] box: run checks on insertions in LUA spaces Kirill Shcherbatov ` (3 preceding siblings ...) 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 4/6] schema: add new system space for CHECK constraints Kirill Shcherbatov @ 2019-05-23 10:19 ` Kirill Shcherbatov 2019-05-26 12:07 ` [tarantool-patches] " Vladislav Shpilevoy 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 6/6] box: user-friendly interface to manage ck constraints Kirill Shcherbatov ` (2 subsequent siblings) 7 siblings, 1 reply; 31+ messages in thread From: Kirill Shcherbatov @ 2019-05-23 10:19 UTC (permalink / raw) To: tarantool-patches, v.shpilevoy; +Cc: Kirill Shcherbatov To perform ck constraints tests before insert or update space operation, we use precompiled VDBE machine associated with each ck constraint, that is executed in on_replace trigger. Each ck constraint VDBE code consists of 1) prologue code that maps new(or updated) tuple via binding, 2) ck constraint code generated by CK constraint AST. In case of ck constraint error the tuple insert/replace operation is aborted and ck constraint error is handled as diag message. Needed for #3691 --- src/box/alter.cc | 64 ++++- src/box/ck_constraint.c | 131 ++++++++++- src/box/ck_constraint.h | 29 ++- src/box/errcode.h | 1 + src/box/space.c | 4 + src/box/space.h | 3 + src/box/sql/expr.c | 25 +- src/box/sql/insert.c | 92 ++------ src/box/sql/sqlInt.h | 36 ++- src/box/sql/vdbe.h | 1 - src/box/sql/vdbeapi.c | 8 - test/box/misc.result | 1 + test/sql-tap/check.test.lua | 32 +-- test/sql-tap/fkey2.test.lua | 4 +- test/sql-tap/table.test.lua | 12 +- test/sql/checks.result | 324 +++++++++++++++++++++++++- test/sql/checks.test.lua | 96 ++++++++ test/sql/errinj.result | 18 +- test/sql/gh-2981-check-autoinc.result | 12 +- test/sql/types.result | 3 +- 20 files changed, 744 insertions(+), 152 deletions(-) diff --git a/src/box/alter.cc b/src/box/alter.cc index 463357c67..e4ded1995 100644 --- a/src/box/alter.cc +++ b/src/box/alter.cc @@ -1421,6 +1421,12 @@ RebuildCkConstraints::space_swap_ck_constraint(struct space *old_space, { rlist_swap(&new_space->ck_constraint, &ck_constraint); rlist_swap(&ck_constraint, &old_space->ck_constraint); + + trigger_clear(&old_space->ck_constraint_trigger); + if (!rlist_empty(&new_space->ck_constraint)) { + trigger_add(&old_space->on_replace, + &new_space->ck_constraint_trigger); + } } void @@ -1450,6 +1456,48 @@ RebuildCkConstraints::~RebuildCkConstraints() } } +/** + * Move CK constraints from old space to the new one. + * Unlike RebuildCkConstraints, this operation doesn't perform + * ck constraints rebuild. This may be used in scenarios where + * space format doesn't change i.e. on index alter or space trim. + */ +class MoveCkConstraints: public AlterSpaceOp +{ + void space_swap_ck_constraint(struct space *old_space, + struct space *new_space); +public: + MoveCkConstraints(struct alter_space *alter) : AlterSpaceOp(alter) {} + virtual void alter(struct alter_space *alter); + virtual void rollback(struct alter_space *alter); +}; + +void +MoveCkConstraints::space_swap_ck_constraint(struct space *old_space, + struct space *new_space) +{ + rlist_swap(&new_space->ck_constraint, + &old_space->ck_constraint); + + trigger_clear(&old_space->ck_constraint_trigger); + if (!rlist_empty(&new_space->ck_constraint)) { + trigger_add(&old_space->on_replace, + &new_space->ck_constraint_trigger); + } +} + +void +MoveCkConstraints::alter(struct alter_space *alter) +{ + space_swap_ck_constraint(alter->old_space, alter->new_space); +} + +void +MoveCkConstraints::rollback(struct alter_space *alter) +{ + space_swap_ck_constraint(alter->new_space, alter->old_space); +} + /* }}} */ /** @@ -2160,7 +2208,7 @@ on_replace_dd_index(struct trigger * /* trigger */, void *event) * old space. */ alter_space_move_indexes(alter, iid + 1, old_space->index_id_max + 1); - (void) new RebuildCkConstraints(alter); + (void) new MoveCkConstraints(alter); /* Add an op to update schema_version on commit. */ (void) new UpdateSchemaVersion(alter); alter_space_do(txn, alter); @@ -2229,7 +2277,7 @@ on_replace_dd_truncate(struct trigger * /* trigger */, void *event) (void) new TruncateIndex(alter, old_index->def->iid); } - (void) new RebuildCkConstraints(alter); + (void) new MoveCkConstraints(alter); alter_space_do(txn, alter); scoped_guard.is_active = false; } @@ -4173,6 +4221,10 @@ on_replace_ck_constraint_rollback(struct trigger *trigger, void *event) assert(space_ck_constraint_by_name(space, ck->def->name, strlen(ck->def->name)) == NULL); rlist_add_entry(&space->ck_constraint, ck, link); + if (rlist_empty(&space->ck_constraint_trigger.link)) { + trigger_add(&space->on_replace, + &space->ck_constraint_trigger); + } } else if (stmt->new_tuple != NULL && stmt->old_tuple == NULL) { /* Rollback INSERT check constraint. */ assert(space != NULL); @@ -4180,6 +4232,8 @@ on_replace_ck_constraint_rollback(struct trigger *trigger, void *event) ck->def->name, strlen(ck->def->name)) != NULL); rlist_del_entry(ck, link); ck_constraint_delete(ck); + if (rlist_empty(&space->ck_constraint)) + trigger_clear(&space->ck_constraint_trigger); } else { /* Rollback REPLACE check constraint. */ assert(space != NULL); @@ -4258,6 +4312,10 @@ on_replace_dd_ck_constraint(struct trigger * /* trigger*/, void *event) if (old_ck_constraint != NULL) rlist_del_entry(old_ck_constraint, link); rlist_add_entry(&space->ck_constraint, new_ck_constraint, link); + if (rlist_empty(&space->ck_constraint_trigger.link)) { + trigger_add(&space->on_replace, + &space->ck_constraint_trigger); + } on_commit->data = old_tuple == NULL ? new_ck_constraint : old_ck_constraint; on_rollback->data = on_commit->data; @@ -4273,6 +4331,8 @@ on_replace_dd_ck_constraint(struct trigger * /* trigger*/, void *event) space_ck_constraint_by_name(space, name, name_len); assert(old_ck_constraint != NULL); rlist_del_entry(old_ck_constraint, link); + if (rlist_empty(&space->ck_constraint)) + trigger_clear(&space->ck_constraint_trigger); on_commit->data = old_ck_constraint; on_rollback->data = old_ck_constraint; } diff --git a/src/box/ck_constraint.c b/src/box/ck_constraint.c index c74687b31..ba2df4b67 100644 --- a/src/box/ck_constraint.c +++ b/src/box/ck_constraint.c @@ -29,10 +29,15 @@ * SUCH DAMAGE. */ #include "box/session.h" +#include "bind.h" #include "ck_constraint.h" #include "errcode.h" +#include "schema.h" +#include "small/region.h" #include "sql.h" #include "sql/sqlInt.h" +#include "sql/vdbeInt.h" +#include "tuple.h" const char *ck_constraint_language_strs[] = {"SQL"}; @@ -57,6 +62,116 @@ ck_constraint_resolve_field_names(struct Expr *expr, return rc; } +/** + * Create a VDBE machine for the ck constraint by a given + * definition and an expression AST. The generated instructions + * consist of prologue code that maps tuple_fetcher via binding + * and ck constraint code that implements a given expression. + * @param ck_constraint_def Check constraint definition to prepare + * an error description. + * @param expr Ck constraint expression AST built for a given + * @a ck_constraint_def, see for (sql_expr_compile and + * ck_constraint_resolve_space_def) implementation. + * @param space_def The space definition of the space this check + * constraint is constructed for. + * @retval not NULL sql_stmt program pointer on success. + * @retval NULL otherwise. + */ +static struct sql_stmt * +ck_constraint_program_compile(struct ck_constraint_def *ck_constraint_def, + struct Expr *expr) +{ + struct sql *db = sql_get(); + struct Parse parser; + sql_parser_create(&parser, db, default_flags); + struct Vdbe *v = sqlGetVdbe(&parser); + if (v == NULL) { + diag_set(OutOfMemory, sizeof(struct Vdbe), "sqlGetVdbe", + "vdbe"); + return NULL; + } + /* + * Generate a prologue code that introduces variables to + * bind tuple_fetcher before execution. + */ + int tuple_fetcher_reg = sqlGetTempReg(&parser); + sqlVdbeAddOp2(v, OP_Variable, ++parser.nVar, tuple_fetcher_reg); + /* Generate ck constraint test code. */ + vdbe_emit_ck_constraint(&parser, expr, ck_constraint_def->name, + ck_constraint_def->expr_str, tuple_fetcher_reg); + + /* Clean-up and restore user-defined sql context. */ + bool is_error = parser.is_aborted; + sql_finish_coding(&parser); + sql_parser_destroy(&parser); + + if (is_error) { + diag_set(ClientError, ER_CREATE_CK_CONSTRAINT, + ck_constraint_def->name, + box_error_message(box_error_last())); + sql_finalize((struct sql_stmt *) v); + return NULL; + } + return (struct sql_stmt *) v; +} + +/** + * Run bytecode implementing check constraint with given + * tuple_fetcher instance. + * @param ck_constraint Ck constraint object to run. + * @param fetcher The initialized tuple_fetcher instance. + * @retval 0 On success, when check constraint test is passed. + * @retval -1 Otherwise. The diag message is set. + */ +static int +ck_constraint_program_run(struct ck_constraint *ck_constraint, + struct tuple_fetcher *fetcher) +{ + if (sql_bind_ptr(ck_constraint->stmt, 1, fetcher) != 0) { + diag_set(ClientError, ER_CK_CONSTRAINT_FAILED, + ck_constraint->def->name, + ck_constraint->def->expr_str); + return -1; + } + /* Checks VDBE can't expire, reset expired flag and go. */ + struct Vdbe *v = (struct Vdbe *) ck_constraint->stmt; + v->expired = 0; + sql_step(ck_constraint->stmt); + /* + * Get VDBE execution state and reset VM to run it + * next time. + */ + return sql_reset(ck_constraint->stmt) != SQL_OK ? -1 : 0; +} + +void +ck_constraint_on_replace_trigger(struct trigger *trigger, void *event) +{ + struct txn *txn = (struct txn *) event; + struct txn_stmt *stmt = txn_current_stmt(txn); + assert(stmt != NULL); + struct tuple *new_tuple = stmt->new_tuple; + if (new_tuple == NULL) + return; + + struct space *space = (struct space *) trigger->data; + uint32_t fetcher_sz = sizeof(struct tuple_fetcher) + + sizeof(uint32_t) * space->def->field_count; + struct tuple_fetcher *fetcher = region_alloc(&fiber()->gc, fetcher_sz); + if (fetcher == NULL) { + diag_set(OutOfMemory, fetcher_sz, "region_alloc", "fetcher"); + diag_raise(); + } + tuple_fetcher_create(fetcher, new_tuple, tuple_data(new_tuple), + new_tuple->bsize); + + struct ck_constraint *ck_constraint; + rlist_foreach_entry(ck_constraint, &space->ck_constraint, link) { + if (ck_constraint_program_run(ck_constraint, fetcher) != 0) + diag_raise(); + } +} + struct ck_constraint * ck_constraint_new(struct ck_constraint_def *ck_constraint_def, struct space_def *space_def) @@ -73,23 +188,29 @@ ck_constraint_new(struct ck_constraint_def *ck_constraint_def, return NULL; } ck_constraint->def = NULL; + ck_constraint->stmt = NULL; ck_constraint->space_id = space_def->id; rlist_create(&ck_constraint->link); - ck_constraint->expr = + struct Expr *expr = sql_expr_compile(sql_get(), ck_constraint_def->expr_str, strlen(ck_constraint_def->expr_str)); - if (ck_constraint->expr == NULL || - ck_constraint_resolve_field_names(ck_constraint->expr, - space_def) != 0) { + if (expr == NULL || + ck_constraint_resolve_field_names(expr, space_def) != 0) { diag_set(ClientError, ER_CREATE_CK_CONSTRAINT, ck_constraint_def->name, box_error_message(box_error_last())); goto error; } + ck_constraint->stmt = + ck_constraint_program_compile(ck_constraint_def, expr); + if (ck_constraint->stmt == NULL) + goto error; + sql_expr_delete(sql_get(), expr, false); ck_constraint->def = ck_constraint_def; return ck_constraint; error: + sql_expr_delete(sql_get(), expr, false); ck_constraint_delete(ck_constraint); return NULL; } @@ -97,7 +218,7 @@ error: void ck_constraint_delete(struct ck_constraint *ck_constraint) { - sql_expr_delete(sql_get(), ck_constraint->expr, false); + sql_finalize(ck_constraint->stmt); free(ck_constraint->def); TRASH(ck_constraint); free(ck_constraint); diff --git a/src/box/ck_constraint.h b/src/box/ck_constraint.h index e20203bb5..f76e2d5f4 100644 --- a/src/box/ck_constraint.h +++ b/src/box/ck_constraint.h @@ -32,6 +32,8 @@ */ #include <stdint.h> +#include "trigger.h" +#include "sql.h" #include "small/rlist.h" #if defined(__cplusplus) @@ -40,6 +42,7 @@ extern "C" { struct space; struct space_def; +struct sql_stmt; struct Expr; /** Supported languages of ck constraint. */ @@ -81,12 +84,11 @@ struct ck_constraint { /** The check constraint definition. */ struct ck_constraint_def *def; /** - * The check constraint expression AST is built for - * ck_constraint::def::expr_str with sql_expr_compile - * and resolved with sql_resolve_self_reference for - * space with space[ck_constraint::space_id] definition. + * Precompiled reusable VDBE program for processing check + * constraints and setting bad exitcode and error + * message when ck condition unsatisfied. */ - struct Expr *expr; + struct sql_stmt *stmt; /** * The id of the space this check constraint is * built for. @@ -157,6 +159,23 @@ ck_constraint_new(struct ck_constraint_def *ck_constraint_def, void ck_constraint_delete(struct ck_constraint *ck_constraint); +/** + * Ck constraint trigger function. It is expected to be executed + * in space::on_replace trigger. + * + * It performs all ck constraints defined for a given space + * running the precompiled bytecode to test a new tuple + * before it will be inserted in destination space. + * The trigger data stores space identifier instead of space + * pointer to make ck constraint independent of specific space + * object version. + * + * Raises an exception when some ck constraint is unsatisfied. + * The diag message is set. + */ +void +ck_constraint_on_replace_trigger(struct trigger *trigger, void *event); + /** * Find check constraint object in space by given name and * name_len. diff --git a/src/box/errcode.h b/src/box/errcode.h index 1f7c81693..e2ec240d2 100644 --- a/src/box/errcode.h +++ b/src/box/errcode.h @@ -248,6 +248,7 @@ struct errcode_record { /*193 */_(ER_CK_DEF_UNSUPPORTED, "%s are prohibited in a ck constraint definition") \ /*194 */_(ER_MULTIKEY_INDEX_MISMATCH, "Field %s is used as multikey in one index and as single key in another") \ /*195 */_(ER_CREATE_CK_CONSTRAINT, "Failed to create check constraint '%s': %s") \ + /*196 */_(ER_CK_CONSTRAINT_FAILED, "Check constraint failed '%s': %s") \ /* * !IMPORTANT! Please follow instructions at start of the file diff --git a/src/box/space.c b/src/box/space.c index a42b3a64b..abf2b3d6b 100644 --- a/src/box/space.c +++ b/src/box/space.c @@ -43,6 +43,7 @@ #include "xrow.h" #include "iproto_constants.h" #include "schema.h" +#include "ck_constraint.h" int access_check_space(struct space *space, user_access_t access) @@ -166,6 +167,8 @@ space_create(struct space *space, struct engine *engine, rlist_create(&space->parent_fk_constraint); rlist_create(&space->child_fk_constraint); rlist_create(&space->ck_constraint); + trigger_create(&space->ck_constraint_trigger, + ck_constraint_on_replace_trigger, space, NULL); return 0; fail_free_indexes: @@ -207,6 +210,7 @@ space_new_ephemeral(struct space_def *def, struct rlist *key_list) void space_delete(struct space *space) { + assert(rlist_empty(&space->ck_constraint_trigger.link)); for (uint32_t j = 0; j <= space->index_id_max; j++) { struct index *index = space->index_map[j]; if (index != NULL) diff --git a/src/box/space.h b/src/box/space.h index 823f291ce..e1246953b 100644 --- a/src/box/space.h +++ b/src/box/space.h @@ -37,6 +37,7 @@ #include "index.h" #include "error.h" #include "diag.h" +#include "trigger.h" #if defined(__cplusplus) extern "C" { @@ -203,6 +204,8 @@ struct space { * ck_constraint::link. */ struct rlist ck_constraint; + /** Trigger that performs ck constraint validation. */ + struct trigger ck_constraint_trigger; /** * Lists of foreign key constraints. In SQL terms child * space is the "from" table i.e. the table that contains diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c index 017b36fb4..7f13098ed 100644 --- a/src/box/sql/expr.c +++ b/src/box/sql/expr.c @@ -3730,16 +3730,23 @@ sqlExprCodeTarget(Parse * pParse, Expr * pExpr, int target) case TK_COLUMN:{ int iTab = pExpr->iTable; int col = pExpr->iColumn; + if (pParse->tuple_fetcher_reg > 0) { + /* Generating CHECK constraints. */ + assert(iTab < 0); + sqlVdbeAddOp3(v, OP_Fetch, + pParse->tuple_fetcher_reg, + col, target); + return target; + } if (iTab < 0) { - if (pParse->ckBase > 0) { - /* Generating CHECK constraints. */ - return col + pParse->ckBase; - } else { - /* Coding an expression that is part of an index where column names - * in the index refer to the table to which the index belongs - */ - iTab = pParse->iSelfTab; - } + /* + * Coding an expression that is + * a part of an index where column + * names in the index refer to + * the table to which the + * index belongs. + */ + iTab = pParse->iSelfTab; } return sqlExprCodeGetColumn(pParse, pExpr->space_def, col, diff --git a/src/box/sql/insert.c b/src/box/sql/insert.c index b28fe5760..a390ed541 100644 --- a/src/box/sql/insert.c +++ b/src/box/sql/insert.c @@ -812,51 +812,26 @@ sqlInsert(Parse * pParse, /* Parser context */ sqlDbFree(db, aRegIdx); } -/* - * Meanings of bits in of pWalker->eCode for checkConstraintUnchanged() - */ -#define CKCNSTRNT_COLUMN 0x01 /* CHECK constraint uses a changing column */ - -/* This is the Walker callback from checkConstraintUnchanged(). Set - * bit 0x01 of pWalker->eCode if - * pWalker->eCode to 0 if this expression node references any of the - * columns that are being modifed by an UPDATE statement. - */ -static int -checkConstraintExprNode(Walker * pWalker, Expr * pExpr) -{ - if (pExpr->op == TK_COLUMN) { - assert(pExpr->iColumn >= 0 || pExpr->iColumn == -1); - if (pExpr->iColumn >= 0) { - if (pWalker->u.aiCol[pExpr->iColumn] >= 0) { - pWalker->eCode |= CKCNSTRNT_COLUMN; - } - } - } - return WRC_Continue; -} - -/* - * pExpr is a CHECK constraint on a row that is being UPDATE-ed. The - * only columns that are modified by the UPDATE are those for which - * aiChng[i]>=0. - * - * Return true if CHECK constraint pExpr does not use any of the - * changing columns. In other words, return true if this CHECK constraint - * can be skipped when validating the new row in the UPDATE statement. - */ -static int -checkConstraintUnchanged(Expr * pExpr, int *aiChng) +void +vdbe_emit_ck_constraint(struct Parse *parser, struct Expr *expr, + const char *name, const char *expr_str, + int tuple_fetcher_reg) { - Walker w; - memset(&w, 0, sizeof(w)); - w.eCode = 0; - w.xExprCallback = checkConstraintExprNode; - w.u.aiCol = aiChng; - sqlWalkExpr(&w, pExpr); - testcase(w.eCode == 0); - testcase(w.eCode == CKCNSTRNT_COLUMN); - return !w.eCode; + parser->tuple_fetcher_reg = tuple_fetcher_reg; + struct Vdbe *v = sqlGetVdbe(parser); + const char *ck_constraint_name = sqlDbStrDup(parser->db, name); + VdbeNoopComment((v, "BEGIN: ck constraint %s test", + ck_constraint_name)); + int check_is_passed = sqlVdbeMakeLabel(v); + sqlExprIfTrue(parser, expr, check_is_passed, SQL_JUMPIFNULL); + sqlMayAbort(parser); + const char *fmt = tnt_errcode_desc(ER_CK_CONSTRAINT_FAILED); + const char *error_msg = tt_sprintf(fmt, ck_constraint_name, expr_str); + sqlVdbeAddOp4(v, OP_Halt, SQL_TARANTOOL_ERROR, ON_CONFLICT_ACTION_ABORT, + 0, sqlDbStrDup(parser->db, error_msg), P4_DYNAMIC); + sqlVdbeChangeP5(v, ER_CK_CONSTRAINT_FAILED); + VdbeNoopComment((v, "END: ck constraint %s test", ck_constraint_name)); + sqlVdbeResolveLabel(v, check_is_passed); } void @@ -926,35 +901,6 @@ vdbe_emit_constraint_checks(struct Parse *parse_context, struct space *space, unreachable(); } } - /* - * For CHECK constraint and for INSERT/UPDATE conflict - * action DEFAULT and ABORT in fact has the same meaning. - */ - if (on_conflict == ON_CONFLICT_ACTION_DEFAULT) - on_conflict = ON_CONFLICT_ACTION_ABORT; - /* Test all CHECK constraints. */ - enum on_conflict_action on_conflict_check = on_conflict; - if (on_conflict == ON_CONFLICT_ACTION_REPLACE) - on_conflict_check = ON_CONFLICT_ACTION_ABORT; - if (!rlist_empty(&space->ck_constraint)) - parse_context->ckBase = new_tuple_reg; - struct ck_constraint *ck_constraint; - rlist_foreach_entry(ck_constraint, &space->ck_constraint, link) { - struct Expr *expr = ck_constraint->expr; - if (is_update && checkConstraintUnchanged(expr, upd_cols) != 0) - continue; - int all_ok = sqlVdbeMakeLabel(v); - sqlExprIfTrue(parse_context, expr, all_ok, SQL_JUMPIFNULL); - if (on_conflict == ON_CONFLICT_ACTION_IGNORE) { - sqlVdbeGoto(v, ignore_label); - } else { - char *name = ck_constraint->def->name; - sqlHaltConstraint(parse_context, SQL_CONSTRAINT_CHECK, - on_conflict_check, name, P4_TRANSIENT, - P5_ConstraintCheck); - } - sqlVdbeResolveLabel(v, all_ok); - } sql_emit_table_types(v, space->def, new_tuple_reg); /* * Other actions except for REPLACE and UPDATE OR IGNORE diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h index abb7b9c2a..d602116da 100644 --- a/src/box/sql/sqlInt.h +++ b/src/box/sql/sqlInt.h @@ -605,6 +605,17 @@ sql_column_value(sql_stmt *, int sql_finalize(sql_stmt * pStmt); +/* + * Terminate the current execution of an SQL statement and reset + * it back to its starting state so that it can be reused. + * + * @param stmt VDBE program, may be NULL. + * @retval SQL_OK On success. + * @retval sql_ret_code Error code on error. + */ +int +sql_reset(struct sql_stmt *stmt); + int sql_exec(sql *, /* An open database */ const char *sql, /* SQL to be evaluated */ @@ -639,7 +650,6 @@ sql_exec(sql *, /* An open database */ #define SQL_IOERR_GETTEMPPATH (SQL_IOERR | (25<<8)) #define SQL_IOERR_CONVPATH (SQL_IOERR | (26<<8)) #define SQL_IOERR_VNODE (SQL_IOERR | (27<<8)) -#define SQL_CONSTRAINT_CHECK (SQL_CONSTRAINT | (1<<8)) #define SQL_CONSTRAINT_FOREIGNKEY (SQL_CONSTRAINT | (3<<8)) #define SQL_CONSTRAINT_FUNCTION (SQL_CONSTRAINT | (4<<8)) #define SQL_CONSTRAINT_NOTNULL (SQL_CONSTRAINT | (5<<8)) @@ -944,7 +954,6 @@ sql_init_db(sql **db); int sql_close(sql *); - /** * Get number of the named parameter in the prepared sql * statement. @@ -2610,7 +2619,11 @@ struct Parse { int nMem; /* Number of memory cells used so far */ int nOpAlloc; /* Number of slots allocated for Vdbe.aOp[] */ int szOpAlloc; /* Bytes of memory space allocated for Vdbe.aOp[] */ - int ckBase; /* Base register of data during check constraints */ + /* + * The register with tuple_fetcher to generate an + * alternative Vdbe code (during check constraints). + */ + int tuple_fetcher_reg; int iSelfTab; /* Table of an index whose exprs are being coded */ int iCacheLevel; /* ColCache valid when aColCache[].iLevel<=iCacheLevel */ int iCacheCnt; /* Counter used to generate aColCache[].lru values */ @@ -3907,6 +3920,23 @@ vdbe_emit_constraint_checks(struct Parse *parse_context, enum on_conflict_action on_conflict, int ignore_label, int *upd_cols); +/** + * Gnerate code to make check constraints tests on tuple insertion + * on INSERT, REPLACE or UPDATE operations. + * @param parser Current parsing context. + * @param expr Check constraint AST. + * @param expr_str Ck constraint expression source string to + * raise an informative error. + * @param name Check constraint name to raise an informative + * error. + * @param tuple_fetcher_reg The VDBE register with prepared + * tuple_fetcher pointer inside is + * initialized with a tuple to be inserted. + */ +void +vdbe_emit_ck_constraint(struct Parse *parser, struct Expr *expr, + const char *name, const char *expr_str, + int tuple_fetcher_reg); /** * This routine generates code to finish the INSERT or UPDATE * operation that was started by a prior call to diff --git a/src/box/sql/vdbe.h b/src/box/sql/vdbe.h index f9bb96f09..09ea8935b 100644 --- a/src/box/sql/vdbe.h +++ b/src/box/sql/vdbe.h @@ -140,7 +140,6 @@ struct SubProgram { /* Error message codes for OP_Halt */ #define P5_ConstraintNotNull 1 #define P5_ConstraintUnique 2 -#define P5_ConstraintCheck 3 #define P5_ConstraintFK 4 /* diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c index 198c2a9d7..d15677df6 100644 --- a/src/box/sql/vdbeapi.c +++ b/src/box/sql/vdbeapi.c @@ -132,14 +132,6 @@ sql_finalize(sql_stmt * pStmt) return rc; } -/* - * Terminate the current execution of an SQL statement and reset it - * back to its starting state so that it can be reused. A success code from - * the prior execution is returned. - * - * This routine sets the error code and string returned by - * sql_errcode(), sql_errmsg() and sql_errmsg16(). - */ int sql_reset(sql_stmt * pStmt) { diff --git a/test/box/misc.result b/test/box/misc.result index 5bf419d4f..33e41b55e 100644 --- a/test/box/misc.result +++ b/test/box/misc.result @@ -524,6 +524,7 @@ t; 193: box.error.CK_DEF_UNSUPPORTED 194: box.error.MULTIKEY_INDEX_MISMATCH 195: box.error.CREATE_CK_CONSTRAINT + 196: box.error.CK_CONSTRAINT_FAILED ... test_run:cmd("setopt delimiter ''"); --- diff --git a/test/sql-tap/check.test.lua b/test/sql-tap/check.test.lua index ede77c630..e1334f435 100755 --- a/test/sql-tap/check.test.lua +++ b/test/sql-tap/check.test.lua @@ -55,7 +55,7 @@ test:do_catchsql_test( INSERT INTO t1 VALUES(6,7, 2); ]], { -- <check-1.3> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" + 1, "Failed to execute SQL statement: Check constraint failed 'CK_CONSTRAINT_1_T1': x<5" -- </check-1.3> }) @@ -75,7 +75,7 @@ test:do_catchsql_test( INSERT INTO t1 VALUES(4,3, 2); ]], { -- <check-1.5> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_2_T1" + 1, "Failed to execute SQL statement: Check constraint failed 'CK_CONSTRAINT_2_T1': y>x" -- </check-1.5> }) @@ -147,7 +147,7 @@ test:do_catchsql_test( UPDATE t1 SET x=7 WHERE x==2 ]], { -- <check-1.12> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" + 1, "Check constraint failed 'CK_CONSTRAINT_1_T1': x<5" -- </check-1.12> }) @@ -167,7 +167,7 @@ test:do_catchsql_test( UPDATE t1 SET x=5 WHERE x==2 ]], { -- <check-1.14> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" + 1, "Check constraint failed 'CK_CONSTRAINT_1_T1': x<5" -- </check-1.14> }) @@ -206,9 +206,9 @@ test:do_execsql_test( [[ CREATE TABLE t2( id INT primary key, - x INTEGER CONSTRAINT one CHECK( typeof(coalesce(x,0))=='integer'), + x SCALAR CONSTRAINT one CHECK( typeof(coalesce(x,0))=='integer'), y REAL CONSTRAINT two CHECK( typeof(coalesce(y,0.1))=='number' ), - z TEXT CONSTRAINT three CHECK( typeof(coalesce(z,''))=='string' ) + z SCALAR CONSTRAINT three CHECK( typeof(coalesce(z,''))=='string' ) ); ]], { -- <check-2.1> @@ -246,7 +246,7 @@ test:do_catchsql_test( INSERT INTO t2 VALUES(3, 1.1, NULL, NULL); ]], { -- <check-2.4> - 1, "Failed to execute SQL statement: CHECK constraint failed: ONE" + 1, "Failed to execute SQL statement: Check constraint failed 'ONE': typeof(coalesce(x,0))=='integer'" -- </check-2.4> }) @@ -256,7 +256,7 @@ test:do_catchsql_test( INSERT INTO t2 VALUES(4, NULL, 5, NULL); ]], { -- <check-2.5> - 1, "Failed to execute SQL statement: CHECK constraint failed: TWO" + 1, "Failed to execute SQL statement: Check constraint failed 'TWO': typeof(coalesce(y,0.1))=='number'" -- </check-2.5> }) @@ -266,7 +266,7 @@ test:do_catchsql_test( INSERT INTO t2 VALUES(5, NULL, NULL, 3.14159); ]], { -- <check-2.6> - 1, "Failed to execute SQL statement: CHECK constraint failed: THREE" + 1, "Failed to execute SQL statement: Check constraint failed 'THREE': typeof(coalesce(z,''))=='string'" -- </check-2.6> }) @@ -413,7 +413,7 @@ test:do_catchsql_test( INSERT INTO t3 VALUES(111,222,333); ]], { -- <check-3.9> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T3" + 1, "Failed to execute SQL statement: Check constraint failed 'CK_CONSTRAINT_1_T3': t3.x<25" -- </check-3.9> }) @@ -484,7 +484,7 @@ test:do_catchsql_test( UPDATE t4 SET x=0, y=1; ]], { -- <check-4.6> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T4" + 1, "Check constraint failed 'CK_CONSTRAINT_1_T4': x+y==11 OR x*y==12 OR x/y BETWEEN 5 AND 8 OR -x==y+10" -- </check-4.6> }) @@ -504,7 +504,7 @@ test:do_catchsql_test( UPDATE t4 SET x=0, y=2; ]], { -- <check-4.9> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T4" + 1, "Check constraint failed 'CK_CONSTRAINT_1_T4': x+y==11 OR x*y==12 OR x/y BETWEEN 5 AND 8 OR -x==y+10" -- </check-4.9> }) @@ -581,7 +581,7 @@ test:do_catchsql_test( UPDATE OR FAIL t1 SET x=7-x, y=y+1; ]], { -- <check-6.5> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" + 1, "Check constraint failed 'CK_CONSTRAINT_1_T1': x<5" -- </check-6.5> }) @@ -603,7 +603,7 @@ test:do_catchsql_test( INSERT OR ROLLBACK INTO t1 VALUES(8,40.0, 10); ]], { -- <check-6.7> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" + 1, "Failed to execute SQL statement: Check constraint failed 'CK_CONSTRAINT_1_T1': x<5" -- </check-6.7> }) @@ -636,7 +636,7 @@ test:do_catchsql_test( REPLACE INTO t1 VALUES(6,7, 11); ]], { -- <check-6.12> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" + 1, "Failed to execute SQL statement: Check constraint failed 'CK_CONSTRAINT_1_T1': x<5" -- </check-6.12> }) @@ -700,7 +700,7 @@ test:do_catchsql_test( 7.3, " INSERT INTO t6 VALUES(11) ", { -- <7.3> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T6" + 1, "Failed to execute SQL statement: Check constraint failed 'CK_CONSTRAINT_1_T6': myfunc(a)" -- </7.3> }) diff --git a/test/sql-tap/fkey2.test.lua b/test/sql-tap/fkey2.test.lua index 695a379a6..def5f8321 100755 --- a/test/sql-tap/fkey2.test.lua +++ b/test/sql-tap/fkey2.test.lua @@ -362,7 +362,7 @@ test:do_catchsql_test( UPDATE ab SET a = 5; ]], { -- <fkey2-3.2> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_EF" + 1, "Check constraint failed 'CK_CONSTRAINT_1_EF': e!=5" -- </fkey2-3.2> }) @@ -382,7 +382,7 @@ test:do_catchsql_test( UPDATE ab SET a = 5; ]], { -- <fkey2-3.4> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_EF" + 1, "Check constraint failed 'CK_CONSTRAINT_1_EF': e!=5" -- </fkey2-3.4> }) diff --git a/test/sql-tap/table.test.lua b/test/sql-tap/table.test.lua index 066662f33..19df1d5df 100755 --- a/test/sql-tap/table.test.lua +++ b/test/sql-tap/table.test.lua @@ -1218,20 +1218,20 @@ test:do_catchsql_test( test:do_catchsql_test( "table-21.3", [[ - INSERT INTO T21 VALUES(1, -1, 1); + INSERT INTO T21 VALUES(2, -1, 1); ]], { -- <table-21.3> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T21" + 1, "Failed to execute SQL statement: Check constraint failed 'CK_CONSTRAINT_1_T21': B > 0" -- </table-21.3> }) test:do_catchsql_test( "table-21.4", [[ - INSERT INTO T21 VALUES(1, 1, -1); + INSERT INTO T21 VALUES(2, 1, -1); ]], { -- <table-21.4> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_2_T21" + 1, "Failed to execute SQL statement: Check constraint failed 'CK_CONSTRAINT_2_T21': C > 0" -- </table-21.4> }) @@ -1372,7 +1372,7 @@ test:do_catchsql_test( INSERT INTO T28 VALUES(0); ]], { -- <table-22.10> - 1, "Failed to execute SQL statement: CHECK constraint failed: CHECK1" + 1, "Failed to execute SQL statement: Check constraint failed 'CHECK1': id != 0" -- </table-22.10> }) @@ -1382,7 +1382,7 @@ test:do_catchsql_test( INSERT INTO T28 VALUES(9); ]], { -- <table-22.11> - 1, "Failed to execute SQL statement: CHECK constraint failed: CHECK2" + 1, "Failed to execute SQL statement: Check constraint failed 'CHECK2': id > 10" -- </table-22.11> }) diff --git a/test/sql/checks.result b/test/sql/checks.result index 5f0098e23..3973d242f 100644 --- a/test/sql/checks.result +++ b/test/sql/checks.result @@ -73,7 +73,12 @@ box.space._ck_constraint:count({}) ... box.execute("INSERT INTO \"test\" VALUES(5);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_01' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_01'': + X<5' +... +box.space.test:insert({5}) +--- +- error: 'Check constraint failed ''CK_CONSTRAINT_01'': X<5' ... box.space._ck_constraint:replace({513, 'CK_CONSTRAINT_01', false, 'SQL', 'X<=5'}) --- @@ -85,7 +90,12 @@ box.execute("INSERT INTO \"test\" VALUES(5);") ... box.execute("INSERT INTO \"test\" VALUES(6);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_01' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_01'': + X<=5' +... +box.space.test:insert({6}) +--- +- error: 'Check constraint failed ''CK_CONSTRAINT_01'': X<=5' ... -- Can't drop table with check constraints. box.space.test:delete({5}) @@ -118,16 +128,28 @@ box.space._ck_constraint:count() ... box.execute("INSERT INTO t1 VALUES (7, 1, 1)") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: ONE' +- error: 'Failed to execute SQL statement: Check constraint failed ''ONE'': x<5' +... +box.space.T1:insert({7, 1, 1}) +--- +- error: 'Check constraint failed ''ONE'': x<5' ... box.execute("INSERT INTO t1 VALUES (2, 1, 1)") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: TWO' +- error: 'Failed to execute SQL statement: Check constraint failed ''TWO'': y>x' +... +box.space.T1:insert({2, 1, 1}) +--- +- error: 'Check constraint failed ''TWO'': y>x' ... box.execute("INSERT INTO t1 VALUES (2, 4, 1)") --- - row_count: 1 ... +box.space.T1:update({1}, {{'+', 1, 5}}) +--- +- error: 'Check constraint failed ''ONE'': x<5' +... box.execute("DROP TABLE t1") --- - row_count: 1 @@ -164,14 +186,14 @@ _ = box.space._ck_constraint:insert({s.id, 'physics', false, 'SQL', 'X<Y'}) ... box.execute("INSERT INTO \"test\" VALUES(2, 1);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: physics' +- error: 'Failed to execute SQL statement: Check constraint failed ''physics'': X<Y' ... s:format({{name='Y', type='integer'}, {name='X', type='integer'}}) --- ... box.execute("INSERT INTO \"test\" VALUES(1, 2);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: physics' +- error: 'Failed to execute SQL statement: Check constraint failed ''physics'': X<Y' ... box.execute("INSERT INTO \"test\" VALUES(2, 1);") --- @@ -182,7 +204,7 @@ s:truncate() ... box.execute("INSERT INTO \"test\" VALUES(1, 2);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: physics' +- error: 'Failed to execute SQL statement: Check constraint failed ''physics'': X<Y' ... s:format({}) --- @@ -214,11 +236,11 @@ _ = box.space._ck_constraint:insert({s.id, 'conflict', false, 'SQL', 'X>10'}) ... box.execute("INSERT INTO \"test\" VALUES(1, 2);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: conflict' +- error: 'Failed to execute SQL statement: Check constraint failed ''conflict'': X>10' ... box.execute("INSERT INTO \"test\" VALUES(11, 11);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: physics' +- error: 'Failed to execute SQL statement: Check constraint failed ''physics'': X<Y' ... box.execute("INSERT INTO \"test\" VALUES(12, 11);") --- @@ -266,11 +288,13 @@ box.space._ck_constraint:select()[1].code ... box.execute("INSERT INTO t1 VALUES('1 a')") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_1_T1'': + x LIKE ''1 a''' ... box.execute("INSERT INTO t1 VALUES('1 a')") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_1_T1'': + x LIKE ''1 a''' ... box.execute("INSERT INTO t1 VALUES('1 a')") --- @@ -280,6 +304,284 @@ box.execute("DROP TABLE t1") --- - row_count: 1 ... +-- +-- Test binding reset on new insertion +-- +s = box.schema.create_space('test', {engine = engine}) +--- +... +_ = s:create_index('pk') +--- +... +s:format({{name='X', type='any'}, {name='Y', type='integer'}, {name='Z', type='integer', is_nullable=true}}) +--- +... +ck_not_null = box.space._ck_constraint:insert({s.id, 'ZnotNULL', false, 'SQL', 'X = 1 AND Z IS NOT NULL'}) +--- +... +s:insert({2, 1}) +--- +- error: 'Check constraint failed ''ZnotNULL'': X = 1 AND Z IS NOT NULL' +... +s:insert({1, 1}) +--- +- error: 'Check constraint failed ''ZnotNULL'': X = 1 AND Z IS NOT NULL' +... +s:insert({1, 1, box.NULL}) +--- +- error: 'Check constraint failed ''ZnotNULL'': X = 1 AND Z IS NOT NULL' +... +s:insert({2, 1, 3}) +--- +- error: 'Check constraint failed ''ZnotNULL'': X = 1 AND Z IS NOT NULL' +... +s:insert({1, 1}) +--- +- error: 'Check constraint failed ''ZnotNULL'': X = 1 AND Z IS NOT NULL' +... +s:insert({1, 1, 3}) +--- +- [1, 1, 3] +... +s:drop() +--- +... +-- +-- Test ck constraint corner cases +-- +s = box.schema.create_space('test', {engine = engine}) +--- +... +_ = s:create_index('pk') +--- +... +s:format({{name='X', type='any'}, {name='Y', type='integer'}, {name='Z', type='integer', is_nullable=true}}) +--- +... +ck_not_null = box.space._ck_constraint:insert({s.id, 'ZnotNULL', false, 'SQL', 'Z IS NOT NULL'}) +--- +... +s:insert({1, 2, box.NULL}) +--- +- error: 'Check constraint failed ''ZnotNULL'': Z IS NOT NULL' +... +s:insert({1, 2}) +--- +- error: 'Check constraint failed ''ZnotNULL'': Z IS NOT NULL' +... +_ = box.space._ck_constraint:delete({s.id, 'ZnotNULL'}) +--- +... +_ = box.space._ck_constraint:insert({s.id, 'XlessY', false, 'SQL', 'X < Y and Y < Z'}) +--- +... +s:insert({'1', 2}) +--- +- error: 'Tuple field 1 type does not match one required by operation: expected unsigned' +... +s:insert({}) +--- +- error: Tuple field 1 required by space format is missing +... +s:insert({2, 1}) +--- +- error: 'Check constraint failed ''XlessY'': X < Y and Y < Z' +... +s:insert({1, 2}) +--- +- [1, 2] +... +s:insert({2, 3, 1}) +--- +- error: 'Check constraint failed ''XlessY'': X < Y and Y < Z' +... +s:insert({2, 3, 4}) +--- +- [2, 3, 4] +... +s:update({2}, {{'+', 2, 3}}) +--- +- error: 'Check constraint failed ''XlessY'': X < Y and Y < Z' +... +s:update({2}, {{'+', 2, 3}, {'+', 3, 3}}) +--- +- [2, 6, 7] +... +s:replace({2, 1, 3}) +--- +- error: 'Check constraint failed ''XlessY'': X < Y and Y < Z' +... +box.snapshot() +--- +- ok +... +s = box.space["test"] +--- +... +s:update({2}, {{'+', 2, 3}}) +--- +- error: 'Check constraint failed ''XlessY'': X < Y and Y < Z' +... +s:update({2}, {{'+', 2, 3}, {'+', 3, 3}}) +--- +- [2, 9, 10] +... +s:replace({2, 1, 3}) +--- +- error: 'Check constraint failed ''XlessY'': X < Y and Y < Z' +... +s:drop() +--- +... +-- +-- Test complex CHECK constraints. +-- +s = box.schema.create_space('test', {engine = engine}) +--- +... +s:format({{name='X', type='integer'}, {name='Y', type='integer'}, {name='Z', type='integer'}}) +--- +... +_ = s:create_index('pk', {parts = {3, 'integer'}}) +--- +... +_ = s:create_index('unique', {parts = {1, 'integer'}}) +--- +... +_ = box.space._ck_constraint:insert({s.id, 'complex1', false, 'SQL', 'x+y==11 OR x*y==12 OR x/y BETWEEN 5 AND 8 OR -x == y+10'}) +--- +... +s:insert({1, 10, 1}) +--- +- [1, 10, 1] +... +s:update({1}, {{'=', 1, 4}, {'=', 2, 3}}) +--- +- [4, 3, 1] +... +s:update({1}, {{'=', 1, 12}, {'=', 2, 2}}) +--- +- [12, 2, 1] +... +s:update({1}, {{'=', 1, 12}, {'=', 2, -22}}) +--- +- [12, -22, 1] +... +s:update({1}, {{'=', 1, 0}, {'=', 2, 1}}) +--- +- error: 'Check constraint failed ''complex1'': x+y==11 OR x*y==12 OR x/y BETWEEN + 5 AND 8 OR -x == y+10' +... +s:get({1}) +--- +- [12, -22, 1] +... +s:update({1}, {{'=', 1, 0}, {'=', 2, 2}}) +--- +- error: 'Check constraint failed ''complex1'': x+y==11 OR x*y==12 OR x/y BETWEEN + 5 AND 8 OR -x == y+10' +... +s:get({1}) +--- +- [12, -22, 1] +... +s:drop() +--- +... +s = box.schema.create_space('test', {engine = engine}) +--- +... +s:format({{name='X', type='integer'}, {name='Z', type='any'}}) +--- +... +_ = s:create_index('pk', {parts = {1, 'integer'}}) +--- +... +_ = box.space._ck_constraint:insert({s.id, 'complex2', false, 'SQL', 'typeof(coalesce(z,0))==\'integer\''}) +--- +... +s:insert({1, 'string'}) +--- +- error: 'Check constraint failed ''complex2'': typeof(coalesce(z,0))==''integer''' +... +s:insert({1, {map=true}}) +--- +- error: 'Check constraint failed ''complex2'': typeof(coalesce(z,0))==''integer''' +... +s:insert({1, {'a', 'r','r','a','y'}}) +--- +- error: 'Check constraint failed ''complex2'': typeof(coalesce(z,0))==''integer''' +... +s:insert({1, 3.14}) +--- +- error: 'Check constraint failed ''complex2'': typeof(coalesce(z,0))==''integer''' +... +s:insert({1, 666}) +--- +- [1, 666] +... +s:drop() +--- +... +-- +-- Test large tuple. +-- +s = box.schema.create_space('test') +--- +... +_ = s:create_index('pk', {parts = {1, 'integer'}}) +--- +... +format65 = {} +--- +... +test_run:cmd("setopt delimiter ';'") +--- +- true +... +for i = 1,66 do + table.insert(format65, {name='X'..i, type='integer', is_nullable = true}) +end +test_run:cmd("setopt delimiter ''"); +--- +... +s:format(format65) +--- +... +_ = box.space._ck_constraint:insert({s.id, 'X1is666andX65is666', false, 'SQL', 'X1 == 666 and X65 == 666 and X63 IS NOT NULL'}) +--- +... +s:insert(s:frommap({X1 = 1, X65 = 1})) +--- +- error: 'Check constraint failed ''X1is666andX65is666'': X1 == 666 and X65 == 666 + and X63 IS NOT NULL' +... +s:insert(s:frommap({X1 = 666, X65 = 1})) +--- +- error: 'Check constraint failed ''X1is666andX65is666'': X1 == 666 and X65 == 666 + and X63 IS NOT NULL' +... +s:insert(s:frommap({X1 = 1, X65 = 666})) +--- +- error: 'Check constraint failed ''X1is666andX65is666'': X1 == 666 and X65 == 666 + and X63 IS NOT NULL' +... +s:insert(s:frommap({X1 = 666, X65 = 666})) +--- +- error: 'Check constraint failed ''X1is666andX65is666'': X1 == 666 and X65 == 666 + and X63 IS NOT NULL' +... +s:insert(s:frommap({X1 = 666, X65 = 666, X63 = 1})) +--- +- [666, null, null, null, null, null, null, null, null, null, null, null, null, null, + null, null, null, null, null, null, null, null, null, null, null, null, null, null, + null, null, null, null, null, null, null, null, null, null, null, null, null, null, + null, null, null, null, null, null, null, null, null, null, null, null, null, null, + null, null, null, null, null, null, 1, null, 666] +... +s:drop() +--- +... test_run:cmd("clear filter") --- - true diff --git a/test/sql/checks.test.lua b/test/sql/checks.test.lua index 3892138fd..c00654f99 100644 --- a/test/sql/checks.test.lua +++ b/test/sql/checks.test.lua @@ -33,9 +33,11 @@ box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', false, 'SQL', 'X<5'}) box.space._ck_constraint:count({}) box.execute("INSERT INTO \"test\" VALUES(5);") +box.space.test:insert({5}) box.space._ck_constraint:replace({513, 'CK_CONSTRAINT_01', false, 'SQL', 'X<=5'}) box.execute("INSERT INTO \"test\" VALUES(5);") box.execute("INSERT INTO \"test\" VALUES(6);") +box.space.test:insert({6}) -- Can't drop table with check constraints. box.space.test:delete({5}) box.space.test.index.pk:drop() @@ -47,8 +49,11 @@ box.space._space:delete({513}) box.execute("CREATE TABLE t1(x INTEGER CONSTRAINT ONE CHECK( x<5 ), y REAL CONSTRAINT TWO CHECK( y>x ), z INTEGER PRIMARY KEY);") box.space._ck_constraint:count() box.execute("INSERT INTO t1 VALUES (7, 1, 1)") +box.space.T1:insert({7, 1, 1}) box.execute("INSERT INTO t1 VALUES (2, 1, 1)") +box.space.T1:insert({2, 1, 1}) box.execute("INSERT INTO t1 VALUES (2, 4, 1)") +box.space.T1:update({1}, {{'+', 1, 5}}) box.execute("DROP TABLE t1") -- Test space creation rollback on spell error in ck constraint. @@ -103,4 +108,95 @@ box.execute("INSERT INTO t1 VALUES('1 a')") box.execute("INSERT INTO t1 VALUES('1 a')") box.execute("DROP TABLE t1") +-- +-- Test binding reset on new insertion +-- +s = box.schema.create_space('test', {engine = engine}) +_ = s:create_index('pk') +s:format({{name='X', type='any'}, {name='Y', type='integer'}, {name='Z', type='integer', is_nullable=true}}) +ck_not_null = box.space._ck_constraint:insert({s.id, 'ZnotNULL', false, 'SQL', 'X = 1 AND Z IS NOT NULL'}) +s:insert({2, 1}) +s:insert({1, 1}) +s:insert({1, 1, box.NULL}) +s:insert({2, 1, 3}) +s:insert({1, 1}) +s:insert({1, 1, 3}) +s:drop() + +-- +-- Test ck constraint corner cases +-- +s = box.schema.create_space('test', {engine = engine}) +_ = s:create_index('pk') +s:format({{name='X', type='any'}, {name='Y', type='integer'}, {name='Z', type='integer', is_nullable=true}}) +ck_not_null = box.space._ck_constraint:insert({s.id, 'ZnotNULL', false, 'SQL', 'Z IS NOT NULL'}) +s:insert({1, 2, box.NULL}) +s:insert({1, 2}) +_ = box.space._ck_constraint:delete({s.id, 'ZnotNULL'}) +_ = box.space._ck_constraint:insert({s.id, 'XlessY', false, 'SQL', 'X < Y and Y < Z'}) +s:insert({'1', 2}) +s:insert({}) +s:insert({2, 1}) +s:insert({1, 2}) +s:insert({2, 3, 1}) +s:insert({2, 3, 4}) +s:update({2}, {{'+', 2, 3}}) +s:update({2}, {{'+', 2, 3}, {'+', 3, 3}}) +s:replace({2, 1, 3}) +box.snapshot() +s = box.space["test"] +s:update({2}, {{'+', 2, 3}}) +s:update({2}, {{'+', 2, 3}, {'+', 3, 3}}) +s:replace({2, 1, 3}) +s:drop() + +-- +-- Test complex CHECK constraints. +-- +s = box.schema.create_space('test', {engine = engine}) +s:format({{name='X', type='integer'}, {name='Y', type='integer'}, {name='Z', type='integer'}}) +_ = s:create_index('pk', {parts = {3, 'integer'}}) +_ = s:create_index('unique', {parts = {1, 'integer'}}) +_ = box.space._ck_constraint:insert({s.id, 'complex1', false, 'SQL', 'x+y==11 OR x*y==12 OR x/y BETWEEN 5 AND 8 OR -x == y+10'}) +s:insert({1, 10, 1}) +s:update({1}, {{'=', 1, 4}, {'=', 2, 3}}) +s:update({1}, {{'=', 1, 12}, {'=', 2, 2}}) +s:update({1}, {{'=', 1, 12}, {'=', 2, -22}}) +s:update({1}, {{'=', 1, 0}, {'=', 2, 1}}) +s:get({1}) +s:update({1}, {{'=', 1, 0}, {'=', 2, 2}}) +s:get({1}) +s:drop() + +s = box.schema.create_space('test', {engine = engine}) +s:format({{name='X', type='integer'}, {name='Z', type='any'}}) +_ = s:create_index('pk', {parts = {1, 'integer'}}) +_ = box.space._ck_constraint:insert({s.id, 'complex2', false, 'SQL', 'typeof(coalesce(z,0))==\'integer\''}) +s:insert({1, 'string'}) +s:insert({1, {map=true}}) +s:insert({1, {'a', 'r','r','a','y'}}) +s:insert({1, 3.14}) +s:insert({1, 666}) +s:drop() + +-- +-- Test large tuple. +-- +s = box.schema.create_space('test') +_ = s:create_index('pk', {parts = {1, 'integer'}}) +format65 = {} +test_run:cmd("setopt delimiter ';'") +for i = 1,66 do + table.insert(format65, {name='X'..i, type='integer', is_nullable = true}) +end +test_run:cmd("setopt delimiter ''"); +s:format(format65) +_ = box.space._ck_constraint:insert({s.id, 'X1is666andX65is666', false, 'SQL', 'X1 == 666 and X65 == 666 and X63 IS NOT NULL'}) +s:insert(s:frommap({X1 = 1, X65 = 1})) +s:insert(s:frommap({X1 = 666, X65 = 1})) +s:insert(s:frommap({X1 = 1, X65 = 666})) +s:insert(s:frommap({X1 = 666, X65 = 666})) +s:insert(s:frommap({X1 = 666, X65 = 666, X63 = 1})) +s:drop() + test_run:cmd("clear filter") diff --git a/test/sql/errinj.result b/test/sql/errinj.result index 414e3c476..28b4b5025 100644 --- a/test/sql/errinj.result +++ b/test/sql/errinj.result @@ -490,7 +490,8 @@ _ = box.space._ck_constraint:insert({s.id, 'CK_CONSTRAINT_01', false, 'SQL', 'X< ... box.execute("INSERT INTO \"test\" VALUES(5);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_01' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_01'': + X<5' ... errinj.set("ERRINJ_WAL_IO", true) --- @@ -521,7 +522,8 @@ _ = box.space._ck_constraint:delete({s.id, 'CK_CONSTRAINT_01'}) ... box.execute("INSERT INTO \"test\" VALUES(6);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_01' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_01'': + X<=5' ... errinj.set("ERRINJ_WAL_IO", false) --- @@ -557,11 +559,13 @@ _ = box.space._ck_constraint:insert({s.id, 'Xgreater10', false, 'SQL', 'X > 10'} ... box.execute("INSERT INTO \"test\" VALUES(1, 2);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: Xgreater10' +- error: 'Failed to execute SQL statement: Check constraint failed ''Xgreater10'': + X > 10' ... box.execute("INSERT INTO \"test\" VALUES(20, 10);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: XlessY' +- error: 'Failed to execute SQL statement: Check constraint failed ''XlessY'': X < + Y' ... box.execute("INSERT INTO \"test\" VALUES(20, 100);") --- @@ -584,11 +588,13 @@ errinj.set("ERRINJ_WAL_IO", false) ... box.execute("INSERT INTO \"test\" VALUES(1, 2);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: Xgreater10' +- error: 'Failed to execute SQL statement: Check constraint failed ''Xgreater10'': + X > 10' ... box.execute("INSERT INTO \"test\" VALUES(20, 10);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: XlessY' +- error: 'Failed to execute SQL statement: Check constraint failed ''XlessY'': X < + Y' ... box.execute("INSERT INTO \"test\" VALUES(20, 100);") --- diff --git a/test/sql/gh-2981-check-autoinc.result b/test/sql/gh-2981-check-autoinc.result index 7384c81e8..e57789897 100644 --- a/test/sql/gh-2981-check-autoinc.result +++ b/test/sql/gh-2981-check-autoinc.result @@ -29,7 +29,8 @@ box.execute("insert into t1 values (18, null);") ... box.execute("insert into t1(s2) values (null);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_1_T1'': + s1 <> 19' ... box.execute("insert into t2 values (18, null);") --- @@ -37,7 +38,8 @@ box.execute("insert into t2 values (18, null);") ... box.execute("insert into t2(s2) values (null);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T2' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_1_T2'': + s1 <> 19 AND s1 <> 25' ... box.execute("insert into t2 values (24, null);") --- @@ -45,7 +47,8 @@ box.execute("insert into t2 values (24, null);") ... box.execute("insert into t2(s2) values (null);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T2' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_1_T2'': + s1 <> 19 AND s1 <> 25' ... box.execute("insert into t3 values (9, null)") --- @@ -53,7 +56,8 @@ box.execute("insert into t3 values (9, null)") ... box.execute("insert into t3(s2) values (null)") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T3' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_1_T3'': + s1 < 10' ... box.execute("DROP TABLE t1") --- diff --git a/test/sql/types.result b/test/sql/types.result index 582785413..ccbdd8c50 100644 --- a/test/sql/types.result +++ b/test/sql/types.result @@ -709,7 +709,8 @@ box.execute("CREATE TABLE t1 (id INT PRIMARY KEY, a BOOLEAN CHECK (a = true));") ... box.execute("INSERT INTO t1 VALUES (1, false);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_1_T1'': + a = true' ... box.execute("INSERT INTO t1 VALUES (1, true);") --- -- 2.21.0 ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 5/6] box: run check constraint tests on space alter 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 5/6] box: run check constraint tests on space alter Kirill Shcherbatov @ 2019-05-26 12:07 ` Vladislav Shpilevoy 2019-05-31 13:45 ` Kirill Shcherbatov 0 siblings, 1 reply; 31+ messages in thread From: Vladislav Shpilevoy @ 2019-05-26 12:07 UTC (permalink / raw) To: tarantool-patches, Kirill Shcherbatov Thanks for the patch! See 9 comments below. In the commit title you somewhy said, that you run CHECKs on alter, but it is not so. CHECKs on a non-empty space disable its alteration. This patch runs CHECKs on DML, not on DDL. Additionally I just realized, that probably the standard explicitly says in which order we should run CHECKs - before ON REPLACE triggers or after? Or it does not matter? Please, investigate. If CHECKs are always first, for example, then we should append CHECKs on_replace trigger in the first place. On 23/05/2019 13:19, Kirill Shcherbatov wrote: > To perform ck constraints tests before insert or update space > operation, we use precompiled VDBE machine associated with > each ck constraint, that is executed in on_replace trigger. > Each ck constraint VDBE code consists of > 1) prologue code that maps new(or updated) tuple via binding, > 2) ck constraint code generated by CK constraint AST. > In case of ck constraint error the tuple insert/replace operation > is aborted and ck constraint error is handled as diag message. > > Needed for #3691 > --- > src/box/alter.cc | 64 ++++- > src/box/ck_constraint.c | 131 ++++++++++- > src/box/ck_constraint.h | 29 ++- > src/box/errcode.h | 1 + > src/box/space.c | 4 + > src/box/space.h | 3 + > src/box/sql/expr.c | 25 +- > src/box/sql/insert.c | 92 ++------ > src/box/sql/sqlInt.h | 36 ++- > src/box/sql/vdbe.h | 1 - > src/box/sql/vdbeapi.c | 8 - > test/box/misc.result | 1 + > test/sql-tap/check.test.lua | 32 +-- > test/sql-tap/fkey2.test.lua | 4 +- > test/sql-tap/table.test.lua | 12 +- > test/sql/checks.result | 324 +++++++++++++++++++++++++- > test/sql/checks.test.lua | 96 ++++++++ > test/sql/errinj.result | 18 +- > test/sql/gh-2981-check-autoinc.result | 12 +- > test/sql/types.result | 3 +- > 20 files changed, 744 insertions(+), 152 deletions(-) > > diff --git a/src/box/alter.cc b/src/box/alter.cc > index 463357c67..e4ded1995 100644 > --- a/src/box/alter.cc > +++ b/src/box/alter.cc > @@ -1421,6 +1421,12 @@ RebuildCkConstraints::space_swap_ck_constraint(struct space *old_space, > { > rlist_swap(&new_space->ck_constraint, &ck_constraint); > rlist_swap(&ck_constraint, &old_space->ck_constraint); > + > + trigger_clear(&old_space->ck_constraint_trigger); > + if (!rlist_empty(&new_space->ck_constraint)) { > + trigger_add(&old_space->on_replace, > + &new_space->ck_constraint_trigger); > + } 1. But it is not a swap. Before it was old_space: old_space.trigger, new_space: nil After it is old_space: new_space.trigger, new_space: nil After one another 'swap' nothing changes - you just always assign a new trigger to the old space. swap(swap(a, b)) should be equal (a, b). In your case you somehow get swap(swap(a, nil)) = (c, nil). What is more, now old_space has 2 pointers at new_space: the new_space pointer itself (via trigger.data), and &new_space.trigger (via on_replace) list. It is not ok, obviously. Probably you decided to do so, because this error is later recovered by space_swap_triggers() - it is a hack, sorry. I think, that in this function you should not even touch on_replace. Lets deal with these problems one-by-one. 1) About old_space referencing new_space via trigger.data. You don't need it. In the trigger function you obtain struct txn_stmt, which already contains struct space pointer. You do not need to assign trigger.data at all. Remove it, please. 2) About old_space referencing &new_space.trigger. Lets store in struct space a pointer at struct trigger for ck constraints. Not the trigger itself, but a pointer. It solves the problem, and allows to swap struct trigger pointers in Move and Rebuild. Also, it allows to do not allocate struct trigger when it is not used. Now it is allocated always, because is a part of struct space. You will allocate that trigger on demand, when a first CK appears. When you will have the things above done, your Move and RebuildCK will not even touch space.on_replace. They will only swap ck lists and struct trigger *ck_trigger pointer in old_ and new_space. Space.on_replace swap will be done by space_swap_triggers(). > } > > diff --git a/src/box/ck_constraint.c b/src/box/ck_constraint.c > index c74687b31..ba2df4b67 100644 > --- a/src/box/ck_constraint.c > +++ b/src/box/ck_constraint.c > @@ -57,6 +62,116 @@ ck_constraint_resolve_field_names(struct Expr *expr, > return rc; > } > > +/** > + * Create a VDBE machine for the ck constraint by a given > + * definition and an expression AST. The generated instructions > + * consist of prologue code that maps tuple_fetcher via binding > + * and ck constraint code that implements a given expression. > + * @param ck_constraint_def Check constraint definition to prepare > + * an error description. > + * @param expr Ck constraint expression AST built for a given > + * @a ck_constraint_def, see for (sql_expr_compile and > + * ck_constraint_resolve_space_def) implementation. > + * @param space_def The space definition of the space this check > + * constraint is constructed for. > + * @retval not NULL sql_stmt program pointer on success. > + * @retval NULL otherwise. > + */ > +static struct sql_stmt * > +ck_constraint_program_compile(struct ck_constraint_def *ck_constraint_def, > + struct Expr *expr) > +{ > + struct sql *db = sql_get(); > + struct Parse parser; > + sql_parser_create(&parser, db, default_flags); > + struct Vdbe *v = sqlGetVdbe(&parser); > + if (v == NULL) { > + diag_set(OutOfMemory, sizeof(struct Vdbe), "sqlGetVdbe", > + "vdbe"); 2. You did not destroy a parser. > + return NULL; > + } > + /* > + * Generate a prologue code that introduces variables to > + * bind tuple_fetcher before execution. > + */ > + int tuple_fetcher_reg = sqlGetTempReg(&parser); > + sqlVdbeAddOp2(v, OP_Variable, ++parser.nVar, tuple_fetcher_reg); > + /* Generate ck constraint test code. */ > + vdbe_emit_ck_constraint(&parser, expr, ck_constraint_def->name, > + ck_constraint_def->expr_str, tuple_fetcher_reg); > + > + /* Clean-up and restore user-defined sql context. */ > + bool is_error = parser.is_aborted; > + sql_finish_coding(&parser); > + sql_parser_destroy(&parser); > + > + if (is_error) { > + diag_set(ClientError, ER_CREATE_CK_CONSTRAINT, > + ck_constraint_def->name, > + box_error_message(box_error_last())); > + sql_finalize((struct sql_stmt *) v); > + return NULL; > + } > + return (struct sql_stmt *) v; > +} > + > +void > +ck_constraint_on_replace_trigger(struct trigger *trigger, void *event) > +{ > + struct txn *txn = (struct txn *) event; > + struct txn_stmt *stmt = txn_current_stmt(txn); > + assert(stmt != NULL); > + struct tuple *new_tuple = stmt->new_tuple; > + if (new_tuple == NULL) > + return; > + > + struct space *space = (struct space *) trigger->data; 3. Take struct space from txn_stmt and do not touch trigger.data. Otherwise the trigger object depends on space, and you can't swap them. > + uint32_t fetcher_sz = sizeof(struct tuple_fetcher) + > + sizeof(uint32_t) * space->def->field_count; > + struct tuple_fetcher *fetcher = region_alloc(&fiber()->gc, fetcher_sz); > + if (fetcher == NULL) { > + diag_set(OutOfMemory, fetcher_sz, "region_alloc", "fetcher"); > + diag_raise(); > + } > + tuple_fetcher_create(fetcher, new_tuple, tuple_data(new_tuple), > + new_tuple->bsize); > + > + struct ck_constraint *ck_constraint; > + rlist_foreach_entry(ck_constraint, &space->ck_constraint, link) { > + if (ck_constraint_program_run(ck_constraint, fetcher) != 0) > + diag_raise(); > + } 4. Perfect, this should work quite swiftly. > diff --git a/src/box/ck_constraint.h b/src/box/ck_constraint.h > index e20203bb5..f76e2d5f4 100644 > --- a/src/box/ck_constraint.h > +++ b/src/box/ck_constraint.h > @@ -32,6 +32,8 @@ > */ > > #include <stdint.h> > +#include "trigger.h" 5. You do not need that header. Just announce struct trigger. > +#include "sql.h" 6. Why do you need it? The only new function here is ck_constraint_on_replace_trigger and its declaration does not depend on SQL. The only new attribute here is struct sql_stmt, but you announced it, and it is ok. > #include "small/rlist.h" > > #if defined(__cplusplus) > diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c > index 017b36fb4..7f13098ed 100644 > --- a/src/box/sql/expr.c > +++ b/src/box/sql/expr.c > @@ -3730,16 +3730,23 @@ sqlExprCodeTarget(Parse * pParse, Expr * pExpr, int target) > case TK_COLUMN:{ > int iTab = pExpr->iTable; > int col = pExpr->iColumn; > + if (pParse->tuple_fetcher_reg > 0) { > + /* Generating CHECK constraints. */ > + assert(iTab < 0); > + sqlVdbeAddOp3(v, OP_Fetch, > + pParse->tuple_fetcher_reg, > + col, target); > + return target; > + } 7. Please, move that code into 'if (iTab < 0)'. Otherwise non-CHECK code will check two conditions - tuple_fetcher_reg and iTab. Before your patch it was checking iTab only. > if (iTab < 0) { > - if (pParse->ckBase > 0) { > - /* Generating CHECK constraints. */ > - return col + pParse->ckBase; > - } else { > - /* Coding an expression that is part of an index where column names > - * in the index refer to the table to which the index belongs > - */ > - iTab = pParse->iSelfTab; > - } > + /* > + * Coding an expression that is > + * a part of an index where column > + * names in the index refer to > + * the table to which the > + * index belongs. > + */ > + iTab = pParse->iSelfTab; > } > return sqlExprCodeGetColumn(pParse, > pExpr->space_def, col, > diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h > index abb7b9c2a..d602116da 100644 > --- a/src/box/sql/sqlInt.h > +++ b/src/box/sql/sqlInt.h > @@ -605,6 +605,17 @@ sql_column_value(sql_stmt *, > int > sql_finalize(sql_stmt * pStmt); > > +/* > + * Terminate the current execution of an SQL statement and reset > + * it back to its starting state so that it can be reused. > + * > + * @param stmt VDBE program, may be NULL. 8. Why do you allow NULL? I grepped and see that it is never NULL. > + * @retval SQL_OK On success. > + * @retval sql_ret_code Error code on error. > + */ > +int > +sql_reset(struct sql_stmt *stmt); > @@ -3907,6 +3920,23 @@ vdbe_emit_constraint_checks(struct Parse *parse_context, > enum on_conflict_action on_conflict, > int ignore_label, int *upd_cols); > > +/** > + * Gnerate code to make check constraints tests on tuple insertion > + * on INSERT, REPLACE or UPDATE operations. > + * @param parser Current parsing context. > + * @param expr Check constraint AST. > + * @param expr_str Ck constraint expression source string to > + * raise an informative error. > + * @param name Check constraint name to raise an informative > + * error. 9. Mismatching order of parameters. > + * @param tuple_fetcher_reg The VDBE register with prepared > + * tuple_fetcher pointer inside is > + * initialized with a tuple to be inserted. > + */ > +void > +vdbe_emit_ck_constraint(struct Parse *parser, struct Expr *expr, > + const char *name, const char *expr_str, > + int tuple_fetcher_reg); ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 5/6] box: run check constraint tests on space alter 2019-05-26 12:07 ` [tarantool-patches] " Vladislav Shpilevoy @ 2019-05-31 13:45 ` Kirill Shcherbatov 2019-06-03 21:15 ` Vladislav Shpilevoy 0 siblings, 1 reply; 31+ messages in thread From: Kirill Shcherbatov @ 2019-05-31 13:45 UTC (permalink / raw) To: tarantool-patches, Vladislav Shpilevoy > In the commit title you somewhy said, that you run > CHECKs on alter, but it is not so. CHECKs on a non-empty > space disable its alteration. This patch runs CHECKs on > DML, not on DDL. I've updated the commit message box: run check constraint on space insertion > Additionally I just realized, that probably the standard > explicitly says in which order we should run CHECKs - before > ON REPLACE triggers or after? Or it does not matter? Please, > investigate. If CHECKs are always first, for example, then > we should append CHECKs on_replace trigger in the first place. Standard says nothing about relative ck constraints and triggers order. There was relative discussion here a few time ago https://github.com/tarantool/tarantool/issues/2356 but about FK constraints. Let's keep it as is. > 1. But it is not a swap. Before it was > > old_space: old_space.trigger, new_space: nil > > After it is > > old_space: new_space.trigger, new_space: nil > > After one another 'swap' nothing changes - you just always > assign a new trigger to the old space. swap(swap(a, b)) > should be equal (a, b). In your case you somehow get > swap(swap(a, nil)) = (c, nil). > > What is more, now old_space has 2 pointers at new_space: > the new_space pointer itself (via trigger.data), and > &new_space.trigger (via on_replace) list. It is not ok, > obviously. Probably you decided to do so, because this error is > later recovered by space_swap_triggers() - it is a hack, sorry. > I think, that in this function you should not even touch > on_replace. > > Lets deal with these problems one-by-one. > > 1) About old_space referencing new_space via trigger.data. > You don't need it. In the trigger function you obtain > struct txn_stmt, which already contains struct space pointer. > You do not need to assign trigger.data at all. Remove it, > please. > > 2) About old_space referencing &new_space.trigger. Lets store > in struct space a pointer at struct trigger for ck constraints. > Not the trigger itself, but a pointer. > It solves the problem, and allows to swap struct trigger pointers > in Move and Rebuild. Also, it allows to do not allocate struct > trigger when it is not used. Now it is allocated always, because > is a part of struct space. You will allocate that trigger on demand, > when a first CK appears. > > When you will have the things above done, your Move and RebuildCK > will not even touch space.on_replace. They will only swap ck lists > and struct trigger *ck_trigger pointer in old_ and new_space. > > Space.on_replace swap will be done by space_swap_triggers(). Done. >> + diag_set(OutOfMemory, sizeof(struct Vdbe), "sqlGetVdbe", >> + "vdbe"); > > 2. You did not destroy a parser. You are right. Fixed. >> + struct space *space = (struct space *) trigger->data; > > 3. Take struct space from txn_stmt and do not touch trigger.data. > Otherwise the trigger object depends on space, and you can't > swap them. Ok. Done. > 4. Perfect, this should work quite swiftly. > 5. You do not need that header. Just announce struct trigger. > >> +#include "sql.h" > 6. Why do you need it? The only new function here is > ck_constraint_on_replace_trigger and its declaration does not > depend on SQL. The only new attribute here is struct sql_stmt, > but you announced it, and it is ok. Ok. Done. > 7. Please, move that code into 'if (iTab < 0)'. Otherwise > non-CHECK code will check two conditions - tuple_fetcher_reg > and iTab. Before your patch it was checking iTab only. Done. >> int >> sql_finalize(sql_stmt * pStmt); >> >> +/* >> + * Terminate the current execution of an SQL statement and reset >> + * it back to its starting state so that it can be reused. >> + * >> + * @param stmt VDBE program, may be NULL. > > 8. Why do you allow NULL? I grepped and see that it is never NULL. I didn't like to touch the routine's body at that moment. ok, refactored now. >> +/** >> + * Gnerate code to make check constraints tests on tuple insertion >> + * on INSERT, REPLACE or UPDATE operations. >> + * @param parser Current parsing context. >> + * @param expr Check constraint AST. >> + * @param expr_str Ck constraint expression source string to >> + * raise an informative error. >> + * @param name Check constraint name to raise an informative >> + * error. > > 9. Mismatching order of parameters. Fixed. ======================================================= To perform ck constraints tests before insert or update space operation, we use precompiled VDBE machine associated with each ck constraint, that is executed in on_replace trigger. Each ck constraint VDBE code consists of 1) prologue code that maps new(or updated) tuple via binding, 2) ck constraint code generated by CK constraint AST. In case of ck constraint error the tuple insert/replace operation is aborted and ck constraint error is handled as diag message. Needed for #3691 --- src/box/alter.cc | 99 +++++++- src/box/ck_constraint.c | 133 ++++++++++- src/box/ck_constraint.h | 28 ++- src/box/errcode.h | 1 + src/box/space.c | 2 + src/box/space.h | 2 + src/box/sql/expr.c | 13 +- src/box/sql/insert.c | 92 ++------ src/box/sql/sqlInt.h | 37 ++- src/box/sql/vdbe.h | 1 - src/box/sql/vdbeapi.c | 29 +-- test/box/misc.result | 1 + test/sql-tap/check.test.lua | 32 +-- test/sql-tap/fkey2.test.lua | 4 +- test/sql-tap/table.test.lua | 12 +- test/sql/checks.result | 324 +++++++++++++++++++++++++- test/sql/checks.test.lua | 96 ++++++++ test/sql/errinj.result | 18 +- test/sql/gh-2981-check-autoinc.result | 12 +- test/sql/types.result | 3 +- 20 files changed, 774 insertions(+), 165 deletions(-) diff --git a/src/box/alter.cc b/src/box/alter.cc index bedeb71cd..7a6975427 100644 --- a/src/box/alter.cc +++ b/src/box/alter.cc @@ -1438,6 +1438,8 @@ RebuildCkConstraints::space_swap_ck_constraint(struct space *old_space, { rlist_swap(&new_space->ck_constraint, &ck_constraint); rlist_swap(&ck_constraint, &old_space->ck_constraint); + SWAP(new_space->ck_constraint_trigger, + old_space->ck_constraint_trigger); } void @@ -1467,6 +1469,44 @@ RebuildCkConstraints::~RebuildCkConstraints() } } +/** + * Move CK constraints from old space to the new one. + * Unlike RebuildCkConstraints, this operation doesn't perform + * ck constraints rebuild. This may be used in scenarios where + * space format doesn't change i.e. on index alter or space trim. + */ +class MoveCkConstraints: public AlterSpaceOp +{ + void space_swap_ck_constraint(struct space *old_space, + struct space *new_space); +public: + MoveCkConstraints(struct alter_space *alter) : AlterSpaceOp(alter) {} + virtual void alter(struct alter_space *alter); + virtual void rollback(struct alter_space *alter); +}; + +void +MoveCkConstraints::space_swap_ck_constraint(struct space *old_space, + struct space *new_space) +{ + rlist_swap(&new_space->ck_constraint, + &old_space->ck_constraint); + SWAP(new_space->ck_constraint_trigger, + old_space->ck_constraint_trigger); +} + +void +MoveCkConstraints::alter(struct alter_space *alter) +{ + space_swap_ck_constraint(alter->old_space, alter->new_space); +} + +void +MoveCkConstraints::rollback(struct alter_space *alter) +{ + space_swap_ck_constraint(alter->new_space, alter->old_space); +} + /* }}} */ /** @@ -2177,7 +2217,7 @@ on_replace_dd_index(struct trigger * /* trigger */, void *event) * old space. */ alter_space_move_indexes(alter, iid + 1, old_space->index_id_max + 1); - (void) new RebuildCkConstraints(alter); + (void) new MoveCkConstraints(alter); /* Add an op to update schema_version on commit. */ (void) new UpdateSchemaVersion(alter); alter_space_do(txn, alter); @@ -2246,7 +2286,7 @@ on_replace_dd_truncate(struct trigger * /* trigger */, void *event) (void) new TruncateIndex(alter, old_index->def->iid); } - (void) new RebuildCkConstraints(alter); + (void) new MoveCkConstraints(alter); alter_space_do(txn, alter); scoped_guard.is_active = false; } @@ -4201,9 +4241,9 @@ on_replace_ck_constraint_rollback(struct trigger *trigger, void *event) { struct txn_stmt *stmt = txn_last_stmt((struct txn *) event); struct ck_constraint *ck = (struct ck_constraint *)trigger->data; - struct space *space = NULL; - if (ck != NULL) - space = space_by_id(ck->def->space_id); + assert(ck != NULL); + struct space *space = space_by_id(ck->def->space_id); + struct trigger *ck_trigger = space->ck_constraint_trigger; if (stmt->old_tuple != NULL && stmt->new_tuple == NULL) { /* Rollback DELETE check constraint. */ assert(ck != NULL); @@ -4211,6 +4251,8 @@ on_replace_ck_constraint_rollback(struct trigger *trigger, void *event) assert(space_ck_constraint_by_name(space, ck->def->name, strlen(ck->def->name)) == NULL); rlist_add_entry(&space->ck_constraint, ck, link); + if (rlist_empty(&ck_trigger->link)) + trigger_add(&space->on_replace, ck_trigger); } else if (stmt->new_tuple != NULL && stmt->old_tuple == NULL) { /* Rollback INSERT check constraint. */ assert(space != NULL); @@ -4218,6 +4260,11 @@ on_replace_ck_constraint_rollback(struct trigger *trigger, void *event) ck->def->name, strlen(ck->def->name)) != NULL); rlist_del_entry(ck, link); ck_constraint_delete(ck); + if (rlist_empty(&space->ck_constraint)) { + trigger_clear(ck_trigger); + ck_trigger->destroy(ck_trigger); + space->ck_constraint_trigger = NULL; + } } else { /* Rollback REPLACE check constraint. */ assert(space != NULL); @@ -4240,8 +4287,23 @@ on_replace_ck_constraint_commit(struct trigger *trigger, void *event) { struct txn_stmt *stmt = txn_last_stmt((struct txn *) event); struct ck_constraint *ck = (struct ck_constraint *)trigger->data; - if (stmt->old_tuple != NULL) + assert(ck != NULL); + struct space *space = space_by_id(ck->def->space_id); + assert(space != NULL); + if (stmt->old_tuple != NULL && stmt->new_tuple == NULL) { + /* Commit DELETE check constraint. */ + struct trigger *ck_trigger = space->ck_constraint_trigger; + assert(ck_trigger != NULL); + if (rlist_empty(&space->ck_constraint)) { + ck_trigger->destroy(ck_trigger); + space->ck_constraint_trigger = NULL; + ck_constraint_delete(ck); + } + } else if (stmt->old_tuple != NULL) { + /* Commit REPLACE check constraint. */ + assert(stmt->new_tuple != NULL); ck_constraint_delete(ck); + } } /** A trigger invoked on replace in the _ck_constraint space. */ @@ -4257,6 +4319,8 @@ on_replace_dd_ck_constraint(struct trigger * /* trigger*/, void *event) tuple_field_u32_xc(old_tuple != NULL ? old_tuple : new_tuple, BOX_CK_CONSTRAINT_FIELD_SPACE_ID); struct space *space = space_cache_find_xc(space_id); + struct trigger *ck_trigger = space->ck_constraint_trigger; + assert(ck_trigger == NULL || !rlist_empty(&ck_trigger->link)); struct trigger *on_rollback = txn_alter_trigger_new(on_replace_ck_constraint_rollback, NULL); struct trigger *on_commit = @@ -4273,7 +4337,7 @@ on_replace_dd_ck_constraint(struct trigger * /* trigger*/, void *event) /* Create or replace check constraint. */ struct ck_constraint_def *ck_def = ck_constraint_def_new_from_tuple(new_tuple); - auto ck_guard = make_scoped_guard([=] { free(ck_def); }); + auto ck_def_guard = make_scoped_guard([=] { free(ck_def); }); /* * FIXME: Ck constraint creation on non-empty * space is not implemented yet. @@ -4288,17 +4352,34 @@ on_replace_dd_ck_constraint(struct trigger * /* trigger*/, void *event) ck_constraint_new(ck_def, space->def); if (new_ck_constraint == NULL) diag_raise(); - ck_guard.is_active = false; + ck_def_guard.is_active = false; + auto ck_guard = make_scoped_guard([=] { + ck_constraint_delete(new_ck_constraint); }); const char *name = new_ck_constraint->def->name; struct ck_constraint *old_ck_constraint = space_ck_constraint_by_name(space, name, strlen(name)); if (old_ck_constraint != NULL) rlist_del_entry(old_ck_constraint, link); rlist_add_entry(&space->ck_constraint, new_ck_constraint, link); + if (ck_trigger == NULL) { + ck_trigger = + (struct trigger *)malloc(sizeof(*ck_trigger)); + if (ck_trigger == NULL) { + tnt_raise(OutOfMemory, sizeof(*ck_trigger), + "malloc", "ck_trigger"); + } + trigger_create(ck_trigger, + ck_constraint_on_replace_trigger, NULL, + (trigger_f0) free); + trigger_add(&space->on_replace, ck_trigger); + space->ck_constraint_trigger = ck_trigger; + } + ck_guard.is_active = false; on_commit->data = old_tuple == NULL ? new_ck_constraint : old_ck_constraint; on_rollback->data = on_commit->data; } else { + assert(ck_trigger != NULL); assert(new_tuple == NULL && old_tuple != NULL); /* Drop check constraint. */ uint32_t name_len; @@ -4310,6 +4391,8 @@ on_replace_dd_ck_constraint(struct trigger * /* trigger*/, void *event) space_ck_constraint_by_name(space, name, name_len); assert(old_ck_constraint != NULL); rlist_del_entry(old_ck_constraint, link); + if (rlist_empty(&space->ck_constraint)) + trigger_clear(ck_trigger); on_commit->data = old_ck_constraint; on_rollback->data = old_ck_constraint; } diff --git a/src/box/ck_constraint.c b/src/box/ck_constraint.c index 69b9793ea..d7935794d 100644 --- a/src/box/ck_constraint.c +++ b/src/box/ck_constraint.c @@ -29,10 +29,15 @@ * SUCH DAMAGE. */ #include "box/session.h" +#include "bind.h" #include "ck_constraint.h" #include "errcode.h" +#include "schema.h" +#include "small/region.h" #include "sql.h" #include "sql/sqlInt.h" +#include "sql/vdbeInt.h" +#include "tuple.h" const char *ck_constraint_language_strs[] = {"SQL"}; @@ -89,6 +94,118 @@ ck_constraint_resolve_field_names(struct Expr *expr, return rc; } +/** + * Create a VDBE machine for the ck constraint by a given + * definition and an expression AST. The generated instructions + * consist of prologue code that maps tuple_fetcher via binding + * and ck constraint code that implements a given expression. + * @param ck_constraint_def Check constraint definition to prepare + * an error description. + * @param expr Ck constraint expression AST built for a given + * @a ck_constraint_def, see for (sql_expr_compile and + * ck_constraint_resolve_space_def) implementation. + * @param space_def The space definition of the space this check + * constraint is constructed for. + * @retval not NULL sql_stmt program pointer on success. + * @retval NULL otherwise. + */ +static struct sql_stmt * +ck_constraint_program_compile(struct ck_constraint_def *ck_constraint_def, + struct Expr *expr) +{ + struct sql *db = sql_get(); + struct Parse parser; + sql_parser_create(&parser, db, default_flags); + struct Vdbe *v = sqlGetVdbe(&parser); + if (v == NULL) { + sql_parser_destroy(&parser); + diag_set(OutOfMemory, sizeof(struct Vdbe), "sqlGetVdbe", + "vdbe"); + return NULL; + } + /* + * Generate a prologue code that introduces variables to + * bind tuple_fetcher before execution. + */ + int tuple_fetcher_reg = sqlGetTempReg(&parser); + sqlVdbeAddOp2(v, OP_Variable, ++parser.nVar, tuple_fetcher_reg); + /* Generate ck constraint test code. */ + vdbe_emit_ck_constraint(&parser, expr, ck_constraint_def->name, + ck_constraint_def->expr_str, tuple_fetcher_reg); + + /* Clean-up and restore user-defined sql context. */ + bool is_error = parser.is_aborted; + sql_finish_coding(&parser); + sql_parser_destroy(&parser); + + if (is_error) { + diag_set(ClientError, ER_CREATE_CK_CONSTRAINT, + ck_constraint_def->name, + box_error_message(box_error_last())); + sql_finalize((struct sql_stmt *) v); + return NULL; + } + return (struct sql_stmt *) v; +} + +/** + * Run bytecode implementing check constraint with given + * tuple_fetcher instance. + * @param ck_constraint Ck constraint object to run. + * @param fetcher The initialized tuple_fetcher instance. + * @retval 0 On success, when check constraint test is passed. + * @retval -1 Otherwise. The diag message is set. + */ +static int +ck_constraint_program_run(struct ck_constraint *ck_constraint, + struct tuple_fetcher *fetcher) +{ + if (sql_bind_ptr(ck_constraint->stmt, 1, fetcher) != 0) { + diag_set(ClientError, ER_CK_CONSTRAINT_FAILED, + ck_constraint->def->name, + ck_constraint->def->expr_str); + return -1; + } + /* Checks VDBE can't expire, reset expired flag and go. */ + struct Vdbe *v = (struct Vdbe *) ck_constraint->stmt; + v->expired = 0; + sql_step(ck_constraint->stmt); + /* + * Get VDBE execution state and reset VM to run it + * next time. + */ + return sql_reset(ck_constraint->stmt) != SQL_OK ? -1 : 0; +} + +void +ck_constraint_on_replace_trigger(struct trigger *trigger, void *event) +{ + (void) trigger; + struct txn *txn = (struct txn *) event; + struct txn_stmt *stmt = txn_current_stmt(txn); + assert(stmt != NULL); + struct tuple *new_tuple = stmt->new_tuple; + if (new_tuple == NULL) + return; + + struct space *space = stmt->space; + assert(space != NULL); + uint32_t fetcher_sz = sizeof(struct tuple_fetcher) + + sizeof(uint32_t) * space->def->field_count; + struct tuple_fetcher *fetcher = region_alloc(&fiber()->gc, fetcher_sz); + if (fetcher == NULL) { + diag_set(OutOfMemory, fetcher_sz, "region_alloc", "fetcher"); + diag_raise(); + } + tuple_fetcher_prepare_tuple(fetcher, new_tuple); + + struct ck_constraint *ck_constraint; + rlist_foreach_entry(ck_constraint, &space->ck_constraint, link) { + if (ck_constraint_program_run(ck_constraint, fetcher) != 0) + diag_raise(); + } +} + struct ck_constraint * ck_constraint_new(struct ck_constraint_def *ck_constraint_def, struct space_def *space_def) @@ -105,22 +222,28 @@ ck_constraint_new(struct ck_constraint_def *ck_constraint_def, return NULL; } ck_constraint->def = NULL; + ck_constraint->stmt = NULL; rlist_create(&ck_constraint->link); - ck_constraint->expr = + struct Expr *expr = sql_expr_compile(sql_get(), ck_constraint_def->expr_str, strlen(ck_constraint_def->expr_str)); - if (ck_constraint->expr == NULL || - ck_constraint_resolve_field_names(ck_constraint->expr, - space_def) != 0) { + if (expr == NULL || + ck_constraint_resolve_field_names(expr, space_def) != 0) { diag_set(ClientError, ER_CREATE_CK_CONSTRAINT, ck_constraint_def->name, box_error_message(box_error_last())); goto error; } + ck_constraint->stmt = + ck_constraint_program_compile(ck_constraint_def, expr); + if (ck_constraint->stmt == NULL) + goto error; + sql_expr_delete(sql_get(), expr, false); ck_constraint->def = ck_constraint_def; return ck_constraint; error: + sql_expr_delete(sql_get(), expr, false); ck_constraint_delete(ck_constraint); return NULL; } @@ -128,7 +251,7 @@ error: void ck_constraint_delete(struct ck_constraint *ck_constraint) { - sql_expr_delete(sql_get(), ck_constraint->expr, false); + sql_finalize(ck_constraint->stmt); ck_constraint_def_delete(ck_constraint->def); TRASH(ck_constraint); free(ck_constraint); diff --git a/src/box/ck_constraint.h b/src/box/ck_constraint.h index 3ae3d5c91..180280d00 100644 --- a/src/box/ck_constraint.h +++ b/src/box/ck_constraint.h @@ -40,7 +40,9 @@ extern "C" { struct space; struct space_def; +struct sql_stmt; struct Expr; +struct trigger; /** Supported languages of ck constraint. */ enum ck_constraint_language { @@ -86,12 +88,11 @@ struct ck_constraint { /** The check constraint definition. */ struct ck_constraint_def *def; /** - * The check constraint expression AST is built for - * ck_constraint::def::expr_str with sql_expr_compile - * and resolved with sql_resolve_self_reference for - * space with space[ck_constraint::space_id] definition. + * Precompiled reusable VDBE program for processing check + * constraints and setting bad exitcode and error + * message when ck condition unsatisfied. */ - struct Expr *expr; + struct sql_stmt *stmt; /** * Organize check constraint structs into linked list * with space::ck_constraint. @@ -185,6 +186,23 @@ ck_constraint_new(struct ck_constraint_def *ck_constraint_def, void ck_constraint_delete(struct ck_constraint *ck_constraint); +/** + * Ck constraint trigger function. It is expected to be executed + * in space::on_replace trigger. + * + * It performs all ck constraints defined for a given space + * running the precompiled bytecode to test a new tuple + * before it will be inserted in destination space. + * The trigger data stores space identifier instead of space + * pointer to make ck constraint independent of specific space + * object version. + * + * Raises an exception when some ck constraint is unsatisfied. + * The diag message is set. + */ +void +ck_constraint_on_replace_trigger(struct trigger *trigger, void *event); + /** * Find check constraint object in space by given name and * name_len. diff --git a/src/box/errcode.h b/src/box/errcode.h index 1f7c81693..e2ec240d2 100644 --- a/src/box/errcode.h +++ b/src/box/errcode.h @@ -248,6 +248,7 @@ struct errcode_record { /*193 */_(ER_CK_DEF_UNSUPPORTED, "%s are prohibited in a ck constraint definition") \ /*194 */_(ER_MULTIKEY_INDEX_MISMATCH, "Field %s is used as multikey in one index and as single key in another") \ /*195 */_(ER_CREATE_CK_CONSTRAINT, "Failed to create check constraint '%s': %s") \ + /*196 */_(ER_CK_CONSTRAINT_FAILED, "Check constraint failed '%s': %s") \ /* * !IMPORTANT! Please follow instructions at start of the file diff --git a/src/box/space.c b/src/box/space.c index a42b3a64b..b6ad87bf7 100644 --- a/src/box/space.c +++ b/src/box/space.c @@ -43,6 +43,7 @@ #include "xrow.h" #include "iproto_constants.h" #include "schema.h" +#include "ck_constraint.h" int access_check_space(struct space *space, user_access_t access) @@ -207,6 +208,7 @@ space_new_ephemeral(struct space_def *def, struct rlist *key_list) void space_delete(struct space *space) { + assert(space->ck_constraint_trigger == NULL); for (uint32_t j = 0; j <= space->index_id_max; j++) { struct index *index = space->index_map[j]; if (index != NULL) diff --git a/src/box/space.h b/src/box/space.h index 7ffe884b3..949f37d45 100644 --- a/src/box/space.h +++ b/src/box/space.h @@ -202,6 +202,8 @@ struct space { * ck_constraint::link. */ struct rlist ck_constraint; + /** Trigger that performs ck constraint validation. */ + struct trigger *ck_constraint_trigger; /** * Lists of foreign key constraints. In SQL terms child * space is the "from" table i.e. the table that contains diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c index 017b36fb4..78f63cbfd 100644 --- a/src/box/sql/expr.c +++ b/src/box/sql/expr.c @@ -3731,9 +3731,16 @@ sqlExprCodeTarget(Parse * pParse, Expr * pExpr, int target) int iTab = pExpr->iTable; int col = pExpr->iColumn; if (iTab < 0) { - if (pParse->ckBase > 0) { - /* Generating CHECK constraints. */ - return col + pParse->ckBase; + if (pParse->tuple_fetcher_reg > 0) { + /* + * Generating CHECK + * constraints. + */ + assert(iTab < 0); + sqlVdbeAddOp3(v, OP_Fetch, + pParse->tuple_fetcher_reg, + col, target); + return target; } else { /* Coding an expression that is part of an index where column names * in the index refer to the table to which the index belongs diff --git a/src/box/sql/insert.c b/src/box/sql/insert.c index c4b629c6c..c32de2be9 100644 --- a/src/box/sql/insert.c +++ b/src/box/sql/insert.c @@ -798,51 +798,26 @@ sqlInsert(Parse * pParse, /* Parser context */ sqlDbFree(db, aRegIdx); } -/* - * Meanings of bits in of pWalker->eCode for checkConstraintUnchanged() - */ -#define CKCNSTRNT_COLUMN 0x01 /* CHECK constraint uses a changing column */ - -/* This is the Walker callback from checkConstraintUnchanged(). Set - * bit 0x01 of pWalker->eCode if - * pWalker->eCode to 0 if this expression node references any of the - * columns that are being modifed by an UPDATE statement. - */ -static int -checkConstraintExprNode(Walker * pWalker, Expr * pExpr) -{ - if (pExpr->op == TK_COLUMN) { - assert(pExpr->iColumn >= 0 || pExpr->iColumn == -1); - if (pExpr->iColumn >= 0) { - if (pWalker->u.aiCol[pExpr->iColumn] >= 0) { - pWalker->eCode |= CKCNSTRNT_COLUMN; - } - } - } - return WRC_Continue; -} - -/* - * pExpr is a CHECK constraint on a row that is being UPDATE-ed. The - * only columns that are modified by the UPDATE are those for which - * aiChng[i]>=0. - * - * Return true if CHECK constraint pExpr does not use any of the - * changing columns. In other words, return true if this CHECK constraint - * can be skipped when validating the new row in the UPDATE statement. - */ -static int -checkConstraintUnchanged(Expr * pExpr, int *aiChng) +void +vdbe_emit_ck_constraint(struct Parse *parser, struct Expr *expr, + const char *name, const char *expr_str, + int tuple_fetcher_reg) { - Walker w; - memset(&w, 0, sizeof(w)); - w.eCode = 0; - w.xExprCallback = checkConstraintExprNode; - w.u.aiCol = aiChng; - sqlWalkExpr(&w, pExpr); - testcase(w.eCode == 0); - testcase(w.eCode == CKCNSTRNT_COLUMN); - return !w.eCode; + parser->tuple_fetcher_reg = tuple_fetcher_reg; + struct Vdbe *v = sqlGetVdbe(parser); + const char *ck_constraint_name = sqlDbStrDup(parser->db, name); + VdbeNoopComment((v, "BEGIN: ck constraint %s test", + ck_constraint_name)); + int check_is_passed = sqlVdbeMakeLabel(v); + sqlExprIfTrue(parser, expr, check_is_passed, SQL_JUMPIFNULL); + sqlMayAbort(parser); + const char *fmt = tnt_errcode_desc(ER_CK_CONSTRAINT_FAILED); + const char *error_msg = tt_sprintf(fmt, ck_constraint_name, expr_str); + sqlVdbeAddOp4(v, OP_Halt, SQL_TARANTOOL_ERROR, ON_CONFLICT_ACTION_ABORT, + 0, sqlDbStrDup(parser->db, error_msg), P4_DYNAMIC); + sqlVdbeChangeP5(v, ER_CK_CONSTRAINT_FAILED); + VdbeNoopComment((v, "END: ck constraint %s test", ck_constraint_name)); + sqlVdbeResolveLabel(v, check_is_passed); } void @@ -912,35 +887,6 @@ vdbe_emit_constraint_checks(struct Parse *parse_context, struct space *space, unreachable(); } } - /* - * For CHECK constraint and for INSERT/UPDATE conflict - * action DEFAULT and ABORT in fact has the same meaning. - */ - if (on_conflict == ON_CONFLICT_ACTION_DEFAULT) - on_conflict = ON_CONFLICT_ACTION_ABORT; - /* Test all CHECK constraints. */ - enum on_conflict_action on_conflict_check = on_conflict; - if (on_conflict == ON_CONFLICT_ACTION_REPLACE) - on_conflict_check = ON_CONFLICT_ACTION_ABORT; - if (!rlist_empty(&space->ck_constraint)) - parse_context->ckBase = new_tuple_reg; - struct ck_constraint *ck_constraint; - rlist_foreach_entry(ck_constraint, &space->ck_constraint, link) { - struct Expr *expr = ck_constraint->expr; - if (is_update && checkConstraintUnchanged(expr, upd_cols) != 0) - continue; - int all_ok = sqlVdbeMakeLabel(v); - sqlExprIfTrue(parse_context, expr, all_ok, SQL_JUMPIFNULL); - if (on_conflict == ON_CONFLICT_ACTION_IGNORE) { - sqlVdbeGoto(v, ignore_label); - } else { - char *name = ck_constraint->def->name; - sqlHaltConstraint(parse_context, SQL_CONSTRAINT_CHECK, - on_conflict_check, name, P4_TRANSIENT, - P5_ConstraintCheck); - } - sqlVdbeResolveLabel(v, all_ok); - } sql_emit_table_types(v, space->def, new_tuple_reg); /* * Other actions except for REPLACE and UPDATE OR IGNORE diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h index abb7b9c2a..e2c760623 100644 --- a/src/box/sql/sqlInt.h +++ b/src/box/sql/sqlInt.h @@ -605,6 +605,17 @@ sql_column_value(sql_stmt *, int sql_finalize(sql_stmt * pStmt); +/* + * Terminate the current execution of an SQL statement and reset + * it back to its starting state so that it can be reused. + * + * @param stmt VDBE program. + * @retval SQL_OK On success. + * @retval sql_ret_code Error code on error. + */ +int +sql_reset(struct sql_stmt *stmt); + int sql_exec(sql *, /* An open database */ const char *sql, /* SQL to be evaluated */ @@ -639,7 +650,6 @@ sql_exec(sql *, /* An open database */ #define SQL_IOERR_GETTEMPPATH (SQL_IOERR | (25<<8)) #define SQL_IOERR_CONVPATH (SQL_IOERR | (26<<8)) #define SQL_IOERR_VNODE (SQL_IOERR | (27<<8)) -#define SQL_CONSTRAINT_CHECK (SQL_CONSTRAINT | (1<<8)) #define SQL_CONSTRAINT_FOREIGNKEY (SQL_CONSTRAINT | (3<<8)) #define SQL_CONSTRAINT_FUNCTION (SQL_CONSTRAINT | (4<<8)) #define SQL_CONSTRAINT_NOTNULL (SQL_CONSTRAINT | (5<<8)) @@ -944,7 +954,6 @@ sql_init_db(sql **db); int sql_close(sql *); - /** * Get number of the named parameter in the prepared sql * statement. @@ -2610,7 +2619,11 @@ struct Parse { int nMem; /* Number of memory cells used so far */ int nOpAlloc; /* Number of slots allocated for Vdbe.aOp[] */ int szOpAlloc; /* Bytes of memory space allocated for Vdbe.aOp[] */ - int ckBase; /* Base register of data during check constraints */ + /* + * The register with tuple_fetcher to generate an + * alternative Vdbe code (during check constraints). + */ + int tuple_fetcher_reg; int iSelfTab; /* Table of an index whose exprs are being coded */ int iCacheLevel; /* ColCache valid when aColCache[].iLevel<=iCacheLevel */ int iCacheCnt; /* Counter used to generate aColCache[].lru values */ @@ -3907,6 +3920,24 @@ vdbe_emit_constraint_checks(struct Parse *parse_context, enum on_conflict_action on_conflict, int ignore_label, int *upd_cols); +/** + * Gnerate code to make check constraints tests on tuple insertion + * on INSERT, REPLACE or UPDATE operations. + * @param parser Current parsing context. + * @param expr Check constraint AST. + * @param name Check constraint name to raise an informative + * error. + * @param expr_str Ck constraint expression source string to + * raise an informative error. + * @param tuple_fetcher_reg The VDBE register with prepared + * tuple_fetcher pointer inside is + * initialized with a tuple to be + * inserted. + */ +void +vdbe_emit_ck_constraint(struct Parse *parser, struct Expr *expr, + const char *name, const char *expr_str, + int tuple_fetcher_reg); /** * This routine generates code to finish the INSERT or UPDATE * operation that was started by a prior call to diff --git a/src/box/sql/vdbe.h b/src/box/sql/vdbe.h index f9bb96f09..09ea8935b 100644 --- a/src/box/sql/vdbe.h +++ b/src/box/sql/vdbe.h @@ -140,7 +140,6 @@ struct SubProgram { /* Error message codes for OP_Halt */ #define P5_ConstraintNotNull 1 #define P5_ConstraintUnique 2 -#define P5_ConstraintCheck 3 #define P5_ConstraintFK 4 /* diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c index 198c2a9d7..ace1eceb1 100644 --- a/src/box/sql/vdbeapi.c +++ b/src/box/sql/vdbeapi.c @@ -132,30 +132,17 @@ sql_finalize(sql_stmt * pStmt) return rc; } -/* - * Terminate the current execution of an SQL statement and reset it - * back to its starting state so that it can be reused. A success code from - * the prior execution is returned. - * - * This routine sets the error code and string returned by - * sql_errcode(), sql_errmsg() and sql_errmsg16(). - */ int sql_reset(sql_stmt * pStmt) { - int rc; - if (pStmt == 0) { - rc = SQL_OK; - } else { - Vdbe *v = (Vdbe *) pStmt; - sql *db = v->db; - checkProfileCallback(db, v); - rc = sqlVdbeReset(v); - sqlVdbeRewind(v); - assert((rc & (db->errMask)) == rc); - rc = sqlApiExit(db, rc); - } - return rc; + assert(pStmt != NULL); + struct Vdbe *v = (Vdbe *) pStmt; + struct sql *db = v->db; + checkProfileCallback(db, v); + int rc = sqlVdbeReset(v); + sqlVdbeRewind(v); + assert((rc & (db->errMask)) == rc); + return sqlApiExit(db, rc); } /* diff --git a/test/box/misc.result b/test/box/misc.result index 5bf419d4f..33e41b55e 100644 --- a/test/box/misc.result +++ b/test/box/misc.result @@ -524,6 +524,7 @@ t; 193: box.error.CK_DEF_UNSUPPORTED 194: box.error.MULTIKEY_INDEX_MISMATCH 195: box.error.CREATE_CK_CONSTRAINT + 196: box.error.CK_CONSTRAINT_FAILED ... test_run:cmd("setopt delimiter ''"); --- diff --git a/test/sql-tap/check.test.lua b/test/sql-tap/check.test.lua index ede77c630..e1334f435 100755 --- a/test/sql-tap/check.test.lua +++ b/test/sql-tap/check.test.lua @@ -55,7 +55,7 @@ test:do_catchsql_test( INSERT INTO t1 VALUES(6,7, 2); ]], { -- <check-1.3> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" + 1, "Failed to execute SQL statement: Check constraint failed 'CK_CONSTRAINT_1_T1': x<5" -- </check-1.3> }) @@ -75,7 +75,7 @@ test:do_catchsql_test( INSERT INTO t1 VALUES(4,3, 2); ]], { -- <check-1.5> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_2_T1" + 1, "Failed to execute SQL statement: Check constraint failed 'CK_CONSTRAINT_2_T1': y>x" -- </check-1.5> }) @@ -147,7 +147,7 @@ test:do_catchsql_test( UPDATE t1 SET x=7 WHERE x==2 ]], { -- <check-1.12> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" + 1, "Check constraint failed 'CK_CONSTRAINT_1_T1': x<5" -- </check-1.12> }) @@ -167,7 +167,7 @@ test:do_catchsql_test( UPDATE t1 SET x=5 WHERE x==2 ]], { -- <check-1.14> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" + 1, "Check constraint failed 'CK_CONSTRAINT_1_T1': x<5" -- </check-1.14> }) @@ -206,9 +206,9 @@ test:do_execsql_test( [[ CREATE TABLE t2( id INT primary key, - x INTEGER CONSTRAINT one CHECK( typeof(coalesce(x,0))=='integer'), + x SCALAR CONSTRAINT one CHECK( typeof(coalesce(x,0))=='integer'), y REAL CONSTRAINT two CHECK( typeof(coalesce(y,0.1))=='number' ), - z TEXT CONSTRAINT three CHECK( typeof(coalesce(z,''))=='string' ) + z SCALAR CONSTRAINT three CHECK( typeof(coalesce(z,''))=='string' ) ); ]], { -- <check-2.1> @@ -246,7 +246,7 @@ test:do_catchsql_test( INSERT INTO t2 VALUES(3, 1.1, NULL, NULL); ]], { -- <check-2.4> - 1, "Failed to execute SQL statement: CHECK constraint failed: ONE" + 1, "Failed to execute SQL statement: Check constraint failed 'ONE': typeof(coalesce(x,0))=='integer'" -- </check-2.4> }) @@ -256,7 +256,7 @@ test:do_catchsql_test( INSERT INTO t2 VALUES(4, NULL, 5, NULL); ]], { -- <check-2.5> - 1, "Failed to execute SQL statement: CHECK constraint failed: TWO" + 1, "Failed to execute SQL statement: Check constraint failed 'TWO': typeof(coalesce(y,0.1))=='number'" -- </check-2.5> }) @@ -266,7 +266,7 @@ test:do_catchsql_test( INSERT INTO t2 VALUES(5, NULL, NULL, 3.14159); ]], { -- <check-2.6> - 1, "Failed to execute SQL statement: CHECK constraint failed: THREE" + 1, "Failed to execute SQL statement: Check constraint failed 'THREE': typeof(coalesce(z,''))=='string'" -- </check-2.6> }) @@ -413,7 +413,7 @@ test:do_catchsql_test( INSERT INTO t3 VALUES(111,222,333); ]], { -- <check-3.9> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T3" + 1, "Failed to execute SQL statement: Check constraint failed 'CK_CONSTRAINT_1_T3': t3.x<25" -- </check-3.9> }) @@ -484,7 +484,7 @@ test:do_catchsql_test( UPDATE t4 SET x=0, y=1; ]], { -- <check-4.6> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T4" + 1, "Check constraint failed 'CK_CONSTRAINT_1_T4': x+y==11 OR x*y==12 OR x/y BETWEEN 5 AND 8 OR -x==y+10" -- </check-4.6> }) @@ -504,7 +504,7 @@ test:do_catchsql_test( UPDATE t4 SET x=0, y=2; ]], { -- <check-4.9> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T4" + 1, "Check constraint failed 'CK_CONSTRAINT_1_T4': x+y==11 OR x*y==12 OR x/y BETWEEN 5 AND 8 OR -x==y+10" -- </check-4.9> }) @@ -581,7 +581,7 @@ test:do_catchsql_test( UPDATE OR FAIL t1 SET x=7-x, y=y+1; ]], { -- <check-6.5> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" + 1, "Check constraint failed 'CK_CONSTRAINT_1_T1': x<5" -- </check-6.5> }) @@ -603,7 +603,7 @@ test:do_catchsql_test( INSERT OR ROLLBACK INTO t1 VALUES(8,40.0, 10); ]], { -- <check-6.7> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" + 1, "Failed to execute SQL statement: Check constraint failed 'CK_CONSTRAINT_1_T1': x<5" -- </check-6.7> }) @@ -636,7 +636,7 @@ test:do_catchsql_test( REPLACE INTO t1 VALUES(6,7, 11); ]], { -- <check-6.12> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1" + 1, "Failed to execute SQL statement: Check constraint failed 'CK_CONSTRAINT_1_T1': x<5" -- </check-6.12> }) @@ -700,7 +700,7 @@ test:do_catchsql_test( 7.3, " INSERT INTO t6 VALUES(11) ", { -- <7.3> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T6" + 1, "Failed to execute SQL statement: Check constraint failed 'CK_CONSTRAINT_1_T6': myfunc(a)" -- </7.3> }) diff --git a/test/sql-tap/fkey2.test.lua b/test/sql-tap/fkey2.test.lua index 695a379a6..def5f8321 100755 --- a/test/sql-tap/fkey2.test.lua +++ b/test/sql-tap/fkey2.test.lua @@ -362,7 +362,7 @@ test:do_catchsql_test( UPDATE ab SET a = 5; ]], { -- <fkey2-3.2> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_EF" + 1, "Check constraint failed 'CK_CONSTRAINT_1_EF': e!=5" -- </fkey2-3.2> }) @@ -382,7 +382,7 @@ test:do_catchsql_test( UPDATE ab SET a = 5; ]], { -- <fkey2-3.4> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_EF" + 1, "Check constraint failed 'CK_CONSTRAINT_1_EF': e!=5" -- </fkey2-3.4> }) diff --git a/test/sql-tap/table.test.lua b/test/sql-tap/table.test.lua index 066662f33..19df1d5df 100755 --- a/test/sql-tap/table.test.lua +++ b/test/sql-tap/table.test.lua @@ -1218,20 +1218,20 @@ test:do_catchsql_test( test:do_catchsql_test( "table-21.3", [[ - INSERT INTO T21 VALUES(1, -1, 1); + INSERT INTO T21 VALUES(2, -1, 1); ]], { -- <table-21.3> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T21" + 1, "Failed to execute SQL statement: Check constraint failed 'CK_CONSTRAINT_1_T21': B > 0" -- </table-21.3> }) test:do_catchsql_test( "table-21.4", [[ - INSERT INTO T21 VALUES(1, 1, -1); + INSERT INTO T21 VALUES(2, 1, -1); ]], { -- <table-21.4> - 1, "Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_2_T21" + 1, "Failed to execute SQL statement: Check constraint failed 'CK_CONSTRAINT_2_T21': C > 0" -- </table-21.4> }) @@ -1372,7 +1372,7 @@ test:do_catchsql_test( INSERT INTO T28 VALUES(0); ]], { -- <table-22.10> - 1, "Failed to execute SQL statement: CHECK constraint failed: CHECK1" + 1, "Failed to execute SQL statement: Check constraint failed 'CHECK1': id != 0" -- </table-22.10> }) @@ -1382,7 +1382,7 @@ test:do_catchsql_test( INSERT INTO T28 VALUES(9); ]], { -- <table-22.11> - 1, "Failed to execute SQL statement: CHECK constraint failed: CHECK2" + 1, "Failed to execute SQL statement: Check constraint failed 'CHECK2': id > 10" -- </table-22.11> }) diff --git a/test/sql/checks.result b/test/sql/checks.result index efe626428..6272cea7a 100644 --- a/test/sql/checks.result +++ b/test/sql/checks.result @@ -73,7 +73,12 @@ box.space._ck_constraint:count({}) ... box.execute("INSERT INTO \"test\" VALUES(5);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_01' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_01'': + X<5' +... +box.space.test:insert({5}) +--- +- error: 'Check constraint failed ''CK_CONSTRAINT_01'': X<5' ... box.space._ck_constraint:replace({513, 'CK_CONSTRAINT_01', false, 'SQL', 'X<=5'}) --- @@ -85,7 +90,12 @@ box.execute("INSERT INTO \"test\" VALUES(5);") ... box.execute("INSERT INTO \"test\" VALUES(6);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_01' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_01'': + X<=5' +... +box.space.test:insert({6}) +--- +- error: 'Check constraint failed ''CK_CONSTRAINT_01'': X<=5' ... -- Can't drop table with check constraints. box.space.test:delete({5}) @@ -118,16 +128,28 @@ box.space._ck_constraint:count() ... box.execute("INSERT INTO t1 VALUES (7, 1, 1)") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: ONE' +- error: 'Failed to execute SQL statement: Check constraint failed ''ONE'': x<5' +... +box.space.T1:insert({7, 1, 1}) +--- +- error: 'Check constraint failed ''ONE'': x<5' ... box.execute("INSERT INTO t1 VALUES (2, 1, 1)") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: TWO' +- error: 'Failed to execute SQL statement: Check constraint failed ''TWO'': y>x' +... +box.space.T1:insert({2, 1, 1}) +--- +- error: 'Check constraint failed ''TWO'': y>x' ... box.execute("INSERT INTO t1 VALUES (2, 4, 1)") --- - row_count: 1 ... +box.space.T1:update({1}, {{'+', 1, 5}}) +--- +- error: 'Check constraint failed ''ONE'': x<5' +... box.execute("DROP TABLE t1") --- - row_count: 1 @@ -164,14 +186,14 @@ _ = box.space._ck_constraint:insert({s.id, 'physics', false, 'SQL', 'X<Y'}) ... box.execute("INSERT INTO \"test\" VALUES(2, 1);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: physics' +- error: 'Failed to execute SQL statement: Check constraint failed ''physics'': X<Y' ... s:format({{name='Y', type='integer'}, {name='X', type='integer'}}) --- ... box.execute("INSERT INTO \"test\" VALUES(1, 2);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: physics' +- error: 'Failed to execute SQL statement: Check constraint failed ''physics'': X<Y' ... box.execute("INSERT INTO \"test\" VALUES(2, 1);") --- @@ -182,7 +204,7 @@ s:truncate() ... box.execute("INSERT INTO \"test\" VALUES(1, 2);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: physics' +- error: 'Failed to execute SQL statement: Check constraint failed ''physics'': X<Y' ... s:format({}) --- @@ -214,11 +236,11 @@ _ = box.space._ck_constraint:insert({s.id, 'conflict', false, 'SQL', 'X>10'}) ... box.execute("INSERT INTO \"test\" VALUES(1, 2);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: conflict' +- error: 'Failed to execute SQL statement: Check constraint failed ''conflict'': X>10' ... box.execute("INSERT INTO \"test\" VALUES(11, 11);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: physics' +- error: 'Failed to execute SQL statement: Check constraint failed ''physics'': X<Y' ... box.execute("INSERT INTO \"test\" VALUES(12, 11);") --- @@ -269,11 +291,13 @@ box.space._ck_constraint:select()[1].code ... box.execute("INSERT INTO t1 VALUES('1 a')") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_1_T1'': + x LIKE ''1 a''' ... box.execute("INSERT INTO t1 VALUES('1 a')") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_1_T1'': + x LIKE ''1 a''' ... box.execute("INSERT INTO t1 VALUES('1 a')") --- @@ -283,6 +307,284 @@ box.execute("DROP TABLE t1") --- - row_count: 1 ... +-- +-- Test binding reset on new insertion +-- +s = box.schema.create_space('test', {engine = engine}) +--- +... +_ = s:create_index('pk') +--- +... +s:format({{name='X', type='any'}, {name='Y', type='integer'}, {name='Z', type='integer', is_nullable=true}}) +--- +... +ck_not_null = box.space._ck_constraint:insert({s.id, 'ZnotNULL', false, 'SQL', 'X = 1 AND Z IS NOT NULL'}) +--- +... +s:insert({2, 1}) +--- +- error: 'Check constraint failed ''ZnotNULL'': X = 1 AND Z IS NOT NULL' +... +s:insert({1, 1}) +--- +- error: 'Check constraint failed ''ZnotNULL'': X = 1 AND Z IS NOT NULL' +... +s:insert({1, 1, box.NULL}) +--- +- error: 'Check constraint failed ''ZnotNULL'': X = 1 AND Z IS NOT NULL' +... +s:insert({2, 1, 3}) +--- +- error: 'Check constraint failed ''ZnotNULL'': X = 1 AND Z IS NOT NULL' +... +s:insert({1, 1}) +--- +- error: 'Check constraint failed ''ZnotNULL'': X = 1 AND Z IS NOT NULL' +... +s:insert({1, 1, 3}) +--- +- [1, 1, 3] +... +s:drop() +--- +... +-- +-- Test ck constraint corner cases +-- +s = box.schema.create_space('test', {engine = engine}) +--- +... +_ = s:create_index('pk') +--- +... +s:format({{name='X', type='any'}, {name='Y', type='integer'}, {name='Z', type='integer', is_nullable=true}}) +--- +... +ck_not_null = box.space._ck_constraint:insert({s.id, 'ZnotNULL', false, 'SQL', 'Z IS NOT NULL'}) +--- +... +s:insert({1, 2, box.NULL}) +--- +- error: 'Check constraint failed ''ZnotNULL'': Z IS NOT NULL' +... +s:insert({1, 2}) +--- +- error: 'Check constraint failed ''ZnotNULL'': Z IS NOT NULL' +... +_ = box.space._ck_constraint:delete({s.id, 'ZnotNULL'}) +--- +... +_ = box.space._ck_constraint:insert({s.id, 'XlessY', false, 'SQL', 'X < Y and Y < Z'}) +--- +... +s:insert({'1', 2}) +--- +- error: 'Tuple field 1 type does not match one required by operation: expected unsigned' +... +s:insert({}) +--- +- error: Tuple field 1 required by space format is missing +... +s:insert({2, 1}) +--- +- error: 'Check constraint failed ''XlessY'': X < Y and Y < Z' +... +s:insert({1, 2}) +--- +- [1, 2] +... +s:insert({2, 3, 1}) +--- +- error: 'Check constraint failed ''XlessY'': X < Y and Y < Z' +... +s:insert({2, 3, 4}) +--- +- [2, 3, 4] +... +s:update({2}, {{'+', 2, 3}}) +--- +- error: 'Check constraint failed ''XlessY'': X < Y and Y < Z' +... +s:update({2}, {{'+', 2, 3}, {'+', 3, 3}}) +--- +- [2, 6, 7] +... +s:replace({2, 1, 3}) +--- +- error: 'Check constraint failed ''XlessY'': X < Y and Y < Z' +... +box.snapshot() +--- +- ok +... +s = box.space["test"] +--- +... +s:update({2}, {{'+', 2, 3}}) +--- +- error: 'Check constraint failed ''XlessY'': X < Y and Y < Z' +... +s:update({2}, {{'+', 2, 3}, {'+', 3, 3}}) +--- +- [2, 9, 10] +... +s:replace({2, 1, 3}) +--- +- error: 'Check constraint failed ''XlessY'': X < Y and Y < Z' +... +s:drop() +--- +... +-- +-- Test complex CHECK constraints. +-- +s = box.schema.create_space('test', {engine = engine}) +--- +... +s:format({{name='X', type='integer'}, {name='Y', type='integer'}, {name='Z', type='integer'}}) +--- +... +_ = s:create_index('pk', {parts = {3, 'integer'}}) +--- +... +_ = s:create_index('unique', {parts = {1, 'integer'}}) +--- +... +_ = box.space._ck_constraint:insert({s.id, 'complex1', false, 'SQL', 'x+y==11 OR x*y==12 OR x/y BETWEEN 5 AND 8 OR -x == y+10'}) +--- +... +s:insert({1, 10, 1}) +--- +- [1, 10, 1] +... +s:update({1}, {{'=', 1, 4}, {'=', 2, 3}}) +--- +- [4, 3, 1] +... +s:update({1}, {{'=', 1, 12}, {'=', 2, 2}}) +--- +- [12, 2, 1] +... +s:update({1}, {{'=', 1, 12}, {'=', 2, -22}}) +--- +- [12, -22, 1] +... +s:update({1}, {{'=', 1, 0}, {'=', 2, 1}}) +--- +- error: 'Check constraint failed ''complex1'': x+y==11 OR x*y==12 OR x/y BETWEEN + 5 AND 8 OR -x == y+10' +... +s:get({1}) +--- +- [12, -22, 1] +... +s:update({1}, {{'=', 1, 0}, {'=', 2, 2}}) +--- +- error: 'Check constraint failed ''complex1'': x+y==11 OR x*y==12 OR x/y BETWEEN + 5 AND 8 OR -x == y+10' +... +s:get({1}) +--- +- [12, -22, 1] +... +s:drop() +--- +... +s = box.schema.create_space('test', {engine = engine}) +--- +... +s:format({{name='X', type='integer'}, {name='Z', type='any'}}) +--- +... +_ = s:create_index('pk', {parts = {1, 'integer'}}) +--- +... +_ = box.space._ck_constraint:insert({s.id, 'complex2', false, 'SQL', 'typeof(coalesce(z,0))==\'integer\''}) +--- +... +s:insert({1, 'string'}) +--- +- error: 'Check constraint failed ''complex2'': typeof(coalesce(z,0))==''integer''' +... +s:insert({1, {map=true}}) +--- +- error: 'Check constraint failed ''complex2'': typeof(coalesce(z,0))==''integer''' +... +s:insert({1, {'a', 'r','r','a','y'}}) +--- +- error: 'Check constraint failed ''complex2'': typeof(coalesce(z,0))==''integer''' +... +s:insert({1, 3.14}) +--- +- error: 'Check constraint failed ''complex2'': typeof(coalesce(z,0))==''integer''' +... +s:insert({1, 666}) +--- +- [1, 666] +... +s:drop() +--- +... +-- +-- Test large tuple. +-- +s = box.schema.create_space('test') +--- +... +_ = s:create_index('pk', {parts = {1, 'integer'}}) +--- +... +format65 = {} +--- +... +test_run:cmd("setopt delimiter ';'") +--- +- true +... +for i = 1,66 do + table.insert(format65, {name='X'..i, type='integer', is_nullable = true}) +end +test_run:cmd("setopt delimiter ''"); +--- +... +s:format(format65) +--- +... +_ = box.space._ck_constraint:insert({s.id, 'X1is666andX65is666', false, 'SQL', 'X1 == 666 and X65 == 666 and X63 IS NOT NULL'}) +--- +... +s:insert(s:frommap({X1 = 1, X65 = 1})) +--- +- error: 'Check constraint failed ''X1is666andX65is666'': X1 == 666 and X65 == 666 + and X63 IS NOT NULL' +... +s:insert(s:frommap({X1 = 666, X65 = 1})) +--- +- error: 'Check constraint failed ''X1is666andX65is666'': X1 == 666 and X65 == 666 + and X63 IS NOT NULL' +... +s:insert(s:frommap({X1 = 1, X65 = 666})) +--- +- error: 'Check constraint failed ''X1is666andX65is666'': X1 == 666 and X65 == 666 + and X63 IS NOT NULL' +... +s:insert(s:frommap({X1 = 666, X65 = 666})) +--- +- error: 'Check constraint failed ''X1is666andX65is666'': X1 == 666 and X65 == 666 + and X63 IS NOT NULL' +... +s:insert(s:frommap({X1 = 666, X65 = 666, X63 = 1})) +--- +- [666, null, null, null, null, null, null, null, null, null, null, null, null, null, + null, null, null, null, null, null, null, null, null, null, null, null, null, null, + null, null, null, null, null, null, null, null, null, null, null, null, null, null, + null, null, null, null, null, null, null, null, null, null, null, null, null, null, + null, null, null, null, null, null, 1, null, 666] +... +s:drop() +--- +... test_run:cmd("clear filter") --- - true diff --git a/test/sql/checks.test.lua b/test/sql/checks.test.lua index 399caf134..dadedb935 100644 --- a/test/sql/checks.test.lua +++ b/test/sql/checks.test.lua @@ -33,9 +33,11 @@ box.space._ck_constraint:insert({513, 'CK_CONSTRAINT_01', false, 'SQL', 'X<5'}) box.space._ck_constraint:count({}) box.execute("INSERT INTO \"test\" VALUES(5);") +box.space.test:insert({5}) box.space._ck_constraint:replace({513, 'CK_CONSTRAINT_01', false, 'SQL', 'X<=5'}) box.execute("INSERT INTO \"test\" VALUES(5);") box.execute("INSERT INTO \"test\" VALUES(6);") +box.space.test:insert({6}) -- Can't drop table with check constraints. box.space.test:delete({5}) box.space.test.index.pk:drop() @@ -47,8 +49,11 @@ box.space._space:delete({513}) box.execute("CREATE TABLE t1(x INTEGER CONSTRAINT ONE CHECK( x<5 ), y REAL CONSTRAINT TWO CHECK( y>x ), z INTEGER PRIMARY KEY);") box.space._ck_constraint:count() box.execute("INSERT INTO t1 VALUES (7, 1, 1)") +box.space.T1:insert({7, 1, 1}) box.execute("INSERT INTO t1 VALUES (2, 1, 1)") +box.space.T1:insert({2, 1, 1}) box.execute("INSERT INTO t1 VALUES (2, 4, 1)") +box.space.T1:update({1}, {{'+', 1, 5}}) box.execute("DROP TABLE t1") -- Test space creation rollback on spell error in ck constraint. @@ -103,4 +108,95 @@ box.execute("INSERT INTO t1 VALUES('1 a')") box.execute("INSERT INTO t1 VALUES('1 a')") box.execute("DROP TABLE t1") +-- +-- Test binding reset on new insertion +-- +s = box.schema.create_space('test', {engine = engine}) +_ = s:create_index('pk') +s:format({{name='X', type='any'}, {name='Y', type='integer'}, {name='Z', type='integer', is_nullable=true}}) +ck_not_null = box.space._ck_constraint:insert({s.id, 'ZnotNULL', false, 'SQL', 'X = 1 AND Z IS NOT NULL'}) +s:insert({2, 1}) +s:insert({1, 1}) +s:insert({1, 1, box.NULL}) +s:insert({2, 1, 3}) +s:insert({1, 1}) +s:insert({1, 1, 3}) +s:drop() + +-- +-- Test ck constraint corner cases +-- +s = box.schema.create_space('test', {engine = engine}) +_ = s:create_index('pk') +s:format({{name='X', type='any'}, {name='Y', type='integer'}, {name='Z', type='integer', is_nullable=true}}) +ck_not_null = box.space._ck_constraint:insert({s.id, 'ZnotNULL', false, 'SQL', 'Z IS NOT NULL'}) +s:insert({1, 2, box.NULL}) +s:insert({1, 2}) +_ = box.space._ck_constraint:delete({s.id, 'ZnotNULL'}) +_ = box.space._ck_constraint:insert({s.id, 'XlessY', false, 'SQL', 'X < Y and Y < Z'}) +s:insert({'1', 2}) +s:insert({}) +s:insert({2, 1}) +s:insert({1, 2}) +s:insert({2, 3, 1}) +s:insert({2, 3, 4}) +s:update({2}, {{'+', 2, 3}}) +s:update({2}, {{'+', 2, 3}, {'+', 3, 3}}) +s:replace({2, 1, 3}) +box.snapshot() +s = box.space["test"] +s:update({2}, {{'+', 2, 3}}) +s:update({2}, {{'+', 2, 3}, {'+', 3, 3}}) +s:replace({2, 1, 3}) +s:drop() + +-- +-- Test complex CHECK constraints. +-- +s = box.schema.create_space('test', {engine = engine}) +s:format({{name='X', type='integer'}, {name='Y', type='integer'}, {name='Z', type='integer'}}) +_ = s:create_index('pk', {parts = {3, 'integer'}}) +_ = s:create_index('unique', {parts = {1, 'integer'}}) +_ = box.space._ck_constraint:insert({s.id, 'complex1', false, 'SQL', 'x+y==11 OR x*y==12 OR x/y BETWEEN 5 AND 8 OR -x == y+10'}) +s:insert({1, 10, 1}) +s:update({1}, {{'=', 1, 4}, {'=', 2, 3}}) +s:update({1}, {{'=', 1, 12}, {'=', 2, 2}}) +s:update({1}, {{'=', 1, 12}, {'=', 2, -22}}) +s:update({1}, {{'=', 1, 0}, {'=', 2, 1}}) +s:get({1}) +s:update({1}, {{'=', 1, 0}, {'=', 2, 2}}) +s:get({1}) +s:drop() + +s = box.schema.create_space('test', {engine = engine}) +s:format({{name='X', type='integer'}, {name='Z', type='any'}}) +_ = s:create_index('pk', {parts = {1, 'integer'}}) +_ = box.space._ck_constraint:insert({s.id, 'complex2', false, 'SQL', 'typeof(coalesce(z,0))==\'integer\''}) +s:insert({1, 'string'}) +s:insert({1, {map=true}}) +s:insert({1, {'a', 'r','r','a','y'}}) +s:insert({1, 3.14}) +s:insert({1, 666}) +s:drop() + +-- +-- Test large tuple. +-- +s = box.schema.create_space('test') +_ = s:create_index('pk', {parts = {1, 'integer'}}) +format65 = {} +test_run:cmd("setopt delimiter ';'") +for i = 1,66 do + table.insert(format65, {name='X'..i, type='integer', is_nullable = true}) +end +test_run:cmd("setopt delimiter ''"); +s:format(format65) +_ = box.space._ck_constraint:insert({s.id, 'X1is666andX65is666', false, 'SQL', 'X1 == 666 and X65 == 666 and X63 IS NOT NULL'}) +s:insert(s:frommap({X1 = 1, X65 = 1})) +s:insert(s:frommap({X1 = 666, X65 = 1})) +s:insert(s:frommap({X1 = 1, X65 = 666})) +s:insert(s:frommap({X1 = 666, X65 = 666})) +s:insert(s:frommap({X1 = 666, X65 = 666, X63 = 1})) +s:drop() + test_run:cmd("clear filter") diff --git a/test/sql/errinj.result b/test/sql/errinj.result index 414e3c476..28b4b5025 100644 --- a/test/sql/errinj.result +++ b/test/sql/errinj.result @@ -490,7 +490,8 @@ _ = box.space._ck_constraint:insert({s.id, 'CK_CONSTRAINT_01', false, 'SQL', 'X< ... box.execute("INSERT INTO \"test\" VALUES(5);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_01' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_01'': + X<5' ... errinj.set("ERRINJ_WAL_IO", true) --- @@ -521,7 +522,8 @@ _ = box.space._ck_constraint:delete({s.id, 'CK_CONSTRAINT_01'}) ... box.execute("INSERT INTO \"test\" VALUES(6);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_01' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_01'': + X<=5' ... errinj.set("ERRINJ_WAL_IO", false) --- @@ -557,11 +559,13 @@ _ = box.space._ck_constraint:insert({s.id, 'Xgreater10', false, 'SQL', 'X > 10'} ... box.execute("INSERT INTO \"test\" VALUES(1, 2);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: Xgreater10' +- error: 'Failed to execute SQL statement: Check constraint failed ''Xgreater10'': + X > 10' ... box.execute("INSERT INTO \"test\" VALUES(20, 10);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: XlessY' +- error: 'Failed to execute SQL statement: Check constraint failed ''XlessY'': X < + Y' ... box.execute("INSERT INTO \"test\" VALUES(20, 100);") --- @@ -584,11 +588,13 @@ errinj.set("ERRINJ_WAL_IO", false) ... box.execute("INSERT INTO \"test\" VALUES(1, 2);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: Xgreater10' +- error: 'Failed to execute SQL statement: Check constraint failed ''Xgreater10'': + X > 10' ... box.execute("INSERT INTO \"test\" VALUES(20, 10);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: XlessY' +- error: 'Failed to execute SQL statement: Check constraint failed ''XlessY'': X < + Y' ... box.execute("INSERT INTO \"test\" VALUES(20, 100);") --- diff --git a/test/sql/gh-2981-check-autoinc.result b/test/sql/gh-2981-check-autoinc.result index 7384c81e8..e57789897 100644 --- a/test/sql/gh-2981-check-autoinc.result +++ b/test/sql/gh-2981-check-autoinc.result @@ -29,7 +29,8 @@ box.execute("insert into t1 values (18, null);") ... box.execute("insert into t1(s2) values (null);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_1_T1'': + s1 <> 19' ... box.execute("insert into t2 values (18, null);") --- @@ -37,7 +38,8 @@ box.execute("insert into t2 values (18, null);") ... box.execute("insert into t2(s2) values (null);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T2' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_1_T2'': + s1 <> 19 AND s1 <> 25' ... box.execute("insert into t2 values (24, null);") --- @@ -45,7 +47,8 @@ box.execute("insert into t2 values (24, null);") ... box.execute("insert into t2(s2) values (null);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T2' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_1_T2'': + s1 <> 19 AND s1 <> 25' ... box.execute("insert into t3 values (9, null)") --- @@ -53,7 +56,8 @@ box.execute("insert into t3 values (9, null)") ... box.execute("insert into t3(s2) values (null)") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T3' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_1_T3'': + s1 < 10' ... box.execute("DROP TABLE t1") --- diff --git a/test/sql/types.result b/test/sql/types.result index 1c9ef5468..9f8569e5a 100644 --- a/test/sql/types.result +++ b/test/sql/types.result @@ -741,7 +741,8 @@ box.execute("CREATE TABLE t1 (id INT PRIMARY KEY, a BOOLEAN CHECK (a = true));") ... box.execute("INSERT INTO t1 VALUES (1, false);") --- -- error: 'Failed to execute SQL statement: CHECK constraint failed: CK_CONSTRAINT_1_T1' +- error: 'Failed to execute SQL statement: Check constraint failed ''CK_CONSTRAINT_1_T1'': + a = true' ... box.execute("INSERT INTO t1 VALUES (1, true);") --- -- 2.21.0 ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 5/6] box: run check constraint tests on space alter 2019-05-31 13:45 ` Kirill Shcherbatov @ 2019-06-03 21:15 ` Vladislav Shpilevoy 0 siblings, 0 replies; 31+ messages in thread From: Vladislav Shpilevoy @ 2019-06-03 21:15 UTC (permalink / raw) To: tarantool-patches, Kirill Shcherbatov Thanks for the fixes! See review fixes on the branch, and at the end of the email. They are quite obvious. ================================================================== diff --git a/src/box/alter.cc b/src/box/alter.cc index 666ec3e6f..b0f8a8c9b 100644 --- a/src/box/alter.cc +++ b/src/box/alter.cc @@ -4356,7 +4356,8 @@ on_replace_dd_ck_constraint(struct trigger * /* trigger*/, void *event) diag_raise(); ck_def_guard.is_active = false; auto ck_guard = make_scoped_guard([=] { - ck_constraint_delete(new_ck_constraint); }); + ck_constraint_delete(new_ck_constraint); + }); const char *name = new_ck_constraint->def->name; struct ck_constraint *old_ck_constraint = space_ck_constraint_by_name(space, name, strlen(name)); diff --git a/src/box/ck_constraint.c b/src/box/ck_constraint.c index 2adbb3742..543c8b388 100644 --- a/src/box/ck_constraint.c +++ b/src/box/ck_constraint.c @@ -101,7 +101,7 @@ ck_constraint_resolve_field_names(struct Expr *expr, * an error description. * @param expr Ck constraint expression AST built for a given * @a ck_constraint_def, see for (sql_expr_compile and - * ck_constraint_resolve_space_def) implementation. + * ck_constraint_resolve_space_def) implementation. * @param space_def The space definition of the space this check * constraint is constructed for. * @retval not NULL sql_stmt program pointer on success. ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] [PATCH v5 6/6] box: user-friendly interface to manage ck constraints 2019-05-23 10:19 [tarantool-patches] [PATCH v5 0/6] box: run checks on insertions in LUA spaces Kirill Shcherbatov ` (4 preceding siblings ...) 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 5/6] box: run check constraint tests on space alter Kirill Shcherbatov @ 2019-05-23 10:19 ` Kirill Shcherbatov 2019-05-26 12:07 ` [tarantool-patches] " Vladislav Shpilevoy 2019-06-03 21:15 ` [tarantool-patches] Re: [PATCH v5 0/6] box: run checks on insertions in LUA spaces Vladislav Shpilevoy 2019-06-06 11:58 ` Kirill Yukhin 7 siblings, 1 reply; 31+ messages in thread From: Kirill Shcherbatov @ 2019-05-23 10:19 UTC (permalink / raw) To: tarantool-patches, v.shpilevoy; +Cc: Kirill Shcherbatov Closes #3691 @TarantoolBot document Title: check constraint for Lua space The check constraint is a type of integrity constraint which specifies a requirement that must be met by tuple before it is inserted into space. The constraint result must be predictable. Expression in check constraint must be <boolean value expression> I.e. return boolean result. Now it is possible to create ck constraints only for empty space having format. Constraint expression is a string that defines relations between top-level tuple fields. Take into account that all names are converted to an uppercase before resolve(like SQL does), use \" sign for names of fields that were created not with SQL. The check constraints are fired on insertion to the Lua space together with Lua space triggers. The execution order of ck constraints checks and space triggers follows their creation sequence. Note: this patch changes the CK constraints execution order for SQL. Previously check of CK constraints integrity was fired before tuple is formed; meanwhile now they are implemented as NoSQL before replace triggers, which are fired right before tuple insertion. In turn, type casts are performed earlier than msgpack serialization. You should be careful with functions that use field types in your check constrains (like typeof()). Consider following situation: ``` box.execute("CREATE TABLE t2(id INT primary key, x INTEGER CHECK (x > 1));") box.execute("INSERT INTO t2 VALUES(3, 1.1)") ``` the last operation would fail because 1.1 is silently cast to integer 1 which is not greater than 1. To create a new CK constraint for a space, use space:create_check_constraint method. All space constraints are shown in space.ck_constraint table. To drop ck constraint, use :drop method. Example: ``` s1 = box.schema.create_space('test1') pk = s1:create_index('pk') ck = s1:create_check_constraint('physics', 'X < Y') s1:insert({2, 1}) -- fail ck:drop() ``` --- src/box/alter.cc | 15 +++++++ src/box/lua/schema.lua | 31 ++++++++++++- src/box/lua/space.cc | 63 +++++++++++++++++++++++++++ test/sql/checks.result | 94 ++++++++++++++++++++++++++++++++++++++++ test/sql/checks.test.lua | 30 +++++++++++++ 5 files changed, 232 insertions(+), 1 deletion(-) diff --git a/src/box/alter.cc b/src/box/alter.cc index e4ded1995..f7da02cd0 100644 --- a/src/box/alter.cc +++ b/src/box/alter.cc @@ -4256,6 +4256,21 @@ on_replace_ck_constraint_commit(struct trigger *trigger, void *event) { struct txn_stmt *stmt = txn_last_stmt((struct txn *) event); struct ck_constraint *ck = (struct ck_constraint *)trigger->data; + + /* + * Configure CK constraint trigger to use it only when + * some ck constraint is defined. + */ + struct space *space = space_by_id(ck->space_id); + assert(space != NULL); + if (rlist_empty(&space->ck_constraint)) + trigger_clear(&space->ck_constraint_trigger); + else if (rlist_empty(&space->ck_constraint_trigger.link)) + trigger_add(&space->on_replace, &space->ck_constraint_trigger); + + /* Export schema changes to Lua. */ + trigger_run_xc(&on_alter_space, space); + if (stmt->old_tuple != NULL) ck_constraint_delete(ck); } diff --git a/src/box/lua/schema.lua b/src/box/lua/schema.lua index f2d76d0af..5504bc666 100644 --- a/src/box/lua/schema.lua +++ b/src/box/lua/schema.lua @@ -1219,6 +1219,15 @@ local function check_primary_index(space) end box.internal.check_primary_index = check_primary_index -- for net.box +-- Helper function to check ck_constraint:method() usage +local function check_ck_constraint_arg(ck_constraint, method) + if type(ck_constraint) ~= 'table' or ck_constraint.name == nil then + local fmt = 'Use ck_constraint:%s(...) instead of ck_constraint.%s(...)' + error(string.format(fmt, method, method)) + end +end +box.internal.check_ck_constraint_arg = check_ck_constraint_arg + box.internal.schema_version = builtin.box_schema_version local function check_iterator_type(opts, key_is_nil) @@ -1567,7 +1576,16 @@ space_mt.auto_increment = function(space, tuple) table.insert(tuple, 1, max + 1) return space:insert(tuple) end - +-- Manage space ck constraints +space_mt.create_check_constraint = function(space, name, code) + check_space_arg(space, 'create_constraint') + if name == nil or code == nil then + box.error(box.error.PROC_LUA, + "Usage: space:create_constraint(name, code)") + end + box.space._ck_constraint:insert({space.id, name, false, 'SQL', code}) + return space.ck_constraint[name] +end space_mt.pairs = function(space, key, opts) check_space_arg(space, 'pairs') local pk = space.index[0] @@ -1613,6 +1631,12 @@ end space_mt.frommap = box.internal.space.frommap space_mt.__index = space_mt +local ck_constraint_mt = {} +ck_constraint_mt.drop = function(ck_constraint) + check_ck_constraint_arg(ck_constraint, 'drop') + box.space._ck_constraint:delete({ck_constraint.space_id, ck_constraint.name}) +end + box.schema.index_mt = base_index_mt box.schema.memtx_index_mt = memtx_index_mt box.schema.vinyl_index_mt = vinyl_index_mt @@ -1661,6 +1685,11 @@ function box.schema.space.bless(space) setmetatable(index, wrap_schema_object_mt(index_mt_name)) end end + for j, ck_constraint in pairs(space.ck_constraint) do + if type(j) == 'string' then + setmetatable(ck_constraint, {__index = ck_constraint_mt}) + end + end end end diff --git a/src/box/lua/space.cc b/src/box/lua/space.cc index 3e8f3b2e5..b375716c6 100644 --- a/src/box/lua/space.cc +++ b/src/box/lua/space.cc @@ -28,6 +28,7 @@ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ +#include "box/ck_constraint.h" #include "box/lua/space.h" #include "box/lua/tuple.h" #include "box/lua/key_def.h" @@ -147,6 +148,66 @@ lbox_space_before_replace(struct lua_State *L) lbox_push_txn_stmt, lbox_pop_txn_stmt); } +/** + * Make ck_constraints available in Lua, via ck_constraint[] + * array. + * Updata a ck_constraint table in the parent space table object + * on the Lua stack. + */ +static void +lbox_ck_constraint(struct lua_State *L, struct space *space, int i) +{ + lua_getfield(L, i, "ck_constraint"); + if (lua_isnil(L, -1)) { + lua_pop(L, 1); + lua_pushstring(L, "ck_constraint"); + lua_newtable(L); + lua_settable(L, i); + lua_getfield(L, i, "ck_constraint"); + } else { + lua_pushnil(L); + while (lua_next(L, -2) != 0) { + size_t name_len; + const char *name = lua_tolstring(L, -2, &name_len); + /* + * Remove ck_constraint only if it was + * deleted. + */ + if (space_ck_constraint_by_name(space, name, + (uint32_t)name_len) == NULL) { + lua_pushlstring(L, name, name_len); + lua_pushnil(L); + lua_settable(L, -5); + } + lua_pop(L, 1); + } + } + struct ck_constraint *ck_constraint = NULL; + rlist_foreach_entry(ck_constraint, &space->ck_constraint, link) { + lua_getfield(L, i, ck_constraint->def->name); + if (lua_isnil(L, -1)) { + lua_pop(L, 1); + lua_pushstring(L, ck_constraint->def->name); + lua_newtable(L); + lua_settable(L, -3); + lua_getfield(L, -1, ck_constraint->def->name); + assert(!lua_isnil(L, -1)); + } + + lua_pushstring(L, ck_constraint->def->name); + lua_setfield(L, -2, "name"); + + lua_pushnumber(L, space->def->id); + lua_setfield(L, -2, "space_id"); + + lua_pushstring(L, ck_constraint->def->expr_str); + lua_setfield(L, -2, "expr_str"); + + lua_setfield(L, -2, ck_constraint->def->name); + } + lua_pop(L, 1); +} + /** * Make a single space available in Lua, * via box.space[] array. @@ -344,6 +405,8 @@ lbox_fillspace(struct lua_State *L, struct space *space, int i) lua_pop(L, 1); /* pop the index field */ + lbox_ck_constraint(L, space, i); + lua_getfield(L, LUA_GLOBALSINDEX, "box"); lua_pushstring(L, "schema"); lua_gettable(L, -2); diff --git a/test/sql/checks.result b/test/sql/checks.result index 3973d242f..e6b526a25 100644 --- a/test/sql/checks.result +++ b/test/sql/checks.result @@ -582,6 +582,100 @@ s:insert(s:frommap({X1 = 666, X65 = 666, X63 = 1})) s:drop() --- ... +-- +-- Test ck constraints LUA integration. +-- +s1 = box.schema.create_space('test1') +--- +... +_ = s1:create_index('pk') +--- +... +s1:format({{name='X', type='any'}, {name='Y', type='integer'}}) +--- +... +s2 = box.schema.create_space('test2') +--- +... +_ = s2:create_index('pk') +--- +... +s2:format({{name='X', type='any'}, {name='Y', type='integer'}}) +--- +... +_ = s1:create_check_constraint('physics', 'X < Y') +--- +... +_ = s1:create_check_constraint('physics', 'X > Y') +--- +- error: Duplicate key exists in unique index 'primary' in space '_ck_constraint' +... +_ = s1:create_check_constraint('greater', 'X > 20') +--- +... +_ = s2:create_check_constraint('physics', 'X > Y') +--- +... +_ = s2:create_check_constraint('greater', 'X > 20') +--- +... +s1.ck_constraint.physics ~= nil +--- +- true +... +s1.ck_constraint.greater ~= nil +--- +- true +... +s2.ck_constraint.physics ~= nil +--- +- true +... +s2.ck_constraint.greater ~= nil +--- +- true +... +s1:insert({2, 1}) +--- +- error: 'Check constraint failed ''greater'': X > 20' +... +s1:insert({21, 20}) +--- +- error: 'Check constraint failed ''physics'': X < Y' +... +s2:insert({1, 2}) +--- +- error: 'Check constraint failed ''greater'': X > 20' +... +s2:insert({21, 22}) +--- +- error: 'Check constraint failed ''physics'': X > Y' +... +s2.ck_constraint.greater:drop() +--- +... +s2.ck_constraint.physics ~= nil +--- +- true +... +s2.ck_constraint.greater == nil +--- +- true +... +s1:insert({2, 1}) +--- +- error: 'Check constraint failed ''greater'': X > 20' +... +s2:insert({1, 2}) +--- +- error: 'Check constraint failed ''physics'': X > Y' +... +s1:drop() +--- +... +s2:drop() +--- +... test_run:cmd("clear filter") --- - true diff --git a/test/sql/checks.test.lua b/test/sql/checks.test.lua index c00654f99..f06b9e647 100644 --- a/test/sql/checks.test.lua +++ b/test/sql/checks.test.lua @@ -199,4 +199,34 @@ s:insert(s:frommap({X1 = 666, X65 = 666})) s:insert(s:frommap({X1 = 666, X65 = 666, X63 = 1})) s:drop() +-- +-- Test ck constraints LUA integration. +-- +s1 = box.schema.create_space('test1') +_ = s1:create_index('pk') +s1:format({{name='X', type='any'}, {name='Y', type='integer'}}) +s2 = box.schema.create_space('test2') +_ = s2:create_index('pk') +s2:format({{name='X', type='any'}, {name='Y', type='integer'}}) +_ = s1:create_check_constraint('physics', 'X < Y') +_ = s1:create_check_constraint('physics', 'X > Y') +_ = s1:create_check_constraint('greater', 'X > 20') +_ = s2:create_check_constraint('physics', 'X > Y') +_ = s2:create_check_constraint('greater', 'X > 20') +s1.ck_constraint.physics ~= nil +s1.ck_constraint.greater ~= nil +s2.ck_constraint.physics ~= nil +s2.ck_constraint.greater ~= nil +s1:insert({2, 1}) +s1:insert({21, 20}) +s2:insert({1, 2}) +s2:insert({21, 22}) +s2.ck_constraint.greater:drop() +s2.ck_constraint.physics ~= nil +s2.ck_constraint.greater == nil +s1:insert({2, 1}) +s2:insert({1, 2}) +s1:drop() +s2:drop() + test_run:cmd("clear filter") -- 2.21.0 ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 6/6] box: user-friendly interface to manage ck constraints 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 6/6] box: user-friendly interface to manage ck constraints Kirill Shcherbatov @ 2019-05-26 12:07 ` Vladislav Shpilevoy 2019-05-31 13:45 ` Kirill Shcherbatov 0 siblings, 1 reply; 31+ messages in thread From: Vladislav Shpilevoy @ 2019-05-26 12:07 UTC (permalink / raw) To: Kirill Shcherbatov, tarantool-patches Thanks for the patch! See 1 comment below. > diff --git a/src/box/lua/space.cc b/src/box/lua/space.cc > index 3e8f3b2e5..b375716c6 100644 > --- a/src/box/lua/space.cc > +++ b/src/box/lua/space.cc > @@ -28,6 +28,7 @@ > * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > * SUCH DAMAGE. > */ > +#include "box/ck_constraint.h" > #include "box/lua/space.h" > #include "box/lua/tuple.h" > #include "box/lua/key_def.h" > @@ -147,6 +148,66 @@ lbox_space_before_replace(struct lua_State *L) > lbox_push_txn_stmt, lbox_pop_txn_stmt); > } > > +/** > + * Make ck_constraints available in Lua, via ck_constraint[] > + * array. > + * Updata a ck_constraint table in the parent space table object > + * on the Lua stack. > + */ > +static void > +lbox_ck_constraint(struct lua_State *L, struct space *space, int i) > +{ > + lua_getfield(L, i, "ck_constraint"); > + if (lua_isnil(L, -1)) { > + lua_pop(L, 1); > + lua_pushstring(L, "ck_constraint"); > + lua_newtable(L); > + lua_settable(L, i); > + lua_getfield(L, i, "ck_constraint"); > + } else { > + lua_pushnil(L); > + while (lua_next(L, -2) != 0) { > + size_t name_len; > + const char *name = lua_tolstring(L, -2, &name_len); > + /* > + * Remove ck_constraint only if it was > + * deleted. > + */ > + if (space_ck_constraint_by_name(space, name, > + (uint32_t)name_len) == NULL) { > + lua_pushlstring(L, name, name_len); > + lua_pushnil(L); > + lua_settable(L, -5); > + } > + lua_pop(L, 1); > + } > + } > + struct ck_constraint *ck_constraint = NULL; > + rlist_foreach_entry(ck_constraint, &space->ck_constraint, link) { > + lua_getfield(L, i, ck_constraint->def->name); > + if (lua_isnil(L, -1)) { > + lua_pop(L, 1); > + lua_pushstring(L, ck_constraint->def->name); > + lua_newtable(L); > + lua_settable(L, -3); > + lua_getfield(L, -1, ck_constraint->def->name); > + assert(!lua_isnil(L, -1)); > + } > + > + lua_pushstring(L, ck_constraint->def->name); > + lua_setfield(L, -2, "name"); > + > + lua_pushnumber(L, space->def->id); > + lua_setfield(L, -2, "space_id"); > + > + lua_pushstring(L, ck_constraint->def->expr_str); > + lua_setfield(L, -2, "expr_str"); 1. I think, 'expr' is better name. What else except 'str' you can expose? - nothing. It means, that '_str' suffix is redundant, IMO. > + > + lua_setfield(L, -2, ck_constraint->def->name); > + } > + lua_pop(L, 1); > +} ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 6/6] box: user-friendly interface to manage ck constraints 2019-05-26 12:07 ` [tarantool-patches] " Vladislav Shpilevoy @ 2019-05-31 13:45 ` Kirill Shcherbatov 0 siblings, 0 replies; 31+ messages in thread From: Kirill Shcherbatov @ 2019-05-31 13:45 UTC (permalink / raw) To: tarantool-patches, Vladislav Shpilevoy > 1. I think, 'expr' is better name. What else except 'str' you > can expose? - nothing. It means, that '_str' suffix is > redundant, IMO. Ok. Done. ================================== Closes #3691 @TarantoolBot document Title: check constraint for Lua space The check constraint is a type of integrity constraint which specifies a requirement that must be met by tuple before it is inserted into space. The constraint result must be predictable. Expression in check constraint must be <boolean value expression> I.e. return boolean result. Now it is possible to create ck constraints only for empty space having format. Constraint expression is a string that defines relations between top-level tuple fields. Take into account that all names are converted to an uppercase before resolve(like SQL does), use \" sign for names of fields that were created not with SQL. The check constraints are fired on insertion to the Lua space together with Lua space triggers. The execution order of ck constraints checks and space triggers follows their creation sequence. Note: this patch changes the CK constraints execution order for SQL. Previously check of CK constraints integrity was fired before tuple is formed; meanwhile now they are implemented as NoSQL before replace triggers, which are fired right before tuple insertion. In turn, type casts are performed earlier than msgpack serialization. You should be careful with functions that use field types in your check constrains (like typeof()). Consider following situation: ``` box.execute("CREATE TABLE t2(id INT primary key, x INTEGER CHECK (x > 1));") box.execute("INSERT INTO t2 VALUES(3, 1.1)") ``` the last operation would fail because 1.1 is silently cast to integer 1 which is not greater than 1. To create a new CK constraint for a space, use space:create_check_constraint method. All space constraints are shown in space.ck_constraint table. To drop ck constraint, use :drop method. Example: ``` s1 = box.schema.create_space('test1') pk = s1:create_index('pk') ck = s1:create_check_constraint('physics', 'X < Y') s1:insert({2, 1}) -- fail ck:drop() ``` --- src/box/alter.cc | 2 + src/box/lua/schema.lua | 31 +++++++++- src/box/lua/space.cc | 63 ++++++++++++++++++++ test/sql/checks.result | 124 +++++++++++++++++++++++++++++++++++++++ test/sql/checks.test.lua | 35 +++++++++++ 5 files changed, 254 insertions(+), 1 deletion(-) diff --git a/src/box/alter.cc b/src/box/alter.cc index 7a6975427..3e8282255 100644 --- a/src/box/alter.cc +++ b/src/box/alter.cc @@ -4304,6 +4304,8 @@ on_replace_ck_constraint_commit(struct trigger *trigger, void *event) assert(stmt->new_tuple != NULL); ck_constraint_delete(ck); } + /* Export schema changes to Lua. */ + trigger_run_xc(&on_alter_space, space); } /** A trigger invoked on replace in the _ck_constraint space. */ diff --git a/src/box/lua/schema.lua b/src/box/lua/schema.lua index 91fae8378..3f080eced 100644 --- a/src/box/lua/schema.lua +++ b/src/box/lua/schema.lua @@ -1309,6 +1309,15 @@ local function check_primary_index(space) end box.internal.check_primary_index = check_primary_index -- for net.box +-- Helper function to check ck_constraint:method() usage +local function check_ck_constraint_arg(ck_constraint, method) + if type(ck_constraint) ~= 'table' or ck_constraint.name == nil then + local fmt = 'Use ck_constraint:%s(...) instead of ck_constraint.%s(...)' + error(string.format(fmt, method, method)) + end +end +box.internal.check_ck_constraint_arg = check_ck_constraint_arg + box.internal.schema_version = builtin.box_schema_version local function check_iterator_type(opts, key_is_nil) @@ -1657,7 +1666,16 @@ space_mt.auto_increment = function(space, tuple) table.insert(tuple, 1, max + 1) return space:insert(tuple) end - +-- Manage space ck constraints +space_mt.create_check_constraint = function(space, name, code) + check_space_arg(space, 'create_constraint') + if name == nil or code == nil then + box.error(box.error.PROC_LUA, + "Usage: space:create_constraint(name, code)") + end + box.space._ck_constraint:insert({space.id, name, false, 'SQL', code}) + return space.ck_constraint[name] +end space_mt.pairs = function(space, key, opts) check_space_arg(space, 'pairs') local pk = space.index[0] @@ -1703,6 +1721,12 @@ end space_mt.frommap = box.internal.space.frommap space_mt.__index = space_mt +local ck_constraint_mt = {} +ck_constraint_mt.drop = function(ck_constraint) + check_ck_constraint_arg(ck_constraint, 'drop') + box.space._ck_constraint:delete({ck_constraint.space_id, ck_constraint.name}) +end + box.schema.index_mt = base_index_mt box.schema.memtx_index_mt = memtx_index_mt box.schema.vinyl_index_mt = vinyl_index_mt @@ -1751,6 +1775,11 @@ function box.schema.space.bless(space) setmetatable(index, wrap_schema_object_mt(index_mt_name)) end end + for j, ck_constraint in pairs(space.ck_constraint) do + if type(j) == 'string' then + setmetatable(ck_constraint, {__index = ck_constraint_mt}) + end + end end end diff --git a/src/box/lua/space.cc b/src/box/lua/space.cc index 509306eae..b2c5937bc 100644 --- a/src/box/lua/space.cc +++ b/src/box/lua/space.cc @@ -28,6 +28,7 @@ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ +#include "box/ck_constraint.h" #include "box/lua/space.h" #include "box/lua/tuple.h" #include "box/lua/key_def.h" @@ -147,6 +148,66 @@ lbox_space_before_replace(struct lua_State *L) lbox_push_txn_stmt, lbox_pop_txn_stmt); } +/** + * Make ck_constraints available in Lua, via ck_constraint[] + * array for space table by given index i. + * Updata a ck_constraint table in the parent space table object + * on the Lua stack. + */ +static void +lbox_push_ck_constraint(struct lua_State *L, struct space *space, int i) +{ + lua_getfield(L, i, "ck_constraint"); + if (lua_isnil(L, -1)) { + lua_pop(L, 1); + lua_pushstring(L, "ck_constraint"); + lua_newtable(L); + lua_settable(L, i); + lua_getfield(L, i, "ck_constraint"); + } else { + lua_pushnil(L); + while (lua_next(L, -2) != 0) { + size_t name_len; + const char *name = lua_tolstring(L, -2, &name_len); + /* + * Remove ck_constraint only if it was + * deleted. + */ + if (space_ck_constraint_by_name(space, name, + (uint32_t)name_len) == NULL) { + lua_pushlstring(L, name, name_len); + lua_pushnil(L); + lua_settable(L, -5); + } + lua_pop(L, 1); + } + } + struct ck_constraint *ck_constraint = NULL; + rlist_foreach_entry(ck_constraint, &space->ck_constraint, link) { + lua_getfield(L, i, ck_constraint->def->name); + if (lua_isnil(L, -1)) { + lua_pop(L, 1); + lua_pushstring(L, ck_constraint->def->name); + lua_newtable(L); + lua_settable(L, -3); + lua_getfield(L, -1, ck_constraint->def->name); + assert(!lua_isnil(L, -1)); + } + + lua_pushstring(L, ck_constraint->def->name); + lua_setfield(L, -2, "name"); + + lua_pushnumber(L, space->def->id); + lua_setfield(L, -2, "space_id"); + + lua_pushstring(L, ck_constraint->def->expr_str); + lua_setfield(L, -2, "expr"); + + lua_setfield(L, -2, ck_constraint->def->name); + } + lua_pop(L, 1); +} + /** * Make a single space available in Lua, * via box.space[] array. @@ -352,6 +413,8 @@ lbox_fillspace(struct lua_State *L, struct space *space, int i) lua_pop(L, 1); /* pop the index field */ + lbox_push_ck_constraint(L, space, i); + lua_getfield(L, LUA_GLOBALSINDEX, "box"); lua_pushstring(L, "schema"); lua_gettable(L, -2); diff --git a/test/sql/checks.result b/test/sql/checks.result index 6272cea7a..ce28637c6 100644 --- a/test/sql/checks.result +++ b/test/sql/checks.result @@ -585,6 +585,130 @@ s:insert(s:frommap({X1 = 666, X65 = 666, X63 = 1})) s:drop() --- ... +-- +-- Test ck constraints LUA integration. +-- +s1 = box.schema.create_space('test1') +--- +... +_ = s1:create_index('pk') +--- +... +s1:format({{name='X', type='any'}, {name='Y', type='integer'}}) +--- +... +s2 = box.schema.create_space('test2') +--- +... +_ = s2:create_index('pk') +--- +... +s2:format({{name='X', type='any'}, {name='Y', type='integer'}}) +--- +... +test_run:cmd("push filter 'space_id: [0-9]+' to 'space_id: <ID>'") +--- +- true +... +_ = s1:create_check_constraint('physics', 'X < Y') +--- +... +_ = s1:create_check_constraint('physics', 'X > Y') +--- +- error: Duplicate key exists in unique index 'primary' in space '_ck_constraint' +... +_ = s1:create_check_constraint('greater', 'X > 20') +--- +... +_ = s2:create_check_constraint('physics', 'X > Y') +--- +... +_ = s2:create_check_constraint('greater', 'X > 20') +--- +... +s1.ck_constraint.physics +--- +- space_id: <ID> + name: physics + expr: X < Y +... +s1.ck_constraint.greater +--- +- space_id: <ID> + name: greater + expr: X > 20 +... +s2.ck_constraint.physics +--- +- space_id: <ID> + name: physics + expr: X > Y +... +s2.ck_constraint.greater +--- +- space_id: <ID> + name: greater + expr: X > 20 +... +s1:insert({2, 1}) +--- +- error: 'Check constraint failed ''greater'': X > 20' +... +s1:insert({21, 20}) +--- +- error: 'Check constraint failed ''physics'': X < Y' +... +s2:insert({1, 2}) +--- +- error: 'Check constraint failed ''greater'': X > 20' +... +s2:insert({21, 22}) +--- +- error: 'Check constraint failed ''physics'': X > Y' +... +s2.ck_constraint.greater:drop() +--- +... +s2.ck_constraint.physics +--- +- space_id: <ID> + name: physics + expr: X > Y +... +s2.ck_constraint.greater +--- +- null +... +s1:insert({2, 1}) +--- +- error: 'Check constraint failed ''greater'': X > 20' +... +s2:insert({1, 2}) +--- +- error: 'Check constraint failed ''physics'': X > Y' +... +s2:insert({2, 1}) +--- +- [2, 1] +... +physics_ck = s2.ck_constraint +--- +... +s1:drop() +--- +... +s2:drop() +--- +... +physics_ck +--- +- [] +... +physics_ck:drop() +--- +- error: '[string "return physics_ck:drop() "]:1: attempt to call method ''drop'' + (a nil value)' +... test_run:cmd("clear filter") --- - true diff --git a/test/sql/checks.test.lua b/test/sql/checks.test.lua index dadedb935..c8efcc7cb 100644 --- a/test/sql/checks.test.lua +++ b/test/sql/checks.test.lua @@ -199,4 +199,39 @@ s:insert(s:frommap({X1 = 666, X65 = 666})) s:insert(s:frommap({X1 = 666, X65 = 666, X63 = 1})) s:drop() +-- +-- Test ck constraints LUA integration. +-- +s1 = box.schema.create_space('test1') +_ = s1:create_index('pk') +s1:format({{name='X', type='any'}, {name='Y', type='integer'}}) +s2 = box.schema.create_space('test2') +_ = s2:create_index('pk') +s2:format({{name='X', type='any'}, {name='Y', type='integer'}}) +test_run:cmd("push filter 'space_id: [0-9]+' to 'space_id: <ID>'") +_ = s1:create_check_constraint('physics', 'X < Y') +_ = s1:create_check_constraint('physics', 'X > Y') +_ = s1:create_check_constraint('greater', 'X > 20') +_ = s2:create_check_constraint('physics', 'X > Y') +_ = s2:create_check_constraint('greater', 'X > 20') +s1.ck_constraint.physics +s1.ck_constraint.greater +s2.ck_constraint.physics +s2.ck_constraint.greater +s1:insert({2, 1}) +s1:insert({21, 20}) +s2:insert({1, 2}) +s2:insert({21, 22}) +s2.ck_constraint.greater:drop() +s2.ck_constraint.physics +s2.ck_constraint.greater +s1:insert({2, 1}) +s2:insert({1, 2}) +s2:insert({2, 1}) +physics_ck = s2.ck_constraint +s1:drop() +s2:drop() +physics_ck +physics_ck:drop() + test_run:cmd("clear filter") -- 2.21.0 ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 0/6] box: run checks on insertions in LUA spaces 2019-05-23 10:19 [tarantool-patches] [PATCH v5 0/6] box: run checks on insertions in LUA spaces Kirill Shcherbatov ` (5 preceding siblings ...) 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 6/6] box: user-friendly interface to manage ck constraints Kirill Shcherbatov @ 2019-06-03 21:15 ` Vladislav Shpilevoy 2019-06-04 7:21 ` Kirill Shcherbatov 2019-06-06 11:58 ` Kirill Yukhin 7 siblings, 1 reply; 31+ messages in thread From: Vladislav Shpilevoy @ 2019-06-03 21:15 UTC (permalink / raw) To: Kirill Shcherbatov, tarantool-patches Please, proceed with next patches. We need to make a single Vdbe for all checks in a separate commit. There is also a new proposal for benchmark. Lets compare SQL CHECKs with tuple validation in Lua before insertion, and see what is faster. Please, write down somewhere all our planned benchmarks so as no to forget something. Totally: 1) bench CHECKs before your patch via pure SQL 2) bench before your patch via checks done in Lua before insertion 3) bench after patch via pure SQL 4) bench via Lua DML After single Vdbe is implemented, bench 3 and 4 again. On 23/05/2019 13:19, Kirill Shcherbatov wrote: > Fire CK constraints for LUA spaces. > To achieve this goal, we reworked data dictionary, to store ck > constraints in separate space _ck_constraints and updated data > migration script to migrate existent data there. This also would > be useful in future to implement ALTER SPACE ADD CONSTRAINT > operation. Now we do not support CK constraint creation on > non-empty space. > Each CK has own precompiled VDBE machine that performs this > check with tuple fields mapped to it's memory with sql_bind() api. > In case of ck constraint conflict detected by this VM we abort > the transaction and return error to user. > Finally, we introduced a LUA-wrapper that provide a user-friendly > way to manage space ck constraints. > > Changes in version 5: > - a new tuple_fetcher class to access tuple fields fast > - refactored _ck_constraint space format and indexes > - one on_replace trigger for all ck constraints > - trim_space_snprintf fixes: do not trim substring in ' or " > - do not wrap ck_constraint methods > - many minor fixes > > v4: https://www.freelists.org/post/tarantool-patches/PATCH-v4-04-box-run-checks-on-insertions-in-LUA-spaces > > Branch: http://github.com/tarantool/tarantool/tree/kshch/gh-3691-checks-on-server-side > Issue: https://github.com/tarantool/tarantool/issues/3691 > > Kirill Shcherbatov (6): > sql: introduce a new method to bind a pointer > sql: refactor OP_Column vdbe instruction > sql: introduce tuple_fetcher class > schema: add new system space for CHECK constraints > box: run check constraint tests on space alter > box: user-friendly interface to manage ck constraints > > src/box/CMakeLists.txt | 1 + > src/box/alter.cc | 342 +++++++++++++- > src/box/alter.h | 1 + > src/box/bootstrap.snap | Bin 4379 -> 4430 bytes > src/box/ck_constraint.c | 238 ++++++++++ > src/box/ck_constraint.h | 196 ++++++++ > src/box/errcode.h | 4 +- > src/box/lua/schema.lua | 35 +- > src/box/lua/space.cc | 65 +++ > src/box/lua/upgrade.lua | 40 ++ > src/box/schema.cc | 8 + > src/box/schema_def.h | 11 + > src/box/space.c | 6 + > src/box/space.h | 8 + > src/box/space_def.c | 98 +--- > src/box/space_def.h | 4 - > src/box/sql.c | 103 +---- > src/box/sql.h | 82 ++-- > src/box/sql/build.c | 220 +++++++-- > src/box/sql/expr.c | 25 +- > src/box/sql/insert.c | 115 ++--- > src/box/sql/parse.y | 2 +- > src/box/sql/parse_def.h | 24 + > src/box/sql/select.c | 11 +- > src/box/sql/sqlInt.h | 50 ++- > src/box/sql/tarantoolInt.h | 5 +- > src/box/sql/tokenize.c | 1 - > src/box/sql/vdbe.c | 308 +++++++------ > src/box/sql/vdbe.h | 1 - > src/box/sql/vdbeInt.h | 20 +- > src/box/sql/vdbeapi.c | 20 +- > src/box/sql/vdbemem.c | 8 + > test/app-tap/tarantoolctl.test.lua | 4 +- > test/box-py/bootstrap.result | 4 + > test/box/access.result | 3 + > test/box/access.test.lua | 1 + > test/box/access_misc.result | 3 + > test/box/access_sysview.result | 6 +- > test/box/alter.result | 5 +- > test/box/misc.result | 2 + > test/sql-tap/check.test.lua | 42 +- > test/sql-tap/fkey2.test.lua | 4 +- > test/sql-tap/sql-errors.test.lua | 2 +- > test/sql-tap/table.test.lua | 12 +- > test/sql/checks.result | 620 ++++++++++++++++++++++++-- > test/sql/checks.test.lua | 228 ++++++++-- > test/sql/errinj.result | 140 ++++++ > test/sql/errinj.test.lua | 45 ++ > test/sql/gh-2981-check-autoinc.result | 12 +- > test/sql/types.result | 3 +- > test/sql/upgrade.result | 19 + > test/sql/upgrade.test.lua | 5 + > test/wal_off/alter.result | 2 +- > 53 files changed, 2561 insertions(+), 653 deletions(-) > create mode 100644 src/box/ck_constraint.c > create mode 100644 src/box/ck_constraint.h > ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 0/6] box: run checks on insertions in LUA spaces 2019-06-03 21:15 ` [tarantool-patches] Re: [PATCH v5 0/6] box: run checks on insertions in LUA spaces Vladislav Shpilevoy @ 2019-06-04 7:21 ` Kirill Shcherbatov 2019-06-04 18:59 ` Vladislav Shpilevoy 0 siblings, 1 reply; 31+ messages in thread From: Kirill Shcherbatov @ 2019-06-04 7:21 UTC (permalink / raw) To: tarantool-patches, Vladislav Shpilevoy > Please, proceed with next patches. We need to > make a single Vdbe for all checks in a separate > commit. > > There is also a new proposal for benchmark. Lets > compare SQL CHECKs with tuple validation in Lua > before insertion, and see what is faster. Please, > write down somewhere all our planned benchmarks so > as no to forget something. > > Totally: > 1) bench CHECKs before your patch via pure SQL > 2) bench before your patch via checks done in Lua > before insertion > 3) bench after patch via pure SQL > 4) bench via Lua DML > > After single Vdbe is implemented, bench 3 and 4 again. Hi! Thank you for review an fixes! They are looking appropriately. I squashed them and fixed commit headers where necessary. I'll try to do what you've mention as soon as possible (don't know, what has higher priority - this or functional indexes fixes). However, can we merge the patch up to this point now? ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 0/6] box: run checks on insertions in LUA spaces 2019-06-04 7:21 ` Kirill Shcherbatov @ 2019-06-04 18:59 ` Vladislav Shpilevoy 0 siblings, 0 replies; 31+ messages in thread From: Vladislav Shpilevoy @ 2019-06-04 18:59 UTC (permalink / raw) To: Kirill Shcherbatov, tarantool-patches, Kirill Yukhin The patchset is LGTM. I've created a follow-up issue: https://github.com/tarantool/tarantool/issues/4272 On 04/06/2019 10:21, Kirill Shcherbatov wrote: >> Please, proceed with next patches. We need to >> make a single Vdbe for all checks in a separate >> commit. >> >> There is also a new proposal for benchmark. Lets >> compare SQL CHECKs with tuple validation in Lua >> before insertion, and see what is faster. Please, >> write down somewhere all our planned benchmarks so >> as no to forget something. >> >> Totally: >> 1) bench CHECKs before your patch via pure SQL >> 2) bench before your patch via checks done in Lua >> before insertion >> 3) bench after patch via pure SQL >> 4) bench via Lua DML >> >> After single Vdbe is implemented, bench 3 and 4 again. > > Hi! Thank you for review an fixes! They are looking appropriately. I squashed them > and fixed commit headers where necessary. > > I'll try to do what you've mention as soon as possible (don't know, what has higher > priority - this or functional indexes fixes). > > However, can we merge the patch up to this point now? > ^ permalink raw reply [flat|nested] 31+ messages in thread
* [tarantool-patches] Re: [PATCH v5 0/6] box: run checks on insertions in LUA spaces 2019-05-23 10:19 [tarantool-patches] [PATCH v5 0/6] box: run checks on insertions in LUA spaces Kirill Shcherbatov ` (6 preceding siblings ...) 2019-06-03 21:15 ` [tarantool-patches] Re: [PATCH v5 0/6] box: run checks on insertions in LUA spaces Vladislav Shpilevoy @ 2019-06-06 11:58 ` Kirill Yukhin 7 siblings, 0 replies; 31+ messages in thread From: Kirill Yukhin @ 2019-06-06 11:58 UTC (permalink / raw) To: tarantool-patches; +Cc: v.shpilevoy, Kirill Shcherbatov Hello, On 23 May 13:19, Kirill Shcherbatov wrote: > Fire CK constraints for LUA spaces. > To achieve this goal, we reworked data dictionary, to store ck > constraints in separate space _ck_constraints and updated data > migration script to migrate existent data there. This also would > be useful in future to implement ALTER SPACE ADD CONSTRAINT > operation. Now we do not support CK constraint creation on > non-empty space. > Each CK has own precompiled VDBE machine that performs this > check with tuple fields mapped to it's memory with sql_bind() api. > In case of ck constraint conflict detected by this VM we abort > the transaction and return error to user. > Finally, we introduced a LUA-wrapper that provide a user-friendly > way to manage space ck constraints. > > Changes in version 5: > - a new tuple_fetcher class to access tuple fields fast > - refactored _ck_constraint space format and indexes > - one on_replace trigger for all ck constraints > - trim_space_snprintf fixes: do not trim substring in ' or " > - do not wrap ck_constraint methods > - many minor fixes > > v4: https://www.freelists.org/post/tarantool-patches/PATCH-v4-04-box-run-checks-on-insertions-in-LUA-spaces > > Branch: http://github.com/tarantool/tarantool/tree/kshch/gh-3691-checks-on-server-side > Issue: https://github.com/tarantool/tarantool/issues/3691 I've checked the patch set into master. -- Regards, Kirill Yukhin ^ permalink raw reply [flat|nested] 31+ messages in thread
end of thread, other threads:[~2019-06-06 11:58 UTC | newest] Thread overview: 31+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-05-23 10:19 [tarantool-patches] [PATCH v5 0/6] box: run checks on insertions in LUA spaces Kirill Shcherbatov 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 1/6] sql: introduce a new method to bind a pointer Kirill Shcherbatov 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 2/6] sql: refactor OP_Column vdbe instruction Kirill Shcherbatov 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 3/6] sql: introduce tuple_fetcher class Kirill Shcherbatov 2019-05-26 12:05 ` [tarantool-patches] " Vladislav Shpilevoy 2019-05-31 13:45 ` Kirill Shcherbatov 2019-05-31 19:45 ` Konstantin Osipov 2019-05-31 19:50 ` Kirill Shcherbatov 2019-05-31 22:36 ` Vladislav Shpilevoy 2019-06-01 5:45 ` Konstantin Osipov 2019-06-02 18:50 ` Kirill Shcherbatov 2019-06-03 21:15 ` Vladislav Shpilevoy 2019-06-05 6:47 ` Konstantin Osipov 2019-06-05 6:48 ` Konstantin Osipov 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 4/6] schema: add new system space for CHECK constraints Kirill Shcherbatov 2019-05-26 12:06 ` [tarantool-patches] " Vladislav Shpilevoy 2019-05-26 13:31 ` n.pettik 2019-05-26 13:32 ` Vladislav Shpilevoy 2019-05-31 13:45 ` Kirill Shcherbatov 2019-06-03 21:15 ` Vladislav Shpilevoy 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 5/6] box: run check constraint tests on space alter Kirill Shcherbatov 2019-05-26 12:07 ` [tarantool-patches] " Vladislav Shpilevoy 2019-05-31 13:45 ` Kirill Shcherbatov 2019-06-03 21:15 ` Vladislav Shpilevoy 2019-05-23 10:19 ` [tarantool-patches] [PATCH v5 6/6] box: user-friendly interface to manage ck constraints Kirill Shcherbatov 2019-05-26 12:07 ` [tarantool-patches] " Vladislav Shpilevoy 2019-05-31 13:45 ` Kirill Shcherbatov 2019-06-03 21:15 ` [tarantool-patches] Re: [PATCH v5 0/6] box: run checks on insertions in LUA spaces Vladislav Shpilevoy 2019-06-04 7:21 ` Kirill Shcherbatov 2019-06-04 18:59 ` Vladislav Shpilevoy 2019-06-06 11:58 ` Kirill Yukhin
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox