[Tarantool-patches] [PATCH v2 00/10] Stacked diagnostics

[Tarantool-patches] [PATCH v2 00/10] Stacked diagnostics

[Nikita Pettik]

Branch: https://github.com/tarantool/tarantool/commits/np/gh-1148-stacked-diag
Issue:
https://github.com/tarantool/tarantool/issues/1148
https://github.com/tarantool/tarantool/issues/4778

Patch-set basically consists of two parts: first one fixes undocumented
behaviour of box.error.new() which sets created error to diagnostics
area. The reason why it is required is described in the corresponding
github ticked. Second part is about stacked diagnostics itself: patch
error structure extending it with double linked list, provide Lua and
C interfaces to interact with it, support stacked diagnostics in IProto
protocol and net.box module (all points according to the rfc: 
https://github.com/tarantool/tarantool/commit/1acd32d98f628431429b427df19caa9d269bb9c8).

Changes in v2:

- renamed 'prev' and 'next' links in struct error to 'cause' and 'effect';
- moved all tests related to box.error module to a separate file (box/error.test.lua);
- removed recursion from error_unref();
- removed box_error_construct() and box_error_add() from public API;
- removed error_prev() getter which was used to access error.prev member in Lua
(now it is accessed directly via error._prev);
- moved diag_add() usages to a separate commit;
- several ref/unref usages corrections in the code;
- added mp_check_* auxiliary checks to iproto_decode_error_stack();
- other minor refactoring (removing unused headers and redundant testing
facilities, fixing comments etc);
- rebased to fresh master branch.

@ChangeLog
* box.error.new() does not longer add created error to Tarantool's diagnostic
area (gh-4778)
* Introduced stacked diagnostick area: now each Lua table representing error
object features .prev member and :set_prev() method. So that errors can be
organized into lists. IProto protocol is extended with new command keys to
support this feature as well (gh-1148).

Kirill Shcherbatov (3):
  box: rfc for stacked diagnostic area
  box: rename diag_add_error to diag_set_error
  iproto: refactor error encoding with mpstream

Nikita Pettik (7):
  test: move box.error tests to box/error.test.lua
  box/error: introduce box.error.set() method
  box/error: don't set error created via box.error.new to diag
  box: introduce stacked diagnostic area
  box: use stacked diagnostic area for functional indexes
  box/error: clarify purpose of reference counting in struct error
  iproto: support error stacked diagnostic area

 doc/rfc/1148-stacked-diagnostics.md | 225 ++++++++
 extra/exports                       |   1 +
 src/box/applier.cc                  |   2 +-
 src/box/error.cc                    |  33 ++
 src/box/error.h                     |  24 +
 src/box/iproto_constants.h          |   6 +
 src/box/key_list.c                  |  16 +-
 src/box/lua/call.c                  |   6 +-
 src/box/lua/error.cc                |  70 ++-
 src/box/lua/net_box.lua             |  32 +-
 src/box/relay.cc                    |   4 +-
 src/box/vy_scheduler.c              |   6 +-
 src/box/xrow.c                      | 158 +++++-
 src/lib/core/diag.c                 |  39 ++
 src/lib/core/diag.h                 | 111 +++-
 src/lib/core/exception.cc           |   3 +-
 src/lib/core/exception.h            |   2 +-
 src/lua/error.c                     |   2 +-
 src/lua/error.h                     |   3 +
 src/lua/error.lua                   |  32 ++
 src/lua/utils.c                     |   2 +-
 test/box-py/iproto.result           |   6 +-
 test/box-py/iproto.test.py          |   6 +-
 test/box/error.result               | 791 ++++++++++++++++++++++++++++
 test/box/error.test.lua             | 216 ++++++++
 test/box/iproto.result              | 141 +++++
 test/box/iproto.test.lua            |  62 +++
 test/box/misc.result                | 423 ---------------
 test/box/misc.test.lua              |  80 ---
 test/box/net.box.result             |  60 +++
 test/box/net.box.test.lua           |  23 +
 test/engine/func_index.result       |  50 +-
 test/engine/func_index.test.lua     |   7 +
 33 files changed, 2042 insertions(+), 600 deletions(-)
 create mode 100644 doc/rfc/1148-stacked-diagnostics.md
 create mode 100644 test/box/error.result
 create mode 100644 test/box/error.test.lua
 create mode 100644 test/box/iproto.result
 create mode 100644 test/box/iproto.test.lua

-- 
2.17.1

[Tarantool-patches] [PATCH v2 01/10] box: rfc for stacked diagnostic area

[Nikita Pettik]

From: Kirill Shcherbatov <kshcherbatov@tarantool.org>

Part of #1148
---
 doc/rfc/1148-stacked-diagnostics.md | 225 ++++++++++++++++++++++++++++
 1 file changed, 225 insertions(+)
 create mode 100644 doc/rfc/1148-stacked-diagnostics.md

diff --git a/doc/rfc/1148-stacked-diagnostics.md b/doc/rfc/1148-stacked-diagnostics.md
new file mode 100644
index 000000000..ccc588d46
--- /dev/null
+++ b/doc/rfc/1148-stacked-diagnostics.md
@@ -0,0 +1,225 @@
+# Stacked Diagnostics
+
+* **Status**: In progress
+* **Start date**: 30-07-2019
+* **Authors**: Kirill Shcherbatov @kshcherbatov kshcherbatov@tarantool.org,
+               Pettik Nikita @Korablev77 korablev@tarantool.org
+* **Issues**: [#1148](https://github.com/tarantool/<repository\>/issues/1148)
+
+
+## Summary
+
+The document describes a stacked diagnostics feature. It is needed for the cases,
+when there is a complex and huge call stack with variety of subsystems in it.
+It may turn out that the most low-level error is not descriptive enough. In turn
+user may want to be able to look at errors along the whole call stack from the
+place where a first error happened. In terms of implementation single fiber's
+error object is going to be replaced with a list of objects forming diagnostic
+stack. Its first element will always be created by the deepest and the most
+basic error. Both C and Lua APIs are extended to support adding errors in stack.
+
+## Background and motivation
+
+Support stacked diagnostics for Tarantool allows to accumulate all errors
+occurred during request processing. It allows to better understand what
+happened, and handle errors appropriately. Consider following example:
+persistent Lua function referenced by functional index has a bug in it's
+definition, Lua handler sets an diag message. Then functional index extractor
+code setups an own, more specialized error.
+
+### Current error diagnostics
+
+Currently Tarantool has `diag_set()` mechanism to set a diagnostic error.
+Object representing error featuring following properties:
+ - type (string) error’s C++ class;
+ - code (number) error’s number;
+ - message (string) error’s message;
+ - file (string) Tarantool source file;
+ - line (number) line number in the Tarantool source file.
+
+The last error raised is exported with `box.error.last()` function.
+
+Type of error is represented by a few C++ classes (all are inherited from
+Exception class). For instance hierarchy for ClientError is following:
+```
+ClientError
+ | LoggedError
+ | AccessDeniedError
+ | UnsupportedIndexFeature
+```
+
+All codes and names of ClientError class are available in box.error.
+User is able to create a new error instance of predefined type using
+box.error.new() function. For example:
+```
+tarantool> t = box.error.new(box.error.CREATE_SPACE, "myspace", "just cause")
+tarantool> t:unpack()
+---
+- type: ClientError
+  code: 9
+  message: 'Failed to create space ''myspace'': just cause'
+  trace:
+  - file: '[string "t = box.error.new(box.error.CREATE_SPACE, "my..."]'
+    line: 1
+```
+
+User is also capable of defining own errors with any code  by means of:
+```
+box.error.new({code = user_code, reason = user_error_msg})
+```
+For instance:
+```
+e = box.error.new({code = 500, reason = 'just cause'})
+```
+
+Error cdata object has `:unpack()`, `:raise()`, `:match(...)`, `:__serialize()`
+methods and `.type`, `.message` and `.trace` fields.
+
+## Proposal
+
+In some cases a diagnostic area should be more complicated than
+one last raised error to provide decent information concerning incident (see
+motivating example above). Without stacked diagnostic area, only last error is
+delivered to user. One way to deal with this problem is to introduce stack
+accumulating all errors happened during request processing.
+
+### C API
+
+Let's keep existent `diag_set()` method as is. It is supposed to replace the
+last error in diagnostic area with a new one. To add new error at the top of
+existing one, let's introduce new method `diag_add()`. It is assumed to keep
+an existent error message in diagnostic area (if any) and sets it as a previous
+error for a recently-constructed error object. Note that `diag_set()` is not
+going to preserve pointer to previous error which is held in error to be
+substituted. To illustrate last point consider example:
+
+```
+0. Errors: <NULL>
+1. diag_set(code = 1)
+Errors: <e1(code = 1) -> NULL>
+2. diag_add(code = 2)
+Errors: <e1(code = 1) -> e2(code = 2) -> NULL>
+3. diag_set(code = 3)
+Errors: <e3(code = 3) -> NULL>
+```
+
+Hence, developer takes responsibility of placing `diag_set()` where the most
+basic error should be raised. For instance, if during request processing
+`diag_add()` is called before `diag_set()` then it will result in inheritance
+of all errors from previous error raise:
+
+```
+-- Processing of request #1
+1. diag_set(code = 1)
+Errors: <e1(code = 1) -> NULL>
+2. diag_add(code = 2)
+Errors: <e1(code = 1) -> e2(code = 2) -> NULL>
+-- End of execution
+
+-- Processing of request #2
+1. diag_add(code = 1)
+Errors: <e1(code = 1) -> e2(code = 2) -> e3(code = 1) -> NULL>
+-- End of execution
+```
+
+As a result, at the end of execution of second request, three errors in
+stack are reported instead of one.
+
+Another way to resolve this issue is to erase diagnostic area before
+request processing. However, it breaks current user-visible behaviour
+since box.error.last() will preserve last occurred error only until execution
+of the next request.
+
+The diagnostic area (now) contains (nothing but) pointer to the top error:
+```
+struct diag {
+  struct error *last;
+};
+
+```
+
+To organize errors in a list let's extend error structure with pointer to
+the previous element. Or alternatively, add member of any data structure
+providing list properties (struct rlist, struct stailq or whatever):
+```
+struct diag {
+  struct stailq *errors;
+};
+
+struct error {
+   ...
+   struct stailq_entry *in_errors;
+};
+```
+When error is set to diagnostics area, its reference counter is incremented;
+on the other hand if error is added (i.e. linked to the head of diagnostics
+area list), its reference counter remains unchanged. The same concerns linking
+two errors: only counter of referenced error is incremented. During error
+destruction (that is the moment when error's reference counter hits 0 value)
+the next error in the list (if any) is also affected: its reference counter
+is decremented as well.
+
+### Lua API
+
+Tarantool returns a last-set (diag::last) error as `cdata` object from central
+diagnostic area to Lua in case of error. User should be unable to modify it
+(since it is considered to be a bad practice - in fact object doesn't belong
+to user). On the other hand, user needs an ability to inspect a collected
+diagnostic information. Hence, let's extend the error object API with a method
+which provides the way to get the previous error (reason): `:prev()` (and
+correspondingly `.prev` field).
+
+```
+-- Return a reason error object for given error object 'e'
+-- (when exists, nil otherwise).
+e:prev(error) == error.prev
+```
+
+Furthermore, let's extend signature of `box.error.new()` with new (optional)
+argument - 'prev' - previous error object:
+
+```
+e1 = box.error.new({code = 111, reason = "just cause"})
+e2 = box.error.new({code = 222, reason = "just cause x2", prev = e1})
+```
+
+User may want to link already existing errors. To achieve this let's add
+`set_prev` method to error object so that one can join two errors:
+```
+e1 = box.error.new({code = 111, reason = "just cause"})
+e2 = box.error.new({code = 222, reason = "just cause x2"})
+...
+e2.set_prev(e1) -- e2.prev == e1
+```
+### Binary protocol
+
+Currently errors are sent as `(IPROTO_ERROR | errcode)` response with an
+string message containing error details as a payload. There are not so many
+options to extend current protocol wihtout breaking backward compatibility
+(which is obviously one of implementation requirements). One way is to extend
+existent binary protocol with a new key IPROTO_ERROR_STACK (or
+IPROTO_ERROR_REASON or simply IPROTO_ERROR_V2):
+```
+{
+        // backward compatibility
+        IPROTO_ERROR: "the most recent error message",
+        // modern error message
+        IPROTO_ERROR_STACK: {
+                {
+                        // the most recent error object
+                        IPROTO_ERROR_CODE: error_code_number,
+                        IPROTO_ERROR_MESSAGE: error_message_string,
+                },
+                ...
+                {
+                        // the oldest (reason) error object
+                },
+        }
+}
+```
+
+IPROTO_ERROR is always sent (as in older versions) in case of error.
+IPROTO_ERROR_STACK is presented in response only if there's at least two
+elements in diagnostic list. Map which contains error stack can be optimized
+in terms of space, so that avoid including error which is already encoded
+in IPROTO_ERROR.
-- 
2.17.1

Re: [Tarantool-patches] [PATCH v2 01/10] box: rfc for stacked diagnostic area

[Konstantin Osipov]

* Nikita Pettik <korablev@tarantool.org> [20/03/25 09:32]:
> From: Kirill Shcherbatov <kshcherbatov@tarantool.org>
> 
> Part of #1148

This RFC is lgtm.

I wonder how it corresponds with capability negotiations. If we
have capability negotiations, we don't need iproto_error_stack
*and* iproto_error messages in the protocol, we can negotiate with
the client whether or not it's ready to receive
iproto_error_stack.

However, I think capability negotiation in client-server protocol
is an overkill. Gradually phasing out old client support is a cleaner
strategy. And simply preserving backward compatibility, which is
already possible with extra fields ignored by old clients, is even better.

-- 
Konstantin Osipov, Moscow, Russia

Re: [Tarantool-patches] [PATCH v2 01/10] box: rfc for stacked diagnostic area

[Nikita Pettik]

On 25 Mar 11:27, Konstantin Osipov wrote:
> * Nikita Pettik <korablev@tarantool.org> [20/03/25 09:32]:
> > From: Kirill Shcherbatov <kshcherbatov@tarantool.org>
> > 
> > Part of #1148
> 
> This RFC is lgtm.
> 
> I wonder how it corresponds with capability negotiations. If we
> have capability negotiations, we don't need iproto_error_stack
> *and* iproto_error messages in the protocol, we can negotiate with
> the client whether or not it's ready to receive
> iproto_error_stack.
> 
> However, I think capability negotiation in client-server protocol
> is an overkill. Gradually phasing out old client support is a cleaner
> strategy. And simply preserving backward compatibility, which is
> already possible with extra fields ignored by old clients, is even better.

Agree, I said it long ago.
 
> -- 
> Konstantin Osipov, Moscow, Russia

[Tarantool-patches] [PATCH v2 02/10] box: rename diag_add_error to diag_set_error

[Nikita Pettik]

From: Kirill Shcherbatov <kshcherbatov@tarantool.org>

Let's rename diag_add_error() to diag_set_error() because it actually
replaces an error object in diagnostic area with a new one and this name
is not representative. Moreover, we are going to introduce a new
diag_add_error() which will place error at the top of stack diagnostic
area.

Needed for #1148
---
 src/box/applier.cc        | 2 +-
 src/box/error.cc          | 2 +-
 src/box/relay.cc          | 4 ++--
 src/box/vy_scheduler.c    | 6 +++---
 src/lib/core/diag.h       | 6 +++---
 src/lib/core/exception.cc | 2 +-
 src/lib/core/exception.h  | 2 +-
 src/lua/utils.c           | 2 +-
 8 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/src/box/applier.cc b/src/box/applier.cc
index 8666a3a98..47a26c366 100644
--- a/src/box/applier.cc
+++ b/src/box/applier.cc
@@ -843,7 +843,7 @@ applier_on_rollback(struct trigger *trigger, void *event)
 	struct applier *applier = (struct applier *)trigger->data;
 	/* Setup a shared error. */
 	if (!diag_is_empty(&replicaset.applier.diag)) {
-		diag_add_error(&applier->diag,
+		diag_set_error(&applier->diag,
 			       diag_last_error(&replicaset.applier.diag));
 	}
 	/* Stop the applier fiber. */
diff --git a/src/box/error.cc b/src/box/error.cc
index 47dce3305..7dfe1b3ee 100644
--- a/src/box/error.cc
+++ b/src/box/error.cc
@@ -82,7 +82,7 @@ box_error_set(const char *file, unsigned line, uint32_t code,
 		error_vformat_msg(e, fmt, ap);
 		va_end(ap);
 	}
-	diag_add_error(&fiber()->diag, e);
+	diag_set_error(&fiber()->diag, e);
 	return -1;
 }
 
diff --git a/src/box/relay.cc b/src/box/relay.cc
index 95245a3cf..c634348a4 100644
--- a/src/box/relay.cc
+++ b/src/box/relay.cc
@@ -479,7 +479,7 @@ relay_set_error(struct relay *relay, struct error *e)
 {
 	/* Don't override existing error. */
 	if (diag_is_empty(&relay->diag))
-		diag_add_error(&relay->diag, e);
+		diag_set_error(&relay->diag, e);
 }
 
 static void
@@ -658,7 +658,7 @@ relay_subscribe_f(va_list ap)
 	 * Don't clear the error for status reporting.
 	 */
 	assert(!diag_is_empty(&relay->diag));
-	diag_add_error(diag_get(), diag_last_error(&relay->diag));
+	diag_set_error(diag_get(), diag_last_error(&relay->diag));
 	diag_log();
 	say_crit("exiting the relay loop");
 
diff --git a/src/box/vy_scheduler.c b/src/box/vy_scheduler.c
index acc909d09..bf4c3fe58 100644
--- a/src/box/vy_scheduler.c
+++ b/src/box/vy_scheduler.c
@@ -619,7 +619,7 @@ vy_scheduler_dump(struct vy_scheduler *scheduler)
 		if (scheduler->is_throttled) {
 			/* Dump error occurred. */
 			struct error *e = diag_last_error(&scheduler->diag);
-			diag_add_error(diag_get(), e);
+			diag_set_error(diag_get(), e);
 			return -1;
 		}
 		fiber_cond_wait(&scheduler->dump_cond);
@@ -693,7 +693,7 @@ vy_scheduler_begin_checkpoint(struct vy_scheduler *scheduler)
 	 */
 	if (scheduler->is_throttled) {
 		struct error *e = diag_last_error(&scheduler->diag);
-		diag_add_error(diag_get(), e);
+		diag_set_error(diag_get(), e);
 		say_error("cannot checkpoint vinyl, "
 			  "scheduler is throttled with: %s", e->errmsg);
 		return -1;
@@ -729,7 +729,7 @@ vy_scheduler_wait_checkpoint(struct vy_scheduler *scheduler)
 		if (scheduler->is_throttled) {
 			/* A dump error occurred, abort checkpoint. */
 			struct error *e = diag_last_error(&scheduler->diag);
-			diag_add_error(diag_get(), e);
+			diag_set_error(diag_get(), e);
 			say_error("vinyl checkpoint failed: %s", e->errmsg);
 			return -1;
 		}
diff --git a/src/lib/core/diag.h b/src/lib/core/diag.h
index f763957c2..7e1e1a174 100644
--- a/src/lib/core/diag.h
+++ b/src/lib/core/diag.h
@@ -168,12 +168,12 @@ diag_clear(struct diag *diag)
 }
 
 /**
- * Add a new error to the diagnostics area
+ * Set a new error to the diagnostics area, replacing existent.
  * \param diag diagnostics area
  * \param e error to add
  */
 static inline void
-diag_add_error(struct diag *diag, struct error *e)
+diag_set_error(struct diag *diag, struct error *e)
 {
 	assert(e != NULL);
 	error_ref(e);
@@ -275,7 +275,7 @@ BuildSocketError(const char *file, unsigned line, const char *socketname,
 	say_debug("%s at %s:%i", #class, __FILE__, __LINE__);		\
 	struct error *e;						\
 	e = Build##class(__FILE__, __LINE__, ##__VA_ARGS__);		\
-	diag_add_error(diag_get(), e);					\
+	diag_set_error(diag_get(), e);					\
 	/* Restore the errno which might have been reset.  */           \
 	errno = save_errno;                                             \
 } while (0)
diff --git a/src/lib/core/exception.cc b/src/lib/core/exception.cc
index 76dcea553..0ab10c4bd 100644
--- a/src/lib/core/exception.cc
+++ b/src/lib/core/exception.cc
@@ -99,7 +99,7 @@ Exception::operator new(size_t size)
 	void *buf = malloc(size);
 	if (buf != NULL)
 		return buf;
-	diag_add_error(diag_get(), &out_of_memory);
+	diag_set_error(diag_get(), &out_of_memory);
 	throw &out_of_memory;
 }
 
diff --git a/src/lib/core/exception.h b/src/lib/core/exception.h
index d6154eb32..1947b4f00 100644
--- a/src/lib/core/exception.h
+++ b/src/lib/core/exception.h
@@ -177,7 +177,7 @@ exception_init();
 #define tnt_error(class, ...) ({					\
 	say_debug("%s at %s:%i", #class, __FILE__, __LINE__);		\
 	class *e = new class(__FILE__, __LINE__, ##__VA_ARGS__);	\
-	diag_add_error(diag_get(), e);					\
+	diag_set_error(diag_get(), e);					\
 	e;								\
 })
 
diff --git a/src/lua/utils.c b/src/lua/utils.c
index de14c778c..54d18ac89 100644
--- a/src/lua/utils.c
+++ b/src/lua/utils.c
@@ -1022,7 +1022,7 @@ luaT_toerror(lua_State *L)
 	struct error *e = luaL_iserror(L, -1);
 	if (e != NULL) {
 		/* Re-throw original error */
-		diag_add_error(&fiber()->diag, e);
+		diag_set_error(&fiber()->diag, e);
 	} else {
 		/* Convert Lua error to a Tarantool exception. */
 		diag_set(LuajitError, luaT_tolstring(L, -1, NULL));
-- 
2.17.1

Re: [Tarantool-patches] [PATCH v2 02/10] box: rename diag_add_error to diag_set_error

[Konstantin Osipov]

* Nikita Pettik <korablev@tarantool.org> [20/03/25 09:32]:
> From: Kirill Shcherbatov <kshcherbatov@tarantool.org>

lgtm/obvious patch. 

-- 
Konstantin Osipov, Moscow, Russia

Re: [Tarantool-patches] [PATCH v2 02/10] box: rename diag_add_error to diag_set_error

[Vladislav Shpilevoy]

Hi! Thanks for the patch!

The commit LGTM. Lets push it right away.

On 25/03/2020 02:42, Nikita Pettik wrote:
> From: Kirill Shcherbatov <kshcherbatov@tarantool.org>
> 
> Let's rename diag_add_error() to diag_set_error() because it actually
> replaces an error object in diagnostic area with a new one and this name
> is not representative. Moreover, we are going to introduce a new
> diag_add_error() which will place error at the top of stack diagnostic
> area.
> 
> Needed for #1148

Re: [Tarantool-patches] [PATCH v2 02/10] box: rename diag_add_error to diag_set_error

[Nikita Pettik]

On 26 Mar 01:22, Vladislav Shpilevoy wrote:
> Hi! Thanks for the patch!
> 
> The commit LGTM. Lets push it right away.

Pushed to master.
 
> On 25/03/2020 02:42, Nikita Pettik wrote:
> > From: Kirill Shcherbatov <kshcherbatov@tarantool.org>
> > 
> > Let's rename diag_add_error() to diag_set_error() because it actually
> > replaces an error object in diagnostic area with a new one and this name
> > is not representative. Moreover, we are going to introduce a new
> > diag_add_error() which will place error at the top of stack diagnostic
> > area.
> > 
> > Needed for #1148

[Tarantool-patches] [PATCH v2 03/10] test: move box.error tests to box/error.test.lua

[Nikita Pettik]

We are going to introduce more tests related to error module, so let's
move all error-related tests from box/misc.test.lua to a separate test
file (box/error.test.lua).

Needed for #1148
---
 test/box/error.result   | 441 ++++++++++++++++++++++++++++++++++++++++
 test/box/error.test.lua |  80 ++++++++
 test/box/misc.result    | 423 --------------------------------------
 test/box/misc.test.lua  |  80 --------
 4 files changed, 521 insertions(+), 503 deletions(-)
 create mode 100644 test/box/error.result
 create mode 100644 test/box/error.test.lua

diff --git a/test/box/error.result b/test/box/error.result
new file mode 100644
index 000000000..9b6c8b462
--- /dev/null
+++ b/test/box/error.result
@@ -0,0 +1,441 @@
+-- test-run result file version 2
+env = require('test_run')
+ | ---
+ | ...
+test_run = env.new()
+ | ---
+ | ...
+
+space = box.schema.space.create('tweedledum')
+ | ---
+ | ...
+index = space:create_index('primary', { type = 'hash' })
+ | ---
+ | ...
+
+box.error({code = 123, reason = 'test'})
+ | ---
+ | - error: test
+ | ...
+box.error(box.error.ILLEGAL_PARAMS, "bla bla")
+ | ---
+ | - error: Illegal parameters, bla bla
+ | ...
+box.error()
+ | ---
+ | - error: Illegal parameters, bla bla
+ | ...
+e = box.error.last()
+ | ---
+ | ...
+e
+ | ---
+ | - Illegal parameters, bla bla
+ | ...
+e:unpack()
+ | ---
+ | - type: ClientError
+ |   code: 1
+ |   message: Illegal parameters, bla bla
+ |   trace:
+ |   - file: '[C]'
+ |     line: 4294967295
+ | ...
+e.type
+ | ---
+ | - ClientError
+ | ...
+e.code
+ | ---
+ | - 1
+ | ...
+e.message
+ | ---
+ | - Illegal parameters, bla bla
+ | ...
+tostring(e)
+ | ---
+ | - Illegal parameters, bla bla
+ | ...
+e = nil
+ | ---
+ | ...
+box.error.clear()
+ | ---
+ | ...
+box.error.last()
+ | ---
+ | - null
+ | ...
+space = box.space.tweedledum
+ | ---
+ | ...
+
+--
+-- gh-2080: box.error() crashes with wrong parameters
+box.error(box.error.UNSUPPORTED, "x", "x%s")
+ | ---
+ | - error: x does not support x%s
+ | ...
+box.error(box.error.UNSUPPORTED, "x")
+ | ---
+ | - error: 'bad argument #3 to ''?'' (no value)'
+ | ...
+box.error(box.error.UNSUPPORTED)
+ | ---
+ | - error: 'box.error(): bad arguments'
+ | ...
+
+--
+-- gh-3031: allow to create an error object with no throwing it.
+--
+e = box.error.new(box.error.UNKNOWN)
+ | ---
+ | ...
+e
+ | ---
+ | - Unknown error
+ | ...
+e = box.error.new(box.error.CREATE_SPACE, "space", "error")
+ | ---
+ | ...
+e
+ | ---
+ | - 'Failed to create space ''space'': error'
+ | ...
+box.error.new()
+ | ---
+ | - error: 'Usage: box.error.new(code, args)'
+ | ...
+
+--
+-- gh-4489: box.error has __concat metamethod
+--
+test_run:cmd("push filter '(.builtin/.*.lua):[0-9]+' to '\\1'")
+ | ---
+ | - true
+ | ...
+e = box.error.new(box.error.UNKNOWN)
+ | ---
+ | ...
+'left side: ' .. e
+ | ---
+ | - 'left side: Unknown error'
+ | ...
+e .. ': right side'
+ | ---
+ | - 'Unknown error: right side'
+ | ...
+e .. nil
+ | ---
+ | - error: 'builtin/error.lua: attempt to concatenate local ''rhs'' (a nil value)'
+ | ...
+nil .. e
+ | ---
+ | - error: 'builtin/error.lua: attempt to concatenate local ''lhs'' (a nil value)'
+ | ...
+e .. box.NULL
+ | ---
+ | - error: 'builtin/error.lua: attempt to concatenate ''string'' and ''void *'''
+ | ...
+box.NULL .. e
+ | ---
+ | - error: 'builtin/error.lua: attempt to concatenate ''void *'' and ''string'''
+ | ...
+123 .. e
+ | ---
+ | - 123Unknown error
+ | ...
+e .. 123
+ | ---
+ | - Unknown error123
+ | ...
+e .. e
+ | ---
+ | - Unknown errorUnknown error
+ | ...
+e .. {}
+ | ---
+ | - error: 'builtin/error.lua: attempt to concatenate local ''rhs'' (a table value)'
+ | ...
+{} .. e
+ | ---
+ | - error: 'builtin/error.lua: attempt to concatenate local ''lhs'' (a table value)'
+ | ...
+-1ULL .. e
+ | ---
+ | - error: 'builtin/error.lua: attempt to concatenate ''uint64_t'' and ''string'''
+ | ...
+e .. -1ULL
+ | ---
+ | - error: 'builtin/error.lua: attempt to concatenate ''string'' and ''uint64_t'''
+ | ...
+1LL .. e
+ | ---
+ | - error: 'builtin/error.lua: attempt to concatenate ''int64_t'' and ''string'''
+ | ...
+e .. 1LL
+ | ---
+ | - error: 'builtin/error.lua: attempt to concatenate ''string'' and ''int64_t'''
+ | ...
+e = nil
+ | ---
+ | ...
+
+--
+-- System errors expose errno as a field.
+--
+_, err = require('fio').open('not_existing_file')
+ | ---
+ | ...
+type(err.errno)
+ | ---
+ | - number
+ | ...
+-- Errors not related to the standard library do
+-- not expose errno.
+err = box.error.new(box.error.PROC_LUA, "errno")
+ | ---
+ | ...
+type(err.errno)
+ | ---
+ | - nil
+ | ...
+
+t = {}
+ | ---
+ | ...
+test_run:cmd("setopt delimiter ';'")
+ | ---
+ | - true
+ | ...
+
+for k,v in pairs(box.error) do
+   if type(v) == 'number' then
+    t[v] = 'box.error.'..tostring(k)
+   end
+end;
+ | ---
+ | ...
+t;
+ | ---
+ | - 0: box.error.UNKNOWN
+ |   1: box.error.ILLEGAL_PARAMS
+ |   2: box.error.MEMORY_ISSUE
+ |   3: box.error.TUPLE_FOUND
+ |   4: box.error.TUPLE_NOT_FOUND
+ |   5: box.error.UNSUPPORTED
+ |   6: box.error.NONMASTER
+ |   7: box.error.READONLY
+ |   8: box.error.INJECTION
+ |   9: box.error.CREATE_SPACE
+ |   10: box.error.SPACE_EXISTS
+ |   11: box.error.DROP_SPACE
+ |   12: box.error.ALTER_SPACE
+ |   13: box.error.INDEX_TYPE
+ |   14: box.error.MODIFY_INDEX
+ |   15: box.error.LAST_DROP
+ |   16: box.error.TUPLE_FORMAT_LIMIT
+ |   17: box.error.DROP_PRIMARY_KEY
+ |   18: box.error.KEY_PART_TYPE
+ |   19: box.error.EXACT_MATCH
+ |   20: box.error.INVALID_MSGPACK
+ |   21: box.error.PROC_RET
+ |   22: box.error.TUPLE_NOT_ARRAY
+ |   23: box.error.FIELD_TYPE
+ |   24: box.error.INDEX_PART_TYPE_MISMATCH
+ |   25: box.error.UPDATE_SPLICE
+ |   26: box.error.UPDATE_ARG_TYPE
+ |   27: box.error.FORMAT_MISMATCH_INDEX_PART
+ |   28: box.error.UNKNOWN_UPDATE_OP
+ |   29: box.error.UPDATE_FIELD
+ |   30: box.error.FUNCTION_TX_ACTIVE
+ |   31: box.error.KEY_PART_COUNT
+ |   32: box.error.PROC_LUA
+ |   33: box.error.NO_SUCH_PROC
+ |   34: box.error.NO_SUCH_TRIGGER
+ |   35: box.error.NO_SUCH_INDEX_ID
+ |   36: box.error.NO_SUCH_SPACE
+ |   37: box.error.NO_SUCH_FIELD_NO
+ |   38: box.error.EXACT_FIELD_COUNT
+ |   39: box.error.FIELD_MISSING
+ |   40: box.error.WAL_IO
+ |   41: box.error.MORE_THAN_ONE_TUPLE
+ |   42: box.error.ACCESS_DENIED
+ |   43: box.error.CREATE_USER
+ |   44: box.error.DROP_USER
+ |   45: box.error.NO_SUCH_USER
+ |   46: box.error.USER_EXISTS
+ |   47: box.error.PASSWORD_MISMATCH
+ |   48: box.error.UNKNOWN_REQUEST_TYPE
+ |   49: box.error.UNKNOWN_SCHEMA_OBJECT
+ |   50: box.error.CREATE_FUNCTION
+ |   51: box.error.NO_SUCH_FUNCTION
+ |   52: box.error.FUNCTION_EXISTS
+ |   53: box.error.BEFORE_REPLACE_RET
+ |   54: box.error.MULTISTATEMENT_TRANSACTION
+ |   55: box.error.TRIGGER_EXISTS
+ |   56: box.error.USER_MAX
+ |   57: box.error.NO_SUCH_ENGINE
+ |   58: box.error.RELOAD_CFG
+ |   59: box.error.CFG
+ |   60: box.error.SAVEPOINT_EMPTY_TX
+ |   61: box.error.NO_SUCH_SAVEPOINT
+ |   62: box.error.UNKNOWN_REPLICA
+ |   63: box.error.REPLICASET_UUID_MISMATCH
+ |   64: box.error.INVALID_UUID
+ |   65: box.error.REPLICASET_UUID_IS_RO
+ |   66: box.error.INSTANCE_UUID_MISMATCH
+ |   68: box.error.INVALID_ORDER
+ |   69: box.error.MISSING_REQUEST_FIELD
+ |   70: box.error.IDENTIFIER
+ |   71: box.error.DROP_FUNCTION
+ |   72: box.error.ITERATOR_TYPE
+ |   73: box.error.REPLICA_MAX
+ |   74: box.error.INVALID_XLOG
+ |   75: box.error.INVALID_XLOG_NAME
+ |   76: box.error.INVALID_XLOG_ORDER
+ |   77: box.error.NO_CONNECTION
+ |   78: box.error.TIMEOUT
+ |   79: box.error.ACTIVE_TRANSACTION
+ |   80: box.error.CURSOR_NO_TRANSACTION
+ |   81: box.error.CROSS_ENGINE_TRANSACTION
+ |   82: box.error.NO_SUCH_ROLE
+ |   83: box.error.ROLE_EXISTS
+ |   84: box.error.CREATE_ROLE
+ |   85: box.error.INDEX_EXISTS
+ |   86: box.error.SESSION_CLOSED
+ |   87: box.error.ROLE_LOOP
+ |   88: box.error.GRANT
+ |   89: box.error.PRIV_GRANTED
+ |   90: box.error.ROLE_GRANTED
+ |   91: box.error.PRIV_NOT_GRANTED
+ |   92: box.error.ROLE_NOT_GRANTED
+ |   93: box.error.MISSING_SNAPSHOT
+ |   94: box.error.CANT_UPDATE_PRIMARY_KEY
+ |   95: box.error.UPDATE_INTEGER_OVERFLOW
+ |   96: box.error.GUEST_USER_PASSWORD
+ |   97: box.error.TRANSACTION_CONFLICT
+ |   98: box.error.UNSUPPORTED_PRIV
+ |   99: box.error.LOAD_FUNCTION
+ |   100: box.error.FUNCTION_LANGUAGE
+ |   101: box.error.RTREE_RECT
+ |   102: box.error.PROC_C
+ |   103: box.error.UNKNOWN_RTREE_INDEX_DISTANCE_TYPE
+ |   104: box.error.PROTOCOL
+ |   105: box.error.UPSERT_UNIQUE_SECONDARY_KEY
+ |   106: box.error.WRONG_INDEX_RECORD
+ |   107: box.error.WRONG_INDEX_PARTS
+ |   108: box.error.WRONG_INDEX_OPTIONS
+ |   109: box.error.WRONG_SCHEMA_VERSION
+ |   110: box.error.MEMTX_MAX_TUPLE_SIZE
+ |   111: box.error.WRONG_SPACE_OPTIONS
+ |   112: box.error.UNSUPPORTED_INDEX_FEATURE
+ |   113: box.error.VIEW_IS_RO
+ |   114: box.error.NO_TRANSACTION
+ |   115: box.error.SYSTEM
+ |   116: box.error.LOADING
+ |   117: box.error.CONNECTION_TO_SELF
+ |   118: box.error.KEY_PART_IS_TOO_LONG
+ |   119: box.error.COMPRESSION
+ |   120: box.error.CHECKPOINT_IN_PROGRESS
+ |   121: box.error.SUB_STMT_MAX
+ |   122: box.error.COMMIT_IN_SUB_STMT
+ |   123: box.error.ROLLBACK_IN_SUB_STMT
+ |   124: box.error.DECOMPRESSION
+ |   125: box.error.INVALID_XLOG_TYPE
+ |   126: box.error.ALREADY_RUNNING
+ |   127: box.error.INDEX_FIELD_COUNT_LIMIT
+ |   128: box.error.LOCAL_INSTANCE_ID_IS_READ_ONLY
+ |   129: box.error.BACKUP_IN_PROGRESS
+ |   130: box.error.READ_VIEW_ABORTED
+ |   131: box.error.INVALID_INDEX_FILE
+ |   132: box.error.INVALID_RUN_FILE
+ |   133: box.error.INVALID_VYLOG_FILE
+ |   134: box.error.CHECKPOINT_ROLLBACK
+ |   135: box.error.VY_QUOTA_TIMEOUT
+ |   136: box.error.PARTIAL_KEY
+ |   137: box.error.TRUNCATE_SYSTEM_SPACE
+ |   138: box.error.LOAD_MODULE
+ |   139: box.error.VINYL_MAX_TUPLE_SIZE
+ |   140: box.error.WRONG_DD_VERSION
+ |   141: box.error.WRONG_SPACE_FORMAT
+ |   142: box.error.CREATE_SEQUENCE
+ |   143: box.error.ALTER_SEQUENCE
+ |   144: box.error.DROP_SEQUENCE
+ |   145: box.error.NO_SUCH_SEQUENCE
+ |   146: box.error.SEQUENCE_EXISTS
+ |   147: box.error.SEQUENCE_OVERFLOW
+ |   148: box.error.NO_SUCH_INDEX_NAME
+ |   149: box.error.SPACE_FIELD_IS_DUPLICATE
+ |   150: box.error.CANT_CREATE_COLLATION
+ |   151: box.error.WRONG_COLLATION_OPTIONS
+ |   152: box.error.NULLABLE_PRIMARY
+ |   153: box.error.NO_SUCH_FIELD_NAME_IN_SPACE
+ |   154: box.error.TRANSACTION_YIELD
+ |   155: box.error.NO_SUCH_GROUP
+ |   156: box.error.SQL_BIND_VALUE
+ |   157: box.error.SQL_BIND_TYPE
+ |   158: box.error.SQL_BIND_PARAMETER_MAX
+ |   159: box.error.SQL_EXECUTE
+ |   160: box.error.UPDATE_DECIMAL_OVERFLOW
+ |   161: box.error.SQL_BIND_NOT_FOUND
+ |   162: box.error.ACTION_MISMATCH
+ |   163: box.error.VIEW_MISSING_SQL
+ |   164: box.error.FOREIGN_KEY_CONSTRAINT
+ |   165: box.error.NO_SUCH_MODULE
+ |   166: box.error.NO_SUCH_COLLATION
+ |   167: box.error.CREATE_FK_CONSTRAINT
+ |   168: box.error.DROP_FK_CONSTRAINT
+ |   169: box.error.NO_SUCH_CONSTRAINT
+ |   170: box.error.CONSTRAINT_EXISTS
+ |   171: box.error.SQL_TYPE_MISMATCH
+ |   172: box.error.ROWID_OVERFLOW
+ |   173: box.error.DROP_COLLATION
+ |   174: box.error.ILLEGAL_COLLATION_MIX
+ |   175: box.error.SQL_NO_SUCH_PRAGMA
+ |   176: box.error.SQL_CANT_RESOLVE_FIELD
+ |   177: box.error.INDEX_EXISTS_IN_SPACE
+ |   178: box.error.INCONSISTENT_TYPES
+ |   179: box.error.SQL_SYNTAX_WITH_POS
+ |   180: box.error.SQL_STACK_OVERFLOW
+ |   181: box.error.SQL_SELECT_WILDCARD
+ |   182: box.error.SQL_STATEMENT_EMPTY
+ |   184: box.error.SQL_SYNTAX_NEAR_TOKEN
+ |   185: box.error.SQL_UNKNOWN_TOKEN
+ |   186: box.error.SQL_PARSER_GENERIC
+ |   187: box.error.SQL_ANALYZE_ARGUMENT
+ |   188: box.error.SQL_COLUMN_COUNT_MAX
+ |   189: box.error.HEX_LITERAL_MAX
+ |   190: box.error.INT_LITERAL_MAX
+ |   191: box.error.SQL_PARSER_LIMIT
+ |   192: box.error.INDEX_DEF_UNSUPPORTED
+ |   193: box.error.CK_DEF_UNSUPPORTED
+ |   194: box.error.MULTIKEY_INDEX_MISMATCH
+ |   195: box.error.CREATE_CK_CONSTRAINT
+ |   196: box.error.CK_CONSTRAINT_FAILED
+ |   197: box.error.SQL_COLUMN_COUNT
+ |   198: box.error.FUNC_INDEX_FUNC
+ |   199: box.error.FUNC_INDEX_FORMAT
+ |   200: box.error.FUNC_INDEX_PARTS
+ |   201: box.error.NO_SUCH_FIELD_NAME
+ |   202: box.error.FUNC_WRONG_ARG_COUNT
+ |   203: box.error.BOOTSTRAP_READONLY
+ |   204: box.error.SQL_FUNC_WRONG_RET_COUNT
+ |   205: box.error.FUNC_INVALID_RETURN_TYPE
+ |   206: box.error.SQL_PARSER_GENERIC_WITH_POS
+ |   207: box.error.REPLICA_NOT_ANON
+ |   208: box.error.CANNOT_REGISTER
+ |   209: box.error.SESSION_SETTING_INVALID_VALUE
+ |   210: box.error.SQL_PREPARE
+ |   211: box.error.WRONG_QUERY_ID
+ |   212: box.error.SEQUENCE_NOT_STARTED
+ | ...
+
+test_run:cmd("setopt delimiter ''");
+ | ---
+ | - true
+ | ...
+space:drop()
+ | ---
+ | ...
diff --git a/test/box/error.test.lua b/test/box/error.test.lua
new file mode 100644
index 000000000..d16cc5672
--- /dev/null
+++ b/test/box/error.test.lua
@@ -0,0 +1,80 @@
+env = require('test_run')
+test_run = env.new()
+
+space = box.schema.space.create('tweedledum')
+index = space:create_index('primary', { type = 'hash' })
+
+box.error({code = 123, reason = 'test'})
+box.error(box.error.ILLEGAL_PARAMS, "bla bla")
+box.error()
+e = box.error.last()
+e
+e:unpack()
+e.type
+e.code
+e.message
+tostring(e)
+e = nil
+box.error.clear()
+box.error.last()
+space = box.space.tweedledum
+
+--
+-- gh-2080: box.error() crashes with wrong parameters
+box.error(box.error.UNSUPPORTED, "x", "x%s")
+box.error(box.error.UNSUPPORTED, "x")
+box.error(box.error.UNSUPPORTED)
+
+--
+-- gh-3031: allow to create an error object with no throwing it.
+--
+e = box.error.new(box.error.UNKNOWN)
+e
+e = box.error.new(box.error.CREATE_SPACE, "space", "error")
+e
+box.error.new()
+
+--
+-- gh-4489: box.error has __concat metamethod
+--
+test_run:cmd("push filter '(.builtin/.*.lua):[0-9]+' to '\\1'")
+e = box.error.new(box.error.UNKNOWN)
+'left side: ' .. e
+e .. ': right side'
+e .. nil
+nil .. e
+e .. box.NULL
+box.NULL .. e
+123 .. e
+e .. 123
+e .. e
+e .. {}
+{} .. e
+-1ULL .. e
+e .. -1ULL
+1LL .. e
+e .. 1LL
+e = nil
+
+--
+-- System errors expose errno as a field.
+--
+_, err = require('fio').open('not_existing_file')
+type(err.errno)
+-- Errors not related to the standard library do
+-- not expose errno.
+err = box.error.new(box.error.PROC_LUA, "errno")
+type(err.errno)
+
+t = {}
+test_run:cmd("setopt delimiter ';'")
+
+for k,v in pairs(box.error) do
+   if type(v) == 'number' then
+    t[v] = 'box.error.'..tostring(k)
+   end
+end;
+t;
+
+test_run:cmd("setopt delimiter ''");
+space:drop()
diff --git a/test/box/misc.result b/test/box/misc.result
index 047591b60..9b76d7665 100644
--- a/test/box/misc.result
+++ b/test/box/misc.result
@@ -93,204 +93,6 @@ t = nil
 ---
 ...
 ----------------
--- # box.error
-----------------
-test_run:cmd("restart server default")
-env = require('test_run')
----
-...
-test_run = env.new()
----
-...
-box.error.last()
----
-- null
-...
-box.error({code = 123, reason = 'test'})
----
-- error: test
-...
-box.error(box.error.ILLEGAL_PARAMS, "bla bla")
----
-- error: Illegal parameters, bla bla
-...
-box.error()
----
-- error: Illegal parameters, bla bla
-...
-e = box.error.last()
----
-...
-e
----
-- Illegal parameters, bla bla
-...
-e:unpack()
----
-- type: ClientError
-  code: 1
-  message: Illegal parameters, bla bla
-  trace:
-  - file: '[C]'
-    line: 4294967295
-...
-e.type
----
-- ClientError
-...
-e.code
----
-- 1
-...
-e.message
----
-- Illegal parameters, bla bla
-...
-tostring(e)
----
-- Illegal parameters, bla bla
-...
-e = nil
----
-...
-box.error.clear()
----
-...
-box.error.last()
----
-- null
-...
-space = box.space.tweedledum
----
-...
---
--- gh-2080: box.error() crashes with wrong parameters
-box.error(box.error.UNSUPPORTED, "x", "x%s")
----
-- error: x does not support x%s
-...
-box.error(box.error.UNSUPPORTED, "x")
----
-- error: 'bad argument #3 to ''?'' (no value)'
-...
-box.error(box.error.UNSUPPORTED)
----
-- error: 'box.error(): bad arguments'
-...
---
--- gh-3031: allow to create an error object with no throwing it.
---
-e = box.error.new(box.error.UNKNOWN)
----
-...
-e
----
-- Unknown error
-...
-e = box.error.new(box.error.CREATE_SPACE, "space", "error")
----
-...
-e
----
-- 'Failed to create space ''space'': error'
-...
-box.error.new()
----
-- error: 'Usage: box.error.new(code, args)'
-...
---
--- gh-4489: box.error has __concat metamethod
---
-test_run:cmd("push filter '(.builtin/.*.lua):[0-9]+' to '\\1'")
----
-- true
-...
-e = box.error.new(box.error.UNKNOWN)
----
-...
-'left side: ' .. e
----
-- 'left side: Unknown error'
-...
-e .. ': right side'
----
-- 'Unknown error: right side'
-...
-e .. nil
----
-- error: 'builtin/error.lua: attempt to concatenate local ''rhs'' (a nil value)'
-...
-nil .. e
----
-- error: 'builtin/error.lua: attempt to concatenate local ''lhs'' (a nil value)'
-...
-e .. box.NULL
----
-- error: 'builtin/error.lua: attempt to concatenate ''string'' and ''void *'''
-...
-box.NULL .. e
----
-- error: 'builtin/error.lua: attempt to concatenate ''void *'' and ''string'''
-...
-123 .. e
----
-- 123Unknown error
-...
-e .. 123
----
-- Unknown error123
-...
-e .. e
----
-- Unknown errorUnknown error
-...
-e .. {}
----
-- error: 'builtin/error.lua: attempt to concatenate local ''rhs'' (a table value)'
-...
-{} .. e
----
-- error: 'builtin/error.lua: attempt to concatenate local ''lhs'' (a table value)'
-...
--1ULL .. e
----
-- error: 'builtin/error.lua: attempt to concatenate ''uint64_t'' and ''string'''
-...
-e .. -1ULL
----
-- error: 'builtin/error.lua: attempt to concatenate ''string'' and ''uint64_t'''
-...
-1LL .. e
----
-- error: 'builtin/error.lua: attempt to concatenate ''int64_t'' and ''string'''
-...
-e .. 1LL
----
-- error: 'builtin/error.lua: attempt to concatenate ''string'' and ''int64_t'''
-...
-e = nil
----
-...
---
--- System errors expose errno as a field.
---
-_, err = require('fio').open('not_existing_file')
----
-...
-type(err.errno)
----
-- number
-...
--- Errors not related to the standard library do
--- not expose errno.
-err = box.error.new(box.error.PROC_LUA, "errno")
----
-...
-type(err.errno)
----
-- nil
-...
-----------------
 -- # box.stat
 ----------------
 t = {}
@@ -413,231 +215,6 @@ type(require('yaml').encode(box.slab.info()));
 ---
 - string
 ...
-----------------
--- # box.error
-----------------
-t = {}
-for k,v in pairs(box.error) do
-   if type(v) == 'number' then
-    t[v] = 'box.error.'..tostring(k)
-   end
-end;
----
-...
-t;
----
-- 0: box.error.UNKNOWN
-  1: box.error.ILLEGAL_PARAMS
-  2: box.error.MEMORY_ISSUE
-  3: box.error.TUPLE_FOUND
-  4: box.error.TUPLE_NOT_FOUND
-  5: box.error.UNSUPPORTED
-  6: box.error.NONMASTER
-  7: box.error.READONLY
-  8: box.error.INJECTION
-  9: box.error.CREATE_SPACE
-  10: box.error.SPACE_EXISTS
-  11: box.error.DROP_SPACE
-  12: box.error.ALTER_SPACE
-  13: box.error.INDEX_TYPE
-  14: box.error.MODIFY_INDEX
-  15: box.error.LAST_DROP
-  16: box.error.TUPLE_FORMAT_LIMIT
-  17: box.error.DROP_PRIMARY_KEY
-  18: box.error.KEY_PART_TYPE
-  19: box.error.EXACT_MATCH
-  20: box.error.INVALID_MSGPACK
-  21: box.error.PROC_RET
-  22: box.error.TUPLE_NOT_ARRAY
-  23: box.error.FIELD_TYPE
-  24: box.error.INDEX_PART_TYPE_MISMATCH
-  25: box.error.UPDATE_SPLICE
-  26: box.error.UPDATE_ARG_TYPE
-  27: box.error.FORMAT_MISMATCH_INDEX_PART
-  28: box.error.UNKNOWN_UPDATE_OP
-  29: box.error.UPDATE_FIELD
-  30: box.error.FUNCTION_TX_ACTIVE
-  31: box.error.KEY_PART_COUNT
-  32: box.error.PROC_LUA
-  33: box.error.NO_SUCH_PROC
-  34: box.error.NO_SUCH_TRIGGER
-  35: box.error.NO_SUCH_INDEX_ID
-  36: box.error.NO_SUCH_SPACE
-  37: box.error.NO_SUCH_FIELD_NO
-  38: box.error.EXACT_FIELD_COUNT
-  39: box.error.FIELD_MISSING
-  40: box.error.WAL_IO
-  41: box.error.MORE_THAN_ONE_TUPLE
-  42: box.error.ACCESS_DENIED
-  43: box.error.CREATE_USER
-  44: box.error.DROP_USER
-  45: box.error.NO_SUCH_USER
-  46: box.error.USER_EXISTS
-  47: box.error.PASSWORD_MISMATCH
-  48: box.error.UNKNOWN_REQUEST_TYPE
-  49: box.error.UNKNOWN_SCHEMA_OBJECT
-  50: box.error.CREATE_FUNCTION
-  51: box.error.NO_SUCH_FUNCTION
-  52: box.error.FUNCTION_EXISTS
-  53: box.error.BEFORE_REPLACE_RET
-  54: box.error.MULTISTATEMENT_TRANSACTION
-  55: box.error.TRIGGER_EXISTS
-  56: box.error.USER_MAX
-  57: box.error.NO_SUCH_ENGINE
-  58: box.error.RELOAD_CFG
-  59: box.error.CFG
-  60: box.error.SAVEPOINT_EMPTY_TX
-  61: box.error.NO_SUCH_SAVEPOINT
-  62: box.error.UNKNOWN_REPLICA
-  63: box.error.REPLICASET_UUID_MISMATCH
-  64: box.error.INVALID_UUID
-  65: box.error.REPLICASET_UUID_IS_RO
-  66: box.error.INSTANCE_UUID_MISMATCH
-  68: box.error.INVALID_ORDER
-  69: box.error.MISSING_REQUEST_FIELD
-  70: box.error.IDENTIFIER
-  71: box.error.DROP_FUNCTION
-  72: box.error.ITERATOR_TYPE
-  73: box.error.REPLICA_MAX
-  74: box.error.INVALID_XLOG
-  75: box.error.INVALID_XLOG_NAME
-  76: box.error.INVALID_XLOG_ORDER
-  77: box.error.NO_CONNECTION
-  78: box.error.TIMEOUT
-  79: box.error.ACTIVE_TRANSACTION
-  80: box.error.CURSOR_NO_TRANSACTION
-  81: box.error.CROSS_ENGINE_TRANSACTION
-  82: box.error.NO_SUCH_ROLE
-  83: box.error.ROLE_EXISTS
-  84: box.error.CREATE_ROLE
-  85: box.error.INDEX_EXISTS
-  86: box.error.SESSION_CLOSED
-  87: box.error.ROLE_LOOP
-  88: box.error.GRANT
-  89: box.error.PRIV_GRANTED
-  90: box.error.ROLE_GRANTED
-  91: box.error.PRIV_NOT_GRANTED
-  92: box.error.ROLE_NOT_GRANTED
-  93: box.error.MISSING_SNAPSHOT
-  94: box.error.CANT_UPDATE_PRIMARY_KEY
-  95: box.error.UPDATE_INTEGER_OVERFLOW
-  96: box.error.GUEST_USER_PASSWORD
-  97: box.error.TRANSACTION_CONFLICT
-  98: box.error.UNSUPPORTED_PRIV
-  99: box.error.LOAD_FUNCTION
-  100: box.error.FUNCTION_LANGUAGE
-  101: box.error.RTREE_RECT
-  102: box.error.PROC_C
-  103: box.error.UNKNOWN_RTREE_INDEX_DISTANCE_TYPE
-  104: box.error.PROTOCOL
-  105: box.error.UPSERT_UNIQUE_SECONDARY_KEY
-  106: box.error.WRONG_INDEX_RECORD
-  107: box.error.WRONG_INDEX_PARTS
-  108: box.error.WRONG_INDEX_OPTIONS
-  109: box.error.WRONG_SCHEMA_VERSION
-  110: box.error.MEMTX_MAX_TUPLE_SIZE
-  111: box.error.WRONG_SPACE_OPTIONS
-  112: box.error.UNSUPPORTED_INDEX_FEATURE
-  113: box.error.VIEW_IS_RO
-  114: box.error.NO_TRANSACTION
-  115: box.error.SYSTEM
-  116: box.error.LOADING
-  117: box.error.CONNECTION_TO_SELF
-  118: box.error.KEY_PART_IS_TOO_LONG
-  119: box.error.COMPRESSION
-  120: box.error.CHECKPOINT_IN_PROGRESS
-  121: box.error.SUB_STMT_MAX
-  122: box.error.COMMIT_IN_SUB_STMT
-  123: box.error.ROLLBACK_IN_SUB_STMT
-  124: box.error.DECOMPRESSION
-  125: box.error.INVALID_XLOG_TYPE
-  126: box.error.ALREADY_RUNNING
-  127: box.error.INDEX_FIELD_COUNT_LIMIT
-  128: box.error.LOCAL_INSTANCE_ID_IS_READ_ONLY
-  129: box.error.BACKUP_IN_PROGRESS
-  130: box.error.READ_VIEW_ABORTED
-  131: box.error.INVALID_INDEX_FILE
-  132: box.error.INVALID_RUN_FILE
-  133: box.error.INVALID_VYLOG_FILE
-  134: box.error.CHECKPOINT_ROLLBACK
-  135: box.error.VY_QUOTA_TIMEOUT
-  136: box.error.PARTIAL_KEY
-  137: box.error.TRUNCATE_SYSTEM_SPACE
-  138: box.error.LOAD_MODULE
-  139: box.error.VINYL_MAX_TUPLE_SIZE
-  140: box.error.WRONG_DD_VERSION
-  141: box.error.WRONG_SPACE_FORMAT
-  142: box.error.CREATE_SEQUENCE
-  143: box.error.ALTER_SEQUENCE
-  144: box.error.DROP_SEQUENCE
-  145: box.error.NO_SUCH_SEQUENCE
-  146: box.error.SEQUENCE_EXISTS
-  147: box.error.SEQUENCE_OVERFLOW
-  148: box.error.NO_SUCH_INDEX_NAME
-  149: box.error.SPACE_FIELD_IS_DUPLICATE
-  150: box.error.CANT_CREATE_COLLATION
-  151: box.error.WRONG_COLLATION_OPTIONS
-  152: box.error.NULLABLE_PRIMARY
-  153: box.error.NO_SUCH_FIELD_NAME_IN_SPACE
-  154: box.error.TRANSACTION_YIELD
-  155: box.error.NO_SUCH_GROUP
-  156: box.error.SQL_BIND_VALUE
-  157: box.error.SQL_BIND_TYPE
-  158: box.error.SQL_BIND_PARAMETER_MAX
-  159: box.error.SQL_EXECUTE
-  160: box.error.UPDATE_DECIMAL_OVERFLOW
-  161: box.error.SQL_BIND_NOT_FOUND
-  162: box.error.ACTION_MISMATCH
-  163: box.error.VIEW_MISSING_SQL
-  164: box.error.FOREIGN_KEY_CONSTRAINT
-  165: box.error.NO_SUCH_MODULE
-  166: box.error.NO_SUCH_COLLATION
-  167: box.error.CREATE_FK_CONSTRAINT
-  168: box.error.DROP_FK_CONSTRAINT
-  169: box.error.NO_SUCH_CONSTRAINT
-  170: box.error.CONSTRAINT_EXISTS
-  171: box.error.SQL_TYPE_MISMATCH
-  172: box.error.ROWID_OVERFLOW
-  173: box.error.DROP_COLLATION
-  174: box.error.ILLEGAL_COLLATION_MIX
-  175: box.error.SQL_NO_SUCH_PRAGMA
-  176: box.error.SQL_CANT_RESOLVE_FIELD
-  177: box.error.INDEX_EXISTS_IN_SPACE
-  178: box.error.INCONSISTENT_TYPES
-  179: box.error.SQL_SYNTAX_WITH_POS
-  180: box.error.SQL_STACK_OVERFLOW
-  181: box.error.SQL_SELECT_WILDCARD
-  182: box.error.SQL_STATEMENT_EMPTY
-  184: box.error.SQL_SYNTAX_NEAR_TOKEN
-  185: box.error.SQL_UNKNOWN_TOKEN
-  186: box.error.SQL_PARSER_GENERIC
-  187: box.error.SQL_ANALYZE_ARGUMENT
-  188: box.error.SQL_COLUMN_COUNT_MAX
-  189: box.error.HEX_LITERAL_MAX
-  190: box.error.INT_LITERAL_MAX
-  191: box.error.SQL_PARSER_LIMIT
-  192: box.error.INDEX_DEF_UNSUPPORTED
-  193: box.error.CK_DEF_UNSUPPORTED
-  194: box.error.MULTIKEY_INDEX_MISMATCH
-  195: box.error.CREATE_CK_CONSTRAINT
-  196: box.error.CK_CONSTRAINT_FAILED
-  197: box.error.SQL_COLUMN_COUNT
-  198: box.error.FUNC_INDEX_FUNC
-  199: box.error.FUNC_INDEX_FORMAT
-  200: box.error.FUNC_INDEX_PARTS
-  201: box.error.NO_SUCH_FIELD_NAME
-  202: box.error.FUNC_WRONG_ARG_COUNT
-  203: box.error.BOOTSTRAP_READONLY
-  204: box.error.SQL_FUNC_WRONG_RET_COUNT
-  205: box.error.FUNC_INVALID_RETURN_TYPE
-  206: box.error.SQL_PARSER_GENERIC_WITH_POS
-  207: box.error.REPLICA_NOT_ANON
-  208: box.error.CANNOT_REGISTER
-  209: box.error.SESSION_SETTING_INVALID_VALUE
-  210: box.error.SQL_PREPARE
-  211: box.error.WRONG_QUERY_ID
-  212: box.error.SEQUENCE_NOT_STARTED
-...
 test_run:cmd("setopt delimiter ''");
 ---
 - true
diff --git a/test/box/misc.test.lua b/test/box/misc.test.lua
index e1c2f990f..350f9f75d 100644
--- a/test/box/misc.test.lua
+++ b/test/box/misc.test.lua
@@ -22,75 +22,6 @@ t = {} for n in pairs(box) do table.insert(t, tostring(n)) end table.sort(t)
 t
 t = nil
 
-----------------
--- # box.error
-----------------
-
-test_run:cmd("restart server default")
-env = require('test_run')
-test_run = env.new()
-box.error.last()
-box.error({code = 123, reason = 'test'})
-box.error(box.error.ILLEGAL_PARAMS, "bla bla")
-box.error()
-e = box.error.last()
-e
-e:unpack()
-e.type
-e.code
-e.message
-tostring(e)
-e = nil
-box.error.clear()
-box.error.last()
-space = box.space.tweedledum
---
--- gh-2080: box.error() crashes with wrong parameters
-box.error(box.error.UNSUPPORTED, "x", "x%s")
-box.error(box.error.UNSUPPORTED, "x")
-box.error(box.error.UNSUPPORTED)
-
---
--- gh-3031: allow to create an error object with no throwing it.
---
-e = box.error.new(box.error.UNKNOWN)
-e
-e = box.error.new(box.error.CREATE_SPACE, "space", "error")
-e
-box.error.new()
-
---
--- gh-4489: box.error has __concat metamethod
---
-test_run:cmd("push filter '(.builtin/.*.lua):[0-9]+' to '\\1'")
-e = box.error.new(box.error.UNKNOWN)
-'left side: ' .. e
-e .. ': right side'
-e .. nil
-nil .. e
-e .. box.NULL
-box.NULL .. e
-123 .. e
-e .. 123
-e .. e
-e .. {}
-{} .. e
--1ULL .. e
-e .. -1ULL
-1LL .. e
-e .. 1LL
-e = nil
-
---
--- System errors expose errno as a field.
---
-_, err = require('fio').open('not_existing_file')
-type(err.errno)
--- Errors not related to the standard library do
--- not expose errno.
-err = box.error.new(box.error.PROC_LUA, "errno")
-type(err.errno)
-
 ----------------
 -- # box.stat
 ----------------
@@ -138,17 +69,6 @@ box.runtime.info().maxalloc > 0;
 --
 type(require('yaml').encode(box.slab.info()));
 
-----------------
--- # box.error
-----------------
-t = {}
-for k,v in pairs(box.error) do
-   if type(v) == 'number' then
-    t[v] = 'box.error.'..tostring(k)
-   end
-end;
-t;
-
 test_run:cmd("setopt delimiter ''");
 
 -- A test case for Bug#901674
-- 
2.17.1

Re: [Tarantool-patches] [PATCH v2 03/10] test: move box.error tests to box/error.test.lua

[Konstantin Osipov]

* Nikita Pettik <korablev@tarantool.org> [20/03/25 09:32]:
> We are going to introduce more tests related to error module, so let's
> move all error-related tests from box/misc.test.lua to a separate test
> file (box/error.test.lua).
> 
> Needed for #1148
Another obvious patch, lgtm.

-- 
Konstantin Osipov, Moscow, Russia

Re: [Tarantool-patches] [PATCH v2 03/10] test: move box.error tests to box/error.test.lua

[Vladislav Shpilevoy]

Thanks for the patch!

The commit LGTM. Lets push it right away.

On 25/03/2020 02:42, Nikita Pettik wrote:
> We are going to introduce more tests related to error module, so let's
> move all error-related tests from box/misc.test.lua to a separate test
> file (box/error.test.lua).
> 
> Needed for #1148

Re: [Tarantool-patches] [PATCH v2 03/10] test: move box.error tests to box/error.test.lua

[Nikita Pettik]

On 26 Mar 01:22, Vladislav Shpilevoy wrote:
> Thanks for the patch!
> 
> The commit LGTM. Lets push it right away.

Pushed to master.
 
> On 25/03/2020 02:42, Nikita Pettik wrote:
> > We are going to introduce more tests related to error module, so let's
> > move all error-related tests from box/misc.test.lua to a separate test
> > file (box/error.test.lua).
> > 
> > Needed for #1148

[Tarantool-patches] [PATCH v2 04/10] box/error: introduce box.error.set() method

[Nikita Pettik]

box.error.set(err) sets err to instance's diagnostics area. Argument err
is supposed to be instance of error object. This method is required
since we are going to avoid adding created via box.error.new() errors to
Tarantool's diagnostic area.

Needed for #1148
Part of #4778
---
 src/box/lua/error.cc    | 14 ++++++++++++++
 src/lua/error.c         |  2 +-
 src/lua/error.h         |  3 +++
 test/box/error.result   | 36 ++++++++++++++++++++++++++++++++++++
 test/box/error.test.lua | 15 +++++++++++++++
 5 files changed, 69 insertions(+), 1 deletion(-)

diff --git a/src/box/lua/error.cc b/src/box/lua/error.cc
index fc53a40f4..640e33910 100644
--- a/src/box/lua/error.cc
+++ b/src/box/lua/error.cc
@@ -154,6 +154,16 @@ luaT_error_clear(lua_State *L)
 	return 0;
 }
 
+static int
+luaT_error_set(lua_State *L)
+{
+	if (lua_gettop(L) == 0)
+		return luaL_error(L, "Usage: box.error.set(error)");
+	struct error *e = luaL_checkerror(L, 1);
+	diag_set_error(&fiber()->diag, e);
+	return 0;
+}
+
 static int
 lbox_errinj_set(struct lua_State *L)
 {
@@ -268,6 +278,10 @@ box_lua_error_init(struct lua_State *L) {
 			lua_pushcfunction(L, luaT_error_new);
 			lua_setfield(L, -2, "new");
 		}
+		{
+			lua_pushcfunction(L, luaT_error_set);
+			lua_setfield(L, -2, "set");
+		}
 		lua_setfield(L, -2, "__index");
 	}
 	lua_setmetatable(L, -2);
diff --git a/src/lua/error.c b/src/lua/error.c
index d82e78dc4..18a990a88 100644
--- a/src/lua/error.c
+++ b/src/lua/error.c
@@ -53,7 +53,7 @@ luaL_iserror(struct lua_State *L, int narg)
 	return e;
 }
 
-static struct error *
+struct error *
 luaL_checkerror(struct lua_State *L, int narg)
 {
 	struct error *error = luaL_iserror(L, narg);
diff --git a/src/lua/error.h b/src/lua/error.h
index 64fa5eba3..16cdaf7fe 100644
--- a/src/lua/error.h
+++ b/src/lua/error.h
@@ -65,6 +65,9 @@ luaT_pusherror(struct lua_State *L, struct error *e);
 struct error *
 luaL_iserror(struct lua_State *L, int narg);
 
+struct error *
+luaL_checkerror(struct lua_State *L, int narg);
+
 void
 tarantool_lua_error_init(struct lua_State *L);
 
diff --git a/test/box/error.result b/test/box/error.result
index 9b6c8b462..31b2ce52b 100644
--- a/test/box/error.result
+++ b/test/box/error.result
@@ -436,6 +436,42 @@ test_run:cmd("setopt delimiter ''");
  | ---
  | - true
  | ...
+
+-- gh-4778: don't add created via box.error.new() errors to
+-- Tarantool's diagnostic area.
+--
+err = box.error.new({code = 111, reason = "cause"})
+ | ---
+ | ...
+assert(box.error.last() ~= err)
+ | ---
+ | - error: assertion failed!
+ | ...
+box.error.set(err)
+ | ---
+ | ...
+assert(box.error.last() == err)
+ | ---
+ | - true
+ | ...
+-- Consider wrong or tricky inputs to box.error.set()
+--
+box.error.set(1)
+ | ---
+ | - error: 'Invalid argument #1 (error expected, got number)'
+ | ...
+box.error.set(nil)
+ | ---
+ | - error: 'Invalid argument #1 (error expected, got nil)'
+ | ...
+box.error.set(box.error.last())
+ | ---
+ | ...
+assert(box.error.last() == err)
+ | ---
+ | - true
+ | ...
+
 space:drop()
  | ---
  | ...
diff --git a/test/box/error.test.lua b/test/box/error.test.lua
index d16cc5672..81591ea7e 100644
--- a/test/box/error.test.lua
+++ b/test/box/error.test.lua
@@ -77,4 +77,19 @@ end;
 t;
 
 test_run:cmd("setopt delimiter ''");
+
+-- gh-4778: don't add created via box.error.new() errors to
+-- Tarantool's diagnostic area.
+--
+err = box.error.new({code = 111, reason = "cause"})
+assert(box.error.last() ~= err)
+box.error.set(err)
+assert(box.error.last() == err)
+-- Consider wrong or tricky inputs to box.error.set()
+--
+box.error.set(1)
+box.error.set(nil)
+box.error.set(box.error.last())
+assert(box.error.last() == err)
+
 space:drop()
-- 
2.17.1

Re: [Tarantool-patches] [PATCH v2 04/10] box/error: introduce box.error.set() method

[Konstantin Osipov]

* Nikita Pettik <korablev@tarantool.org> [20/03/25 09:32]:
> box.error.set(err) sets err to instance's diagnostics area. Argument err
> is supposed to be instance of error object. This method is required
> since we are going to avoid adding created via box.error.new() errors to
> Tarantool's diagnostic area.

I agree that it's a good idea to separate creation and addition in
Lua.

I don't know if it breaks anything, but please search on github
for box.error.new. This is not easy to do, since github doesn't
support '.' in searches, so perhaps you should do:

for each (repo in github/tarntool and moonlibs)
    git clone repo
    cd repo
    git grep -r 'box.error.new' .

Most likely there is some obscure place  you break.

-- 
Konstantin Osipov, Moscow, Russia

Re: [Tarantool-patches] [PATCH v2 04/10] box/error: introduce box.error.set() method

[Nikita Pettik]

On 25 Mar 11:33, Konstantin Osipov wrote:
> * Nikita Pettik <korablev@tarantool.org> [20/03/25 09:32]:
> > box.error.set(err) sets err to instance's diagnostics area. Argument err
> > is supposed to be instance of error object. This method is required
> > since we are going to avoid adding created via box.error.new() errors to
> > Tarantool's diagnostic area.
> 
> I agree that it's a good idea to separate creation and addition in
> Lua.
> 
> I don't know if it breaks anything, but please search on github
> for box.error.new. This is not easy to do, since github doesn't
> support '.' in searches, so perhaps you should do:

As far as I see, it doesn't break anything: test are passing
as expected; it is not in direct contradictions to documentation.
 
> for each (repo in github/tarntool and moonlibs)
>     git clone repo
>     cd repo
>     git grep -r 'box.error.new' .
> 
> Most likely there is some obscure place  you break.
> 
> -- 
> Konstantin Osipov, Moscow, Russia

Re: [Tarantool-patches] [PATCH v2 04/10] box/error: introduce box.error.set() method

[Vladislav Shpilevoy]

On 25/03/2020 02:43, Nikita Pettik wrote:
> box.error.set(err) sets err to instance's diagnostics area. Argument err
> is supposed to be instance of error object. This method is required
> since we are going to avoid adding created via box.error.new() errors to
> Tarantool's diagnostic area.
> 
> Needed for #1148
> Part of #4778
> ---
>  src/box/lua/error.cc    | 14 ++++++++++++++
>  src/lua/error.c         |  2 +-
>  src/lua/error.h         |  3 +++
>  test/box/error.result   | 36 ++++++++++++++++++++++++++++++++++++
>  test/box/error.test.lua | 15 +++++++++++++++
>  5 files changed, 69 insertions(+), 1 deletion(-)
> 
> diff --git a/src/box/lua/error.cc b/src/box/lua/error.cc
> index fc53a40f4..640e33910 100644
> --- a/src/box/lua/error.cc
> +++ b/src/box/lua/error.cc
> @@ -154,6 +154,16 @@ luaT_error_clear(lua_State *L)
>  	return 0;
>  }
>  
> +static int
> +luaT_error_set(lua_State *L)

Better specify 'struct' before 'lua_State'.

After that the commit LGTM. Lets push it after this
amendment.

Re: [Tarantool-patches] [PATCH v2 04/10] box/error: introduce box.error.set() method

[Nikita Pettik]

On 26 Mar 01:22, Vladislav Shpilevoy wrote:
> 
> 
> On 25/03/2020 02:43, Nikita Pettik wrote:
> > box.error.set(err) sets err to instance's diagnostics area. Argument err
> > is supposed to be instance of error object. This method is required
> > since we are going to avoid adding created via box.error.new() errors to
> > Tarantool's diagnostic area.
> > 
> > Needed for #1148
> > Part of #4778
> > ---
> >  src/box/lua/error.cc    | 14 ++++++++++++++
> >  src/lua/error.c         |  2 +-
> >  src/lua/error.h         |  3 +++
> >  test/box/error.result   | 36 ++++++++++++++++++++++++++++++++++++
> >  test/box/error.test.lua | 15 +++++++++++++++
> >  5 files changed, 69 insertions(+), 1 deletion(-)
> > 
> > diff --git a/src/box/lua/error.cc b/src/box/lua/error.cc
> > index fc53a40f4..640e33910 100644
> > --- a/src/box/lua/error.cc
> > +++ b/src/box/lua/error.cc
> > @@ -154,6 +154,16 @@ luaT_error_clear(lua_State *L)
> >  	return 0;
> >  }
> >  
> > +static int
> > +luaT_error_set(lua_State *L)
> 
> Better specify 'struct' before 'lua_State'.

Ok, fixed.
 
> After that the commit LGTM. Lets push it after this
> amendment.

Pushed to master.

[Tarantool-patches] [PATCH v2 05/10] box/error: don't set error created via box.error.new to diag

[Nikita Pettik]

To achieve this let's refactor luaT_error_create() to return error
object instead of setting it via box_error_set().
luaT_error_create() is used both to handle box.error() and
box.error.new() invocations, and box.error() is still expected to set
error to diagnostic area. So, luaT_error_call() which implements
box.error() processing at the end calls diag_set_error().
It is worth mentioning that net.box module relied on the fact that
box.error.new() set error to diagnostic area: otherwise request errors
don't get to diagnostic area on client side.

Needed for #1148
Closes #4778

@TarantoolBot document
Title: Don't promote error created via box.error.new to diagnostic area

Now box.error.new() only creates error object, but doesn't set it to
Tarantool's diagnostic area:
```
box.error.clear()
e = box.error.new({code = 111, reason = "cause"})
assert(box.error.last() == nil)
---
- true
...
```
To set error in diagnostic area explicitly box.error.set() has been
introduced. It accepts error object which is set as last system error
(i.e. becomes available via box.error.last()).
Finally, box.error.new() does not longer accept error object as an
argument (this was undocumented feature).
Note that patch does not affect box.error(), which still pushed error to
diagnostic area. This fact is reflected in docs:
'''
Emulate a request error, with text based on one of the pre-defined
Tarantool errors...
'''
---
 src/box/error.cc        | 16 ++++++++++++
 src/box/error.h         |  8 ++++++
 src/box/lua/error.cc    | 56 +++++++++++++++++++++++------------------
 test/box/error.result   | 32 ++++++++++++++++++++++-
 test/box/error.test.lua | 16 ++++++++++++
 5 files changed, 103 insertions(+), 25 deletions(-)

diff --git a/src/box/error.cc b/src/box/error.cc
index 7dfe1b3ee..824a4617c 100644
--- a/src/box/error.cc
+++ b/src/box/error.cc
@@ -86,6 +86,22 @@ box_error_set(const char *file, unsigned line, uint32_t code,
 	return -1;
 }
 
+struct error *
+box_error_construct(const char *file, unsigned line, uint32_t code,
+		    const char *fmt, ...)
+{
+	struct error *e = BuildClientError(file, line, ER_UNKNOWN);
+	ClientError *client_error = type_cast(ClientError, e);
+	if (client_error != NULL) {
+		client_error->m_errcode = code;
+		va_list ap;
+		va_start(ap, fmt);
+		error_vformat_msg(e, fmt, ap);
+		va_end(ap);
+	}
+	return e;
+}
+
 /* }}} */
 
 struct rmean *rmean_error = NULL;
diff --git a/src/box/error.h b/src/box/error.h
index b8c7cf73d..ef9cf3e76 100644
--- a/src/box/error.h
+++ b/src/box/error.h
@@ -137,6 +137,14 @@ box_error_set(const char *file, unsigned line, uint32_t code,
 
 /** \endcond public */
 
+/**
+ * Construct error object without setting it in the diagnostics
+ * area. On the memory allocation fail returns NULL.
+ */
+struct error *
+box_error_construct(const char *file, unsigned line, uint32_t code,
+		    const char *fmt, ...);
+
 extern const struct type_info type_ClientError;
 extern const struct type_info type_XlogError;
 extern const struct type_info type_AccessDeniedError;
diff --git a/src/box/lua/error.cc b/src/box/lua/error.cc
index 640e33910..ff285d7eb 100644
--- a/src/box/lua/error.cc
+++ b/src/box/lua/error.cc
@@ -42,7 +42,14 @@ extern "C" {
 #include "lua/utils.h"
 #include "box/error.h"
 
-static void
+/**
+ * Parse Lua arguments (they can come as single table
+ * f({code : number, reason : string}) or as separate members
+ * f(code, reason)) and construct struct error with given values.
+ * In case one of arguments is missing its corresponding field
+ * in struct error is filled with default value.
+ */
+static struct error *
 luaT_error_create(lua_State *L, int top_base)
 {
 	uint32_t code = 0;
@@ -69,25 +76,19 @@ luaT_error_create(lua_State *L, int top_base)
 			reason = lua_tostring(L, -1);
 		} else if (strchr(reason, '%') != NULL) {
 			/* Missing arguments to format string */
-			luaL_error(L, "box.error(): bad arguments");
-		}
-	} else if (top == top_base) {
-		if (lua_istable(L, top_base)) {
-			/* A special case that rethrows raw error (used by net.box) */
-			lua_getfield(L, top_base, "code");
-			code = lua_tonumber(L, -1);
-			lua_pop(L, 1);
-			lua_getfield(L, top_base, "reason");
-			reason = lua_tostring(L, -1);
-			if (reason == NULL)
-				reason = "";
-			lua_pop(L, 1);
-		} else if (luaL_iserror(L, top_base)) {
-			lua_error(L);
-			return;
+			return NULL;
 		}
+	} else if (top == top_base && lua_istable(L, top_base)) {
+		lua_getfield(L, top_base, "code");
+		code = lua_tonumber(L, -1);
+		lua_pop(L, 1);
+		lua_getfield(L, top_base, "reason");
+		reason = lua_tostring(L, -1);
+		if (reason == NULL)
+			reason = "";
+		lua_pop(L, 1);
 	} else {
-		luaL_error(L, "box.error(): bad arguments");
+		return NULL;
 	}
 
 raise:
@@ -101,8 +102,7 @@ raise:
 		}
 		line = info.currentline;
 	}
-	say_debug("box.error() at %s:%i", file, line);
-	box_error_set(file, line, code, "%s", reason);
+	return box_error_construct(file, line, code, "%s", reason);
 }
 
 static int
@@ -111,10 +111,15 @@ luaT_error_call(lua_State *L)
 	if (lua_gettop(L) <= 1) {
 		/* Re-throw saved exceptions if any. */
 		if (box_error_last())
-			luaT_error(L);
+			return luaT_error(L);
 		return 0;
 	}
-	luaT_error_create(L, 2);
+	if (lua_gettop(L) == 2 && luaL_iserror(L, 2))
+		return lua_error(L);
+	struct error *e = luaT_error_create(L, 2);
+	if (e == NULL)
+		return luaL_error(L, "box.error(): bad arguments");
+	diag_set_error(&fiber()->diag, e);
 	return luaT_error(L);
 }
 
@@ -139,9 +144,12 @@ luaT_error_new(lua_State *L)
 {
 	if (lua_gettop(L) == 0)
 		return luaL_error(L, "Usage: box.error.new(code, args)");
-	luaT_error_create(L, 1);
+	struct error *e = luaT_error_create(L, 1);
+	if (e == NULL)
+		return luaL_error(L, "box.error.new(): bad arguments");
 	lua_settop(L, 0);
-	return luaT_error_last(L);
+	luaT_pusherror(L, e);
+	return 1;
 }
 
 static int
diff --git a/test/box/error.result b/test/box/error.result
index 31b2ce52b..a19a7044f 100644
--- a/test/box/error.result
+++ b/test/box/error.result
@@ -445,7 +445,7 @@ err = box.error.new({code = 111, reason = "cause"})
  | ...
 assert(box.error.last() ~= err)
  | ---
- | - error: assertion failed!
+ | - true
  | ...
 box.error.set(err)
  | ---
@@ -471,6 +471,36 @@ assert(box.error.last() == err)
  | ---
  | - true
  | ...
+-- Check that box.error.new() does not set error to diag.
+--
+box.error.clear()
+ | ---
+ | ...
+err = box.error.new(1, "cause")
+ | ---
+ | ...
+assert(box.error.last() == nil)
+ | ---
+ | - true
+ | ...
+
+-- box.error.new() does not accept error objects.
+--
+box.error.new(err)
+ | ---
+ | - error: 'box.error.new(): bad arguments'
+ | ...
+
+-- box.error() is supposed to re-throw last diagnostic error.
+-- Make sure it does not fail if there's no errors at all
+-- (in diagnostics area).
+--
+box.error.clear()
+ | ---
+ | ...
+box.error()
+ | ---
+ | ...
 
 space:drop()
  | ---
diff --git a/test/box/error.test.lua b/test/box/error.test.lua
index 81591ea7e..a0b7d3e78 100644
--- a/test/box/error.test.lua
+++ b/test/box/error.test.lua
@@ -91,5 +91,21 @@ box.error.set(1)
 box.error.set(nil)
 box.error.set(box.error.last())
 assert(box.error.last() == err)
+-- Check that box.error.new() does not set error to diag.
+--
+box.error.clear()
+err = box.error.new(1, "cause")
+assert(box.error.last() == nil)
+
+-- box.error.new() does not accept error objects.
+--
+box.error.new(err)
+
+-- box.error() is supposed to re-throw last diagnostic error.
+-- Make sure it does not fail if there's no errors at all
+-- (in diagnostics area).
+--
+box.error.clear()
+box.error()
 
 space:drop()
-- 
2.17.1

Re: [Tarantool-patches] [PATCH v2 05/10] box/error: don't set error created via box.error.new to diag

[Konstantin Osipov]

* Nikita Pettik <korablev@tarantool.org> [20/03/25 09:32]:
> To achieve this let's refactor luaT_error_create() to return error
> object instead of setting it via box_error_set().
> luaT_error_create() is used both to handle box.error() and
> box.error.new() invocations, and box.error() is still expected to set
> error to diagnostic area. So, luaT_error_call() which implements
> box.error() processing at the end calls diag_set_error().
> It is worth mentioning that net.box module relied on the fact that
> box.error.new() set error to diagnostic area: otherwise request errors
> don't get to diagnostic area on client side.
> 

I know there was a discussion about construct vs create vs new.

Let me remind about approved name pairs:

obj_create/obj_destroy - constructor/destructor, without
allocation/deallocation.

obj_new/obj_delete - constructor/destructor with
allocation/deallocation

init/free - library initialization/destruction

_construct is not in the vocabulary. I'd avoid it.

I know this is bikeshed, pls feel free to ignore in scope of this
patch.  And sorry for jumping in late.

> +struct error *
> +box_error_construct(const char *file, unsigned line, uint32_t code,
> +		    const char *fmt, ...)
> +{
> +	struct error *e = BuildClientError(file, line, ER_UNKNOWN);
> +	ClientError *client_error = type_cast(ClientError, e);
> +	if (client_error != NULL) {
> +		client_error->m_errcode = code;
> +		va_list ap;
> +		va_start(ap, fmt);
> +		error_vformat_msg(e, fmt, ap);
> +		va_end(ap);
> +	}
> +	return e;
> +}

if box_error_new() is already taken by the public api, options are
box_client_error_new() or box_build_client_error()?


-- 
Konstantin Osipov, Moscow, Russia

Re: [Tarantool-patches] [PATCH v2 05/10] box/error: don't set error created via box.error.new to diag

[Nikita Pettik]

On 26 Mar 19:50, Konstantin Osipov wrote:
> * Nikita Pettik <korablev@tarantool.org> [20/03/25 09:32]:
> > To achieve this let's refactor luaT_error_create() to return error
> > object instead of setting it via box_error_set().
> > luaT_error_create() is used both to handle box.error() and
> > box.error.new() invocations, and box.error() is still expected to set
> > error to diagnostic area. So, luaT_error_call() which implements
> > box.error() processing at the end calls diag_set_error().
> > It is worth mentioning that net.box module relied on the fact that
> > box.error.new() set error to diagnostic area: otherwise request errors
> > don't get to diagnostic area on client side.
> > 
> 
> I know there was a discussion about construct vs create vs new.

It was not really discussion, I accidentally didn't notice that
box_error_construct() in fact allocated memory (meanwhile I thought
it didn't). I've already renamed it to box_error_new().

> Let me remind about approved name pairs:
> 
> obj_create/obj_destroy - constructor/destructor, without
> allocation/deallocation.
> 
> obj_new/obj_delete - constructor/destructor with
> allocation/deallocation
> 
> init/free - library initialization/destruction
> 
> _construct is not in the vocabulary. I'd avoid it.

Ok, Vlad already guided me (but it would be nice to see this
manual in our dev docs).
 
> I know this is bikeshed, pls feel free to ignore in scope of this
> patch.  And sorry for jumping in late.
> 
> > +struct error *
> > +box_error_construct(const char *file, unsigned line, uint32_t code,
> > +		    const char *fmt, ...)
> > +{
> > +	struct error *e = BuildClientError(file, line, ER_UNKNOWN);
> > +	ClientError *client_error = type_cast(ClientError, e);
> > +	if (client_error != NULL) {
> > +		client_error->m_errcode = code;
> > +		va_list ap;
> > +		va_start(ap, fmt);
> > +		error_vformat_msg(e, fmt, ap);
> > +		va_end(ap);
> > +	}
> > +	return e;
> > +}
> 
> if box_error_new() is already taken by the public api, options are
> box_client_error_new() or box_build_client_error()?

No, box_error_new() name is still available. I've renamed
box_error_construct() to box_error_new() and pushed branch
containing this changes.

> 
> -- 
> Konstantin Osipov, Moscow, Russia

Re: [Tarantool-patches] [PATCH v2 05/10] box/error: don't set error created via box.error.new to diag

[Nikita Pettik]

On 26 Mar 17:59, Nikita Pettik wrote:
> On 26 Mar 19:50, Konstantin Osipov wrote:
> > * Nikita Pettik <korablev@tarantool.org> [20/03/25 09:32]:
> > > To achieve this let's refactor luaT_error_create() to return error
> > > object instead of setting it via box_error_set().
> > > luaT_error_create() is used both to handle box.error() and
> > > box.error.new() invocations, and box.error() is still expected to set
> > > error to diagnostic area. So, luaT_error_call() which implements
> > > box.error() processing at the end calls diag_set_error().
> > > It is worth mentioning that net.box module relied on the fact that
> > > box.error.new() set error to diagnostic area: otherwise request errors
> > > don't get to diagnostic area on client side.
> > > 
> > 
> > I know there was a discussion about construct vs create vs new.
> 
> It was not really discussion, I accidentally didn't notice that
> box_error_construct() in fact allocated memory (meanwhile I thought
> it didn't). I've already renamed it to box_error_new().
> 
> > Let me remind about approved name pairs:
> > 
> > obj_create/obj_destroy - constructor/destructor, without
> > allocation/deallocation.
> > 
> > obj_new/obj_delete - constructor/destructor with
> > allocation/deallocation
> > 
> > init/free - library initialization/destruction
> > 
> > _construct is not in the vocabulary. I'd avoid it.
> 
> Ok, Vlad already guided me (but it would be nice to see this
> manual in our dev docs).

Oh, sorry, my bad. It is documented convention:
tarantool.io/en/doc/1.10/dev_guide/c_style_guide/

> > I know this is bikeshed, pls feel free to ignore in scope of this
> > patch.  And sorry for jumping in late.
> > 
> > > +struct error *
> > > +box_error_construct(const char *file, unsigned line, uint32_t code,
> > > +		    const char *fmt, ...)
> > > +{
> > > +	struct error *e = BuildClientError(file, line, ER_UNKNOWN);
> > > +	ClientError *client_error = type_cast(ClientError, e);
> > > +	if (client_error != NULL) {
> > > +		client_error->m_errcode = code;
> > > +		va_list ap;
> > > +		va_start(ap, fmt);
> > > +		error_vformat_msg(e, fmt, ap);
> > > +		va_end(ap);
> > > +	}
> > > +	return e;
> > > +}
> > 
> > if box_error_new() is already taken by the public api, options are
> > box_client_error_new() or box_build_client_error()?
> 
> No, box_error_new() name is still available. I've renamed
> box_error_construct() to box_error_new() and pushed branch
> containing this changes.
> 
> > 
> > -- 
> > Konstantin Osipov, Moscow, Russia

Re: [Tarantool-patches] [PATCH v2 05/10] box/error: don't set error created via box.error.new to diag

[Alexander Turenko]

> > I know there was a discussion about construct vs create vs new.
> 
> It was not really discussion, I accidentally didn't notice that
> box_error_construct() in fact allocated memory (meanwhile I thought
> it didn't). I've already renamed it to box_error_new().
> 
> > Let me remind about approved name pairs:
> > 
> > obj_create/obj_destroy - constructor/destructor, without
> > allocation/deallocation.
> > 
> > obj_new/obj_delete - constructor/destructor with
> > allocation/deallocation
> > 
> > init/free - library initialization/destruction
> > 
> > _construct is not in the vocabulary. I'd avoid it.
> 
> Ok, Vlad already guided me (but it would be nice to see this
> manual in our dev docs).

Should be there already: https://github.com/tarantool/doc/issues/1032

Re: [Tarantool-patches] [PATCH v2 05/10] box/error: don't set error created via box.error.new to diag

[Vladislav Shpilevoy]

Hi! Thanks for the patch!

See 2 comments below.

On 25/03/2020 02:43, Nikita Pettik wrote:
> To achieve this let's refactor luaT_error_create() to return error
> object instead of setting it via box_error_set().
> luaT_error_create() is used both to handle box.error() and
> box.error.new() invocations, and box.error() is still expected to set
> error to diagnostic area. So, luaT_error_call() which implements
> box.error() processing at the end calls diag_set_error().
> It is worth mentioning that net.box module relied on the fact that
> box.error.new() set error to diagnostic area: otherwise request errors
> don't get to diagnostic area on client side.
> 
> Needed for #1148
> Closes #4778
> 
> @TarantoolBot document
> Title: Don't promote error created via box.error.new to diagnostic area
> 
> Now box.error.new() only creates error object, but doesn't set it to
> Tarantool's diagnostic area:
> ```
> box.error.clear()
> e = box.error.new({code = 111, reason = "cause"})
> assert(box.error.last() == nil)
> ---
> - true
> ...
> ```
> To set error in diagnostic area explicitly box.error.set() has been
> introduced. It accepts error object which is set as last system error
> (i.e. becomes available via box.error.last()).
> Finally, box.error.new() does not longer accept error object as an
> argument (this was undocumented feature).
> Note that patch does not affect box.error(), which still pushed error to

1. 'pushed' -> 'pushes'.

> diff --git a/src/box/lua/error.cc b/src/box/lua/error.cc
> index 640e33910..ff285d7eb 100644
> --- a/src/box/lua/error.cc
> +++ b/src/box/lua/error.cc
> @@ -139,9 +144,12 @@ luaT_error_new(lua_State *L)
>  {
>  	if (lua_gettop(L) == 0)
>  		return luaL_error(L, "Usage: box.error.new(code, args)");
> -	luaT_error_create(L, 1);
> +	struct error *e = luaT_error_create(L, 1);
> +	if (e == NULL)
> +		return luaL_error(L, "box.error.new(): bad arguments");

2. I think it would be better to return the same 'Usage' error
in case of a problem with arguments. Not two different messages
meaning essentially the same. Or both should return 'bad arguments'.
However with both 'Usage' the diff would be 1 line smaller.

Other than these comments, LGTM. If you fix them both - then push right
away.

Re: [Tarantool-patches] [PATCH v2 05/10] box/error: don't set error created via box.error.new to diag

[Nikita Pettik]

On 27 Mar 01:19, Vladislav Shpilevoy wrote:
> Hi! Thanks for the patch!
> 
> See 2 comments below.
> 
> On 25/03/2020 02:43, Nikita Pettik wrote:
> > To achieve this let's refactor luaT_error_create() to return error
> > object instead of setting it via box_error_set().
> > luaT_error_create() is used both to handle box.error() and
> > box.error.new() invocations, and box.error() is still expected to set
> > error to diagnostic area. So, luaT_error_call() which implements
> > box.error() processing at the end calls diag_set_error().
> > It is worth mentioning that net.box module relied on the fact that
> > box.error.new() set error to diagnostic area: otherwise request errors
> > don't get to diagnostic area on client side.
> > 
> > Needed for #1148
> > Closes #4778
> > 
> > @TarantoolBot document
> > Title: Don't promote error created via box.error.new to diagnostic area
> > 
> > Now box.error.new() only creates error object, but doesn't set it to
> > Tarantool's diagnostic area:
> > ```
> > box.error.clear()
> > e = box.error.new({code = 111, reason = "cause"})
> > assert(box.error.last() == nil)
> > ---
> > - true
> > ...
> > ```
> > To set error in diagnostic area explicitly box.error.set() has been
> > introduced. It accepts error object which is set as last system error
> > (i.e. becomes available via box.error.last()).
> > Finally, box.error.new() does not longer accept error object as an
> > argument (this was undocumented feature).
> > Note that patch does not affect box.error(), which still pushed error to
> 
> 1. 'pushed' -> 'pushes'.
> 
> > diff --git a/src/box/lua/error.cc b/src/box/lua/error.cc
> > index 640e33910..ff285d7eb 100644
> > --- a/src/box/lua/error.cc
> > +++ b/src/box/lua/error.cc
> > @@ -139,9 +144,12 @@ luaT_error_new(lua_State *L)
> >  {
> >  	if (lua_gettop(L) == 0)
> >  		return luaL_error(L, "Usage: box.error.new(code, args)");
> > -	luaT_error_create(L, 1);
> > +	struct error *e = luaT_error_create(L, 1);
> > +	if (e == NULL)
> > +		return luaL_error(L, "box.error.new(): bad arguments");
> 
> 2. I think it would be better to return the same 'Usage' error
> in case of a problem with arguments. Not two different messages
> meaning essentially the same. Or both should return 'bad arguments'.
> However with both 'Usage' the diff would be 1 line smaller.
> 
> Other than these comments, LGTM. If you fix them both - then push right
> away.

Pushed to master, updated changelog.

[Tarantool-patches] [PATCH v2 06/10] box: introduce stacked diagnostic area

[Nikita Pettik]

In terms of implementation, now struct error objects can be organized
into double-linked lists. To achieve this pointers to the next and
previous elements (cause and effort correspondingly) have been added to
struct error. It is worth mentioning that already existing rlist and
stailq list implementations are not suitable: rlist is cycled list, as a
result it is impossible to start iteration over the list from random
list entry and finish it at the logical end of the list; stailq is
single-linked list leaving no possibility to remove elements from the
middle of the list.

As a part of C interface, box_error_add() has been introduced. In
contrast to box_error_set() it does not replace last raised error, but
instead it adds error to the list of diagnostic errors having already
been set. If error is to be deleted (its reference counter hits 0 value)
it is unlinked from the list it belongs to and destroyed. Meanwhile,
error destruction leads to decrement of reference counter of its
previous error and so on.

To organize errors into lists in Lua, table representing error object in
Lua now has .prev field (corresponding to 'previous' error) and method
:set_prev(e). The latter accepts error object (i.e. created via
box.error.new() or box.error.last()) and nil value. Both field .prev and
:set_prev() method are implemented as ffi functions. Also note that
cycles are not allowed while organizing errors into lists:
e1 -> e2 -> e3; e3:set_prev(e1) -- would lead to error.

Part of #1148
---
 extra/exports                   |   1 +
 src/box/key_list.c              |   4 +-
 src/box/lua/call.c              |   4 +-
 src/lib/core/diag.c             |  39 +++++
 src/lib/core/diag.h             |  96 ++++++++++-
 src/lib/core/exception.cc       |   1 +
 src/lua/error.lua               |  32 ++++
 test/box/error.result           | 284 ++++++++++++++++++++++++++++++++
 test/box/error.test.lua         | 105 ++++++++++++
 test/engine/func_index.result   |   6 -
 test/engine/func_index.test.lua |   1 -
 11 files changed, 559 insertions(+), 14 deletions(-)

diff --git a/extra/exports b/extra/exports
index cbb5adcf4..9323996c1 100644
--- a/extra/exports
+++ b/extra/exports
@@ -234,6 +234,7 @@ box_error_message
 box_error_last
 box_error_clear
 box_error_set
+error_set_prev
 box_latch_new
 box_latch_delete
 box_latch_lock
diff --git a/src/box/key_list.c b/src/box/key_list.c
index 3d736b55f..81eb501a5 100644
--- a/src/box/key_list.c
+++ b/src/box/key_list.c
@@ -64,7 +64,7 @@ key_list_iterator_create(struct key_list_iterator *it, struct tuple *tuple,
 		/* Can't evaluate function. */
 		struct space *space = space_by_id(index_def->space_id);
 		diag_set(ClientError, ER_FUNC_INDEX_FUNC, index_def->name,
-			 space ? space_name(space) : "",
+			 space != NULL ? space_name(space) : "",
 			 diag_last_error(diag_get())->errmsg);
 		return -1;
 	}
@@ -75,7 +75,7 @@ key_list_iterator_create(struct key_list_iterator *it, struct tuple *tuple,
 		struct space *space = space_by_id(index_def->space_id);
 		/* Can't get a result returned by function . */
 		diag_set(ClientError, ER_FUNC_INDEX_FUNC, index_def->name,
-			 space ? space_name(space) : "",
+			 space != NULL ? space_name(space) : "",
 			 diag_last_error(diag_get())->errmsg);
 		return -1;
 	}
diff --git a/src/box/lua/call.c b/src/box/lua/call.c
index f1bbde7f0..92575374d 100644
--- a/src/box/lua/call.c
+++ b/src/box/lua/call.c
@@ -688,8 +688,8 @@ func_persistent_lua_load(struct func_lua *func)
 		if (prepare_lua_sandbox(tarantool_L, default_sandbox_exports,
 					nelem(default_sandbox_exports)) != 0) {
 			diag_set(ClientError, ER_LOAD_FUNCTION,
-				func->base.def->name,
-				diag_last_error(diag_get())->errmsg);
+				 func->base.def->name,
+				 diag_last_error(diag_get())->errmsg);
 			goto end;
 		}
 	} else {
diff --git a/src/lib/core/diag.c b/src/lib/core/diag.c
index c350abb4a..57da5da44 100644
--- a/src/lib/core/diag.c
+++ b/src/lib/core/diag.c
@@ -34,6 +34,43 @@
 /* Must be set by the library user */
 struct error_factory *error_factory = NULL;
 
+int
+error_set_prev(struct error *e, struct error *prev)
+{
+	/*
+	 * Make sure that adding error won't result in cycles.
+	 * Don't bother with sophisticated cycle-detection
+	 * algorithms, simple iteration is OK since as a rule
+	 * list contains a dozen errors at maximum.
+	 */
+	struct error *tmp = prev;
+	while (tmp != NULL) {
+		if (tmp == e)
+			return -1;
+		tmp = tmp->cause;
+	}
+	/*
+	 * At once error can feature only one reason.
+	 * So unlink previous 'cause' node.
+	 */
+	if (e->cause != NULL) {
+		e->cause->effect = NULL;
+		error_unref(e->cause);
+	}
+	/* Set new 'prev' node. */
+	e->cause = prev;
+	/*
+	 * Unlink new 'effect' node from its old list of 'cause'
+	 * errors. nil can be also passed as an argument.
+	 */
+	if (prev != NULL) {
+		error_unlink_effect(prev);
+		prev->effect = e;
+		error_ref(prev);
+	}
+	return 0;
+}
+
 void
 error_create(struct error *e,
 	     error_f destroy, error_f raise, error_f log,
@@ -53,6 +90,8 @@ error_create(struct error *e,
 		e->line = 0;
 	}
 	e->errmsg[0] = '\0';
+	e->cause = NULL;
+	e->effect = NULL;
 }
 
 struct diag *
diff --git a/src/lib/core/diag.h b/src/lib/core/diag.h
index 7e1e1a174..675b9c6f1 100644
--- a/src/lib/core/diag.h
+++ b/src/lib/core/diag.h
@@ -84,6 +84,23 @@ struct error {
 	char file[DIAG_FILENAME_MAX];
 	/* Error description. */
 	char errmsg[DIAG_ERRMSG_MAX];
+	/**
+	 * Link to the cause and effect of given error. The cause
+	 * creates the effect:
+	 * e1 = box.error.new({code = 0, reason = 'e1'})
+	 * e2 = box.error.new({code = 0, reason = 'e2'})
+	 * e1:set_prev(e2) -- Now e2 is the cause of e1 and e1 is
+	 * the effect of e2.
+	 * Only cause keeps reference to avoid cyclic dependence.
+	 * RLIST implementation is not really suitable here
+	 * since it is organized as circular list. In such
+	 * a case it is impossible to start an iteration
+	 * from any node and finish at the logical end of the
+	 * list. Double-linked list is required to allow deletion
+	 * from the middle of the list.
+	 */
+	struct error *cause;
+	struct error *effect;
 };
 
 static inline void
@@ -96,11 +113,56 @@ static inline void
 error_unref(struct error *e)
 {
 	assert(e->refs > 0);
-	--e->refs;
-	if (e->refs == 0)
-		e->destroy(e);
+	struct error *to_delete = e;
+	while (--to_delete->refs == 0) {
+		/* Unlink error from lists completely.*/
+		struct error *cause = to_delete->cause;
+		if (to_delete->effect != NULL)
+			to_delete->effect->cause = to_delete->cause;
+		if (to_delete->cause != NULL)
+			to_delete->cause->effect = to_delete->effect;
+		to_delete->cause = NULL;
+		to_delete->effect = NULL;
+		to_delete->destroy(to_delete);
+		if (cause == NULL)
+			return;
+		to_delete = cause;
+	}
+}
+
+/**
+ * Unlink error from its effect. For instance:
+ * e1 -> e2 -> e3 -> e4 (e1:set_prev(e2); e2:set_prev(e3) ...)
+ * unlink(e3): e1 -> e2 -> NULL; e3 -> e4 -> NULL
+ */
+static inline void
+error_unlink_effect(struct error *e)
+{
+	if (e->effect != NULL) {
+		assert(e->refs > 1);
+		error_unref(e);
+		e->effect->cause = NULL;
+	}
+	e->effect = NULL;
 }
 
+/**
+ * Set previous error: cut @a prev from its previous 'tail' of
+ * causes and link to the one @a e belongs to. Note that all
+ * previous errors starting from @a prev->cause are transferred
+ * with it as well (i.e. causes for given error are not erased).
+ * For instance:
+ * e1 -> e2 -> NULL; e3 -> e4 -> NULL;
+ * e2:set_effect(e3): e1 -> e2 -> e3 -> e4 -> NULL
+ *
+ * @a effect can be  NULL. To be used as ffi method in lua/error.lua.
+ *
+ * @retval -1 in case adding @a effect results in list cycles;
+ *          0 otherwise.
+ */
+int
+error_set_prev(struct error *e, struct error *prev);
+
 NORETURN static inline void
 error_raise(struct error *e)
 {
@@ -178,6 +240,25 @@ diag_set_error(struct diag *diag, struct error *e)
 	assert(e != NULL);
 	error_ref(e);
 	diag_clear(diag);
+	error_unlink_effect(e);
+	diag->last = e;
+}
+
+/**
+ * Add a new error to the diagnostics area. It is added to the
+ * tail, so that list forms stack.
+ * @param diag Diagnostics area.
+ * @param e Error to be added.
+ */
+static inline void
+diag_add_error(struct diag *diag, struct error *e)
+{
+	assert(e != NULL);
+	error_ref(e);
+	error_unlink_effect(e);
+	e->cause = diag->last;
+	if (diag->last != NULL)
+		diag->last->effect = e;
 	diag->last = e;
 }
 
@@ -280,6 +361,15 @@ BuildSocketError(const char *file, unsigned line, const char *socketname,
 	errno = save_errno;                                             \
 } while (0)
 
+#define diag_add(class, ...) do {					\
+	int save_errno = errno;						\
+	say_debug("%s at %s:%i", #class, __FILE__, __LINE__);		\
+	struct error *e;						\
+	e = Build##class(__FILE__, __LINE__, ##__VA_ARGS__);		\
+	diag_add_error(diag_get(), e);					\
+	errno = save_errno;						\
+} while (0)
+
 #if defined(__cplusplus)
 } /* extern "C" */
 #endif /* defined(__cplusplus) */
diff --git a/src/lib/core/exception.cc b/src/lib/core/exception.cc
index 0ab10c4bd..180cb0e97 100644
--- a/src/lib/core/exception.cc
+++ b/src/lib/core/exception.cc
@@ -114,6 +114,7 @@ Exception::~Exception()
 	if (this != &out_of_memory) {
 		assert(refs == 0);
 	}
+	TRASH((struct error *) this);
 }
 
 Exception::Exception(const struct type_info *type_arg, const char *file,
diff --git a/src/lua/error.lua b/src/lua/error.lua
index 7f249864a..bdc9c714d 100644
--- a/src/lua/error.lua
+++ b/src/lua/error.lua
@@ -24,12 +24,17 @@ struct error {
     char _file[DIAG_FILENAME_MAX];
     /* Error description. */
     char _errmsg[DIAG_ERRMSG_MAX];
+    struct error *_cause;
+    struct error *_effect;
 };
 
 char *
 exception_get_string(struct error *e, const struct method_info *method);
 int
 exception_get_int(struct error *e, const struct method_info *method);
+
+int
+error_set_prev(struct error *e, struct error *prev);
 ]]
 
 local REFLECTION_CACHE = {}
@@ -95,11 +100,37 @@ local function error_errno(err)
     return e
 end
 
+local function error_prev(err)
+    local e = err._cause;
+    if e ~= nil then
+        return e
+    else
+        return nil
+    end
+end
+
+local function error_set_prev(err, prev)
+    -- First argument must be error.
+    if not ffi.istype('struct error', err) then
+        error("Usage: error1:set_prev(error2)")
+    end
+    -- Second argument must be error or nil.
+    if not ffi.istype('struct error', prev) and prev ~= nil then
+        error("Usage: error1:set_prev(error2)")
+    end
+    local ok = ffi.C.error_set_prev(err, prev);
+    if ok ~= 0 then
+        error("Cycles are not allowed")
+    end
+
+end
+
 local error_fields = {
     ["type"]        = error_type;
     ["message"]     = error_message;
     ["trace"]       = error_trace;
     ["errno"]       = error_errno;
+    ["prev"]        = error_prev;
 }
 
 local function error_unpack(err)
@@ -143,6 +174,7 @@ local error_methods = {
     ["raise"] = error_raise;
     ["match"] = error_match; -- Tarantool 1.6 backward compatibility
     ["__serialize"] = error_serialize;
+    ["set_prev"] = error_set_prev;
 }
 
 local function error_index(err, key)
diff --git a/test/box/error.result b/test/box/error.result
index a19a7044f..ff2b8b270 100644
--- a/test/box/error.result
+++ b/test/box/error.result
@@ -502,6 +502,290 @@ box.error()
  | ---
  | ...
 
+-- gh-1148: errors can be arranged into list (so called
+-- stacked diagnostics).
+--
+e1 = box.error.new({code = 111, reason = "cause"})
+ | ---
+ | ...
+assert(e1.prev == nil)
+ | ---
+ | - true
+ | ...
+e1:set_prev(e1)
+ | ---
+ | - error: 'builtin/error.lua: Cycles are not allowed'
+ | ...
+assert(e1.prev == nil)
+ | ---
+ | - true
+ | ...
+e2 = box.error.new({code = 111, reason = "cause of cause"})
+ | ---
+ | ...
+e1:set_prev(e2)
+ | ---
+ | ...
+assert(e1.prev == e2)
+ | ---
+ | - true
+ | ...
+e2:set_prev(e1)
+ | ---
+ | - error: 'builtin/error.lua: Cycles are not allowed'
+ | ...
+assert(e2.prev == nil)
+ | ---
+ | - true
+ | ...
+-- At this point stack is following: e1 -> e2
+-- Let's test following cases:
+-- 1. e3 -> e2, e1 -> NULL (e3:set_prev(e2))
+-- 2. e1 -> e3, e2 -> NULL (e1:set_prev(e3))
+-- 3. e3 -> e1 -> e2 (e3:set_prev(e1))
+-- 4. e1 -> e2 -> e3 (e2:set_prev(e3))
+--
+e3 = box.error.new({code = 111, reason = "another cause"})
+ | ---
+ | ...
+e3:set_prev(e2)
+ | ---
+ | ...
+assert(e3.prev == e2)
+ | ---
+ | - true
+ | ...
+assert(e2.prev == nil)
+ | ---
+ | - true
+ | ...
+assert(e1.prev == nil)
+ | ---
+ | - true
+ | ...
+
+-- Reset stack to e1 -> e2 and test case 2.
+--
+e1:set_prev(e2)
+ | ---
+ | ...
+assert(e2.prev == nil)
+ | ---
+ | - true
+ | ...
+assert(e3.prev == nil)
+ | ---
+ | - true
+ | ...
+e1:set_prev(e3)
+ | ---
+ | ...
+assert(e2.prev == nil)
+ | ---
+ | - true
+ | ...
+assert(e1.prev == e3)
+ | ---
+ | - true
+ | ...
+assert(e3.prev == nil)
+ | ---
+ | - true
+ | ...
+
+-- Reset stack to e1 -> e2 and test case 3.
+--
+e1:set_prev(e2)
+ | ---
+ | ...
+assert(e1.prev == e2)
+ | ---
+ | - true
+ | ...
+assert(e2.prev == nil)
+ | ---
+ | - true
+ | ...
+assert(e3.prev == nil)
+ | ---
+ | - true
+ | ...
+e3:set_prev(e1)
+ | ---
+ | ...
+assert(e1.prev == e2)
+ | ---
+ | - true
+ | ...
+assert(e2.prev == nil)
+ | ---
+ | - true
+ | ...
+assert(e3.prev == e1)
+ | ---
+ | - true
+ | ...
+
+-- Unlink errors and test case 4.
+--
+e1:set_prev(nil)
+ | ---
+ | ...
+e2:set_prev(nil)
+ | ---
+ | ...
+e3:set_prev(nil)
+ | ---
+ | ...
+e1:set_prev(e2)
+ | ---
+ | ...
+e2:set_prev(e3)
+ | ---
+ | ...
+assert(e1.prev == e2)
+ | ---
+ | - true
+ | ...
+assert(e2.prev == e3)
+ | ---
+ | - true
+ | ...
+assert(e3.prev == nil)
+ | ---
+ | - true
+ | ...
+
+-- Test circle detecting. At the moment stack is
+-- following: e1 -> e2 -> e3
+--
+e3:set_prev(e1)
+ | ---
+ | - error: 'builtin/error.lua: Cycles are not allowed'
+ | ...
+assert(e3.prev == nil)
+ | ---
+ | - true
+ | ...
+e3:set_prev(e2)
+ | ---
+ | - error: 'builtin/error.lua: Cycles are not allowed'
+ | ...
+assert(e3.prev == nil)
+ | ---
+ | - true
+ | ...
+
+-- Test splitting list into two ones.
+-- After that we will get two lists: e1->e2->e5 and e3->e4
+--
+e4 = box.error.new({code = 111, reason = "yet another cause"})
+ | ---
+ | ...
+e5 = box.error.new({code = 111, reason = "and another one"})
+ | ---
+ | ...
+e3:set_prev(e4)
+ | ---
+ | ...
+e2:set_prev(e5)
+ | ---
+ | ...
+assert(e1.prev == e2)
+ | ---
+ | - true
+ | ...
+assert(e2.prev == e5)
+ | ---
+ | - true
+ | ...
+assert(e3.prev == e4)
+ | ---
+ | - true
+ | ...
+assert(e5.prev == nil)
+ | ---
+ | - true
+ | ...
+assert(e4.prev == nil)
+ | ---
+ | - true
+ | ...
+
+-- Another splitting option: e1->e2 and e5->e3->e4
+-- But firstly restore to one single list e1->e2->e3->e4
+--
+e2:set_prev(e3)
+ | ---
+ | ...
+e5:set_prev(e3)
+ | ---
+ | ...
+assert(e1.prev == e2)
+ | ---
+ | - true
+ | ...
+assert(e2.prev == nil)
+ | ---
+ | - true
+ | ...
+assert(e5.prev == e3)
+ | ---
+ | - true
+ | ...
+assert(e3.prev == e4)
+ | ---
+ | - true
+ | ...
+assert(e4.prev == nil)
+ | ---
+ | - true
+ | ...
+
+-- In case error is destroyed, it unrefs reference counter
+-- of its previous error. In turn, box.error.clear() refs/unrefs
+-- only head and doesn't touch other errors.
+--
+e2:set_prev(nil)
+ | ---
+ | ...
+box.error.set(e1)
+ | ---
+ | ...
+assert(box.error.last() == e1)
+ | ---
+ | - true
+ | ...
+assert(box.error.last().prev == e2)
+ | ---
+ | - true
+ | ...
+box.error.clear()
+ | ---
+ | ...
+assert(box.error.last() == nil)
+ | ---
+ | - true
+ | ...
+assert(e1.prev == e2)
+ | ---
+ | - true
+ | ...
+assert(e2.code  == 111)
+ | ---
+ | - true
+ | ...
+box.error.set(e1)
+ | ---
+ | ...
+box.error.clear()
+ | ---
+ | ...
+assert(e1.prev == e2)
+ | ---
+ | - true
+ | ...
+
 space:drop()
  | ---
  | ...
diff --git a/test/box/error.test.lua b/test/box/error.test.lua
index a0b7d3e78..1fdd6ed98 100644
--- a/test/box/error.test.lua
+++ b/test/box/error.test.lua
@@ -108,4 +108,109 @@ box.error.new(err)
 box.error.clear()
 box.error()
 
+-- gh-1148: errors can be arranged into list (so called
+-- stacked diagnostics).
+--
+e1 = box.error.new({code = 111, reason = "cause"})
+assert(e1.prev == nil)
+e1:set_prev(e1)
+assert(e1.prev == nil)
+e2 = box.error.new({code = 111, reason = "cause of cause"})
+e1:set_prev(e2)
+assert(e1.prev == e2)
+e2:set_prev(e1)
+assert(e2.prev == nil)
+-- At this point stack is following: e1 -> e2
+-- Let's test following cases:
+-- 1. e3 -> e2, e1 -> NULL (e3:set_prev(e2))
+-- 2. e1 -> e3, e2 -> NULL (e1:set_prev(e3))
+-- 3. e3 -> e1 -> e2 (e3:set_prev(e1))
+-- 4. e1 -> e2 -> e3 (e2:set_prev(e3))
+--
+e3 = box.error.new({code = 111, reason = "another cause"})
+e3:set_prev(e2)
+assert(e3.prev == e2)
+assert(e2.prev == nil)
+assert(e1.prev == nil)
+
+-- Reset stack to e1 -> e2 and test case 2.
+--
+e1:set_prev(e2)
+assert(e2.prev == nil)
+assert(e3.prev == nil)
+e1:set_prev(e3)
+assert(e2.prev == nil)
+assert(e1.prev == e3)
+assert(e3.prev == nil)
+
+-- Reset stack to e1 -> e2 and test case 3.
+--
+e1:set_prev(e2)
+assert(e1.prev == e2)
+assert(e2.prev == nil)
+assert(e3.prev == nil)
+e3:set_prev(e1)
+assert(e1.prev == e2)
+assert(e2.prev == nil)
+assert(e3.prev == e1)
+
+-- Unlink errors and test case 4.
+--
+e1:set_prev(nil)
+e2:set_prev(nil)
+e3:set_prev(nil)
+e1:set_prev(e2)
+e2:set_prev(e3)
+assert(e1.prev == e2)
+assert(e2.prev == e3)
+assert(e3.prev == nil)
+
+-- Test circle detecting. At the moment stack is
+-- following: e1 -> e2 -> e3
+--
+e3:set_prev(e1)
+assert(e3.prev == nil)
+e3:set_prev(e2)
+assert(e3.prev == nil)
+
+-- Test splitting list into two ones.
+-- After that we will get two lists: e1->e2->e5 and e3->e4
+--
+e4 = box.error.new({code = 111, reason = "yet another cause"})
+e5 = box.error.new({code = 111, reason = "and another one"})
+e3:set_prev(e4)
+e2:set_prev(e5)
+assert(e1.prev == e2)
+assert(e2.prev == e5)
+assert(e3.prev == e4)
+assert(e5.prev == nil)
+assert(e4.prev == nil)
+
+-- Another splitting option: e1->e2 and e5->e3->e4
+-- But firstly restore to one single list e1->e2->e3->e4
+--
+e2:set_prev(e3)
+e5:set_prev(e3)
+assert(e1.prev == e2)
+assert(e2.prev == nil)
+assert(e5.prev == e3)
+assert(e3.prev == e4)
+assert(e4.prev == nil)
+
+-- In case error is destroyed, it unrefs reference counter
+-- of its previous error. In turn, box.error.clear() refs/unrefs
+-- only head and doesn't touch other errors.
+--
+e2:set_prev(nil)
+box.error.set(e1)
+assert(box.error.last() == e1)
+assert(box.error.last().prev == e2)
+box.error.clear()
+assert(box.error.last() == nil)
+assert(e1.prev == e2)
+assert(e2.code  == 111)
+box.error.set(e1)
+box.error.clear()
+assert(e1.prev == e2)
+
 space:drop()
diff --git a/test/engine/func_index.result b/test/engine/func_index.result
index 84cb83022..159158f1c 100644
--- a/test/engine/func_index.result
+++ b/test/engine/func_index.result
@@ -261,12 +261,6 @@ box.schema.func.create('runtimeerror', {body = lua_code, is_deterministic = true
 idx = s:create_index('idx', {func = box.func.runtimeerror.id, parts = {{1, 'string'}}})
  | ---
  | ...
-s:insert({1})
- | ---
- | - error: 'Failed to build a key for functional index ''idx'' of space ''withdata'':
- |     [string "return function(tuple)                 local ..."]:1: attempt to call
- |     global ''require'' (a nil value)'
- | ...
 idx:drop()
  | ---
  | ...
diff --git a/test/engine/func_index.test.lua b/test/engine/func_index.test.lua
index f31162c97..c3c3a7029 100644
--- a/test/engine/func_index.test.lua
+++ b/test/engine/func_index.test.lua
@@ -98,7 +98,6 @@ lua_code = [[function(tuple)
 test_run:cmd("setopt delimiter ''");
 box.schema.func.create('runtimeerror', {body = lua_code, is_deterministic = true, is_sandboxed = true})
 idx = s:create_index('idx', {func = box.func.runtimeerror.id, parts = {{1, 'string'}}})
-s:insert({1})
 idx:drop()
 
 -- Remove old persistent functions
-- 
2.17.1

Re: [Tarantool-patches] [PATCH v2 06/10] box: introduce stacked diagnostic area

[Konstantin Osipov]

* Nikita Pettik <korablev@tarantool.org> [20/03/25 09:32]:
> +	/*
> +	 * Make sure that adding error won't result in cycles.
> +	 * Don't bother with sophisticated cycle-detection
> +	 * algorithms, simple iteration is OK since as a rule
> +	 * list contains a dozen errors at maximum.
> +	 */
> +	struct error *tmp = prev;
> +	while (tmp != NULL) {
> +		if (tmp == e)
> +			return -1;
> +		tmp = tmp->cause;
> +	}

If you have to do it, better use rlist, rlist_del(e) rlist_add(e),
this will make sure there are no cycles and the cost is constant.

> +	if (prev != NULL) {
> +		error_unlink_effect(prev);
> +		prev->effect = e;
> +		error_ref(prev);
> +	}

cause and effect can be just wrappers around rlist_prev/next, if
you have to. But error stack is not always cause and effect:
imagine an SQL update which runs a bunch of triggers which all set
add some errors to the diag, which one of these triggers errors is
the cause of update error?.


-- 
Konstantin Osipov, Moscow, Russia

Re: [Tarantool-patches] [PATCH v2 06/10] box: introduce stacked diagnostic area

[Nikita Pettik]

On 26 Mar 19:54, Konstantin Osipov wrote:
> * Nikita Pettik <korablev@tarantool.org> [20/03/25 09:32]:
> > +	/*
> > +	 * Make sure that adding error won't result in cycles.
> > +	 * Don't bother with sophisticated cycle-detection
> > +	 * algorithms, simple iteration is OK since as a rule
> > +	 * list contains a dozen errors at maximum.
> > +	 */
> > +	struct error *tmp = prev;
> > +	while (tmp != NULL) {
> > +		if (tmp == e)
> > +			return -1;
> > +		tmp = tmp->cause;
> > +	}
> 
> If you have to do it, better use rlist, rlist_del(e) rlist_add(e),
> this will make sure there are no cycles and the cost is constant.

rlist implementation is not really suitable for organizing errors
into list. See comment in the code:

	 * RLIST implementation is not really suitable here
	 * since it is organized as circular list. In such
	 * a case it is impossible to start an iteration
	 * from any node and finish at the logical end of the
	 * list.
 
> > +	if (prev != NULL) {
> > +		error_unlink_effect(prev);
> > +		prev->effect = e;
> > +		error_ref(prev);
> > +	}
> 
> cause and effect can be just wrappers around rlist_prev/next, if
> you have to. But error stack is not always cause and effect:
> imagine an SQL update which runs a bunch of triggers which all set
> add some errors to the diag, which one of these triggers errors is
> the cause of update error?.

I guess it is ok to stop execution after first trigger sets an error..?
They are fired one by one, not all at once.

> 
> -- 
> Konstantin Osipov, Moscow, Russia

Re: [Tarantool-patches] [PATCH v2 06/10] box: introduce stacked diagnostic area

[Konstantin Osipov]

* Nikita Pettik <korablev@tarantool.org> [20/03/26 21:04]:
> > If you have to do it, better use rlist, rlist_del(e) rlist_add(e),
> > this will make sure there are no cycles and the cost is constant.
> 
> rlist implementation is not really suitable for organizing errors
> into list. See comment in the code:
> 
> 	 * RLIST implementation is not really suitable here
> 	 * since it is organized as circular list. In such
> 	 * a case it is impossible to start an iteration
> 	 * from any node and finish at the logical end of the
> 	 * list.

I missed it, sorry. No, it's still possible, you can stor struct
rlist head in the diagnostics, and whenever you meet the head, you
reached the end.

Basically an end marker.


-- 
Konstantin Osipov, Moscow, Russia

Re: [Tarantool-patches] [PATCH v2 06/10] box: introduce stacked diagnostic area

[Vladislav Shpilevoy]

Hi! Thanks for the patch!

See 7 comments below.

On 25/03/2020 02:43, Nikita Pettik wrote:
> In terms of implementation, now struct error objects can be organized
> into double-linked lists. To achieve this pointers to the next and
> previous elements (cause and effort correspondingly) have been added to

1. effort -> effect.

> struct error. It is worth mentioning that already existing rlist and
> stailq list implementations are not suitable: rlist is cycled list, as a
> result it is impossible to start iteration over the list from random
> list entry and finish it at the logical end of the list; stailq is
> single-linked list leaving no possibility to remove elements from the
> middle of the list.
> 
> diff --git a/src/box/key_list.c b/src/box/key_list.c
> index 3d736b55f..81eb501a5 100644
> --- a/src/box/key_list.c
> +++ b/src/box/key_list.c
> @@ -64,7 +64,7 @@ key_list_iterator_create(struct key_list_iterator *it, struct tuple *tuple,
>  		/* Can't evaluate function. */
>  		struct space *space = space_by_id(index_def->space_id);
>  		diag_set(ClientError, ER_FUNC_INDEX_FUNC, index_def->name,
> -			 space ? space_name(space) : "",
> +			 space != NULL ? space_name(space) : "",
>  			 diag_last_error(diag_get())->errmsg);
>  		return -1;
>  	}
> @@ -75,7 +75,7 @@ key_list_iterator_create(struct key_list_iterator *it, struct tuple *tuple,
>  		struct space *space = space_by_id(index_def->space_id);
>  		/* Can't get a result returned by function . */
>  		diag_set(ClientError, ER_FUNC_INDEX_FUNC, index_def->name,
> -			 space ? space_name(space) : "",
> +			 space != NULL ? space_name(space) : "",
>  			 diag_last_error(diag_get())->errmsg);
>  		return -1;
>  	}
> diff --git a/src/box/lua/call.c b/src/box/lua/call.c
> index f1bbde7f0..92575374d 100644
> --- a/src/box/lua/call.c
> +++ b/src/box/lua/call.c
> @@ -688,8 +688,8 @@ func_persistent_lua_load(struct func_lua *func)
>  		if (prepare_lua_sandbox(tarantool_L, default_sandbox_exports,
>  					nelem(default_sandbox_exports)) != 0) {
>  			diag_set(ClientError, ER_LOAD_FUNCTION,
> -				func->base.def->name,
> -				diag_last_error(diag_get())->errmsg);
> +				 func->base.def->name,
> +				 diag_last_error(diag_get())->errmsg);
>  			goto end;

2. Do you really need the 3 hunks above? Seems like unnecessary
refactoring.

>  		}
>  	} else {
> diff --git a/src/lib/core/diag.c b/src/lib/core/diag.c
> index c350abb4a..57da5da44 100644
> --- a/src/lib/core/diag.c
> +++ b/src/lib/core/diag.c
> @@ -34,6 +34,43 @@
>  /* Must be set by the library user */
>  struct error_factory *error_factory = NULL;
>  
> +int
> +error_set_prev(struct error *e, struct error *prev)
> +{
> +	/*
> +	 * Make sure that adding error won't result in cycles.
> +	 * Don't bother with sophisticated cycle-detection
> +	 * algorithms, simple iteration is OK since as a rule
> +	 * list contains a dozen errors at maximum.
> +	 */
> +	struct error *tmp = prev;
> +	while (tmp != NULL) {
> +		if (tmp == e)
> +			return -1;
> +		tmp = tmp->cause;
> +	}

3. Or we could remove 'prev' from its current stack gracefully,
since the list is doubly linked. Then cycles become impossible.
Is there a reason not to do that? I mean, we remove 'prev'
from its old stack by changing its 'effect' already anyway.

Can do the same for the other pointer, and no need to check
for cycles anymore.

I can only think of a problem when someone makes

    e1 = box.error.new()
    e2 = box.error.new()
    e2:set_prev(e1)

and then

    e4 = box.error.new()
    e4:set_prev(e1)

assuming that e1 will be copied. But it will steal e1 from
its old stack. Even with your current solution. Maybe to make
it more protected against dummies we should forbid moving error
from one stack to another? So an error can't change its 'cause'
and 'effect' once they become not NULL. In that way we eliminate
any uncertainties and in future can decide whether we want to copy
in set_prev() when necessary, or move from previous stack. Or we
will keep this forbidden because probably no one does moving
between stacks, nor the example I showed above. This also
significantly simplifies tests and reduces probability of having
a bug there. What are your thoughts?

> +	/*
> +	 * At once error can feature only one reason.
> +	 * So unlink previous 'cause' node.
> +	 */
> +	if (e->cause != NULL) {
> +		e->cause->effect = NULL;
> +		error_unref(e->cause);
> +	}
> +	/* Set new 'prev' node. */
> +	e->cause = prev;
> +	/*
> +	 * Unlink new 'effect' node from its old list of 'cause'
> +	 * errors. nil can be also passed as an argument.
> +	 */
> +	if (prev != NULL) {
> +		error_unlink_effect(prev);
> +		prev->effect = e;
> +		error_ref(prev);

4. Better call error_ref() before error_unlink_effect(). To
make it impossible for prev to be deleted inside unlink() and
be used after free. I can't find how can be it done now, but by
moving ref() a bit above we will make it impossible in future
as well.

> +	}
> +	return 0;
> +}
> diff --git a/src/lib/core/diag.h b/src/lib/core/diag.h
> index 7e1e1a174..675b9c6f1 100644
> --- a/src/lib/core/diag.h
> +++ b/src/lib/core/diag.h> @@ -96,11 +113,56 @@ static inline void
>  
> +/**
> + * Set previous error: cut @a prev from its previous 'tail' of
> + * causes and link to the one @a e belongs to. Note that all
> + * previous errors starting from @a prev->cause are transferred
> + * with it as well (i.e. causes for given error are not erased).
> + * For instance:
> + * e1 -> e2 -> NULL; e3 -> e4 -> NULL;
> + * e2:set_effect(e3): e1 -> e2 -> e3 -> e4 -> NULL
> + *
> + * @a effect can be  NULL. To be used as ffi method in lua/error.lua.

5. Out of 66 and double whitespace before 'NULL'.

> + *
> + * @retval -1 in case adding @a effect results in list cycles;
> + *          0 otherwise.
> + */
> +int
> +error_set_prev(struct error *e, struct error *prev);
> +
> diff --git a/test/box/error.result b/test/box/error.result
> index a19a7044f..ff2b8b270 100644
> --- a/test/box/error.result
> +++ b/test/box/error.result
> @@ -502,6 +502,290 @@ box.error()
> +-- In case error is destroyed, it unrefs reference counter
> +-- of its previous error. In turn, box.error.clear() refs/unrefs
> +-- only head and doesn't touch other errors.
> +--
> +e2:set_prev(nil)
> + | ---
> + | ...
> +box.error.set(e1)
> + | ---
> + | ...
> +assert(box.error.last() == e1)
> + | ---
> + | - true
> + | ...
> +assert(box.error.last().prev == e2)
> + | ---
> + | - true
> + | ...
> +box.error.clear()
> + | ---
> + | ...
> +assert(box.error.last() == nil)
> + | ---
> + | - true
> + | ...
> +assert(e1.prev == e2)
> + | ---
> + | - true
> + | ...
> +assert(e2.code  == 111)

6. Double whitespace. Macbook keyboard, huh?

> diff --git a/test/engine/func_index.result b/test/engine/func_index.result
> index 84cb83022..159158f1c 100644
> --- a/test/engine/func_index.result
> +++ b/test/engine/func_index.result
> @@ -261,12 +261,6 @@ box.schema.func.create('runtimeerror', {body = lua_code, is_deterministic = true
>  idx = s:create_index('idx', {func = box.func.runtimeerror.id, parts = {{1, 'string'}}})
>   | ---
>   | ...
> -s:insert({1})
> - | ---
> - | - error: 'Failed to build a key for functional index ''idx'' of space ''withdata'':
> - |     [string "return function(tuple)                 local ..."]:1: attempt to call
> - |     global ''require'' (a nil value)'
> - | ...
>  idx:drop()

7. Why do you drop it here? The commit does not change any
existing places setting errors. So it should not lead to
any test changes.

>   | ---
>   | ...

Re: [Tarantool-patches] [PATCH v2 06/10] box: introduce stacked diagnostic area

[Nikita Pettik]

On 28 Mar 19:40, Vladislav Shpilevoy wrote:
> Hi! Thanks for the patch!
> 
> See 7 comments below.
> 
> On 25/03/2020 02:43, Nikita Pettik wrote:
> > In terms of implementation, now struct error objects can be organized
> > into double-linked lists. To achieve this pointers to the next and
> > previous elements (cause and effort correspondingly) have been added to
> 
> 1. effort -> effect.

Fixed.
 
> > struct error. It is worth mentioning that already existing rlist and
> > stailq list implementations are not suitable: rlist is cycled list, as a
> > result it is impossible to start iteration over the list from random
> > list entry and finish it at the logical end of the list; stailq is
> > single-linked list leaving no possibility to remove elements from the
> > middle of the list.
> > 
> > diff --git a/src/box/key_list.c b/src/box/key_list.c
> > index 3d736b55f..81eb501a5 100644
> > --- a/src/box/key_list.c
> > +++ b/src/box/key_list.c
> > @@ -64,7 +64,7 @@ key_list_iterator_create(struct key_list_iterator *it, struct tuple *tuple,
> >  		/* Can't evaluate function. */
> >  		struct space *space = space_by_id(index_def->space_id);
> >  		diag_set(ClientError, ER_FUNC_INDEX_FUNC, index_def->name,
> > -			 space ? space_name(space) : "",
> > +			 space != NULL ? space_name(space) : "",
> >  			 diag_last_error(diag_get())->errmsg);
> >  		return -1;
> >  	}
> > @@ -75,7 +75,7 @@ key_list_iterator_create(struct key_list_iterator *it, struct tuple *tuple,
> >  		struct space *space = space_by_id(index_def->space_id);
> >  		/* Can't get a result returned by function . */
> >  		diag_set(ClientError, ER_FUNC_INDEX_FUNC, index_def->name,
> > -			 space ? space_name(space) : "",
> > +			 space != NULL ? space_name(space) : "",
> >  			 diag_last_error(diag_get())->errmsg);
> >  		return -1;
> >  	}
> > diff --git a/src/box/lua/call.c b/src/box/lua/call.c
> > index f1bbde7f0..92575374d 100644
> > --- a/src/box/lua/call.c
> > +++ b/src/box/lua/call.c
> > @@ -688,8 +688,8 @@ func_persistent_lua_load(struct func_lua *func)
> >  		if (prepare_lua_sandbox(tarantool_L, default_sandbox_exports,
> >  					nelem(default_sandbox_exports)) != 0) {
> >  			diag_set(ClientError, ER_LOAD_FUNCTION,
> > -				func->base.def->name,
> > -				diag_last_error(diag_get())->errmsg);
> > +				 func->base.def->name,
> > +				 diag_last_error(diag_get())->errmsg);
> >  			goto end;
> 
> 2. Do you really need the 3 hunks above? Seems like unnecessary
> refactoring.

Fixed (extra diff occured due to splitting into two patches).

> > diff --git a/src/lib/core/diag.c b/src/lib/core/diag.c
> > index c350abb4a..57da5da44 100644
> > --- a/src/lib/core/diag.c
> > +++ b/src/lib/core/diag.c
> > @@ -34,6 +34,43 @@
> >  /* Must be set by the library user */
> >  struct error_factory *error_factory = NULL;
> >  
> > +int
> > +error_set_prev(struct error *e, struct error *prev)
> > +{
> > +	/*
> > +	 * Make sure that adding error won't result in cycles.
> > +	 * Don't bother with sophisticated cycle-detection
> > +	 * algorithms, simple iteration is OK since as a rule
> > +	 * list contains a dozen errors at maximum.
> > +	 */
> > +	struct error *tmp = prev;
> > +	while (tmp != NULL) {
> > +		if (tmp == e)
> > +			return -1;
> > +		tmp = tmp->cause;
> > +	}
> 
> 3. Or we could remove 'prev' from its current stack gracefully,
> since the list is doubly linked. Then cycles become impossible.
> Is there a reason not to do that? I mean, we remove 'prev'
> from its old stack by changing its 'effect' already anyway.
> 
> Can do the same for the other pointer, and no need to check
> for cycles anymore.
> I can only think of a problem when someone makes
> 
>     e1 = box.error.new()
>     e2 = box.error.new()
>     e2:set_prev(e1)
> 
> and then
> 
>     e4 = box.error.new()
>     e4:set_prev(e1)
> 
> assuming that e1 will be copied. But it will steal e1 from
> its old stack. Even with your current solution. Maybe to make
> it more protected against dummies we should forbid moving error
> from one stack to another? So an error can't change its 'cause'
> and 'effect' once they become not NULL. In that way we eliminate
> any uncertainties and in future can decide whether we want to copy
> in set_prev() when necessary, or move from previous stack. Or we
> will keep this forbidden because probably no one does moving
> between stacks, nor the example I showed above. This also
> significantly simplifies tests and reduces probability of having
> a bug there. What are your thoughts?

I'm okay with current approach. It is all about documentation - 
the better feature is documented, the easier it turns out to
be in terms of usage. There's always way to misuse any feature
however well designed. If you insist on some changes - let me know.

> > +	/*
> > +	 * At once error can feature only one reason.
> > +	 * So unlink previous 'cause' node.
> > +	 */
> > +	if (e->cause != NULL) {
> > +		e->cause->effect = NULL;
> > +		error_unref(e->cause);
> > +	}
> > +	/* Set new 'prev' node. */
> > +	e->cause = prev;
> > +	/*
> > +	 * Unlink new 'effect' node from its old list of 'cause'
> > +	 * errors. nil can be also passed as an argument.
> > +	 */
> > +	if (prev != NULL) {
> > +		error_unlink_effect(prev);
> > +		prev->effect = e;
> > +		error_ref(prev);
> 
> 4. Better call error_ref() before error_unlink_effect(). To
> make it impossible for prev to be deleted inside unlink() and
> be used after free. I can't find how can be it done now, but by
> moving ref() a bit above we will make it impossible in future
> as well.

Agree, fixed.

> > diff --git a/src/lib/core/diag.h b/src/lib/core/diag.h
> > index 7e1e1a174..675b9c6f1 100644
> > --- a/src/lib/core/diag.h
> > +++ b/src/lib/core/diag.h> @@ -96,11 +113,56 @@ static inline void
> >  
> > +/**
> > + * Set previous error: cut @a prev from its previous 'tail' of
> > + * causes and link to the one @a e belongs to. Note that all
> > + * previous errors starting from @a prev->cause are transferred
> > + * with it as well (i.e. causes for given error are not erased).
> > + * For instance:
> > + * e1 -> e2 -> NULL; e3 -> e4 -> NULL;
> > + * e2:set_effect(e3): e1 -> e2 -> e3 -> e4 -> NULL
> > + *
> > + * @a effect can be  NULL. To be used as ffi method in lua/error.lua.
> 
> 5. Out of 66 and double whitespace before 'NULL'.

Fixed.
 
> > diff --git a/test/box/error.result b/test/box/error.result
> > index a19a7044f..ff2b8b270 100644
> > --- a/test/box/error.result
> > +++ b/test/box/error.result
> > @@ -502,6 +502,290 @@ box.error()
> > +-- In case error is destroyed, it unrefs reference counter
> > +-- of its previous error. In turn, box.error.clear() refs/unrefs
> > +-- only head and doesn't touch other errors.
> > +--
> > +e2:set_prev(nil)
> > + | ---
> > + | ...
> > +box.error.set(e1)
> > + | ---
> > + | ...
> > +assert(box.error.last() == e1)
> > + | ---
> > + | - true
> > + | ...
> > +assert(box.error.last().prev == e2)
> > + | ---
> > + | - true
> > + | ...
> > +box.error.clear()
> > + | ---
> > + | ...
> > +assert(box.error.last() == nil)
> > + | ---
> > + | - true
> > + | ...
> > +assert(e1.prev == e2)
> > + | ---
> > + | - true
> > + | ...
> > +assert(e2.code  == 111)
> 
> 6. Double whitespace. Macbook keyboard, huh?

Fixed.

> > diff --git a/test/engine/func_index.result b/test/engine/func_index.result
> > index 84cb83022..159158f1c 100644
> > --- a/test/engine/func_index.result
> > +++ b/test/engine/func_index.result
> > @@ -261,12 +261,6 @@ box.schema.func.create('runtimeerror', {body = lua_code, is_deterministic = true
> >  idx = s:create_index('idx', {func = box.func.runtimeerror.id, parts = {{1, 'string'}}})
> >   | ---
> >   | ...
> > -s:insert({1})
> > - | ---
> > - | - error: 'Failed to build a key for functional index ''idx'' of space ''withdata'':
> > - |     [string "return function(tuple)                 local ..."]:1: attempt to call
> > - |     global ''require'' (a nil value)'
> > - | ...
> >  idx:drop()
> 
> 7. Why do you drop it here? The commit does not change any
> existing places setting errors. So it should not lead to
> any test changes.

Sorry, removed (accidental diff due to splitting into two commits).

Re: [Tarantool-patches] [PATCH v2 06/10] box: introduce stacked diagnostic area

[Vladislav Shpilevoy]

[-- Attachment #1: Type: text/plain, Size: 18079 bytes --]

Hi! Thanks for the fixes!

See 2 comments below.

> static inline void
> error_unref(struct error *e)
> {
> 	assert(e->refs > 0);
> 	struct error *to_delete = e;
> 	while (--to_delete->refs == 0) {
> 		/* Unlink error from lists completely.*/
> 		struct error *cause = to_delete->cause;
> 		if (to_delete->effect != NULL)
> 			to_delete->effect->cause = to_delete->cause;

1. Looks like 'effect' should be always NULL here. Because
if it is not NULL, then effect->cause == e, and it would keep a
reference, making e->refs == 0 impossible. I applied the diff
below and all the tests passed.

====================
diff --git a/src/lib/core/diag.h b/src/lib/core/diag.h
index 3a817a659..276bf2362 100644
--- a/src/lib/core/diag.h
+++ b/src/lib/core/diag.h
@@ -117,12 +117,11 @@ error_unref(struct error *e)
 	while (--to_delete->refs == 0) {
 		/* Unlink error from lists completely.*/
 		struct error *cause = to_delete->cause;
-		if (to_delete->effect != NULL)
-			to_delete->effect->cause = to_delete->cause;
-		if (to_delete->cause != NULL)
-			to_delete->cause->effect = to_delete->effect;
-		to_delete->cause = NULL;
-		to_delete->effect = NULL;
+		assert(to_delete->effect == NULL);
+		if (to_delete->cause != NULL) {
+			to_delete->cause->effect = NULL;
+			to_delete->cause = NULL;
+		}
 		to_delete->destroy(to_delete);
 		if (cause == NULL)
 			return;
====================

> 		if (to_delete->cause != NULL)
> 			to_delete->cause->effect = to_delete->effect;
> 		to_delete->cause = NULL;
> 		to_delete->effect = NULL;
> 		to_delete->destroy(to_delete);
> 		if (cause == NULL)
> 			return;
> 		to_delete = cause;
> 	}
> }>>> diff --git a/src/lib/core/diag.c b/src/lib/core/diag.c
>>> index c350abb4a..57da5da44 100644
>>> --- a/src/lib/core/diag.c
>>> +++ b/src/lib/core/diag.c
>>> @@ -34,6 +34,43 @@
>>>  /* Must be set by the library user */
>>>  struct error_factory *error_factory = NULL;
>>>  
>>> +int
>>> +error_set_prev(struct error *e, struct error *prev)
>>> +{
>>> +	/*
>>> +	 * Make sure that adding error won't result in cycles.
>>> +	 * Don't bother with sophisticated cycle-detection
>>> +	 * algorithms, simple iteration is OK since as a rule
>>> +	 * list contains a dozen errors at maximum.
>>> +	 */
>>> +	struct error *tmp = prev;
>>> +	while (tmp != NULL) {
>>> +		if (tmp == e)
>>> +			return -1;
>>> +		tmp = tmp->cause;
>>> +	}
>>
>> 3. Or we could remove 'prev' from its current stack gracefully,
>> since the list is doubly linked. Then cycles become impossible.
>> Is there a reason not to do that? I mean, we remove 'prev'
>> from its old stack by changing its 'effect' already anyway.
>>
>> Can do the same for the other pointer, and no need to check
>> for cycles anymore.
>> I can only think of a problem when someone makes
>>
>>     e1 = box.error.new()
>>     e2 = box.error.new()
>>     e2:set_prev(e1)
>>
>> and then
>>
>>     e4 = box.error.new()
>>     e4:set_prev(e1)
>>
>> assuming that e1 will be copied. But it will steal e1 from
>> its old stack. Even with your current solution. Maybe to make
>> it more protected against dummies we should forbid moving error
>> from one stack to another? So an error can't change its 'cause'
>> and 'effect' once they become not NULL. In that way we eliminate
>> any uncertainties and in future can decide whether we want to copy
>> in set_prev() when necessary, or move from previous stack. Or we
>> will keep this forbidden because probably no one does moving
>> between stacks, nor the example I showed above. This also
>> significantly simplifies tests and reduces probability of having
>> a bug there. What are your thoughts?
> 
> I'm okay with current approach. It is all about documentation - 
> the better feature is documented, the easier it turns out to
> be in terms of usage. There's always way to misuse any feature
> however well designed. If you insist on some changes - let me know.

2. I don't insist. Can only recommend. And I would strongly recommend
to cut the ability to move errors between stacks. I see only pros in
cutting that. Because it is never needed (I can't imagine a sane case),
can be surprising for someone, and complicates the code. The server
code. Makes it easier to leave a bug here. Another reason is that it is
easier to add something later if someone will need it, than to forbid
something after introduction.

Besides, it seems we could even make the error list unidirectional then.
Instead of 'effect' pointer we could keep a flag saying whether this error
belongs to a stack. No more complex handling of when to do ref/unref/cycles
and so on. Even bigger simplification.

I decided to try that and got a patch. See it attached to the email as
'unidirect.txt', and below. I didn't push it on the branch so as not to
change next commits.

On the summary, I made the list unidirectional, removed the cycle
detection and forbidden any chance of creating it by banning change of
error object when it is not on top of the stack. So a stack should be
built always in one order - from the cause of all causes to top. The
only thing you can do is to cut a tail of a stack. All more complex
actions which may be needed in very rare cases still can be implemented
by users using the provided basic API.

That made the patch 110 lines shorter.


Also a small comment about cycle detection in your patch. Here I
would strongly-strongly recommend to avoid it when it is clearly
not necessary.

You can't create a cycle, if e->effect and prev->effect are NULL.
Which is the case in 99% of cases. Even if prev->cause is not NULL,
you still can't create a cycle when both effects are NULL. See diff
below and attached as 'opt-cycle.txt'. That skips 'prev' iteration
even if it is a whole another stack, but 'prev' is its top.

Optimized cycle detection:
====================
diff --git a/src/lib/core/diag.c b/src/lib/core/diag.c
index ed8beb58d..64ff5e4d6 100644
--- a/src/lib/core/diag.c
+++ b/src/lib/core/diag.c
@@ -43,11 +43,28 @@ error_set_prev(struct error *e, struct error *prev)
 	 * algorithms, simple iteration is OK since as a rule
 	 * list contains a dozen errors at maximum.
 	 */
-	struct error *tmp = prev;
-	while (tmp != NULL) {
-		if (tmp == e)
+	if (prev != NULL) {
+		if (e == prev)
 			return -1;
-		tmp = tmp->cause;
+		if (prev->effect != NULL || e->effect != NULL) {
+			/*
+			 * e and prev already compared, so start
+			 * from prev->cause.
+			 */
+			struct error *tmp = prev->cause;
+			while (tmp != NULL) {
+				if (tmp == e)
+					return -1;
+				tmp = tmp->cause;
+			}
+			/*
+			 * Unlink new 'effect' node from its old
+			 * list of 'cause' errors.
+			 */
+			error_unlink_effect(prev);
+		}
+		error_ref(prev);
+		prev->effect = e;
 	}
 	/*
 	 * At once error can feature only one reason.
@@ -59,15 +76,6 @@ error_set_prev(struct error *e, struct error *prev)
 	}
 	/* Set new 'prev' node. */
 	e->cause = prev;
-	/*
-	 * Unlink new 'effect' node from its old list of 'cause'
-	 * errors. nil can be also passed as an argument.
-	 */
-	if (prev != NULL) {
-		error_ref(prev);
-		error_unlink_effect(prev);
-		prev->effect = e;
-	}
 	return 0;
 }
====================


Simplified error list patch:
====================
diff --git a/src/lib/core/diag.c b/src/lib/core/diag.c
index ed8beb58d..69a19d91f 100644
--- a/src/lib/core/diag.c
+++ b/src/lib/core/diag.c
@@ -37,37 +37,32 @@ struct error_factory *error_factory = NULL;
 int
 error_set_prev(struct error *e, struct error *prev)
 {
-	/*
-	 * Make sure that adding error won't result in cycles.
-	 * Don't bother with sophisticated cycle-detection
-	 * algorithms, simple iteration is OK since as a rule
-	 * list contains a dozen errors at maximum.
-	 */
-	struct error *tmp = prev;
-	while (tmp != NULL) {
-		if (tmp == e)
+	if (e == prev)
+		return -1;
+	if (prev != NULL) {
+		/*
+		 * It is not allowed to change middle of the error
+		 * stack. Except when the tail is cut. Not
+		 * replaced. Reason is to make the code simpler,
+		 * and avoid any necessity to detect cycles. In
+		 * that implementation cycles are not allowed, and
+		 * this guarantee costs nothing.
+		 */
+		if (prev->has_effect || e->has_effect)
 			return -1;
-		tmp = tmp->cause;
+		prev->has_effect = true;
+		error_ref(prev);
 	}
 	/*
 	 * At once error can feature only one reason.
 	 * So unlink previous 'cause' node.
 	 */
 	if (e->cause != NULL) {
-		e->cause->effect = NULL;
+		e->cause->has_effect = false;
 		error_unref(e->cause);
 	}
 	/* Set new 'prev' node. */
 	e->cause = prev;
-	/*
-	 * Unlink new 'effect' node from its old list of 'cause'
-	 * errors. nil can be also passed as an argument.
-	 */
-	if (prev != NULL) {
-		error_ref(prev);
-		error_unlink_effect(prev);
-		prev->effect = e;
-	}
 	return 0;
 }
 
@@ -91,7 +86,7 @@ error_create(struct error *e,
 	}
 	e->errmsg[0] = '\0';
 	e->cause = NULL;
-	e->effect = NULL;
+	e->has_effect = false;
 }
 
 struct diag *
diff --git a/src/lib/core/diag.h b/src/lib/core/diag.h
index 3a817a659..52987befa 100644
--- a/src/lib/core/diag.h
+++ b/src/lib/core/diag.h
@@ -85,22 +85,20 @@ struct error {
 	/* Error description. */
 	char errmsg[DIAG_ERRMSG_MAX];
 	/**
-	 * Link to the cause and effect of given error. The cause
-	 * creates the effect:
+	 * Cause of the given error..
 	 * e1 = box.error.new({code = 0, reason = 'e1'})
 	 * e2 = box.error.new({code = 0, reason = 'e2'})
-	 * e1:set_prev(e2) -- Now e2 is the cause of e1 and e1 is
-	 * the effect of e2.
-	 * Only cause keeps reference to avoid cyclic dependence.
-	 * RLIST implementation is not really suitable here
-	 * since it is organized as circular list. In such
-	 * a case it is impossible to start an iteration
-	 * from any node and finish at the logical end of the
-	 * list. Double-linked list is required to allow deletion
-	 * from the middle of the list.
+	 * e1:set_prev(e2) -- Now e2 is the cause of e1.
 	 */
 	struct error *cause;
-	struct error *effect;
+	/**
+	 * Flag whether this error is not top of the error stack.
+	 * I.e. it has an 'effect'. Effect is e1 in the example
+	 * above. The flag is used to prevent changing middle of
+	 * the stack (except when it is cut off - this easy to
+	 * support, and can't create a cycle).
+	 */
+	bool has_effect;
 };
 
 static inline void
@@ -115,51 +113,24 @@ error_unref(struct error *e)
 	assert(e->refs > 0);
 	struct error *to_delete = e;
 	while (--to_delete->refs == 0) {
-		/* Unlink error from lists completely.*/
 		struct error *cause = to_delete->cause;
-		if (to_delete->effect != NULL)
-			to_delete->effect->cause = to_delete->cause;
-		if (to_delete->cause != NULL)
-			to_delete->cause->effect = to_delete->effect;
 		to_delete->cause = NULL;
-		to_delete->effect = NULL;
 		to_delete->destroy(to_delete);
 		if (cause == NULL)
 			return;
+		cause->has_effect = false;
 		to_delete = cause;
 	}
 }
 
 /**
- * Unlink error from its effect. For instance:
- * e1 -> e2 -> e3 -> e4 (e1:set_prev(e2); e2:set_prev(e3) ...)
- * unlink(e3): e1 -> e2 -> NULL; e3 -> e4 -> NULL
- */
-static inline void
-error_unlink_effect(struct error *e)
-{
-	if (e->effect != NULL) {
-		assert(e->refs > 1);
-		error_unref(e);
-		e->effect->cause = NULL;
-	}
-	e->effect = NULL;
-}
-
-/**
- * Set previous error: cut @a prev from its previous 'tail' of
- * causes and link to the one @a e belongs to. Note that all
- * previous errors starting from @a prev->cause are transferred
- * with it as well (i.e. causes for given error are not erased).
- * For instance:
- * e1 -> e2 -> NULL; e3 -> e4 -> NULL;
- * e2:set_effect(e3): e1 -> e2 -> e3 -> e4 -> NULL
- *
- * @a effect can be NULL. To be used as ffi method in
- * lua/error.lua.
+ * Set previous error. It can be NULL to make @a not having any
+ * previous error. In case @a prev is not NULL, it should not be
+ * already belong to another stack, and @a e should be top of the
+ * stack.
  *
- * @retval -1 in case adding @a effect results in list cycles;
- *          0 otherwise.
+ * @retval -1 In case adding @a prev is not possible.
+ * @retval 0 Success.
  */
 int
 error_set_prev(struct error *e, struct error *prev);
@@ -241,7 +212,6 @@ diag_set_error(struct diag *diag, struct error *e)
 	assert(e != NULL);
 	error_ref(e);
 	diag_clear(diag);
-	error_unlink_effect(e);
 	diag->last = e;
 }
 
@@ -255,11 +225,11 @@ static inline void
 diag_add_error(struct diag *diag, struct error *e)
 {
 	assert(e != NULL);
+	assert(e->cause == NULL);
 	error_ref(e);
-	error_unlink_effect(e);
 	e->cause = diag->last;
 	if (diag->last != NULL)
-		diag->last->effect = e;
+		diag->last->has_effect = true;
 	diag->last = e;
 }
 
diff --git a/src/lua/error.lua b/src/lua/error.lua
index bdc9c714d..3d0b75689 100644
--- a/src/lua/error.lua
+++ b/src/lua/error.lua
@@ -25,7 +25,7 @@ struct error {
     /* Error description. */
     char _errmsg[DIAG_ERRMSG_MAX];
     struct error *_cause;
-    struct error *_effect;
+    bool has_effect;
 };
 
 char *
diff --git a/test/box/error.result b/test/box/error.result
index 4f0f30491..300def647 100644
--- a/test/box/error.result
+++ b/test/box/error.result
@@ -502,6 +502,10 @@ box.error()
  | ---
  | ...
 
+space:drop()
+ | ---
+ | ...
+
 -- gh-1148: errors can be arranged into list (so called
 -- stacked diagnostics).
 --
@@ -540,14 +544,17 @@ assert(e2.prev == nil)
  | ...
 -- At this point stack is following: e1 -> e2
 -- Let's test following cases:
--- 1. e3 -> e2, e1 -> NULL (e3:set_prev(e2))
+-- 1. e3 -> e2, e1 -> NULL (e1:set_prev(nil), e3:set_prev(e2))
 -- 2. e1 -> e3, e2 -> NULL (e1:set_prev(e3))
 -- 3. e3 -> e1 -> e2 (e3:set_prev(e1))
--- 4. e1 -> e2 -> e3 (e2:set_prev(e3))
+-- 4. e1 -> e2 -> e3 (e1:set_prev(nil) e2:set_prev(e3) e1:set_prev(e2))
 --
 e3 = box.error.new({code = 111, reason = "another cause"})
  | ---
  | ...
+e1:set_prev(nil)
+ | ---
+ | ...
 e3:set_prev(e2)
  | ---
  | ...
@@ -566,6 +573,9 @@ assert(e1.prev == nil)
 
 -- Reset stack to e1 -> e2 and test case 2.
 --
+e3:set_prev(nil)
+ | ---
+ | ...
 e1:set_prev(e2)
  | ---
  | ...
@@ -628,19 +638,19 @@ assert(e3.prev == e1)
 
 -- Unlink errors and test case 4.
 --
-e1:set_prev(nil)
+e3:set_prev(nil)
  | ---
  | ...
 e2:set_prev(nil)
  | ---
  | ...
-e3:set_prev(nil)
+e1:set_prev(nil)
  | ---
  | ...
-e1:set_prev(e2)
+e2:set_prev(e3)
  | ---
  | ...
-e2:set_prev(e3)
+e1:set_prev(e2)
  | ---
  | ...
 assert(e1.prev == e2)
@@ -676,72 +686,6 @@ assert(e3.prev == nil)
  | - true
  | ...
 
--- Test splitting list into two ones.
--- After that we will get two lists: e1->e2->e5 and e3->e4
---
-e4 = box.error.new({code = 111, reason = "yet another cause"})
- | ---
- | ...
-e5 = box.error.new({code = 111, reason = "and another one"})
- | ---
- | ...
-e3:set_prev(e4)
- | ---
- | ...
-e2:set_prev(e5)
- | ---
- | ...
-assert(e1.prev == e2)
- | ---
- | - true
- | ...
-assert(e2.prev == e5)
- | ---
- | - true
- | ...
-assert(e3.prev == e4)
- | ---
- | - true
- | ...
-assert(e5.prev == nil)
- | ---
- | - true
- | ...
-assert(e4.prev == nil)
- | ---
- | - true
- | ...
-
--- Another splitting option: e1->e2 and e5->e3->e4
--- But firstly restore to one single list e1->e2->e3->e4
---
-e2:set_prev(e3)
- | ---
- | ...
-e5:set_prev(e3)
- | ---
- | ...
-assert(e1.prev == e2)
- | ---
- | - true
- | ...
-assert(e2.prev == nil)
- | ---
- | - true
- | ...
-assert(e5.prev == e3)
- | ---
- | - true
- | ...
-assert(e3.prev == e4)
- | ---
- | - true
- | ...
-assert(e4.prev == nil)
- | ---
- | - true
- | ...
-
 -- In case error is destroyed, it unrefs reference counter
 -- of its previous error. In turn, box.error.clear() refs/unrefs
 -- only head and doesn't touch other errors.
@@ -785,7 +729,3 @@ assert(e1.prev == e2)
  | ---
  | - true
  | ...
-
-space:drop()
- | ---
- | ...
diff --git a/test/box/error.test.lua b/test/box/error.test.lua
index 66a22db90..04e00168e 100644
--- a/test/box/error.test.lua
+++ b/test/box/error.test.lua
@@ -108,6 +108,8 @@ box.error.new(err)
 box.error.clear()
 box.error()
 
+space:drop()
+
 -- gh-1148: errors can be arranged into list (so called
 -- stacked diagnostics).
 --
@@ -122,12 +124,13 @@ e2:set_prev(e1)
 assert(e2.prev == nil)
 -- At this point stack is following: e1 -> e2
 -- Let's test following cases:
--- 1. e3 -> e2, e1 -> NULL (e3:set_prev(e2))
+-- 1. e3 -> e2, e1 -> NULL (e1:set_prev(nil), e3:set_prev(e2))
 -- 2. e1 -> e3, e2 -> NULL (e1:set_prev(e3))
 -- 3. e3 -> e1 -> e2 (e3:set_prev(e1))
--- 4. e1 -> e2 -> e3 (e2:set_prev(e3))
+-- 4. e1 -> e2 -> e3 (e1:set_prev(nil) e2:set_prev(e3) e1:set_prev(e2))
 --
 e3 = box.error.new({code = 111, reason = "another cause"})
+e1:set_prev(nil)
 e3:set_prev(e2)
 assert(e3.prev == e2)
 assert(e2.prev == nil)
@@ -135,6 +138,7 @@ assert(e1.prev == nil)
 
 -- Reset stack to e1 -> e2 and test case 2.
 --
+e3:set_prev(nil)
 e1:set_prev(e2)
 assert(e2.prev == nil)
 assert(e3.prev == nil)
@@ -156,11 +160,11 @@ assert(e3.prev == e1)
 
 -- Unlink errors and test case 4.
 --
-e1:set_prev(nil)
-e2:set_prev(nil)
 e3:set_prev(nil)
-e1:set_prev(e2)
+e2:set_prev(nil)
+e1:set_prev(nil)
 e2:set_prev(e3)
+e1:set_prev(e2)
 assert(e1.prev == e2)
 assert(e2.prev == e3)
 assert(e3.prev == nil)
@@ -173,30 +177,6 @@ assert(e3.prev == nil)
 e3:set_prev(e2)
 assert(e3.prev == nil)
 
--- Test splitting list into two ones.
--- After that we will get two lists: e1->e2->e5 and e3->e4
---
-e4 = box.error.new({code = 111, reason = "yet another cause"})
-e5 = box.error.new({code = 111, reason = "and another one"})
-e3:set_prev(e4)
-e2:set_prev(e5)
-assert(e1.prev == e2)
-assert(e2.prev == e5)
-assert(e3.prev == e4)
-assert(e5.prev == nil)
-assert(e4.prev == nil)
-
--- Another splitting option: e1->e2 and e5->e3->e4
--- But firstly restore to one single list e1->e2->e3->e4
---
-e2:set_prev(e3)
-e5:set_prev(e3)
-assert(e1.prev == e2)
-assert(e2.prev == nil)
-assert(e5.prev == e3)
-assert(e3.prev == e4)
-assert(e4.prev == nil)
-
 -- In case error is destroyed, it unrefs reference counter
 -- of its previous error. In turn, box.error.clear() refs/unrefs
 -- only head and doesn't touch other errors.
@@ -212,5 +192,3 @@ assert(e2.code == 111)
 box.error.set(e1)
 box.error.clear()
 assert(e1.prev == e2)
-
-space:drop()

[-- Attachment #2: opt-cycle.txt --]
[-- Type: text/plain, Size: 749 bytes --]

diff --git a/src/lib/core/diag.h b/src/lib/core/diag.h
index 3a817a659..276bf2362 100644
--- a/src/lib/core/diag.h
+++ b/src/lib/core/diag.h
@@ -117,12 +117,11 @@ error_unref(struct error *e)
 	while (--to_delete->refs == 0) {
 		/* Unlink error from lists completely.*/
 		struct error *cause = to_delete->cause;
-		if (to_delete->effect != NULL)
-			to_delete->effect->cause = to_delete->cause;
-		if (to_delete->cause != NULL)
-			to_delete->cause->effect = to_delete->effect;
-		to_delete->cause = NULL;
-		to_delete->effect = NULL;
+		assert(to_delete->effect == NULL);
+		if (to_delete->cause != NULL) {
+			to_delete->cause->effect = NULL;
+			to_delete->cause = NULL;
+		}
 		to_delete->destroy(to_delete);
 		if (cause == NULL)
 			return;

[-- Attachment #3: unidirect.txt --]
[-- Type: text/plain, Size: 11079 bytes --]

commit eaada025e478aeb66d0b8f8f31c91045b0f0089f
Author: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Date:   Thu Apr 2 00:52:54 2020 +0200

    Diag stack simplification

diff --git a/src/lib/core/diag.c b/src/lib/core/diag.c
index ed8beb58d..69a19d91f 100644
--- a/src/lib/core/diag.c
+++ b/src/lib/core/diag.c
@@ -37,37 +37,32 @@ struct error_factory *error_factory = NULL;
 int
 error_set_prev(struct error *e, struct error *prev)
 {
-	/*
-	 * Make sure that adding error won't result in cycles.
-	 * Don't bother with sophisticated cycle-detection
-	 * algorithms, simple iteration is OK since as a rule
-	 * list contains a dozen errors at maximum.
-	 */
-	struct error *tmp = prev;
-	while (tmp != NULL) {
-		if (tmp == e)
+	if (e == prev)
+		return -1;
+	if (prev != NULL) {
+		/*
+		 * It is not allowed to change middle of the error
+		 * stack. Except when the tail is cut. Not
+		 * replaced. Reason is to make the code simpler,
+		 * and avoid any necessity to detect cycles. In
+		 * that implementation cycles are not allowed, and
+		 * this guarantee costs nothing.
+		 */
+		if (prev->has_effect || e->has_effect)
 			return -1;
-		tmp = tmp->cause;
+		prev->has_effect = true;
+		error_ref(prev);
 	}
 	/*
 	 * At once error can feature only one reason.
 	 * So unlink previous 'cause' node.
 	 */
 	if (e->cause != NULL) {
-		e->cause->effect = NULL;
+		e->cause->has_effect = false;
 		error_unref(e->cause);
 	}
 	/* Set new 'prev' node. */
 	e->cause = prev;
-	/*
-	 * Unlink new 'effect' node from its old list of 'cause'
-	 * errors. nil can be also passed as an argument.
-	 */
-	if (prev != NULL) {
-		error_ref(prev);
-		error_unlink_effect(prev);
-		prev->effect = e;
-	}
 	return 0;
 }
 
@@ -91,7 +86,7 @@ error_create(struct error *e,
 	}
 	e->errmsg[0] = '\0';
 	e->cause = NULL;
-	e->effect = NULL;
+	e->has_effect = false;
 }
 
 struct diag *
diff --git a/src/lib/core/diag.h b/src/lib/core/diag.h
index 3a817a659..52987befa 100644
--- a/src/lib/core/diag.h
+++ b/src/lib/core/diag.h
@@ -85,22 +85,20 @@ struct error {
 	/* Error description. */
 	char errmsg[DIAG_ERRMSG_MAX];
 	/**
-	 * Link to the cause and effect of given error. The cause
-	 * creates the effect:
+	 * Cause of the given error..
 	 * e1 = box.error.new({code = 0, reason = 'e1'})
 	 * e2 = box.error.new({code = 0, reason = 'e2'})
-	 * e1:set_prev(e2) -- Now e2 is the cause of e1 and e1 is
-	 * the effect of e2.
-	 * Only cause keeps reference to avoid cyclic dependence.
-	 * RLIST implementation is not really suitable here
-	 * since it is organized as circular list. In such
-	 * a case it is impossible to start an iteration
-	 * from any node and finish at the logical end of the
-	 * list. Double-linked list is required to allow deletion
-	 * from the middle of the list.
+	 * e1:set_prev(e2) -- Now e2 is the cause of e1.
 	 */
 	struct error *cause;
-	struct error *effect;
+	/**
+	 * Flag whether this error is not top of the error stack.
+	 * I.e. it has an 'effect'. Effect is e1 in the example
+	 * above. The flag is used to prevent changing middle of
+	 * the stack (except when it is cut off - this easy to
+	 * support, and can't create a cycle).
+	 */
+	bool has_effect;
 };
 
 static inline void
@@ -115,51 +113,24 @@ error_unref(struct error *e)
 	assert(e->refs > 0);
 	struct error *to_delete = e;
 	while (--to_delete->refs == 0) {
-		/* Unlink error from lists completely.*/
 		struct error *cause = to_delete->cause;
-		if (to_delete->effect != NULL)
-			to_delete->effect->cause = to_delete->cause;
-		if (to_delete->cause != NULL)
-			to_delete->cause->effect = to_delete->effect;
 		to_delete->cause = NULL;
-		to_delete->effect = NULL;
 		to_delete->destroy(to_delete);
 		if (cause == NULL)
 			return;
+		cause->has_effect = false;
 		to_delete = cause;
 	}
 }
 
 /**
- * Unlink error from its effect. For instance:
- * e1 -> e2 -> e3 -> e4 (e1:set_prev(e2); e2:set_prev(e3) ...)
- * unlink(e3): e1 -> e2 -> NULL; e3 -> e4 -> NULL
- */
-static inline void
-error_unlink_effect(struct error *e)
-{
-	if (e->effect != NULL) {
-		assert(e->refs > 1);
-		error_unref(e);
-		e->effect->cause = NULL;
-	}
-	e->effect = NULL;
-}
-
-/**
- * Set previous error: cut @a prev from its previous 'tail' of
- * causes and link to the one @a e belongs to. Note that all
- * previous errors starting from @a prev->cause are transferred
- * with it as well (i.e. causes for given error are not erased).
- * For instance:
- * e1 -> e2 -> NULL; e3 -> e4 -> NULL;
- * e2:set_effect(e3): e1 -> e2 -> e3 -> e4 -> NULL
- *
- * @a effect can be NULL. To be used as ffi method in
- * lua/error.lua.
+ * Set previous error. It can be NULL to make @a not having any
+ * previous error. In case @a prev is not NULL, it should not be
+ * already belong to another stack, and @a e should be top of the
+ * stack.
  *
- * @retval -1 in case adding @a effect results in list cycles;
- *          0 otherwise.
+ * @retval -1 In case adding @a prev is not possible.
+ * @retval 0 Success.
  */
 int
 error_set_prev(struct error *e, struct error *prev);
@@ -241,7 +212,6 @@ diag_set_error(struct diag *diag, struct error *e)
 	assert(e != NULL);
 	error_ref(e);
 	diag_clear(diag);
-	error_unlink_effect(e);
 	diag->last = e;
 }
 
@@ -255,11 +225,11 @@ static inline void
 diag_add_error(struct diag *diag, struct error *e)
 {
 	assert(e != NULL);
+	assert(e->cause == NULL);
 	error_ref(e);
-	error_unlink_effect(e);
 	e->cause = diag->last;
 	if (diag->last != NULL)
-		diag->last->effect = e;
+		diag->last->has_effect = true;
 	diag->last = e;
 }
 
diff --git a/src/lua/error.lua b/src/lua/error.lua
index bdc9c714d..3d0b75689 100644
--- a/src/lua/error.lua
+++ b/src/lua/error.lua
@@ -25,7 +25,7 @@ struct error {
     /* Error description. */
     char _errmsg[DIAG_ERRMSG_MAX];
     struct error *_cause;
-    struct error *_effect;
+    bool has_effect;
 };
 
 char *
diff --git a/test/box/error.result b/test/box/error.result
index 4f0f30491..300def647 100644
--- a/test/box/error.result
+++ b/test/box/error.result
@@ -502,6 +502,10 @@ box.error()
  | ---
  | ...
 
+space:drop()
+ | ---
+ | ...
+
 -- gh-1148: errors can be arranged into list (so called
 -- stacked diagnostics).
 --
@@ -540,14 +544,17 @@ assert(e2.prev == nil)
  | ...
 -- At this point stack is following: e1 -> e2
 -- Let's test following cases:
--- 1. e3 -> e2, e1 -> NULL (e3:set_prev(e2))
+-- 1. e3 -> e2, e1 -> NULL (e1:set_prev(nil), e3:set_prev(e2))
 -- 2. e1 -> e3, e2 -> NULL (e1:set_prev(e3))
 -- 3. e3 -> e1 -> e2 (e3:set_prev(e1))
--- 4. e1 -> e2 -> e3 (e2:set_prev(e3))
+-- 4. e1 -> e2 -> e3 (e1:set_prev(nil) e2:set_prev(e3) e1:set_prev(e2))
 --
 e3 = box.error.new({code = 111, reason = "another cause"})
  | ---
  | ...
+e1:set_prev(nil)
+ | ---
+ | ...
 e3:set_prev(e2)
  | ---
  | ...
@@ -566,6 +573,9 @@ assert(e1.prev == nil)
 
 -- Reset stack to e1 -> e2 and test case 2.
 --
+e3:set_prev(nil)
+ | ---
+ | ...
 e1:set_prev(e2)
  | ---
  | ...
@@ -628,19 +638,19 @@ assert(e3.prev == e1)
 
 -- Unlink errors and test case 4.
 --
-e1:set_prev(nil)
+e3:set_prev(nil)
  | ---
  | ...
 e2:set_prev(nil)
  | ---
  | ...
-e3:set_prev(nil)
+e1:set_prev(nil)
  | ---
  | ...
-e1:set_prev(e2)
+e2:set_prev(e3)
  | ---
  | ...
-e2:set_prev(e3)
+e1:set_prev(e2)
  | ---
  | ...
 assert(e1.prev == e2)
@@ -676,72 +686,6 @@ assert(e3.prev == nil)
  | - true
  | ...
 
--- Test splitting list into two ones.
--- After that we will get two lists: e1->e2->e5 and e3->e4
---
-e4 = box.error.new({code = 111, reason = "yet another cause"})
- | ---
- | ...
-e5 = box.error.new({code = 111, reason = "and another one"})
- | ---
- | ...
-e3:set_prev(e4)
- | ---
- | ...
-e2:set_prev(e5)
- | ---
- | ...
-assert(e1.prev == e2)
- | ---
- | - true
- | ...
-assert(e2.prev == e5)
- | ---
- | - true
- | ...
-assert(e3.prev == e4)
- | ---
- | - true
- | ...
-assert(e5.prev == nil)
- | ---
- | - true
- | ...
-assert(e4.prev == nil)
- | ---
- | - true
- | ...
-
--- Another splitting option: e1->e2 and e5->e3->e4
--- But firstly restore to one single list e1->e2->e3->e4
---
-e2:set_prev(e3)
- | ---
- | ...
-e5:set_prev(e3)
- | ---
- | ...
-assert(e1.prev == e2)
- | ---
- | - true
- | ...
-assert(e2.prev == nil)
- | ---
- | - true
- | ...
-assert(e5.prev == e3)
- | ---
- | - true
- | ...
-assert(e3.prev == e4)
- | ---
- | - true
- | ...
-assert(e4.prev == nil)
- | ---
- | - true
- | ...
-
 -- In case error is destroyed, it unrefs reference counter
 -- of its previous error. In turn, box.error.clear() refs/unrefs
 -- only head and doesn't touch other errors.
@@ -785,7 +729,3 @@ assert(e1.prev == e2)
  | ---
  | - true
  | ...
-
-space:drop()
- | ---
- | ...
diff --git a/test/box/error.test.lua b/test/box/error.test.lua
index 66a22db90..04e00168e 100644
--- a/test/box/error.test.lua
+++ b/test/box/error.test.lua
@@ -108,6 +108,8 @@ box.error.new(err)
 box.error.clear()
 box.error()
 
+space:drop()
+
 -- gh-1148: errors can be arranged into list (so called
 -- stacked diagnostics).
 --
@@ -122,12 +124,13 @@ e2:set_prev(e1)
 assert(e2.prev == nil)
 -- At this point stack is following: e1 -> e2
 -- Let's test following cases:
--- 1. e3 -> e2, e1 -> NULL (e3:set_prev(e2))
+-- 1. e3 -> e2, e1 -> NULL (e1:set_prev(nil), e3:set_prev(e2))
 -- 2. e1 -> e3, e2 -> NULL (e1:set_prev(e3))
 -- 3. e3 -> e1 -> e2 (e3:set_prev(e1))
--- 4. e1 -> e2 -> e3 (e2:set_prev(e3))
+-- 4. e1 -> e2 -> e3 (e1:set_prev(nil) e2:set_prev(e3) e1:set_prev(e2))
 --
 e3 = box.error.new({code = 111, reason = "another cause"})
+e1:set_prev(nil)
 e3:set_prev(e2)
 assert(e3.prev == e2)
 assert(e2.prev == nil)
@@ -135,6 +138,7 @@ assert(e1.prev == nil)
 
 -- Reset stack to e1 -> e2 and test case 2.
 --
+e3:set_prev(nil)
 e1:set_prev(e2)
 assert(e2.prev == nil)
 assert(e3.prev == nil)
@@ -156,11 +160,11 @@ assert(e3.prev == e1)
 
 -- Unlink errors and test case 4.
 --
-e1:set_prev(nil)
-e2:set_prev(nil)
 e3:set_prev(nil)
-e1:set_prev(e2)
+e2:set_prev(nil)
+e1:set_prev(nil)
 e2:set_prev(e3)
+e1:set_prev(e2)
 assert(e1.prev == e2)
 assert(e2.prev == e3)
 assert(e3.prev == nil)
@@ -173,30 +177,6 @@ assert(e3.prev == nil)
 e3:set_prev(e2)
 assert(e3.prev == nil)
 
--- Test splitting list into two ones.
--- After that we will get two lists: e1->e2->e5 and e3->e4
---
-e4 = box.error.new({code = 111, reason = "yet another cause"})
-e5 = box.error.new({code = 111, reason = "and another one"})
-e3:set_prev(e4)
-e2:set_prev(e5)
-assert(e1.prev == e2)
-assert(e2.prev == e5)
-assert(e3.prev == e4)
-assert(e5.prev == nil)
-assert(e4.prev == nil)
-
--- Another splitting option: e1->e2 and e5->e3->e4
--- But firstly restore to one single list e1->e2->e3->e4
---
-e2:set_prev(e3)
-e5:set_prev(e3)
-assert(e1.prev == e2)
-assert(e2.prev == nil)
-assert(e5.prev == e3)
-assert(e3.prev == e4)
-assert(e4.prev == nil)
-
 -- In case error is destroyed, it unrefs reference counter
 -- of its previous error. In turn, box.error.clear() refs/unrefs
 -- only head and doesn't touch other errors.
@@ -212,5 +192,3 @@ assert(e2.code == 111)
 box.error.set(e1)
 box.error.clear()
 assert(e1.prev == e2)
-
-space:drop()

Re: [Tarantool-patches] [PATCH v2 06/10] box: introduce stacked diagnostic area

[Nikita Pettik]

On 02 Apr 02:29, Vladislav Shpilevoy wrote:
> Hi! Thanks for the fixes!
> 
> See 2 comments below.
> 
> > static inline void
> > error_unref(struct error *e)
> > {
> > 	assert(e->refs > 0);
> > 	struct error *to_delete = e;
> > 	while (--to_delete->refs == 0) {
> > 		/* Unlink error from lists completely.*/
> > 		struct error *cause = to_delete->cause;
> > 		if (to_delete->effect != NULL)
> > 			to_delete->effect->cause = to_delete->cause;
> 
> 1. Looks like 'effect' should be always NULL here. Because
> if it is not NULL, then effect->cause == e, and it would keep a
> reference, making e->refs == 0 impossible. I applied the diff
> below and all the tests passed.

Agree, applied.
 
> ====================
> diff --git a/src/lib/core/diag.h b/src/lib/core/diag.h
> index 3a817a659..276bf2362 100644
> --- a/src/lib/core/diag.h
> +++ b/src/lib/core/diag.h
> @@ -117,12 +117,11 @@ error_unref(struct error *e)
>  	while (--to_delete->refs == 0) {
>  		/* Unlink error from lists completely.*/
>  		struct error *cause = to_delete->cause;
> -		if (to_delete->effect != NULL)
> -			to_delete->effect->cause = to_delete->cause;
> -		if (to_delete->cause != NULL)
> -			to_delete->cause->effect = to_delete->effect;
> -		to_delete->cause = NULL;
> -		to_delete->effect = NULL;
> +		assert(to_delete->effect == NULL);
> +		if (to_delete->cause != NULL) {
> +			to_delete->cause->effect = NULL;
> +			to_delete->cause = NULL;
> +		}
>  		to_delete->destroy(to_delete);
>  		if (cause == NULL)
>  			return;
> ====================
> > }>>> diff --git a/src/lib/core/diag.c b/src/lib/core/diag.c
> >>> index c350abb4a..57da5da44 100644
> >>> --- a/src/lib/core/diag.c
> >>> +++ b/src/lib/core/diag.c
> >>> @@ -34,6 +34,43 @@
> >>>  /* Must be set by the library user */
> >>>  struct error_factory *error_factory = NULL;
> >>>  
> >>> +int
> >>> +error_set_prev(struct error *e, struct error *prev)
> >>> +{
> >>> +	/*
> >>> +	 * Make sure that adding error won't result in cycles.
> >>> +	 * Don't bother with sophisticated cycle-detection
> >>> +	 * algorithms, simple iteration is OK since as a rule
> >>> +	 * list contains a dozen errors at maximum.
> >>> +	 */
> >>> +	struct error *tmp = prev;
> >>> +	while (tmp != NULL) {
> >>> +		if (tmp == e)
> >>> +			return -1;
> >>> +		tmp = tmp->cause;
> >>> +	}
> >>
> >> 3. Or we could remove 'prev' from its current stack gracefully,
> >> since the list is doubly linked. Then cycles become impossible.
> >> Is there a reason not to do that? I mean, we remove 'prev'
> >> from its old stack by changing its 'effect' already anyway.
> >>
> >> Can do the same for the other pointer, and no need to check
> >> for cycles anymore.
> >> I can only think of a problem when someone makes
> >>
> >>     e1 = box.error.new()
> >>     e2 = box.error.new()
> >>     e2:set_prev(e1)
> >>
> >> and then
> >>
> >>     e4 = box.error.new()
> >>     e4:set_prev(e1)
> >>
> >> assuming that e1 will be copied. But it will steal e1 from
> >> its old stack. Even with your current solution. Maybe to make
> >> it more protected against dummies we should forbid moving error
> >> from one stack to another? So an error can't change its 'cause'
> >> and 'effect' once they become not NULL. In that way we eliminate
> >> any uncertainties and in future can decide whether we want to copy
> >> in set_prev() when necessary, or move from previous stack. Or we
> >> will keep this forbidden because probably no one does moving
> >> between stacks, nor the example I showed above. This also
> >> significantly simplifies tests and reduces probability of having
> >> a bug there. What are your thoughts?
> > 
> > I'm okay with current approach. It is all about documentation - 
> > the better feature is documented, the easier it turns out to
> > be in terms of usage. There's always way to misuse any feature
> > however well designed. If you insist on some changes - let me know.
> 
> 2. I don't insist. Can only recommend. And I would strongly recommend
> to cut the ability to move errors between stacks. I see only pros in
> cutting that. Because it is never needed (I can't imagine a sane case),
> can be surprising for someone, and complicates the code. The server
> code. Makes it easier to leave a bug here. Another reason is that it is
> easier to add something later if someone will need it, than to forbid
> something after introduction.
> 
> Besides, it seems we could even make the error list unidirectional then.
> Instead of 'effect' pointer we could keep a flag saying whether this error
> belongs to a stack. No more complex handling of when to do ref/unref/cycles
> and so on. Even bigger simplification.
> 
> I decided to try that and got a patch. See it attached to the email as
> 'unidirect.txt', and below. I didn't push it on the branch so as not to
> change next commits.
> 
> On the summary, I made the list unidirectional, removed the cycle
> detection and forbidden any chance of creating it by banning change of
> error object when it is not on top of the stack. So a stack should be
> built always in one order - from the cause of all causes to top. The
> only thing you can do is to cut a tail of a stack. All more complex
> actions which may be needed in very rare cases still can be implemented
> by users using the provided basic API.
> 
> That made the patch 110 lines shorter.

Size of the patch does not always define its readability.
 
> Also a small comment about cycle detection in your patch. Here I
> would strongly-strongly recommend to avoid it when it is clearly
> not necessary.

To be honest, I don't understand why do you care about this opimization so
much. Linking errors into list is unlikely to be hot execution path, so nobody
should bother with such minor improvement here. Instead, keeping code clear,
simple and straightforward (your diff contains more branches and looks a bit
more complicated to me) must be the main goal. Applied diff below only due to
your recommendation.
 
> You can't create a cycle, if e->effect and prev->effect are NULL.
> Which is the case in 99% of cases. Even if prev->cause is not NULL,
> you still can't create a cycle when both effects are NULL. See diff
> below and attached as 'opt-cycle.txt'. That skips 'prev' iteration
> even if it is a whole another stack, but 'prev' is its top.
> 
> Optimized cycle detection:
> ====================
> diff --git a/src/lib/core/diag.c b/src/lib/core/diag.c
> index ed8beb58d..64ff5e4d6 100644
> --- a/src/lib/core/diag.c
> +++ b/src/lib/core/diag.c
> @@ -43,11 +43,28 @@ error_set_prev(struct error *e, struct error *prev)
>  	 * algorithms, simple iteration is OK since as a rule
>  	 * list contains a dozen errors at maximum.
>  	 */
> -	struct error *tmp = prev;
> -	while (tmp != NULL) {
> -		if (tmp == e)
> +	if (prev != NULL) {
> +		if (e == prev)
>  			return -1;
> -		tmp = tmp->cause;
> +		if (prev->effect != NULL || e->effect != NULL) {
> +			/*
> +			 * e and prev already compared, so start
> +			 * from prev->cause.
> +			 */
> +			struct error *tmp = prev->cause;
> +			while (tmp != NULL) {
> +				if (tmp == e)
> +					return -1;
> +				tmp = tmp->cause;
> +			}
> +			/*
> +			 * Unlink new 'effect' node from its old
> +			 * list of 'cause' errors.
> +			 */
> +			error_unlink_effect(prev);
> +		}
> +		error_ref(prev);
> +		prev->effect = e;
>  	}
>  	/*
>  	 * At once error can feature only one reason.
> 
> 
> Simplified error list patch:

Still, I haven't changed my mind. I didn't apply diff below, sorry.
I'll leave it to the second reviewer.

> ====================
> diff --git a/src/lib/core/diag.c b/src/lib/core/diag.c
> index ed8beb58d..69a19d91f 100644
> --- a/src/lib/core/diag.c
> +++ b/src/lib/core/diag.c
> @@ -37,37 +37,32 @@ struct error_factory *error_factory = NULL;
>  int
>  error_set_prev(struct error *e, struct error *prev)
>  {
> -	/*
> -	 * Make sure that adding error won't result in cycles.
> -	 * Don't bother with sophisticated cycle-detection
> -	 * algorithms, simple iteration is OK since as a rule
> -	 * list contains a dozen errors at maximum.
> -	 */
> -	struct error *tmp = prev;
> -	while (tmp != NULL) {
> -		if (tmp == e)
> +	if (e == prev)
> +		return -1;
> +	if (prev != NULL) {
> +		/*
> +		 * It is not allowed to change middle of the error
> +		 * stack. Except when the tail is cut. Not
> +		 * replaced. Reason is to make the code simpler,
> +		 * and avoid any necessity to detect cycles. In
> +		 * that implementation cycles are not allowed, and
> +		 * this guarantee costs nothing.
> +		 */
> +		if (prev->has_effect || e->has_effect)
>  			return -1;
> -		tmp = tmp->cause;
> +		prev->has_effect = true;
> +		error_ref(prev);
>  	}
>  	/*
>  	 * At once error can feature only one reason.
>  	 * So unlink previous 'cause' node.
>  	 */
>  	if (e->cause != NULL) {
> -		e->cause->effect = NULL;
> +		e->cause->has_effect = false;
>  		error_unref(e->cause);
>  	}
>  	/* Set new 'prev' node. */
>  	e->cause = prev;
> -	/*
> -	 * Unlink new 'effect' node from its old list of 'cause'
> -	 * errors. nil can be also passed as an argument.
> -	 */
> -	if (prev != NULL) {
> -		error_ref(prev);
> -		error_unlink_effect(prev);
> -		prev->effect = e;
> -	}
>  	return 0;
>  }
>  
> @@ -91,7 +86,7 @@ error_create(struct error *e,
>  	}
>  	e->errmsg[0] = '\0';
>  	e->cause = NULL;
> -	e->effect = NULL;
> +	e->has_effect = false;
>  }
>  
>  struct diag *
> diff --git a/src/lib/core/diag.h b/src/lib/core/diag.h
> index 3a817a659..52987befa 100644
> --- a/src/lib/core/diag.h
> +++ b/src/lib/core/diag.h
> @@ -85,22 +85,20 @@ struct error {
>  	/* Error description. */
>  	char errmsg[DIAG_ERRMSG_MAX];
>  	/**
> -	 * Link to the cause and effect of given error. The cause
> -	 * creates the effect:
> +	 * Cause of the given error..
>  	 * e1 = box.error.new({code = 0, reason = 'e1'})
>  	 * e2 = box.error.new({code = 0, reason = 'e2'})
> -	 * e1:set_prev(e2) -- Now e2 is the cause of e1 and e1 is
> -	 * the effect of e2.
> -	 * Only cause keeps reference to avoid cyclic dependence.
> -	 * RLIST implementation is not really suitable here
> -	 * since it is organized as circular list. In such
> -	 * a case it is impossible to start an iteration
> -	 * from any node and finish at the logical end of the
> -	 * list. Double-linked list is required to allow deletion
> -	 * from the middle of the list.
> +	 * e1:set_prev(e2) -- Now e2 is the cause of e1.
>  	 */
>  	struct error *cause;
> -	struct error *effect;
> +	/**
> +	 * Flag whether this error is not top of the error stack.
> +	 * I.e. it has an 'effect'. Effect is e1 in the example
> +	 * above. The flag is used to prevent changing middle of
> +	 * the stack (except when it is cut off - this easy to
> +	 * support, and can't create a cycle).
> +	 */
> +	bool has_effect;
>  };
>  
>  static inline void
> @@ -115,51 +113,24 @@ error_unref(struct error *e)
>  	assert(e->refs > 0);
>  	struct error *to_delete = e;
>  	while (--to_delete->refs == 0) {
> -		/* Unlink error from lists completely.*/
>  		struct error *cause = to_delete->cause;
> -		if (to_delete->effect != NULL)
> -			to_delete->effect->cause = to_delete->cause;
> -		if (to_delete->cause != NULL)
> -			to_delete->cause->effect = to_delete->effect;
>  		to_delete->cause = NULL;
> -		to_delete->effect = NULL;
>  		to_delete->destroy(to_delete);
>  		if (cause == NULL)
>  			return;
> +		cause->has_effect = false;
>  		to_delete = cause;
>  	}
>  }
>  
>  /**
> - * Unlink error from its effect. For instance:
> - * e1 -> e2 -> e3 -> e4 (e1:set_prev(e2); e2:set_prev(e3) ...)
> - * unlink(e3): e1 -> e2 -> NULL; e3 -> e4 -> NULL
> - */
> -static inline void
> -error_unlink_effect(struct error *e)
> -{
> -	if (e->effect != NULL) {
> -		assert(e->refs > 1);
> -		error_unref(e);
> -		e->effect->cause = NULL;
> -	}
> -	e->effect = NULL;
> -}
> -
> -/**
> - * Set previous error: cut @a prev from its previous 'tail' of
> - * causes and link to the one @a e belongs to. Note that all
> - * previous errors starting from @a prev->cause are transferred
> - * with it as well (i.e. causes for given error are not erased).
> - * For instance:
> - * e1 -> e2 -> NULL; e3 -> e4 -> NULL;
> - * e2:set_effect(e3): e1 -> e2 -> e3 -> e4 -> NULL
> - *
> - * @a effect can be NULL. To be used as ffi method in
> - * lua/error.lua.
> + * Set previous error. It can be NULL to make @a not having any
> + * previous error. In case @a prev is not NULL, it should not be
> + * already belong to another stack, and @a e should be top of the
> + * stack.
>   *
> - * @retval -1 in case adding @a effect results in list cycles;
> - *          0 otherwise.
> + * @retval -1 In case adding @a prev is not possible.
> + * @retval 0 Success.
>   */
>  int
>  error_set_prev(struct error *e, struct error *prev);
> @@ -241,7 +212,6 @@ diag_set_error(struct diag *diag, struct error *e)
>  	assert(e != NULL);
>  	error_ref(e);
>  	diag_clear(diag);
> -	error_unlink_effect(e);
>  	diag->last = e;
>  }
>  
> @@ -255,11 +225,11 @@ static inline void
>  diag_add_error(struct diag *diag, struct error *e)
>  {
>  	assert(e != NULL);
> +	assert(e->cause == NULL);
>  	error_ref(e);
> -	error_unlink_effect(e);
>  	e->cause = diag->last;
>  	if (diag->last != NULL)
> -		diag->last->effect = e;
> +		diag->last->has_effect = true;
>  	diag->last = e;
>  }
>  
> diff --git a/src/lua/error.lua b/src/lua/error.lua
> index bdc9c714d..3d0b75689 100644
> --- a/src/lua/error.lua
> +++ b/src/lua/error.lua
> @@ -25,7 +25,7 @@ struct error {
>      /* Error description. */
>      char _errmsg[DIAG_ERRMSG_MAX];
>      struct error *_cause;
> -    struct error *_effect;
> +    bool has_effect;
>  };
>  
>  char *
> diff --git a/test/box/error.result b/test/box/error.result
> index 4f0f30491..300def647 100644
> --- a/test/box/error.result
> +++ b/test/box/error.result
> @@ -502,6 +502,10 @@ box.error()
>   | ---
>   | ...
>  
> +space:drop()
> + | ---
> + | ...
> +
>  -- gh-1148: errors can be arranged into list (so called
>  -- stacked diagnostics).
>  --
> @@ -540,14 +544,17 @@ assert(e2.prev == nil)
>   | ...
>  -- At this point stack is following: e1 -> e2
>  -- Let's test following cases:
> --- 1. e3 -> e2, e1 -> NULL (e3:set_prev(e2))
> +-- 1. e3 -> e2, e1 -> NULL (e1:set_prev(nil), e3:set_prev(e2))
>  -- 2. e1 -> e3, e2 -> NULL (e1:set_prev(e3))
>  -- 3. e3 -> e1 -> e2 (e3:set_prev(e1))
> --- 4. e1 -> e2 -> e3 (e2:set_prev(e3))
> +-- 4. e1 -> e2 -> e3 (e1:set_prev(nil) e2:set_prev(e3) e1:set_prev(e2))
>  --
>  e3 = box.error.new({code = 111, reason = "another cause"})
>   | ---
>   | ...
> +e1:set_prev(nil)
> + | ---
> + | ...
>  e3:set_prev(e2)
>   | ---
>   | ...
> @@ -566,6 +573,9 @@ assert(e1.prev == nil)
>  
>  -- Reset stack to e1 -> e2 and test case 2.
>  --
> +e3:set_prev(nil)
> + | ---
> + | ...
>  e1:set_prev(e2)
>   | ---
>   | ...
> @@ -628,19 +638,19 @@ assert(e3.prev == e1)
>  
>  -- Unlink errors and test case 4.
>  --
> -e1:set_prev(nil)
> +e3:set_prev(nil)
>   | ---
>   | ...
>  e2:set_prev(nil)
>   | ---
>   | ...
> -e3:set_prev(nil)
> +e1:set_prev(nil)
>   | ---
>   | ...
> -e1:set_prev(e2)
> +e2:set_prev(e3)
>   | ---
>   | ...
> -e2:set_prev(e3)
> +e1:set_prev(e2)
>   | ---
>   | ...
>  assert(e1.prev == e2)
> @@ -676,72 +686,6 @@ assert(e3.prev == nil)
>   | - true
>   | ...
>  
> --- Test splitting list into two ones.
> --- After that we will get two lists: e1->e2->e5 and e3->e4
> ---
> -e4 = box.error.new({code = 111, reason = "yet another cause"})
> - | ---
> - | ...
> -e5 = box.error.new({code = 111, reason = "and another one"})
> - | ---
> - | ...
> -e3:set_prev(e4)
> - | ---
> - | ...
> -e2:set_prev(e5)
> - | ---
> - | ...
> -assert(e1.prev == e2)
> - | ---
> - | - true
> - | ...
> -assert(e2.prev == e5)
> - | ---
> - | - true
> - | ...
> -assert(e3.prev == e4)
> - | ---
> - | - true
> - | ...
> -assert(e5.prev == nil)
> - | ---
> - | - true
> - | ...
> -assert(e4.prev == nil)
> - | ---
> - | - true
> - | ...
> -
> --- Another splitting option: e1->e2 and e5->e3->e4
> --- But firstly restore to one single list e1->e2->e3->e4
> ---
> -e2:set_prev(e3)
> - | ---
> - | ...
> -e5:set_prev(e3)
> - | ---
> - | ...
> -assert(e1.prev == e2)
> - | ---
> - | - true
> - | ...
> -assert(e2.prev == nil)
> - | ---
> - | - true
> - | ...
> -assert(e5.prev == e3)
> - | ---
> - | - true
> - | ...
> -assert(e3.prev == e4)
> - | ---
> - | - true
> - | ...
> -assert(e4.prev == nil)
> - | ---
> - | - true
> - | ...
> -
>  -- In case error is destroyed, it unrefs reference counter
>  -- of its previous error. In turn, box.error.clear() refs/unrefs
>  -- only head and doesn't touch other errors.
> @@ -785,7 +729,3 @@ assert(e1.prev == e2)
>   | ---
>   | - true
>   | ...
> -
> -space:drop()
> - | ---
> - | ...
> diff --git a/test/box/error.test.lua b/test/box/error.test.lua
> index 66a22db90..04e00168e 100644
> --- a/test/box/error.test.lua
> +++ b/test/box/error.test.lua
> @@ -108,6 +108,8 @@ box.error.new(err)
>  box.error.clear()
>  box.error()
>  
> +space:drop()
> +
>  -- gh-1148: errors can be arranged into list (so called
>  -- stacked diagnostics).
>  --
> @@ -122,12 +124,13 @@ e2:set_prev(e1)
>  assert(e2.prev == nil)
>  -- At this point stack is following: e1 -> e2
>  -- Let's test following cases:
> --- 1. e3 -> e2, e1 -> NULL (e3:set_prev(e2))
> +-- 1. e3 -> e2, e1 -> NULL (e1:set_prev(nil), e3:set_prev(e2))
>  -- 2. e1 -> e3, e2 -> NULL (e1:set_prev(e3))
>  -- 3. e3 -> e1 -> e2 (e3:set_prev(e1))
> --- 4. e1 -> e2 -> e3 (e2:set_prev(e3))
> +-- 4. e1 -> e2 -> e3 (e1:set_prev(nil) e2:set_prev(e3) e1:set_prev(e2))
>  --
>  e3 = box.error.new({code = 111, reason = "another cause"})
> +e1:set_prev(nil)
>  e3:set_prev(e2)
>  assert(e3.prev == e2)
>  assert(e2.prev == nil)
> @@ -135,6 +138,7 @@ assert(e1.prev == nil)
>  
>  -- Reset stack to e1 -> e2 and test case 2.
>  --
> +e3:set_prev(nil)
>  e1:set_prev(e2)
>  assert(e2.prev == nil)
>  assert(e3.prev == nil)
> @@ -156,11 +160,11 @@ assert(e3.prev == e1)
>  
>  -- Unlink errors and test case 4.
>  --
> -e1:set_prev(nil)
> -e2:set_prev(nil)
>  e3:set_prev(nil)
> -e1:set_prev(e2)
> +e2:set_prev(nil)
> +e1:set_prev(nil)
>  e2:set_prev(e3)
> +e1:set_prev(e2)
>  assert(e1.prev == e2)
>  assert(e2.prev == e3)
>  assert(e3.prev == nil)
> @@ -173,30 +177,6 @@ assert(e3.prev == nil)
>  e3:set_prev(e2)
>  assert(e3.prev == nil)
>  
> --- Test splitting list into two ones.
> --- After that we will get two lists: e1->e2->e5 and e3->e4
> ---
> -e4 = box.error.new({code = 111, reason = "yet another cause"})
> -e5 = box.error.new({code = 111, reason = "and another one"})
> -e3:set_prev(e4)
> -e2:set_prev(e5)
> -assert(e1.prev == e2)
> -assert(e2.prev == e5)
> -assert(e3.prev == e4)
> -assert(e5.prev == nil)
> -assert(e4.prev == nil)
> -
> --- Another splitting option: e1->e2 and e5->e3->e4
> --- But firstly restore to one single list e1->e2->e3->e4
> ---
> -e2:set_prev(e3)
> -e5:set_prev(e3)
> -assert(e1.prev == e2)
> -assert(e2.prev == nil)
> -assert(e5.prev == e3)
> -assert(e3.prev == e4)
> -assert(e4.prev == nil)
> -
>  -- In case error is destroyed, it unrefs reference counter
>  -- of its previous error. In turn, box.error.clear() refs/unrefs
>  -- only head and doesn't touch other errors.
> @@ -212,5 +192,3 @@ assert(e2.code == 111)
>  box.error.set(e1)
>  box.error.clear()
>  assert(e1.prev == e2)
> -
> -space:drop()

> diff --git a/src/lib/core/diag.h b/src/lib/core/diag.h
> index 3a817a659..276bf2362 100644
> --- a/src/lib/core/diag.h
> +++ b/src/lib/core/diag.h
> @@ -117,12 +117,11 @@ error_unref(struct error *e)
>  	while (--to_delete->refs == 0) {
>  		/* Unlink error from lists completely.*/
>  		struct error *cause = to_delete->cause;
> -		if (to_delete->effect != NULL)
> -			to_delete->effect->cause = to_delete->cause;
> -		if (to_delete->cause != NULL)
> -			to_delete->cause->effect = to_delete->effect;
> -		to_delete->cause = NULL;
> -		to_delete->effect = NULL;
> +		assert(to_delete->effect == NULL);
> +		if (to_delete->cause != NULL) {
> +			to_delete->cause->effect = NULL;
> +			to_delete->cause = NULL;
> +		}
>  		to_delete->destroy(to_delete);
>  		if (cause == NULL)
>  			return;

> commit eaada025e478aeb66d0b8f8f31c91045b0f0089f
> Author: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
> Date:   Thu Apr 2 00:52:54 2020 +0200
> 
>     Diag stack simplification
> 
> diff --git a/src/lib/core/diag.c b/src/lib/core/diag.c
> index ed8beb58d..69a19d91f 100644
> --- a/src/lib/core/diag.c
> +++ b/src/lib/core/diag.c
> @@ -37,37 +37,32 @@ struct error_factory *error_factory = NULL;
>  int
>  error_set_prev(struct error *e, struct error *prev)
>  {
> -	/*
> -	 * Make sure that adding error won't result in cycles.
> -	 * Don't bother with sophisticated cycle-detection
> -	 * algorithms, simple iteration is OK since as a rule
> -	 * list contains a dozen errors at maximum.
> -	 */
> -	struct error *tmp = prev;
> -	while (tmp != NULL) {
> -		if (tmp == e)
> +	if (e == prev)
> +		return -1;
> +	if (prev != NULL) {
> +		/*
> +		 * It is not allowed to change middle of the error
> +		 * stack. Except when the tail is cut. Not
> +		 * replaced. Reason is to make the code simpler,
> +		 * and avoid any necessity to detect cycles. In
> +		 * that implementation cycles are not allowed, and
> +		 * this guarantee costs nothing.
> +		 */
> +		if (prev->has_effect || e->has_effect)
>  			return -1;
> -		tmp = tmp->cause;
> +		prev->has_effect = true;
> +		error_ref(prev);
>  	}
>  	/*
>  	 * At once error can feature only one reason.
>  	 * So unlink previous 'cause' node.
>  	 */
>  	if (e->cause != NULL) {
> -		e->cause->effect = NULL;
> +		e->cause->has_effect = false;
>  		error_unref(e->cause);
>  	}
>  	/* Set new 'prev' node. */
>  	e->cause = prev;
> -	/*
> -	 * Unlink new 'effect' node from its old list of 'cause'
> -	 * errors. nil can be also passed as an argument.
> -	 */
> -	if (prev != NULL) {
> -		error_ref(prev);
> -		error_unlink_effect(prev);
> -		prev->effect = e;
> -	}
>  	return 0;
>  }
>  
> @@ -91,7 +86,7 @@ error_create(struct error *e,
>  	}
>  	e->errmsg[0] = '\0';
>  	e->cause = NULL;
> -	e->effect = NULL;
> +	e->has_effect = false;
>  }
>  
>  struct diag *
> diff --git a/src/lib/core/diag.h b/src/lib/core/diag.h
> index 3a817a659..52987befa 100644
> --- a/src/lib/core/diag.h
> +++ b/src/lib/core/diag.h
> @@ -85,22 +85,20 @@ struct error {
>  	/* Error description. */
>  	char errmsg[DIAG_ERRMSG_MAX];
>  	/**
> -	 * Link to the cause and effect of given error. The cause
> -	 * creates the effect:
> +	 * Cause of the given error..
>  	 * e1 = box.error.new({code = 0, reason = 'e1'})
>  	 * e2 = box.error.new({code = 0, reason = 'e2'})
> -	 * e1:set_prev(e2) -- Now e2 is the cause of e1 and e1 is
> -	 * the effect of e2.
> -	 * Only cause keeps reference to avoid cyclic dependence.
> -	 * RLIST implementation is not really suitable here
> -	 * since it is organized as circular list. In such
> -	 * a case it is impossible to start an iteration
> -	 * from any node and finish at the logical end of the
> -	 * list. Double-linked list is required to allow deletion
> -	 * from the middle of the list.
> +	 * e1:set_prev(e2) -- Now e2 is the cause of e1.
>  	 */
>  	struct error *cause;
> -	struct error *effect;
> +	/**
> +	 * Flag whether this error is not top of the error stack.
> +	 * I.e. it has an 'effect'. Effect is e1 in the example
> +	 * above. The flag is used to prevent changing middle of
> +	 * the stack (except when it is cut off - this easy to
> +	 * support, and can't create a cycle).
> +	 */
> +	bool has_effect;
>  };
>  
>  static inline void
> @@ -115,51 +113,24 @@ error_unref(struct error *e)
>  	assert(e->refs > 0);
>  	struct error *to_delete = e;
>  	while (--to_delete->refs == 0) {
> -		/* Unlink error from lists completely.*/
>  		struct error *cause = to_delete->cause;
> -		if (to_delete->effect != NULL)
> -			to_delete->effect->cause = to_delete->cause;
> -		if (to_delete->cause != NULL)
> -			to_delete->cause->effect = to_delete->effect;
>  		to_delete->cause = NULL;
> -		to_delete->effect = NULL;
>  		to_delete->destroy(to_delete);
>  		if (cause == NULL)
>  			return;
> +		cause->has_effect = false;
>  		to_delete = cause;
>  	}
>  }
>  
>  /**
> - * Unlink error from its effect. For instance:
> - * e1 -> e2 -> e3 -> e4 (e1:set_prev(e2); e2:set_prev(e3) ...)
> - * unlink(e3): e1 -> e2 -> NULL; e3 -> e4 -> NULL
> - */
> -static inline void
> -error_unlink_effect(struct error *e)
> -{
> -	if (e->effect != NULL) {
> -		assert(e->refs > 1);
> -		error_unref(e);
> -		e->effect->cause = NULL;
> -	}
> -	e->effect = NULL;
> -}
> -
> -/**
> - * Set previous error: cut @a prev from its previous 'tail' of
> - * causes and link to the one @a e belongs to. Note that all
> - * previous errors starting from @a prev->cause are transferred
> - * with it as well (i.e. causes for given error are not erased).
> - * For instance:
> - * e1 -> e2 -> NULL; e3 -> e4 -> NULL;
> - * e2:set_effect(e3): e1 -> e2 -> e3 -> e4 -> NULL
> - *
> - * @a effect can be NULL. To be used as ffi method in
> - * lua/error.lua.
> + * Set previous error. It can be NULL to make @a not having any
> + * previous error. In case @a prev is not NULL, it should not be
> + * already belong to another stack, and @a e should be top of the
> + * stack.
>   *
> - * @retval -1 in case adding @a effect results in list cycles;
> - *          0 otherwise.
> + * @retval -1 In case adding @a prev is not possible.
> + * @retval 0 Success.
>   */
>  int
>  error_set_prev(struct error *e, struct error *prev);
> @@ -241,7 +212,6 @@ diag_set_error(struct diag *diag, struct error *e)
>  	assert(e != NULL);
>  	error_ref(e);
>  	diag_clear(diag);
> -	error_unlink_effect(e);
>  	diag->last = e;
>  }
>  
> @@ -255,11 +225,11 @@ static inline void
>  diag_add_error(struct diag *diag, struct error *e)
>  {
>  	assert(e != NULL);
> +	assert(e->cause == NULL);
>  	error_ref(e);
> -	error_unlink_effect(e);
>  	e->cause = diag->last;
>  	if (diag->last != NULL)
> -		diag->last->effect = e;
> +		diag->last->has_effect = true;
>  	diag->last = e;
>  }
>  
> diff --git a/src/lua/error.lua b/src/lua/error.lua
> index bdc9c714d..3d0b75689 100644
> --- a/src/lua/error.lua
> +++ b/src/lua/error.lua
> @@ -25,7 +25,7 @@ struct error {
>      /* Error description. */
>      char _errmsg[DIAG_ERRMSG_MAX];
>      struct error *_cause;
> -    struct error *_effect;
> +    bool has_effect;
>  };
>  
>  char *
> diff --git a/test/box/error.result b/test/box/error.result
> index 4f0f30491..300def647 100644
> --- a/test/box/error.result
> +++ b/test/box/error.result
> @@ -502,6 +502,10 @@ box.error()
>   | ---
>   | ...
>  
> +space:drop()
> + | ---
> + | ...
> +
>  -- gh-1148: errors can be arranged into list (so called
>  -- stacked diagnostics).
>  --
> @@ -540,14 +544,17 @@ assert(e2.prev == nil)
>   | ...
>  -- At this point stack is following: e1 -> e2
>  -- Let's test following cases:
> --- 1. e3 -> e2, e1 -> NULL (e3:set_prev(e2))
> +-- 1. e3 -> e2, e1 -> NULL (e1:set_prev(nil), e3:set_prev(e2))
>  -- 2. e1 -> e3, e2 -> NULL (e1:set_prev(e3))
>  -- 3. e3 -> e1 -> e2 (e3:set_prev(e1))
> --- 4. e1 -> e2 -> e3 (e2:set_prev(e3))
> +-- 4. e1 -> e2 -> e3 (e1:set_prev(nil) e2:set_prev(e3) e1:set_prev(e2))
>  --
>  e3 = box.error.new({code = 111, reason = "another cause"})
>   | ---
>   | ...
> +e1:set_prev(nil)
> + | ---
> + | ...
>  e3:set_prev(e2)
>   | ---
>   | ...
> @@ -566,6 +573,9 @@ assert(e1.prev == nil)
>  
>  -- Reset stack to e1 -> e2 and test case 2.
>  --
> +e3:set_prev(nil)
> + | ---
> + | ...
>  e1:set_prev(e2)
>   | ---
>   | ...
> @@ -628,19 +638,19 @@ assert(e3.prev == e1)
>  
>  -- Unlink errors and test case 4.
>  --
> -e1:set_prev(nil)
> +e3:set_prev(nil)
>   | ---
>   | ...
>  e2:set_prev(nil)
>   | ---
>   | ...
> -e3:set_prev(nil)
> +e1:set_prev(nil)
>   | ---
>   | ...
> -e1:set_prev(e2)
> +e2:set_prev(e3)
>   | ---
>   | ...
> -e2:set_prev(e3)
> +e1:set_prev(e2)
>   | ---
>   | ...
>  assert(e1.prev == e2)
> @@ -676,72 +686,6 @@ assert(e3.prev == nil)
>   | - true
>   | ...
>  
> --- Test splitting list into two ones.
> --- After that we will get two lists: e1->e2->e5 and e3->e4
> ---
> -e4 = box.error.new({code = 111, reason = "yet another cause"})
> - | ---
> - | ...
> -e5 = box.error.new({code = 111, reason = "and another one"})
> - | ---
> - | ...
> -e3:set_prev(e4)
> - | ---
> - | ...
> -e2:set_prev(e5)
> - | ---
> - | ...
> -assert(e1.prev == e2)
> - | ---
> - | - true
> - | ...
> -assert(e2.prev == e5)
> - | ---
> - | - true
> - | ...
> -assert(e3.prev == e4)
> - | ---
> - | - true
> - | ...
> -assert(e5.prev == nil)
> - | ---
> - | - true
> - | ...
> -assert(e4.prev == nil)
> - | ---
> - | - true
> - | ...
> -
> --- Another splitting option: e1->e2 and e5->e3->e4
> --- But firstly restore to one single list e1->e2->e3->e4
> ---
> -e2:set_prev(e3)
> - | ---
> - | ...
> -e5:set_prev(e3)
> - | ---
> - | ...
> -assert(e1.prev == e2)
> - | ---
> - | - true
> - | ...
> -assert(e2.prev == nil)
> - | ---
> - | - true
> - | ...
> -assert(e5.prev == e3)
> - | ---
> - | - true
> - | ...
> -assert(e3.prev == e4)
> - | ---
> - | - true
> - | ...
> -assert(e4.prev == nil)
> - | ---
> - | - true
> - | ...
> -
>  -- In case error is destroyed, it unrefs reference counter
>  -- of its previous error. In turn, box.error.clear() refs/unrefs
>  -- only head and doesn't touch other errors.
> @@ -785,7 +729,3 @@ assert(e1.prev == e2)
>   | ---
>   | - true
>   | ...
> -
> -space:drop()
> - | ---
> - | ...
> diff --git a/test/box/error.test.lua b/test/box/error.test.lua
> index 66a22db90..04e00168e 100644
> --- a/test/box/error.test.lua
> +++ b/test/box/error.test.lua
> @@ -108,6 +108,8 @@ box.error.new(err)
>  box.error.clear()
>  box.error()
>  
> +space:drop()
> +
>  -- gh-1148: errors can be arranged into list (so called
>  -- stacked diagnostics).
>  --
> @@ -122,12 +124,13 @@ e2:set_prev(e1)
>  assert(e2.prev == nil)
>  -- At this point stack is following: e1 -> e2
>  -- Let's test following cases:
> --- 1. e3 -> e2, e1 -> NULL (e3:set_prev(e2))
> +-- 1. e3 -> e2, e1 -> NULL (e1:set_prev(nil), e3:set_prev(e2))
>  -- 2. e1 -> e3, e2 -> NULL (e1:set_prev(e3))
>  -- 3. e3 -> e1 -> e2 (e3:set_prev(e1))
> --- 4. e1 -> e2 -> e3 (e2:set_prev(e3))
> +-- 4. e1 -> e2 -> e3 (e1:set_prev(nil) e2:set_prev(e3) e1:set_prev(e2))
>  --
>  e3 = box.error.new({code = 111, reason = "another cause"})
> +e1:set_prev(nil)
>  e3:set_prev(e2)
>  assert(e3.prev == e2)
>  assert(e2.prev == nil)
> @@ -135,6 +138,7 @@ assert(e1.prev == nil)
>  
>  -- Reset stack to e1 -> e2 and test case 2.
>  --
> +e3:set_prev(nil)
>  e1:set_prev(e2)
>  assert(e2.prev == nil)
>  assert(e3.prev == nil)
> @@ -156,11 +160,11 @@ assert(e3.prev == e1)
>  
>  -- Unlink errors and test case 4.
>  --
> -e1:set_prev(nil)
> -e2:set_prev(nil)
>  e3:set_prev(nil)
> -e1:set_prev(e2)
> +e2:set_prev(nil)
> +e1:set_prev(nil)
>  e2:set_prev(e3)
> +e1:set_prev(e2)
>  assert(e1.prev == e2)
>  assert(e2.prev == e3)
>  assert(e3.prev == nil)
> @@ -173,30 +177,6 @@ assert(e3.prev == nil)
>  e3:set_prev(e2)
>  assert(e3.prev == nil)
>  
> --- Test splitting list into two ones.
> --- After that we will get two lists: e1->e2->e5 and e3->e4
> ---
> -e4 = box.error.new({code = 111, reason = "yet another cause"})
> -e5 = box.error.new({code = 111, reason = "and another one"})
> -e3:set_prev(e4)
> -e2:set_prev(e5)
> -assert(e1.prev == e2)
> -assert(e2.prev == e5)
> -assert(e3.prev == e4)
> -assert(e5.prev == nil)
> -assert(e4.prev == nil)
> -
> --- Another splitting option: e1->e2 and e5->e3->e4
> --- But firstly restore to one single list e1->e2->e3->e4
> ---
> -e2:set_prev(e3)
> -e5:set_prev(e3)
> -assert(e1.prev == e2)
> -assert(e2.prev == nil)
> -assert(e5.prev == e3)
> -assert(e3.prev == e4)
> -assert(e4.prev == nil)
> -
>  -- In case error is destroyed, it unrefs reference counter
>  -- of its previous error. In turn, box.error.clear() refs/unrefs
>  -- only head and doesn't touch other errors.
> @@ -212,5 +192,3 @@ assert(e2.code == 111)
>  box.error.set(e1)
>  box.error.clear()
>  assert(e1.prev == e2)
> -
> -space:drop()

Re: [Tarantool-patches] [PATCH v2 06/10] box: introduce stacked diagnostic area

[Vladislav Shpilevoy]

Hi! Thanks for the fixes!

>> Also a small comment about cycle detection in your patch. Here I
>> would strongly-strongly recommend to avoid it when it is clearly
>> not necessary.
> 
> To be honest, I don't understand why do you care about this opimization so
> much. Linking errors into list is unlikely to be hot execution path, so nobody
> should bother with such minor improvement here.

I am really afraid, that our solution team will find a way to
shoot in their legs. For example, by doing some weird banking
application producing error stacks with 50 errors and more, or
they will add many errors when will forward them in the network
from node to node (they actually want this, and are going to do
that when we introduce custom errors), and will waste notable
time on cycle checking. And probably will even hit the cycle
error, and will complain. This is what I am afraid of. I am
trying to save every CPU cycle.

> Instead, keeping code clear,
> simple and straightforward (your diff contains more branches and looks a bit
> more complicated to me) must be the main goal. Applied diff below only due to
> your recommendation.
>  
> 
> Still, I haven't changed my mind. I didn't apply diff below, sorry.
> I'll leave it to the second reviewer.

Not sure there will be a second reviewer. From what I understand, the
release is begin of next week. I am ok, lets leave it then as is.


Sorry, I still couldn't finish with this commit. Made another
amendment. See diff below, and on top of this commit on the branch.

What I did is I made diag_add_error() assert, when the diag is
empty. And when we are trying to add not a topmost error.

To understand why see that case. Assume, we have a place which
does diag_set() somewhere deep. And we have code calling this
place and doing diag_add(). While diag_set() is there, we can't
end up in infinitely growing stack if that error hits again and
again. Diag_set() will drop the old stack.

Then the diag_set() place is removed/changed in a way, that it
does not set a diag, or also makes diag_add(). We will not notice
this in the tests, and the stack will have a chance of growing
infinitely.

So I added an assertion to catch such changes right away. I hope
you agree. I added one another tiny commit on top of the last patch
to make box_error_add() ok with that. This is a public function,
and in there it is ok to both add and set.

Also I added a test on what happens when you box.error.set() middle
of an error stack.

====================
diff --git a/src/lib/core/diag.h b/src/lib/core/diag.h
index 276bf2362..10d59d240 100644
--- a/src/lib/core/diag.h
+++ b/src/lib/core/diag.h
@@ -254,11 +254,12 @@ static inline void
 diag_add_error(struct diag *diag, struct error *e)
 {
 	assert(e != NULL);
+	assert(diag->last != NULL);
+	assert(e->effect == NULL);
+	assert(diag->last->effect == NULL);
 	error_ref(e);
-	error_unlink_effect(e);
 	e->cause = diag->last;
-	if (diag->last != NULL)
-		diag->last->effect = e;
+	diag->last->effect = e;
 	diag->last = e;
 }
 
diff --git a/test/box/error.result b/test/box/error.result
index a4e8a8d8d..3d07f6e64 100644
--- a/test/box/error.result
+++ b/test/box/error.result
@@ -789,3 +789,22 @@ assert(e1.prev == e2)
  | ---
  | - true
  | ...
+
+-- Set middle of an error stack into the diagnostics area.
+e1:set_prev(e2)
+ | ---
+ | ...
+e2:set_prev(e3)
+ | ---
+ | ...
+box.error.set(e2)
+ | ---
+ | ...
+assert(e1.prev == nil)
+ | ---
+ | - true
+ | ...
+assert(e2.prev == e3)
+ | ---
+ | - true
+ | ...
diff --git a/test/box/error.test.lua b/test/box/error.test.lua
index f03fb4d36..ed7eb7565 100644
--- a/test/box/error.test.lua
+++ b/test/box/error.test.lua
@@ -214,3 +214,10 @@ assert(e2.code == 111)
 box.error.set(e1)
 box.error.clear()
 assert(e1.prev == e2)
+
+-- Set middle of an error stack into the diagnostics area.
+e1:set_prev(e2)
+e2:set_prev(e3)
+box.error.set(e2)
+assert(e1.prev == nil)
+assert(e2.prev == e3)

Re: [Tarantool-patches] [PATCH v2 06/10] box: introduce stacked diagnostic area

[Nikita Pettik]

On 03 Apr 00:20, Vladislav Shpilevoy wrote:
> Hi! Thanks for the fixes!
> 
> >> Also a small comment about cycle detection in your patch. Here I
> >> would strongly-strongly recommend to avoid it when it is clearly
> >> not necessary.
> > 
> > To be honest, I don't understand why do you care about this opimization so
> > much. Linking errors into list is unlikely to be hot execution path, so nobody
> > should bother with such minor improvement here.
> 
> I am really afraid, that our solution team will find a way to
> shoot in their legs. For example, by doing some weird banking
> application producing error stacks with 50 errors and more, or
> they will add many errors when will forward them in the network
> from node to node (they actually want this, and are going to do
> that when we introduce custom errors), and will waste notable
> time on cycle checking. And probably will even hit the cycle
> error, and will complain. This is what I am afraid of. I am
> trying to save every CPU cycle.
> 
> > Instead, keeping code clear,
> > simple and straightforward (your diff contains more branches and looks a bit
> > more complicated to me) must be the main goal. Applied diff below only due to
> > your recommendation.
> >  
> > 
> > Still, I haven't changed my mind. I didn't apply diff below, sorry.
> > I'll leave it to the second reviewer.
> 
> Not sure there will be a second reviewer. From what I understand, the
> release is begin of next week. I am ok, lets leave it then as is.

There's still 56 open issues assigned to 2.4 milestone. I doubt
that all of them will be closed till Monday...
 
> Sorry, I still couldn't finish with this commit. Made another
> amendment. See diff below, and on top of this commit on the branch.
> 
> What I did is I made diag_add_error() assert, when the diag is
> empty. And when we are trying to add not a topmost error.

Ok, as far as I understand, to ensure that diag_set() is always
called before diag_add().
 
> To understand why see that case. Assume, we have a place which
> does diag_set() somewhere deep. And we have code calling this
> place and doing diag_add(). While diag_set() is there, we can't
> end up in infinitely growing stack if that error hits again and
> again. Diag_set() will drop the old stack.

Agree.

> Then the diag_set() place is removed/changed in a way, that it
> does not set a diag, or also makes diag_add(). We will not notice
> this in the tests, and the stack will have a chance of growing
> infinitely.
> 
> So I added an assertion to catch such changes right away. I hope
> you agree. I added one another tiny commit on top of the last patch
> to make box_error_add() ok with that. This is a public function,
> and in there it is ok to both add and set.
> 
> Also I added a test on what happens when you box.error.set() middle
> of an error stack.
> 
> ====================
> diff --git a/src/lib/core/diag.h b/src/lib/core/diag.h
> index 276bf2362..10d59d240 100644
> --- a/src/lib/core/diag.h
> +++ b/src/lib/core/diag.h
> @@ -254,11 +254,12 @@ static inline void
>  diag_add_error(struct diag *diag, struct error *e)
>  {
>  	assert(e != NULL);
> +	assert(diag->last != NULL);
> +	assert(e->effect == NULL);

I added a few comments, check them out:

diff --git a/src/lib/core/diag.h b/src/lib/core/diag.h
index 10d59d240..9c238f5a2 100644
--- a/src/lib/core/diag.h
+++ b/src/lib/core/diag.h
@@ -254,7 +254,12 @@ static inline void
 diag_add_error(struct diag *diag, struct error *e)
 {
        assert(e != NULL);
+       /* diag_set_error() should be called before. */
        assert(diag->last != NULL);
+       /*
+        * e should be the bottom of its own stack.
+        * Otherwise some errors may be lost.
+        */
        assert(e->effect == NULL);
        assert(diag->last->effect == NULL);
        error_ref(e);


> +	assert(diag->last->effect == NULL);
>  	error_ref(e);
> -	error_unlink_effect(e);
>  	e->cause = diag->last;
> -	if (diag->last != NULL)
> -		diag->last->effect = e;
> +	diag->last->effect = e;
>  	diag->last = e;
>  }
>  
> diff --git a/test/box/error.result b/test/box/error.result
> index a4e8a8d8d..3d07f6e64 100644
> --- a/test/box/error.result
> +++ b/test/box/error.result
> @@ -789,3 +789,22 @@ assert(e1.prev == e2)
>   | ---
>   | - true
>   | ...
> +
> +-- Set middle of an error stack into the diagnostics area.
> +e1:set_prev(e2)
> + | ---
> + | ...
> +e2:set_prev(e3)
> + | ---
> + | ...
> +box.error.set(e2)
> + | ---
> + | ...
> +assert(e1.prev == nil)
> + | ---
> + | - true
> + | ...
> +assert(e2.prev == e3)
> + | ---
> + | - true
> + | ...
> diff --git a/test/box/error.test.lua b/test/box/error.test.lua
> index f03fb4d36..ed7eb7565 100644
> --- a/test/box/error.test.lua
> +++ b/test/box/error.test.lua
> @@ -214,3 +214,10 @@ assert(e2.code == 111)
>  box.error.set(e1)
>  box.error.clear()
>  assert(e1.prev == e2)
> +
> +-- Set middle of an error stack into the diagnostics area.
> +e1:set_prev(e2)
> +e2:set_prev(e3)
> +box.error.set(e2)
> +assert(e1.prev == nil)
> +assert(e2.prev == e3)

Re: [Tarantool-patches] [PATCH v2 06/10] box: introduce stacked diagnostic area

[Vladislav Shpilevoy]

Hi! Thanks for the fixes!

On 03/04/2020 03:54, Nikita Pettik wrote:
> On 03 Apr 00:20, Vladislav Shpilevoy wrote:
>> Hi! Thanks for the fixes!
>>
>>>> Also a small comment about cycle detection in your patch. Here I
>>>> would strongly-strongly recommend to avoid it when it is clearly
>>>> not necessary.
>>>
>>> To be honest, I don't understand why do you care about this opimization so
>>> much. Linking errors into list is unlikely to be hot execution path, so nobody
>>> should bother with such minor improvement here.
>>
>> I am really afraid, that our solution team will find a way to
>> shoot in their legs. For example, by doing some weird banking
>> application producing error stacks with 50 errors and more, or
>> they will add many errors when will forward them in the network
>> from node to node (they actually want this, and are going to do
>> that when we introduce custom errors), and will waste notable
>> time on cycle checking. And probably will even hit the cycle
>> error, and will complain. This is what I am afraid of. I am
>> trying to save every CPU cycle.
>>
>>> Instead, keeping code clear,
>>> simple and straightforward (your diff contains more branches and looks a bit
>>> more complicated to me) must be the main goal. Applied diff below only due to
>>> your recommendation.
>>>  
>>>
>>> Still, I haven't changed my mind. I didn't apply diff below, sorry.
>>> I'll leave it to the second reviewer.
>>
>> Not sure there will be a second reviewer. From what I understand, the
>> release is begin of next week. I am ok, lets leave it then as is.
> 
> There's still 56 open issues assigned to 2.4 milestone. I doubt
> that all of them will be closed till Monday...

Didn't know that. In this case I don't know how is the release
supposed to be published on the next week. Probably a second review
can be got then. However I don't know who to ask. Maybe Alexander L.,
since he is back in the team, and he knows the core.

>> Sorry, I still couldn't finish with this commit. Made another
>> amendment. See diff below, and on top of this commit on the branch.
>>
>> What I did is I made diag_add_error() assert, when the diag is
>> empty. And when we are trying to add not a topmost error.
> 
> Ok, as far as I understand, to ensure that diag_set() is always
> called before diag_add().

Exactly.

> I added a few comments, check them out:

Good.

Re: [Tarantool-patches] [PATCH v2 06/10] box: introduce stacked diagnostic area

[Vladislav Shpilevoy]

Two more comments.

> diff --git a/test/box/error.test.lua b/test/box/error.test.lua
> index a0b7d3e78..1fdd6ed98 100644
> --- a/test/box/error.test.lua
> +++ b/test/box/error.test.lua
> @@ -108,4 +108,109 @@ box.error.new(err)
> +
>  space:drop()

1. You probably need to keep this 'space:drop()' after
the test related to it.

2. Also I did some double checking if there are no leaks. And came
up with that test:

====================
diff --git a/extra/exports b/extra/exports
index 9323996c1..f581026fb 100644
--- a/extra/exports
+++ b/extra/exports
@@ -43,6 +43,7 @@ tnt_iconv_close
 tnt_iconv
 exception_get_string
 exception_get_int
+get_error_count
 
 tarantool_lua_ibuf
 uuid_nil
diff --git a/src/lib/core/diag.c b/src/lib/core/diag.c
index 57da5da44..f5d3c8a18 100644
--- a/src/lib/core/diag.c
+++ b/src/lib/core/diag.c
@@ -31,6 +31,14 @@
 #include "diag.h"
 #include "fiber.h"
 
+int diag_error_count = 0;
+
+int
+get_error_count(void)
+{
+	return __atomic_load_n(&diag_error_count, __ATOMIC_SEQ_CST);
+}
+
 /* Must be set by the library user */
 struct error_factory *error_factory = NULL;
 
@@ -76,6 +84,7 @@ error_create(struct error *e,
 	     error_f destroy, error_f raise, error_f log,
 	     const struct type_info *type, const char *file, unsigned line)
 {
+	__atomic_add_fetch(&diag_error_count, 1, __ATOMIC_SEQ_CST);
 	e->destroy = destroy;
 	e->raise = raise;
 	e->log = log;
@@ -92,6 +101,7 @@ error_create(struct error *e,
 	e->errmsg[0] = '\0';
 	e->cause = NULL;
 	e->effect = NULL;
+
 }
 
 struct diag *
diff --git a/src/lib/core/diag.h b/src/lib/core/diag.h
index 665f492fa..bd4ddcf3e 100644
--- a/src/lib/core/diag.h
+++ b/src/lib/core/diag.h
@@ -47,11 +47,16 @@ enum {
 	DIAG_FILENAME_MAX = 256
 };
 
+extern int diag_error_count;
+
 struct type_info;
 struct error;
 
 typedef void (*error_f)(struct error *e);
 
+int
+get_error_count(void);
+
 /**
  * Error diagnostics needs to be equally usable in C and C++
  * code. This is why there is a common infrastructure for errors.
@@ -136,6 +141,7 @@ error_unref(struct error *e)
 		if (cause == NULL)
 			return;
 		to_delete = cause;
+		__atomic_add_fetch(&diag_error_count, -1, __ATOMIC_SEQ_CST);
 	}
 }
 
diff --git a/test/box/error.test.lua b/test/box/error.test.lua
index 1fdd6ed98..8f884c372 100644
--- a/test/box/error.test.lua
+++ b/test/box/error.test.lua
@@ -78,6 +78,25 @@ t;
 
 test_run:cmd("setopt delimiter ''");
 
+ffi = require('ffi')
+
+ffi.cdef[[						\
+	int 						\
+	get_error_count(void); 				\
+]]
+
+e1 = nil
+e2 = nil
+e3 = nil
+e4 = nil
+e5 = nil
+err = nil
+box.error.clear()
+collectgarbage('collect')
+collectgarbage('collect')
+
+errcount = ffi.C.get_error_count()
+
 -- gh-4778: don't add created via box.error.new() errors to
 -- Tarantool's diagnostic area.
 --
@@ -213,4 +232,20 @@ box.error.set(e1)
 box.error.clear()
 assert(e1.prev == e2)
 
+e1 = nil
+e2 = nil
+e3 = nil
+e4 = nil
+e5 = nil
+err = nil
+box.error.clear()
+
+collectgarbage('collect')
+collectgarbage('collect')
+
+errcount2 = ffi.C.get_error_count()
+
+errcount
+errcount2
+
 space:drop()
====================

errcount is 13, errcount2 is 17. Something is probably wrong.
I thought these were errors created before line

    gh-4778: don't add created via box.error.new() errors to

But the difference still should be 0. Because I nullified and
cleared all variables you use before and after the test. I tried
commenting out all the code before this line, and still get a
not 0 difference. I ask you to check why some errors are not
deleted.

Re: [Tarantool-patches] [PATCH v2 06/10] box: introduce stacked diagnostic area

[Nikita Pettik]

On 28 Mar 19:59, Vladislav Shpilevoy wrote:
> Two more comments.
> 
> > diff --git a/test/box/error.test.lua b/test/box/error.test.lua
> > index a0b7d3e78..1fdd6ed98 100644
> > --- a/test/box/error.test.lua
> > +++ b/test/box/error.test.lua
> > @@ -108,4 +108,109 @@ box.error.new(err)
> > +
> >  space:drop()
> 
> 1. You probably need to keep this 'space:drop()' after
> the test related to it.

Isn't it too late?:) I mean test it is related to is finished at
line 20, meanwhile space:drop() is already at line 111 (box/error.test.lua
is already pushed).

> 2. Also I did some double checking if there are no leaks. And came
> up with that test:
> ====================
> diff --git a/extra/exports b/extra/exports
> index 9323996c1..f581026fb 100644
> --- a/extra/exports
> +++ b/extra/exports
> @@ -43,6 +43,7 @@ tnt_iconv_close
>  tnt_iconv
>  exception_get_string
>  exception_get_int
> +get_error_count
>  
>  tarantool_lua_ibuf
>  uuid_nil
> diff --git a/src/lib/core/diag.c b/src/lib/core/diag.c
> index 57da5da44..f5d3c8a18 100644
> --- a/src/lib/core/diag.c
> +++ b/src/lib/core/diag.c
> @@ -31,6 +31,14 @@
>  #include "diag.h"
>  #include "fiber.h"
>  
> +int diag_error_count = 0;
> +
> +int
> +get_error_count(void)
> +{
> +	return __atomic_load_n(&diag_error_count, __ATOMIC_SEQ_CST);
> +}
> +
>  /* Must be set by the library user */
>  struct error_factory *error_factory = NULL;
>  
> @@ -76,6 +84,7 @@ error_create(struct error *e,
>  	     error_f destroy, error_f raise, error_f log,
>  	     const struct type_info *type, const char *file, unsigned line)
>  {
> +	__atomic_add_fetch(&diag_error_count, 1, __ATOMIC_SEQ_CST);
>  	e->destroy = destroy;
>  	e->raise = raise;
>  	e->log = log;
> @@ -92,6 +101,7 @@ error_create(struct error *e,
>  	e->errmsg[0] = '\0';
>  	e->cause = NULL;
>  	e->effect = NULL;
> +
>  }
>  
>  struct diag *
> diff --git a/src/lib/core/diag.h b/src/lib/core/diag.h
> index 665f492fa..bd4ddcf3e 100644
> --- a/src/lib/core/diag.h
> +++ b/src/lib/core/diag.h
> @@ -47,11 +47,16 @@ enum {
>  	DIAG_FILENAME_MAX = 256
>  };
>  
> +extern int diag_error_count;
> +
>  struct type_info;
>  struct error;
>  
>  typedef void (*error_f)(struct error *e);
>  
> +int
> +get_error_count(void);
> +
>  /**
>   * Error diagnostics needs to be equally usable in C and C++
>   * code. This is why there is a common infrastructure for errors.
> @@ -136,6 +141,7 @@ error_unref(struct error *e)
>  		if (cause == NULL)
>  			return;
>  		to_delete = cause;
> +		__atomic_add_fetch(&diag_error_count, -1, __ATOMIC_SEQ_CST);
>  	}
>  }
>  
> diff --git a/test/box/error.test.lua b/test/box/error.test.lua
> index 1fdd6ed98..8f884c372 100644
> --- a/test/box/error.test.lua
> +++ b/test/box/error.test.lua
> @@ -78,6 +78,25 @@ t;
>  
>  test_run:cmd("setopt delimiter ''");
>  
> +ffi = require('ffi')
> +
> +ffi.cdef[[						\
> +	int 						\
> +	get_error_count(void); 				\
> +]]
> +
> +e1 = nil
> +e2 = nil
> +e3 = nil
> +e4 = nil
> +e5 = nil
> +err = nil
> +box.error.clear()
> +collectgarbage('collect')
> +collectgarbage('collect')
> +
> +errcount = ffi.C.get_error_count()
> +
>  -- gh-4778: don't add created via box.error.new() errors to
>  -- Tarantool's diagnostic area.
>  --
> @@ -213,4 +232,20 @@ box.error.set(e1)
>  box.error.clear()
>  assert(e1.prev == e2)
>  
> +e1 = nil
> +e2 = nil
> +e3 = nil
> +e4 = nil
> +e5 = nil
> +err = nil
> +box.error.clear()
> +
> +collectgarbage('collect')
> +collectgarbage('collect')
> +
> +errcount2 = ffi.C.get_error_count()
> +
> +errcount
> +errcount2
> +
>  space:drop()
> ====================
> 
> errcount is 13, errcount2 is 17. Something is probably wrong.
> I thought these were errors created before line

Apllied your diff with one change and every launch I get:

[001] +errcount
[001] + | ---
[001] + | - 1
[001] + | ...
[001] +errcount2
[001] + | ---
[001] + | - 1
[001] + | ...
[001] +

One error I guess is that which gets stuck in diag. The change is:

@@ -124,6 +129,7 @@ error_unref(struct error *e)
                to_delete->cause = NULL;
                to_delete->effect = NULL;
                to_delete->destroy(to_delete);
+               __atomic_add_fetch(&diag_error_count, -1, __ATOMIC_SEQ_CST);
                if (cause == NULL)
                        return;
                to_delete = cause;
 
I.e. we should accout destroy before checking cause since for
the last error in the list it is NULL, ergo it won't be accounted.

Nit: could you please send diff as a separate mail attachment next time?
The thing is I have to manually extract it to a separate file
instead of being capable of applying it as a patch via git am/apply.
Thanks.

>     gh-4778: don't add created via box.error.new() errors to
> 
> But the difference still should be 0. Because I nullified and
> cleared all variables you use before and after the test. I tried
> commenting out all the code before this line, and still get a
> not 0 difference. I ask you to check why some errors are not
> deleted.

Re: [Tarantool-patches] [PATCH v2 06/10] box: introduce stacked diagnostic area

[Vladislav Shpilevoy]

Thanks for the answers!

On 31/03/2020 19:44, Nikita Pettik wrote:
> On 28 Mar 19:59, Vladislav Shpilevoy wrote:
>> Two more comments.
>>
>>> diff --git a/test/box/error.test.lua b/test/box/error.test.lua
>>> index a0b7d3e78..1fdd6ed98 100644
>>> --- a/test/box/error.test.lua
>>> +++ b/test/box/error.test.lua
>>> @@ -108,4 +108,109 @@ box.error.new(err)
>>> +
>>>  space:drop()
>>
>> 1. You probably need to keep this 'space:drop()' after
>> the test related to it.
> 
> Isn't it too late?:) I mean test it is related to is finished at
> line 20, meanwhile space:drop() is already at line 111 (box/error.test.lua
> is already pushed).

You still can add 1148 tests after space:drop, not before. In
that case it at least won't become worse than it is.

> Apllied your diff with one change and every launch I get:
> 
> [001] +errcount
> [001] + | ---
> [001] + | - 1
> [001] + | ...
> [001] +errcount2
> [001] + | ---
> [001] + | - 1
> [001] + | ...
> [001] +
> 
> One error I guess is that which gets stuck in diag. The change is:
> 
> @@ -124,6 +129,7 @@ error_unref(struct error *e)
>                 to_delete->cause = NULL;
>                 to_delete->effect = NULL;
>                 to_delete->destroy(to_delete);
> +               __atomic_add_fetch(&diag_error_count, -1, __ATOMIC_SEQ_CST);
>                 if (cause == NULL)
>                         return;
>                 to_delete = cause;
>  
> I.e. we should accout destroy before checking cause since for
> the last error in the list it is NULL, ergo it won't be accounted.

Shame on me, you are right. I should decrement it exactly on
destroy. Then there is no leaks!

> Nit: could you please send diff as a separate mail attachment next time?
> The thing is I have to manually extract it to a separate file
> instead of being capable of applying it as a patch via git am/apply.
> Thanks.

Sure.

Re: [Tarantool-patches] [PATCH v2 06/10] box: introduce stacked diagnostic area

[Nikita Pettik]

On 02 Apr 02:29, Vladislav Shpilevoy wrote:
> Thanks for the answers!
> 
> On 31/03/2020 19:44, Nikita Pettik wrote:
> > On 28 Mar 19:59, Vladislav Shpilevoy wrote:
> >> Two more comments.
> >>
> >>> diff --git a/test/box/error.test.lua b/test/box/error.test.lua
> >>> index a0b7d3e78..1fdd6ed98 100644
> >>> --- a/test/box/error.test.lua
> >>> +++ b/test/box/error.test.lua
> >>> @@ -108,4 +108,109 @@ box.error.new(err)
> >>> +
> >>>  space:drop()
> >>
> >> 1. You probably need to keep this 'space:drop()' after
> >> the test related to it.
> > 
> > Isn't it too late?:) I mean test it is related to is finished at
> > line 20, meanwhile space:drop() is already at line 111 (box/error.test.lua
> > is already pushed).
> 
> You still can add 1148 tests after space:drop, not before. In
> that case it at least won't become worse than it is.

Ok, placed space:drop() before #1148 test cases.
 

[Tarantool-patches] [PATCH v2 07/10] box: use stacked diagnostic area for functional indexes

[Nikita Pettik]

Since we've introduced stacked diagnostic in previous commit, let's use
it in the code implementing functional indexes.

Part of #1148
---
 src/box/key_list.c              | 12 ++++----
 src/box/lua/call.c              |  4 +--
 test/engine/func_index.result   | 52 +++++++++++++++++++++++++++++----
 test/engine/func_index.test.lua |  8 +++++
 4 files changed, 62 insertions(+), 14 deletions(-)

diff --git a/src/box/key_list.c b/src/box/key_list.c
index 81eb501a5..a766ce0ec 100644
--- a/src/box/key_list.c
+++ b/src/box/key_list.c
@@ -63,9 +63,9 @@ key_list_iterator_create(struct key_list_iterator *it, struct tuple *tuple,
 	if (rc != 0) {
 		/* Can't evaluate function. */
 		struct space *space = space_by_id(index_def->space_id);
-		diag_set(ClientError, ER_FUNC_INDEX_FUNC, index_def->name,
+		diag_add(ClientError, ER_FUNC_INDEX_FUNC, index_def->name,
 			 space != NULL ? space_name(space) : "",
-			 diag_last_error(diag_get())->errmsg);
+			 "can't evaluate function");
 		return -1;
 	}
 	uint32_t key_data_sz;
@@ -74,9 +74,9 @@ key_list_iterator_create(struct key_list_iterator *it, struct tuple *tuple,
 	if (key_data == NULL) {
 		struct space *space = space_by_id(index_def->space_id);
 		/* Can't get a result returned by function . */
-		diag_set(ClientError, ER_FUNC_INDEX_FUNC, index_def->name,
+		diag_add(ClientError, ER_FUNC_INDEX_FUNC, index_def->name,
 			 space != NULL ? space_name(space) : "",
-			 diag_last_error(diag_get())->errmsg);
+			 "can't get a value returned by function");
 		return -1;
 	}
 
@@ -170,9 +170,9 @@ key_list_iterator_next(struct key_list_iterator *it, const char **value)
 		 * The key doesn't follow functional index key
 		 * definition.
 		 */
-		diag_set(ClientError, ER_FUNC_INDEX_FORMAT, it->index_def->name,
+		diag_add(ClientError, ER_FUNC_INDEX_FORMAT, it->index_def->name,
 			 space ? space_name(space) : "",
-			 diag_last_error(diag_get())->errmsg);
+			 "key does not follow functional index definition");
 		return -1;
 	}
 
diff --git a/src/box/lua/call.c b/src/box/lua/call.c
index 92575374d..5d3579eff 100644
--- a/src/box/lua/call.c
+++ b/src/box/lua/call.c
@@ -687,9 +687,9 @@ func_persistent_lua_load(struct func_lua *func)
 	if (func->base.def->is_sandboxed) {
 		if (prepare_lua_sandbox(tarantool_L, default_sandbox_exports,
 					nelem(default_sandbox_exports)) != 0) {
-			diag_set(ClientError, ER_LOAD_FUNCTION,
+			diag_add(ClientError, ER_LOAD_FUNCTION,
 				 func->base.def->name,
-				 diag_last_error(diag_get())->errmsg);
+				 "can't prepare a Lua sandbox");
 			goto end;
 		}
 	} else {
diff --git a/test/engine/func_index.result b/test/engine/func_index.result
index 159158f1c..8f92fcf11 100644
--- a/test/engine/func_index.result
+++ b/test/engine/func_index.result
@@ -5,6 +5,10 @@ test_run = require('test_run').new()
 engine = test_run:get_cfg('engine')
  | ---
  | ...
+test_run:cmd("push filter \"file: .*\" to \"file: <filename>\"")
+ | ---
+ | - true
+ | ...
 
 --
 -- gh-1260: Func index.
@@ -158,8 +162,7 @@ idx = s:create_index('idx', {func = box.func.invalidreturn1.id, parts = {{1, 'un
 s:insert({1})
  | ---
  | - error: 'Key format doesn''t match one defined in functional index ''idx'' of space
- |     ''withdata'': Supplied key type of part 0 does not match index part type: expected
- |     unsigned'
+ |     ''withdata'': key does not follow functional index definition'
  | ...
 idx:drop()
  | ---
@@ -197,8 +200,7 @@ idx = s:create_index('idx', {func = box.func.invalidreturn3.id, parts = {{1, 'un
 s:insert({1})
  | ---
  | - error: 'Key format doesn''t match one defined in functional index ''idx'' of space
- |     ''withdata'': Supplied key type of part 0 does not match index part type: expected
- |     unsigned'
+ |     ''withdata'': key does not follow functional index definition'
  | ...
 idx:drop()
  | ---
@@ -217,8 +219,7 @@ idx = s:create_index('idx', {func = box.func.invalidreturn4.id, parts = {{1, 'un
 s:insert({1})
  | ---
  | - error: 'Key format doesn''t match one defined in functional index ''idx'' of space
- |     ''withdata'': Supplied key type of part 0 does not match index part type: expected
- |     unsigned'
+ |     ''withdata'': key does not follow functional index definition'
  | ...
 idx:drop()
  | ---
@@ -261,6 +262,45 @@ box.schema.func.create('runtimeerror', {body = lua_code, is_deterministic = true
 idx = s:create_index('idx', {func = box.func.runtimeerror.id, parts = {{1, 'string'}}})
  | ---
  | ...
+s:insert({1})
+ | ---
+ | - error: 'Failed to build a key for functional index ''idx'' of space ''withdata'':
+ |     can''t evaluate function'
+ | ...
+e = box.error.last()
+ | ---
+ | ...
+e:unpack()
+ | ---
+ | - code: 198
+ |   trace:
+ |   - file: <filename>
+ |     line: 68
+ |   type: ClientError
+ |   message: 'Failed to build a key for functional index ''idx'' of space ''withdata'':
+ |     can''t evaluate function'
+ |   prev: '[string "return function(tuple)                 local ..."]:1: attempt to
+ |     call global ''require'' (a nil value)'
+ | ...
+e = e.prev
+ | ---
+ | ...
+e:unpack()
+ | ---
+ | - type: LuajitError
+ |   message: '[string "return function(tuple)                 local ..."]:1: attempt
+ |     to call global ''require'' (a nil value)'
+ |   trace:
+ |   - file: <filename>
+ |     line: 1028
+ | ...
+e = e.prev
+ | ---
+ | ...
+e == nil
+ | ---
+ | - true
+ | ...
 idx:drop()
  | ---
  | ...
diff --git a/test/engine/func_index.test.lua b/test/engine/func_index.test.lua
index c3c3a7029..0e4043260 100644
--- a/test/engine/func_index.test.lua
+++ b/test/engine/func_index.test.lua
@@ -1,5 +1,6 @@
 test_run = require('test_run').new()
 engine = test_run:get_cfg('engine')
+test_run:cmd("push filter \"file: .*\" to \"file: <filename>\"")
 
 --
 -- gh-1260: Func index.
@@ -98,6 +99,13 @@ lua_code = [[function(tuple)
 test_run:cmd("setopt delimiter ''");
 box.schema.func.create('runtimeerror', {body = lua_code, is_deterministic = true, is_sandboxed = true})
 idx = s:create_index('idx', {func = box.func.runtimeerror.id, parts = {{1, 'string'}}})
+s:insert({1})
+e = box.error.last()
+e:unpack()
+e = e.prev
+e:unpack()
+e = e.prev
+e == nil
 idx:drop()
 
 -- Remove old persistent functions
-- 
2.17.1

Re: [Tarantool-patches] [PATCH v2 07/10] box: use stacked diagnostic area for functional indexes

[Vladislav Shpilevoy]

Hi! Thanks for the patch!

> diff --git a/test/engine/func_index.result b/test/engine/func_index.result
> index 159158f1c..8f92fcf11 100644
> --- a/test/engine/func_index.result
> +++ b/test/engine/func_index.result
> @@ -5,6 +5,10 @@ test_run = require('test_run').new()
>  engine = test_run:get_cfg('engine')
>   | ---
>   | ...
> +test_run:cmd("push filter \"file: .*\" to \"file: <filename>\"")

Better hide 'line: *' pattern too.

Re: [Tarantool-patches] [PATCH v2 07/10] box: use stacked diagnostic area for functional indexes

[Nikita Pettik]

On 31 Mar 01:24, Vladislav Shpilevoy wrote:
> Hi! Thanks for the patch!
> 
> > diff --git a/test/engine/func_index.result b/test/engine/func_index.result
> > index 159158f1c..8f92fcf11 100644
> > --- a/test/engine/func_index.result
> > +++ b/test/engine/func_index.result
> > @@ -5,6 +5,10 @@ test_run = require('test_run').new()
> >  engine = test_run:get_cfg('engine')
> >   | ---
> >   | ...
> > +test_run:cmd("push filter \"file: .*\" to \"file: <filename>\"")
> 
> Better hide 'line: *' pattern too.

Ok:

diff --git a/test/engine/func_index.test.lua b/test/engine/func_index.test.lua
index 0e4043260..5db588c1f 100644
--- a/test/engine/func_index.test.lua
+++ b/test/engine/func_index.test.lua
@@ -1,6 +1,7 @@
 test_run = require('test_run').new()
 engine = test_run:get_cfg('engine')
 test_run:cmd("push filter \"file: .*\" to \"file: <filename>\"")
+test_run:cmd("push filter \"line: .*\" to \"line: <line>\"")
 
 --
 -- gh-1260: Func index.

[Tarantool-patches] [PATCH v2 08/10] box/error: clarify purpose of reference counting in struct error

[Nikita Pettik]

---
 src/lib/core/diag.h | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/src/lib/core/diag.h b/src/lib/core/diag.h
index 675b9c6f1..665f492fa 100644
--- a/src/lib/core/diag.h
+++ b/src/lib/core/diag.h
@@ -71,6 +71,15 @@ struct error {
 	error_f raise;
 	error_f log;
 	const struct type_info *type;
+	/**
+	 * Reference counting is basically required since
+	 * instances of this structure are available in Lua
+	 * as well (as cdata with overloaded fields and methods
+	 * by the means of introspection). Thus, it may turn
+	 * out that Lua's GC attempts at releasing object
+	 * meanwhile it is still used in C internals or vice
+	 * versa. For details see luaT_pusherror().
+	 */
 	int refs;
 	/**
 	 * Errno at the moment of the error
-- 
2.17.1

Re: [Tarantool-patches] [PATCH v2 08/10] box/error: clarify purpose of reference counting in struct error

[Vladislav Shpilevoy]

LGTM, you can push this out of order.

On 25/03/2020 02:43, Nikita Pettik wrote:
> ---
>  src/lib/core/diag.h | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/src/lib/core/diag.h b/src/lib/core/diag.h
> index 675b9c6f1..665f492fa 100644
> --- a/src/lib/core/diag.h
> +++ b/src/lib/core/diag.h
> @@ -71,6 +71,15 @@ struct error {
>  	error_f raise;
>  	error_f log;
>  	const struct type_info *type;
> +	/**
> +	 * Reference counting is basically required since
> +	 * instances of this structure are available in Lua
> +	 * as well (as cdata with overloaded fields and methods
> +	 * by the means of introspection). Thus, it may turn
> +	 * out that Lua's GC attempts at releasing object
> +	 * meanwhile it is still used in C internals or vice
> +	 * versa. For details see luaT_pusherror().
> +	 */
>  	int refs;
>  	/**
>  	 * Errno at the moment of the error
> 

[Tarantool-patches] [PATCH v2 09/10] iproto: refactor error encoding with mpstream

[Nikita Pettik]

From: Kirill Shcherbatov <kshcherbatov@tarantool.org>

Refactor iproto_reply_error and iproto_write_error with a new
mpstream-based helper mpstream_iproto_encode_error that encodes
error object for iproto protocol on a given stream object.
Previously each routine implemented an own error encoding, but
with the increasing complexity of encode operation with following
patches we need a uniform way to do it.

The iproto_write_error routine starts using region location
to use region-based mpstream. It is not a problem itself, because
errors reporting is not really performance-critical path.

Needed for #1148
---
 src/box/xrow.c | 70 +++++++++++++++++++++++++++++++++++---------------
 1 file changed, 49 insertions(+), 21 deletions(-)

diff --git a/src/box/xrow.c b/src/box/xrow.c
index 5e3cb0709..256dd4d91 100644
--- a/src/box/xrow.c
+++ b/src/box/xrow.c
@@ -42,6 +42,7 @@
 #include "vclock.h"
 #include "scramble.h"
 #include "iproto_constants.h"
+#include "mpstream.h"
 
 static_assert(IPROTO_DATA < 0x7f && IPROTO_METADATA < 0x7f &&
 	      IPROTO_SQL_INFO < 0x7f, "encoded IPROTO_BODY keys must fit into "\
@@ -381,10 +382,6 @@ static const struct iproto_body_bin iproto_body_bin = {
 	0x81, IPROTO_DATA, 0xdd, 0
 };
 
-static const struct iproto_body_bin iproto_error_bin = {
-	0x81, IPROTO_ERROR, 0xdb, 0
-};
-
 /** Return a 4-byte numeric error code, with status flags. */
 static inline uint32_t
 iproto_encode_error(uint32_t error)
@@ -478,46 +475,77 @@ iproto_reply_vote(struct obuf *out, const struct ballot *ballot,
 	return 0;
 }
 
+static void
+mpstream_error_handler(void *error_ctx)
+{
+	*(bool *)error_ctx = true;
+}
+
+static void
+mpstream_iproto_encode_error(struct mpstream *stream, const struct error *error)
+{
+	mpstream_encode_map(stream, 2);
+	mpstream_encode_uint(stream, IPROTO_ERROR);
+	mpstream_encode_str(stream, error->errmsg);
+}
+
 int
 iproto_reply_error(struct obuf *out, const struct error *e, uint64_t sync,
 		   uint32_t schema_version)
 {
-	uint32_t msg_len = strlen(e->errmsg);
-	uint32_t errcode = box_error_code(e);
-
-	struct iproto_body_bin body = iproto_error_bin;
 	char *header = (char *)obuf_alloc(out, IPROTO_HEADER_LEN);
 	if (header == NULL)
 		return -1;
 
+	bool is_error = false;
+	struct mpstream stream;
+	mpstream_init(&stream, out, obuf_reserve_cb, obuf_alloc_cb,
+		      mpstream_error_handler, &is_error);
+
+	uint32_t used = obuf_size(out);
+	mpstream_iproto_encode_error(&stream, e);
+	mpstream_flush(&stream);
+
+	uint32_t errcode = box_error_code(e);
 	iproto_header_encode(header, iproto_encode_error(errcode), sync,
-			     schema_version, sizeof(body) + msg_len);
-	body.v_data_len = mp_bswap_u32(msg_len);
+			     schema_version, obuf_size(out) - used);
+
 	/* Malformed packet appears to be a lesser evil than abort. */
-	return obuf_dup(out, &body, sizeof(body)) != sizeof(body) ||
-	       obuf_dup(out, e->errmsg, msg_len) != msg_len ? -1 : 0;
+	return is_error;
 }
 
 void
 iproto_write_error(int fd, const struct error *e, uint32_t schema_version,
 		   uint64_t sync)
 {
-	uint32_t msg_len = strlen(e->errmsg);
-	uint32_t errcode = box_error_code(e);
+	bool is_error = false;
+	struct mpstream stream;
+	struct region *region = &fiber()->gc;
+	mpstream_init(&stream, region, region_reserve_cb, region_alloc_cb,
+		      mpstream_error_handler, &is_error);
+
+	size_t region_svp = region_used(region);
+	mpstream_iproto_encode_error(&stream, e);
+	mpstream_flush(&stream);
+	if (is_error)
+		goto cleanup;
+
+	size_t payload_size = region_used(region) - region_svp;
+	char *payload = region_join(region, payload_size);
+	if (payload == NULL)
+		goto cleanup;
 
+	uint32_t errcode = box_error_code(e);
 	char header[IPROTO_HEADER_LEN];
-	struct iproto_body_bin body = iproto_error_bin;
-
 	iproto_header_encode(header, iproto_encode_error(errcode), sync,
-			     schema_version, sizeof(body) + msg_len);
-
-	body.v_data_len = mp_bswap_u32(msg_len);
+			     schema_version, payload_size);
 
 	ssize_t unused;
 	unused = write(fd, header, sizeof(header));
-	unused = write(fd, &body, sizeof(body));
-	unused = write(fd, e->errmsg, msg_len);
+	unused = write(fd, payload, payload_size);
 	(void) unused;
+cleanup:
+	region_truncate(region, region_svp);
 }
 
 int
-- 
2.17.1

Re: [Tarantool-patches] [PATCH v2 09/10] iproto: refactor error encoding with mpstream

[Vladislav Shpilevoy]

Thanks for the patch!

> diff --git a/src/box/xrow.c b/src/box/xrow.c
> index 5e3cb0709..256dd4d91 100644
> --- a/src/box/xrow.c
> +++ b/src/box/xrow.c
> @@ -478,46 +475,77 @@ iproto_reply_vote(struct obuf *out, const struct ballot *ballot,
>  int
>  iproto_reply_error(struct obuf *out, const struct error *e, uint64_t sync,
>  		   uint32_t schema_version)
>  {
> -	uint32_t msg_len = strlen(e->errmsg);
> -	uint32_t errcode = box_error_code(e);
> -
> -	struct iproto_body_bin body = iproto_error_bin;
>  	char *header = (char *)obuf_alloc(out, IPROTO_HEADER_LEN);
>  	if (header == NULL)
>  		return -1;
>  
> +	bool is_error = false;
> +	struct mpstream stream;
> +	mpstream_init(&stream, out, obuf_reserve_cb, obuf_alloc_cb,
> +		      mpstream_error_handler, &is_error);
> +
> +	uint32_t used = obuf_size(out);
> +	mpstream_iproto_encode_error(&stream, e);
> +	mpstream_flush(&stream);
> +
> +	uint32_t errcode = box_error_code(e);
>  	iproto_header_encode(header, iproto_encode_error(errcode), sync,
> -			     schema_version, sizeof(body) + msg_len);
> -	body.v_data_len = mp_bswap_u32(msg_len);
> +			     schema_version, obuf_size(out) - used);
> +
>  	/* Malformed packet appears to be a lesser evil than abort. */
> -	return obuf_dup(out, &body, sizeof(body)) != sizeof(body) ||
> -	       obuf_dup(out, e->errmsg, msg_len) != msg_len ? -1 : 0;
> +	return is_error;

The function is supposed to return 0/-1. So better keep this
convention. You can declare is_error as 'int rc', and set it to
-1 in mpstream_error_handler(). Should be fine. Or make it

    return is_error ? -1 : 0;

Re: [Tarantool-patches] [PATCH v2 09/10] iproto: refactor error encoding with mpstream

[Nikita Pettik]

On 31 Mar 01:24, Vladislav Shpilevoy wrote:
> Thanks for the patch!
> 
> > diff --git a/src/box/xrow.c b/src/box/xrow.c
> > index 5e3cb0709..256dd4d91 100644
> > --- a/src/box/xrow.c
> > +++ b/src/box/xrow.c
> > @@ -478,46 +475,77 @@ iproto_reply_vote(struct obuf *out, const struct ballot *ballot,
> >  int
> >  iproto_reply_error(struct obuf *out, const struct error *e, uint64_t sync,
> >  		   uint32_t schema_version)
> >  {
> > -	uint32_t msg_len = strlen(e->errmsg);
> > -	uint32_t errcode = box_error_code(e);
> > -
> > -	struct iproto_body_bin body = iproto_error_bin;
> >  	char *header = (char *)obuf_alloc(out, IPROTO_HEADER_LEN);
> >  	if (header == NULL)
> >  		return -1;
> >  
> > +	bool is_error = false;
> > +	struct mpstream stream;
> > +	mpstream_init(&stream, out, obuf_reserve_cb, obuf_alloc_cb,
> > +		      mpstream_error_handler, &is_error);
> > +
> > +	uint32_t used = obuf_size(out);
> > +	mpstream_iproto_encode_error(&stream, e);
> > +	mpstream_flush(&stream);
> > +
> > +	uint32_t errcode = box_error_code(e);
> >  	iproto_header_encode(header, iproto_encode_error(errcode), sync,
> > -			     schema_version, sizeof(body) + msg_len);
> > -	body.v_data_len = mp_bswap_u32(msg_len);
> > +			     schema_version, obuf_size(out) - used);
> > +
> >  	/* Malformed packet appears to be a lesser evil than abort. */
> > -	return obuf_dup(out, &body, sizeof(body)) != sizeof(body) ||
> > -	       obuf_dup(out, e->errmsg, msg_len) != msg_len ? -1 : 0;
> > +	return is_error;
> 
> The function is supposed to return 0/-1. So better keep this
> convention. You can declare is_error as 'int rc', and set it to
> -1 in mpstream_error_handler(). Should be fine. Or make it
> 
>     return is_error ? -1 : 0;

I prefer the latter option. Fixed.

[Tarantool-patches] [PATCH v2 10/10] iproto: support error stacked diagnostic area

[Nikita Pettik]

This patch introduces support of stacked errors in IProto protocol and
in net.box module.

@TarantoolBot document
Title: Stacked error diagnostic area

Starting from now errors can be organized into lists. To achieve this
Lua table representing error object is extended with .prev field and
e:set_prev(err) method. .prev field returns previous error if any exist.
e:set_prev(err) method expects err to be error object or nil and sets
err as previous error of e. For instance:

e1 = box.error.new({code = 111, reason = "cause"})
e2 = box.error.new({code = 111, reason = "cause of cause"})

e1:set_prev(e2)
assert(e1.prev == e2) -- true

Cycles are not allowed for error lists:

e2:set_prev(e1)
- error: 'builtin/error.lua: Cycles are not allowed'

Nil is valid input to :set_prev() method:

e1:set_prev(nil)
assert(e1.prev == nil) -- true

Note that error can be 'previous' only to the one error at once:

e1:set_prev(e2)
e3:set_prev(e2)
assert(e1.prev == nil) -- true
assert(e3.prev == e2) -- true

Setting previous error does not erase its own previous members:

-- e1 -> e2 -> e3 -> e4
e1:set_prev(e2)
e2:set_prev(e3)
e3:set_prev(e4)
e2:set_prev(e5)
-- Now there are two lists: e1->e2->e5 and e3->e4
assert(e1.prev == e2) -- true
assert(e2.prev == e5) -- true
assert(e3.prev == e4) -- true

Alternatively:

e1:set_prev(e2)
e2:set_prev(e3)
e3:set_prev(e4)
e5:set_prev(e3)
-- Now there are two lists: e1->e2 and e5->e3->e4
assert(e1.prev == e2) -- true
assert(e2.prev == nil) -- true
assert(e5.prev == e3) -- true
assert(e3.prev == e4) -- true

Stacked diagnostics is also supported by IProto protocol. Now responses
containing errors always (even if there's only one error to be returned)
include new IProto key: IPROTO_ERROR_STACK (0x51). So, body corresponding to
error response now looks like:
MAP{IPROTO_ERROR : string, IPROTO_ERROR_STACK : ARRAY[MAP{ERROR_CODE : uint, ERROR_MESSAGE : string}, MAP{...}, ...]}

where IPROTO_ERROR is 0x31 key, IPROTO_ERROR_STACK is 0x51, ERROR_CODE
is 0x01 and ERROR_MESSAGE is 0x02.
Instances of older versions (without support of stacked errors in
protocol) simply ignore unknown keys and still rely only on IPROTO_ERROR
key.

Closes #1148
---
 src/box/error.cc           |  17 +++++
 src/box/error.h            |  16 +++++
 src/box/iproto_constants.h |   6 ++
 src/box/lua/net_box.lua    |  32 ++++++++-
 src/box/xrow.c             |  88 ++++++++++++++++++++---
 test/box-py/iproto.result  |   6 +-
 test/box-py/iproto.test.py |   6 +-
 test/box/iproto.result     | 141 +++++++++++++++++++++++++++++++++++++
 test/box/iproto.test.lua   |  62 ++++++++++++++++
 test/box/net.box.result    |  60 ++++++++++++++++
 test/box/net.box.test.lua  |  23 ++++++
 11 files changed, 440 insertions(+), 17 deletions(-)
 create mode 100644 test/box/iproto.result
 create mode 100644 test/box/iproto.test.lua

diff --git a/src/box/error.cc b/src/box/error.cc
index 824a4617c..e3197c8e6 100644
--- a/src/box/error.cc
+++ b/src/box/error.cc
@@ -102,6 +102,23 @@ box_error_construct(const char *file, unsigned line, uint32_t code,
 	return e;
 }
 
+int
+box_error_add(const char *file, unsigned line, uint32_t code,
+	      const char *fmt, ...)
+{
+	struct error *e = BuildClientError(file, line, ER_UNKNOWN);
+	ClientError *client_error = type_cast(ClientError, e);
+	if (client_error) {
+		client_error->m_errcode = code;
+		va_list ap;
+		va_start(ap, fmt);
+		error_vformat_msg(e, fmt, ap);
+		va_end(ap);
+	}
+	diag_add_error(&fiber()->diag, e);
+	return -1;
+}
+
 /* }}} */
 
 struct rmean *rmean_error = NULL;
diff --git a/src/box/error.h b/src/box/error.h
index ef9cf3e76..24fc157bb 100644
--- a/src/box/error.h
+++ b/src/box/error.h
@@ -137,6 +137,22 @@ box_error_set(const char *file, unsigned line, uint32_t code,
 
 /** \endcond public */
 
+/**
+ * Add error to the diagnostic area. In contrast to box_error_set()
+ * it does not replace previous error being set, but rather link
+ * them into list.
+ *
+ * \param code IPROTO error code (enum \link box_error_code \endlink)
+ * \param format (const char * ) - printf()-like format string
+ * \param ... - format arguments
+ * \returns -1 for convention use
+ *
+ * \sa enum box_error_code
+ */
+int
+box_error_add(const char *file, unsigned line, uint32_t code,
+	      const char *fmt, ...);
+
 /**
  * Construct error object without setting it in the diagnostics
  * area. On the memory allocation fail returns NULL.
diff --git a/src/box/iproto_constants.h b/src/box/iproto_constants.h
index f9d413a31..7ed829645 100644
--- a/src/box/iproto_constants.h
+++ b/src/box/iproto_constants.h
@@ -126,6 +126,7 @@ enum iproto_key {
 	/* Leave a gap between SQL keys and additional request keys */
 	IPROTO_REPLICA_ANON = 0x50,
 	IPROTO_ID_FILTER = 0x51,
+	IPROTO_ERROR_STACK = 0x52,
 	IPROTO_KEY_MAX
 };
 
@@ -150,6 +151,11 @@ enum iproto_ballot_key {
 	IPROTO_BALLOT_IS_LOADING = 0x04,
 };
 
+enum iproto_error_key {
+	IPROTO_ERROR_CODE = 0x01,
+	IPROTO_ERROR_MESSAGE = 0x02,
+};
+
 #define bit(c) (1ULL<<IPROTO_##c)
 
 #define IPROTO_HEAD_BMAP (bit(REQUEST_TYPE) | bit(SYNC) | bit(REPLICA_ID) |\
diff --git a/src/box/lua/net_box.lua b/src/box/lua/net_box.lua
index 3f611c027..f6ec669c7 100644
--- a/src/box/lua/net_box.lua
+++ b/src/box/lua/net_box.lua
@@ -44,6 +44,9 @@ local SQL_INFO_ROW_COUNT_KEY = 0
 local IPROTO_FIELD_NAME_KEY = 0
 local IPROTO_DATA_KEY      = 0x30
 local IPROTO_ERROR_KEY     = 0x31
+local IPROTO_ERROR_STACK   = 0x51
+local IPROTO_ERROR_CODE    = 0x01
+local IPROTO_ERROR_MESSAGE = 0x02
 local IPROTO_GREETING_SIZE = 128
 local IPROTO_CHUNK_KEY     = 128
 local IPROTO_OK_KEY        = 0
@@ -277,8 +280,24 @@ local function create_transport(host, port, user, password, callback,
     --
     function request_index:result()
         if self.errno then
-            return nil, box.error.new({code = self.errno,
-                                       reason = self.response})
+            if type(self.response) == 'table' then
+                -- Decode and fill in error stack.
+                local prev = nil
+                for i = #self.response, 1, -1 do
+                    local error = self.response[i]
+                    local code = error[IPROTO_ERROR_CODE]
+                    local msg = error[IPROTO_ERROR_MESSAGE]
+                    assert(type(code) == 'number')
+                    assert(type(msg) == 'string')
+                    local new_err = box.error.new({code = code, reason = msg})
+                    new_err:set_prev(prev)
+                    prev = new_err
+                end
+                return nil, prev
+            else
+                return nil, box.error.new({code = self.errno,
+                                           reason = self.response})
+            end
         elseif not self.id then
             return self.response
         elseif not worker_fiber then
@@ -559,7 +578,14 @@ local function create_transport(host, port, user, password, callback,
             body, body_end_check = decode(body_rpos)
             assert(body_end == body_end_check, "invalid xrow length")
             request.errno = band(status, IPROTO_ERRNO_MASK)
-            request.response = body[IPROTO_ERROR_KEY]
+            -- IPROTO_ERROR_STACK comprises error encoded with
+            -- IPROTO_ERROR_KEY, so we may ignore content of that key.
+            if body[IPROTO_ERROR_STACK] ~= nil then
+                request.response = body[IPROTO_ERROR_STACK]
+                assert(type(request.response) == 'table')
+            else
+                request.response = body[IPROTO_ERROR_KEY]
+            end
             request.cond:broadcast()
             return
         end
diff --git a/src/box/xrow.c b/src/box/xrow.c
index 256dd4d91..ceb38b040 100644
--- a/src/box/xrow.c
+++ b/src/box/xrow.c
@@ -487,6 +487,19 @@ mpstream_iproto_encode_error(struct mpstream *stream, const struct error *error)
 	mpstream_encode_map(stream, 2);
 	mpstream_encode_uint(stream, IPROTO_ERROR);
 	mpstream_encode_str(stream, error->errmsg);
+
+	uint32_t err_cnt = 0;
+	for (const struct error *it = error; it != NULL; it = it->cause)
+		err_cnt++;
+	mpstream_encode_uint(stream, IPROTO_ERROR_STACK);
+	mpstream_encode_array(stream, err_cnt);
+	for (const struct error *it = error; it != NULL; it = it->cause) {
+		mpstream_encode_map(stream, 2);
+		mpstream_encode_uint(stream, IPROTO_ERROR_CODE);
+		mpstream_encode_uint(stream, box_error_code(it));
+		mpstream_encode_uint(stream, IPROTO_ERROR_MESSAGE);
+		mpstream_encode_str(stream, it->errmsg);
+	}
 }
 
 int
@@ -1070,19 +1083,66 @@ xrow_encode_auth(struct xrow_header *packet, const char *salt, size_t salt_len,
 	return 0;
 }
 
+static int
+iproto_decode_error_stack(const char **pos, const char *end)
+{
+	const char *reason = NULL;
+	static_assert(TT_STATIC_BUF_LEN >= DIAG_ERRMSG_MAX, "static buffer is "\
+		      "expected to be larger than error message max length");
+	/*
+	 * Erase previously set diag errors. It is required since
+	 * box_error_add() does not replace previous errors.
+	 */
+	box_error_clear();
+	if (mp_check_array(*pos, end) != 0)
+		return -1;
+	uint32_t stack_sz = mp_decode_array(pos);
+	for (uint32_t i = 0; i < stack_sz; i++) {
+		uint32_t code = 0;
+		if (mp_typeof(**pos) != MP_MAP || mp_check_map(*pos, end) != 0)
+			return -1;
+		uint32_t map_sz = mp_decode_map(pos);
+		for (uint32_t key_idx = 0; key_idx < map_sz; key_idx++) {
+			if (mp_typeof(**pos) != MP_UINT ||
+			    mp_check_uint(*pos, end) != 0)
+				return -1;
+			uint8_t key = mp_decode_uint(pos);
+			if (key == IPROTO_ERROR_CODE) {
+				if (mp_typeof(**pos) != MP_UINT ||
+				    mp_check_uint(*pos, end) != 0)
+					return -1;
+				code = mp_decode_uint(pos);
+			} else if (key == IPROTO_ERROR_MESSAGE) {
+				if (mp_typeof(**pos) != MP_STR ||
+				    mp_check_strl(*pos, end) != 0)
+					return -1;
+				uint32_t len;
+				const char *str = mp_decode_str(pos, &len);
+				reason = tt_cstr(str, len);
+			} else {
+				mp_next(pos);
+				continue;
+			}
+			box_error_add(__FILE__, __LINE__, code, reason);
+		}
+	}
+	return 0;
+}
+
 void
 xrow_decode_error(struct xrow_header *row)
 {
 	uint32_t code = row->type & (IPROTO_TYPE_ERROR - 1);
 
 	char error[DIAG_ERRMSG_MAX] = { 0 };
-	const char *pos;
+	const char *pos, *end;
 	uint32_t map_size;
 
 	if (row->bodycnt == 0)
 		goto error;
 	pos = (char *) row->body[0].iov_base;
-	if (mp_check(&pos, pos + row->body[0].iov_len))
+	end = pos + row->body[0].iov_len;
+	if (mp_check(&pos, end))
 		goto error;
 
 	pos = (char *) row->body[0].iov_base;
@@ -1096,15 +1156,27 @@ xrow_decode_error(struct xrow_header *row)
 			continue;
 		}
 		uint8_t key = mp_decode_uint(&pos);
-		if (key != IPROTO_ERROR || mp_typeof(*pos) != MP_STR) {
-			mp_next(&pos); /* value */
+		if (key == IPROTO_ERROR && mp_typeof(*pos) == MP_STR) {
+			/*
+			 * Obsolete way of sending error responses.
+			 * To be deprecated but still should be supported
+			 * to not break backward compatibility.
+			 */
+			uint32_t len;
+			const char *str = mp_decode_str(&pos, &len);
+			snprintf(error, sizeof(error), "%.*s", len, str);
+			box_error_set(__FILE__, __LINE__, code, error);
+		} else if (key == IPROTO_ERROR_STACK ) {
+			if (mp_check_array(pos, end) != 0)
+				goto error;
+			if (iproto_decode_error_stack(&pos, end) != 0)
+				continue;
+		} else {
+			mp_next(&pos);
 			continue;
 		}
-
-		uint32_t len;
-		const char *str = mp_decode_str(&pos, &len);
-		snprintf(error, sizeof(error), "%.*s", len, str);
 	}
+	return;
 
 error:
 	box_error_set(__FILE__, __LINE__, code, error);
diff --git a/test/box-py/iproto.result b/test/box-py/iproto.result
index 900e6e24f..04e2e220c 100644
--- a/test/box-py/iproto.result
+++ b/test/box-py/iproto.result
@@ -169,9 +169,9 @@ box.schema.user.revoke('guest', 'read,write,execute', 'universe')
 # Test bugs gh-272, gh-1654 if the packet was incorrect, respond with
 # an error code and do not close connection
 
-sync=0, {49: 'Invalid MsgPack - packet header'}
-sync=1234, {49: "Missing mandatory field 'space id' in request"}
-sync=5678, {49: "Read access to space '_user' is denied for user 'guest'"}
+sync=0, Invalid MsgPack - packet header
+sync=1234, Missing mandatory field 'space id' in request
+sync=5678, Read access to space '_user' is denied for user 'guest'
 space = box.schema.space.create('test_index_base', { id = 568 })
 ---
 ...
diff --git a/test/box-py/iproto.test.py b/test/box-py/iproto.test.py
index 77637d8ed..cdd1a71c5 100644
--- a/test/box-py/iproto.test.py
+++ b/test/box-py/iproto.test.py
@@ -355,15 +355,15 @@ s = c._socket
 header = { "hello": "world"}
 body = { "bug": 272 }
 resp = test_request(header, body)
-print 'sync=%d, %s' % (resp['header'][IPROTO_SYNC], resp['body'])
+print 'sync=%d, %s' % (resp['header'][IPROTO_SYNC], resp['body'].get(IPROTO_ERROR))
 header = { IPROTO_CODE : REQUEST_TYPE_SELECT }
 header[IPROTO_SYNC] = 1234
 resp = test_request(header, body)
-print 'sync=%d, %s' % (resp['header'][IPROTO_SYNC], resp['body'])
+print 'sync=%d, %s' % (resp['header'][IPROTO_SYNC], resp['body'].get(IPROTO_ERROR))
 header[IPROTO_SYNC] = 5678
 body = { IPROTO_SPACE_ID: 304, IPROTO_KEY: [], IPROTO_LIMIT: 1 }
 resp = test_request(header, body)
-print 'sync=%d, %s' % (resp['header'][IPROTO_SYNC], resp['body'])
+print 'sync=%d, %s' % (resp['header'][IPROTO_SYNC], resp['body'].get(IPROTO_ERROR))
 c.close()
 
 
diff --git a/test/box/iproto.result b/test/box/iproto.result
new file mode 100644
index 000000000..bb369be5a
--- /dev/null
+++ b/test/box/iproto.result
@@ -0,0 +1,141 @@
+test_run = require('test_run').new()
+---
+...
+net_box = require('net.box')
+---
+...
+urilib = require('uri')
+---
+...
+msgpack = require('msgpack')
+---
+...
+IPROTO_REQUEST_TYPE   = 0x00
+---
+...
+IPROTO_INSERT         = 0x02
+---
+...
+IPROTO_SYNC           = 0x01
+---
+...
+IPROTO_SPACE_ID       = 0x10
+---
+...
+IPROTO_TUPLE          = 0x21
+---
+...
+IPROTO_ERROR          = 0x31
+---
+...
+IPROTO_ERROR_STACK    = 0x52
+---
+...
+IPROTO_ERROR_CODE     = 0x01
+---
+...
+IPROTO_ERROR_MESSAGE  = 0x02
+---
+...
+IPROTO_OK             = 0x00
+---
+...
+IPROTO_SCHEMA_VERSION = 0x05
+---
+...
+IPROTO_STATUS_KEY     = 0x00
+---
+...
+-- gh-1148: test capabilities of stacked diagnostics bypassing net.box.
+--
+lua_func = [[function(tuple) local json = require('json') return json.encode(tuple) end]]
+---
+...
+box.schema.func.create('f1', {body = lua_func, is_deterministic = true, is_sandboxed = true})
+---
+...
+s = box.schema.space.create('s')
+---
+...
+pk = s:create_index('pk')
+---
+...
+idx = s:create_index('idx', {func = box.func.f1.id, parts = {{1, 'string'}}})
+---
+...
+box.schema.user.grant('guest', 'read, write, execute', 'universe')
+---
+...
+next_request_id = 16
+---
+...
+header = { \
+    [IPROTO_REQUEST_TYPE] = IPROTO_INSERT, \
+    [IPROTO_SYNC]         = next_request_id, \
+}
+---
+...
+body = { \
+    [IPROTO_SPACE_ID] = s.id, \
+    [IPROTO_TUPLE]    = box.tuple.new({1}) \
+}
+---
+...
+uri = urilib.parse(box.cfg.listen)
+---
+...
+sock = net_box.establish_connection(uri.host, uri.service)
+---
+...
+response = iproto_request(sock, header, body)
+---
+...
+sock:close()
+---
+- true
+...
+-- Both keys (obsolete and stack ones) are present in response.
+--
+assert(response.body[IPROTO_ERROR_STACK] ~= nil)
+---
+- true
+...
+assert(response.body[IPROTO_ERROR] ~= nil)
+---
+- true
+...
+err = response.body[IPROTO_ERROR_STACK][1]
+---
+...
+assert(err[IPROTO_ERROR_MESSAGE] == body[IPROTO_ERROR])
+---
+- error: assertion failed!
+...
+err = response.body[IPROTO_ERROR_STACK][2]
+---
+...
+assert(err[IPROTO_ERROR_CODE] ~= nil)
+---
+- true
+...
+assert(type(err[IPROTO_ERROR_CODE]) == 'number')
+---
+- true
+...
+assert(err[IPROTO_ERROR_MESSAGE] ~= nil)
+---
+- true
+...
+assert(type(err[IPROTO_ERROR_MESSAGE]) == 'string')
+---
+- true
+...
+box.schema.user.revoke('guest', 'read,write,execute', 'universe')
+---
+...
+s:drop()
+---
+...
+box.func.f1:drop()
+---
+...
diff --git a/test/box/iproto.test.lua b/test/box/iproto.test.lua
new file mode 100644
index 000000000..c16fe84c1
--- /dev/null
+++ b/test/box/iproto.test.lua
@@ -0,0 +1,62 @@
+test_run = require('test_run').new()
+net_box = require('net.box')
+urilib = require('uri')
+msgpack = require('msgpack')
+
+IPROTO_REQUEST_TYPE   = 0x00
+IPROTO_INSERT         = 0x02
+IPROTO_SYNC           = 0x01
+IPROTO_SPACE_ID       = 0x10
+IPROTO_TUPLE          = 0x21
+IPROTO_ERROR          = 0x31
+IPROTO_ERROR_STACK    = 0x52
+IPROTO_ERROR_CODE     = 0x01
+IPROTO_ERROR_MESSAGE  = 0x02
+IPROTO_OK             = 0x00
+IPROTO_SCHEMA_VERSION = 0x05
+IPROTO_STATUS_KEY     = 0x00
+
+-- gh-1148: test capabilities of stacked diagnostics bypassing net.box.
+--
+lua_func = [[function(tuple) local json = require('json') return json.encode(tuple) end]]
+
+box.schema.func.create('f1', {body = lua_func, is_deterministic = true, is_sandboxed = true})
+s = box.schema.space.create('s')
+pk = s:create_index('pk')
+idx = s:create_index('idx', {func = box.func.f1.id, parts = {{1, 'string'}}})
+
+box.schema.user.grant('guest', 'read, write, execute', 'universe')
+
+next_request_id = 16
+header = { \
+    [IPROTO_REQUEST_TYPE] = IPROTO_INSERT, \
+    [IPROTO_SYNC]         = next_request_id, \
+}
+
+body = { \
+    [IPROTO_SPACE_ID] = s.id, \
+    [IPROTO_TUPLE]    = box.tuple.new({1}) \
+}
+
+uri = urilib.parse(box.cfg.listen)
+sock = net_box.establish_connection(uri.host, uri.service)
+
+response = iproto_request(sock, header, body)
+sock:close()
+
+-- Both keys (obsolete and stack ones) are present in response.
+--
+assert(response.body[IPROTO_ERROR_STACK] ~= nil)
+assert(response.body[IPROTO_ERROR] ~= nil)
+
+err = response.body[IPROTO_ERROR_STACK][1]
+assert(err[IPROTO_ERROR_MESSAGE] == body[IPROTO_ERROR])
+err = response.body[IPROTO_ERROR_STACK][2]
+assert(err[IPROTO_ERROR_CODE] ~= nil)
+assert(type(err[IPROTO_ERROR_CODE]) == 'number')
+assert(err[IPROTO_ERROR_MESSAGE] ~= nil)
+assert(type(err[IPROTO_ERROR_MESSAGE]) == 'string')
+
+box.schema.user.revoke('guest', 'read,write,execute', 'universe')
+s:drop()
+box.func.f1:drop()
diff --git a/test/box/net.box.result b/test/box/net.box.result
index e3dabf7d9..eeefc0e3f 100644
--- a/test/box/net.box.result
+++ b/test/box/net.box.result
@@ -3902,6 +3902,66 @@ sock:close()
 ---
 - true
 ...
+-- gh-1148: test stacked diagnostics.
+--
+test_run:cmd("push filter \"file: .*\" to \"file: <filename>\"")
+---
+- true
+...
+lua_code = [[function(tuple) local json = require('json') return json.encode(tuple) end]]
+---
+...
+box.schema.func.create('f1', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+---
+...
+s = box.schema.space.create('s')
+---
+...
+pk = s:create_index('pk')
+---
+...
+idx = s:create_index('idx', {func = box.func.f1.id, parts = {{1, 'string'}}})
+---
+...
+box.schema.user.grant('guest', 'read,write,execute', 'universe')
+---
+...
+c = net.connect(box.cfg.listen)
+---
+...
+f = function(...) return c.space.s:insert(...) end
+---
+...
+_, e = pcall(f, {1})
+---
+...
+assert(e ~= nil)
+---
+- true
+...
+e:unpack().message
+---
+- 'Failed to build a key for functional index ''idx'' of space ''s'': can''t evaluate
+  function'
+...
+e.prev.message
+---
+- error: '[string "return e.prev.message "]:1: attempt to index field ''prev'' (a
+    nil value)'
+...
+box.schema.user.revoke('guest', 'read,write,execute', 'universe')
+---
+...
+s:drop()
+---
+...
+box.func.f1:drop()
+---
+...
+test_run:cmd("clear filter")
+---
+- true
+...
 test_run:wait_log('default', 'Got a corrupted row.*', nil, 10)
 ---
 - 'Got a corrupted row:'
diff --git a/test/box/net.box.test.lua b/test/box/net.box.test.lua
index 8e65ff470..7e3571d43 100644
--- a/test/box/net.box.test.lua
+++ b/test/box/net.box.test.lua
@@ -1574,6 +1574,29 @@ data = string.fromhex('3C'..string.rep(require('digest').sha1_hex('bcde'), 3))
 sock:write(data)
 sock:close()
 
+-- gh-1148: test stacked diagnostics.
+--
+test_run:cmd("push filter \"file: .*\" to \"file: <filename>\"")
+lua_code = [[function(tuple) local json = require('json') return json.encode(tuple) end]]
+box.schema.func.create('f1', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+s = box.schema.space.create('s')
+pk = s:create_index('pk')
+idx = s:create_index('idx', {func = box.func.f1.id, parts = {{1, 'string'}}})
+
+box.schema.user.grant('guest', 'read,write,execute', 'universe')
+c = net.connect(box.cfg.listen)
+f = function(...) return c.space.s:insert(...) end
+_, e = pcall(f, {1})
+assert(e ~= nil)
+
+e:unpack().message
+e.prev.message
+
+box.schema.user.revoke('guest', 'read,write,execute', 'universe')
+s:drop()
+box.func.f1:drop()
+test_run:cmd("clear filter")
+
 test_run:wait_log('default', 'Got a corrupted row.*', nil, 10)
 test_run:wait_log('default', '00000000:.*', nil, 10)
 test_run:wait_log('default', '00000010:.*', nil, 10)
-- 
2.17.1

Re: [Tarantool-patches] [PATCH v2 10/10] iproto: support error stacked diagnostic area

[Vladislav Shpilevoy]

Thanks for the patchset!

On future - it would be good to have diffs and answers on
my comments inlined. Otherwise I need to jump between the
new email, the old one, and the commit's diff to check what
and how was fixed. And I likely forget something.

See 10 comments below.

On 25/03/2020 02:43, Nikita Pettik wrote:
> This patch introduces support of stacked errors in IProto protocol and
> in net.box module.
> 
> @TarantoolBot document
> Title: Stacked error diagnostic area
> 
> Starting from now errors can be organized into lists. To achieve this
> Lua table representing error object is extended with .prev field and
> e:set_prev(err) method. .prev field returns previous error if any exist.
> e:set_prev(err) method expects err to be error object or nil and sets
> err as previous error of e. For instance:
> 
> e1 = box.error.new({code = 111, reason = "cause"})
> e2 = box.error.new({code = 111, reason = "cause of cause"})

1. I would wrap these code samples into ```. Otherwise you may
screw the whole ticket formatting accidentally. You can check how
will it look by clicking 'New issue', pasting the request, and
clicking 'Preview'.

Currently it looks not the best. For example,

    - error: 'builtin/error.lua: Cycles are not allowed'

is turned into a bullet point.

> e1:set_prev(e2)
> assert(e1.prev == e2) -- true
> 
> Cycles are not allowed for error lists:
> 
> e2:set_prev(e1)
> - error: 'builtin/error.lua: Cycles are not allowed'
> 
> Nil is valid input to :set_prev() method:
> 
> e1:set_prev(nil)
> assert(e1.prev == nil) -- true
> 
> Note that error can be 'previous' only to the one error at once:
> 
> e1:set_prev(e2)
> e3:set_prev(e2)
> assert(e1.prev == nil) -- true
> assert(e3.prev == e2) -- true
> 
> Setting previous error does not erase its own previous members:
> 
> -- e1 -> e2 -> e3 -> e4
> e1:set_prev(e2)
> e2:set_prev(e3)
> e3:set_prev(e4)
> e2:set_prev(e5)
> -- Now there are two lists: e1->e2->e5 and e3->e4
> assert(e1.prev == e2) -- true
> assert(e2.prev == e5) -- true
> assert(e3.prev == e4) -- true
> 
> Alternatively:
> 
> e1:set_prev(e2)
> e2:set_prev(e3)
> e3:set_prev(e4)
> e5:set_prev(e3)
> -- Now there are two lists: e1->e2 and e5->e3->e4
> assert(e1.prev == e2) -- true
> assert(e2.prev == nil) -- true
> assert(e5.prev == e3) -- true
> assert(e3.prev == e4) -- true
> 
> Stacked diagnostics is also supported by IProto protocol. Now responses
> containing errors always (even if there's only one error to be returned)
> include new IProto key: IPROTO_ERROR_STACK (0x51). So, body corresponding to
> error response now looks like:
> MAP{IPROTO_ERROR : string, IPROTO_ERROR_STACK : ARRAY[MAP{ERROR_CODE : uint, ERROR_MESSAGE : string}, MAP{...}, ...]}
> 
> where IPROTO_ERROR is 0x31 key, IPROTO_ERROR_STACK is 0x51, ERROR_CODE
> is 0x01 and ERROR_MESSAGE is 0x02.
> Instances of older versions (without support of stacked errors in
> protocol) simply ignore unknown keys and still rely only on IPROTO_ERROR
> key.
> 
> Closes #1148

2. Propose to move this to before the docbot request. So as not
to include it into the doc ticket.

> diff --git a/src/box/xrow.c b/src/box/xrow.c
> index 256dd4d91..ceb38b040 100644
> --- a/src/box/xrow.c
> +++ b/src/box/xrow.c
> @@ -1070,19 +1083,66 @@ xrow_encode_auth(struct xrow_header *packet, const char *salt, size_t salt_len,
>  	return 0;
>  }
>  
> +static int
> +iproto_decode_error_stack(const char **pos, const char *end)
> +{
> +	const char *reason = NULL;
> +	static_assert(TT_STATIC_BUF_LEN >= DIAG_ERRMSG_MAX, "static buffer is "\
> +		      "expected to be larger than error message max length");
> +	/*
> +	 * Erase previously set diag errors. It is required since
> +	 * box_error_add() does not replace previous errors.
> +	 */
> +	box_error_clear();
> +	if (mp_check_array(*pos, end) != 0)

3. mp_check() functions return > 0 on error.

When you call mp_check_array() without checking if it is
MP_ARRAY, it will crash. Also it will crash if *pos == end
in the checks below. But it does not matter anymore.

I noticed that we don't need MessagePack validness checking
here. I see now that mp_check() is called by some upper function
always. You need only check that MP_ types are correct. That it
is MP_ARRAY, keys are MP_UINT and so on.

I don't see any tests on that. Are there some? Maybe at least
unit tests, in xrow.cc? Although a proper test would try to
send an error stack through applier. Because xrow_decode_error()
is used only there.

> +		return -1;
> +	uint32_t stack_sz = mp_decode_array(pos);
> +	for (uint32_t i = 0; i < stack_sz; i++) {
> +		uint32_t code = 0;
> +		if (mp_typeof(**pos) != MP_MAP || mp_check_map(*pos, end) != 0)
> +			return -1;
> +		uint32_t map_sz = mp_decode_map(pos);
> +		for (uint32_t key_idx = 0; key_idx < map_sz; key_idx++) {
> +			if (mp_typeof(**pos) != MP_UINT ||
> +			    mp_check_uint(*pos, end) != 0)
> +				return -1;
> +			uint8_t key = mp_decode_uint(pos);
> +			if (key == IPROTO_ERROR_CODE) {
> +				if (mp_typeof(**pos) != MP_UINT ||
> +				    mp_check_uint(*pos, end) != 0)
> +					return -1;
> +				code = mp_decode_uint(pos);
> +			} else if (key == IPROTO_ERROR_MESSAGE) {
> +				if (mp_typeof(**pos) != MP_STR ||
> +				    mp_check_strl(*pos, end) != 0)
> +					return -1;
> +				uint32_t len;
> +				const char *str = mp_decode_str(pos, &len);
> +				reason = tt_cstr(str, len);
> +			} else {
> +				mp_next(pos);
> +				continue;
> +			}
> +			box_error_add(__FILE__, __LINE__, code, reason);
> +		}
> +	}
> +	return 0;
> +}
> diff --git a/test/box/iproto.result b/test/box/iproto.result
> new file mode 100644
> index 000000000..bb369be5a
> --- /dev/null
> +++ b/test/box/iproto.result
> @@ -0,0 +1,141 @@
> +test_run = require('test_run').new()

4. test_run objects seems to be not necessary here.

> +---
> +...
> +net_box = require('net.box')
> +---
> +...
> +urilib = require('uri')
> +---
> +...
> +msgpack = require('msgpack')
> +---
> +...
> +IPROTO_REQUEST_TYPE   = 0x00
> +---
> +...
> +IPROTO_INSERT         = 0x02
> +---
> +...
> +IPROTO_SYNC           = 0x01
> +---
> +...
> +IPROTO_SPACE_ID       = 0x10
> +---
> +...
> +IPROTO_TUPLE          = 0x21
> +---
> +...
> +IPROTO_ERROR          = 0x31
> +---
> +...
> +IPROTO_ERROR_STACK    = 0x52
> +---
> +...
> +IPROTO_ERROR_CODE     = 0x01
> +---
> +...
> +IPROTO_ERROR_MESSAGE  = 0x02
> +---
> +...
> +IPROTO_OK             = 0x00
> +---
> +...
> +IPROTO_SCHEMA_VERSION = 0x05
> +---
> +...
> +IPROTO_STATUS_KEY     = 0x00

5. The last 3 keys are unused.

> +---
> +...
> +-- gh-1148: test capabilities of stacked diagnostics bypassing net.box.
> +--
> +lua_func = [[function(tuple) local json = require('json') return json.encode(tuple) end]]
> +---
> +...
> +box.schema.func.create('f1', {body = lua_func, is_deterministic = true, is_sandboxed = true})
> +---
> +...
> +s = box.schema.space.create('s')
> +---
> +...
> +pk = s:create_index('pk')
> +---
> +...
> +idx = s:create_index('idx', {func = box.func.f1.id, parts = {{1, 'string'}}})
> +---
> +...
> +box.schema.user.grant('guest', 'read, write, execute', 'universe')
> +---
> +...
> +next_request_id = 16
> +---
> +...
> +header = { \
> +    [IPROTO_REQUEST_TYPE] = IPROTO_INSERT, \
> +    [IPROTO_SYNC]         = next_request_id, \
> +}
> +---
> +...
> +body = { \
> +    [IPROTO_SPACE_ID] = s.id, \
> +    [IPROTO_TUPLE]    = box.tuple.new({1}) \
> +}
> +---
> +...
> +uri = urilib.parse(box.cfg.listen)
> +---
> +...
> +sock = net_box.establish_connection(uri.host, uri.service)
> +---
> +...
> +response = iproto_request(sock, header, body)
> +---
> +...
> +sock:close()
> +---
> +- true
> +...
> +-- Both keys (obsolete and stack ones) are present in response.
> +--
> +assert(response.body[IPROTO_ERROR_STACK] ~= nil)
> +---
> +- true
> +...
> +assert(response.body[IPROTO_ERROR] ~= nil)
> +---
> +- true
> +...
> +err = response.body[IPROTO_ERROR_STACK][1]
> +---
> +...
> +assert(err[IPROTO_ERROR_MESSAGE] == body[IPROTO_ERROR])
> +---
> +- error: assertion failed!

6.Shouldn't it be true? You meant response.body, not just body?

> +...
> +err = response.body[IPROTO_ERROR_STACK][2]
> +---
> +...
> +assert(err[IPROTO_ERROR_CODE] ~= nil)
> +---
> +- true
> +...
> +assert(type(err[IPROTO_ERROR_CODE]) == 'number')
> +---
> +- true
> +...
> +assert(err[IPROTO_ERROR_MESSAGE] ~= nil)
> +---
> +- true
> +...
> +assert(type(err[IPROTO_ERROR_MESSAGE]) == 'string')

7. This test covers the previous assertion. type(nil)
is nil, so it won't be equal to 'string'.

> +---
> +- true
> +...
> +box.schema.user.revoke('guest', 'read,write,execute', 'universe')
> +---
> +...
> +s:drop()
> +---
> diff --git a/test/box/net.box.result b/test/box/net.box.result
> index e3dabf7d9..eeefc0e3f 100644
> --- a/test/box/net.box.result
> +++ b/test/box/net.box.result
> @@ -3902,6 +3902,66 @@ sock:close()
>  ---
>  - true
>  ...
> +-- gh-1148: test stacked diagnostics.
> +--
> +test_run:cmd("push filter \"file: .*\" to \"file: <filename>\"")

8. I removed the filter and nothing changed. Why do you
need it?

> +---
> +- true
> +...
> +lua_code = [[function(tuple) local json = require('json') return json.encode(tuple) end]]
> +---
> +...
> +box.schema.func.create('f1', {body = lua_code, is_deterministic = true, is_sandboxed = true})
> +---
> +...
> +s = box.schema.space.create('s')
> +---
> +...
> +pk = s:create_index('pk')
> +---
> +...
> +idx = s:create_index('idx', {func = box.func.f1.id, parts = {{1, 'string'}}})

9. Functional index is an overkill for such a simple test. You
may prefer creating a stored function, which creates a stack of 2
errors, and calls box.error(). That would be simpler IMO, and you can
test more than 2 errors too. The same for the iproto.test.lua. To get
that error you can use IPROTO_CALL. No need to create spaces, indexes,
bother with some 'determenistic sandboxed' shit. No even need to call
box.schema.func.create assuming you have 'universe execute' access.

> +---
> +...
> +box.schema.user.grant('guest', 'read,write,execute', 'universe')
> +---
> +...
> +c = net.connect(box.cfg.listen)
> +---
> +...
> +f = function(...) return c.space.s:insert(...) end
> +---
> +...
> +_, e = pcall(f, {1})
> +---
> +...
> +assert(e ~= nil)
> +---
> +- true
> +...
> +e:unpack().message
> +---
> +- 'Failed to build a key for functional index ''idx'' of space ''s'': can''t evaluate
> +  function'
> +...
> +e.prev.message
> +---
> +- error: '[string "return e.prev.message "]:1: attempt to index field ''prev'' (a
> +    nil value)'

10. Something is wrong here, no?

Re: [Tarantool-patches] [PATCH v2 10/10] iproto: support error stacked diagnostic area

[Nikita Pettik]

On 31 Mar 01:24, Vladislav Shpilevoy wrote:
> Thanks for the patchset!
> 
> On future - it would be good to have diffs and answers on
> my comments inlined. Otherwise I need to jump between the
> new email, the old one, and the commit's diff to check what
> and how was fixed. And I likely forget something.

It's quite complicated to extract certain diff while working
on whole patch-set and changes are big enough. Will try next time.
(All non-optional requested fixes are done; I answered explicitly
on skipped comments).
 
> See 10 comments below.
> 
> On 25/03/2020 02:43, Nikita Pettik wrote:
> > This patch introduces support of stacked errors in IProto protocol and
> > in net.box module.
> > 
> > @TarantoolBot document
> > Title: Stacked error diagnostic area
> > 
> > Starting from now errors can be organized into lists. To achieve this
> > Lua table representing error object is extended with .prev field and
> > e:set_prev(err) method. .prev field returns previous error if any exist.
> > e:set_prev(err) method expects err to be error object or nil and sets
> > err as previous error of e. For instance:
> > 
> > e1 = box.error.new({code = 111, reason = "cause"})
> > e2 = box.error.new({code = 111, reason = "cause of cause"})
> 
> 1. I would wrap these code samples into ```. Otherwise you may
> screw the whole ticket formatting accidentally. You can check how
> will it look by clicking 'New issue', pasting the request, and
> clicking 'Preview'.

Fixed.

> > 
> > Closes #1148
> 
> 2. Propose to move this to before the docbot request. So as not
> to include it into the doc ticket.

Fixed.
 
> > diff --git a/src/box/xrow.c b/src/box/xrow.c
> > index 256dd4d91..ceb38b040 100644
> > --- a/src/box/xrow.c
> > +++ b/src/box/xrow.c
> > @@ -1070,19 +1083,66 @@ xrow_encode_auth(struct xrow_header *packet, const char *salt, size_t salt_len,
> >  	return 0;
> >  }
> >  
> > +static int
> > +iproto_decode_error_stack(const char **pos, const char *end)
> > +{
> > +	const char *reason = NULL;
> > +	static_assert(TT_STATIC_BUF_LEN >= DIAG_ERRMSG_MAX, "static buffer is "\
> > +		      "expected to be larger than error message max length");
> > +	/*
> > +	 * Erase previously set diag errors. It is required since
> > +	 * box_error_add() does not replace previous errors.
> > +	 */
> > +	box_error_clear();
> > +	if (mp_check_array(*pos, end) != 0)
> 
> 3. mp_check() functions return > 0 on error.
> 
> When you call mp_check_array() without checking if it is
> MP_ARRAY, it will crash. Also it will crash if *pos == end
> in the checks below. But it does not matter anymore.
> 
> I noticed that we don't need MessagePack validness checking
> here. I see now that mp_check() is called by some upper function
> always. You need only check that MP_ types are correct. That it
> is MP_ARRAY, keys are MP_UINT and so on.

Fixed:

diff --git a/src/box/xrow.c b/src/box/xrow.c
index e84c6f758..596c765b7 100644
--- a/src/box/xrow.c
+++ b/src/box/xrow.c
@@ -1084,7 +1084,7 @@ xrow_encode_auth(struct xrow_header *packet, const char *salt, size_t salt_len,
 }
 
 static int
-iproto_decode_error_stack(const char **pos, const char *end)
+iproto_decode_error_stack(const char **pos)
 {
 	const char *reason = NULL;
 	static_assert(TT_STATIC_BUF_LEN >= DIAG_ERRMSG_MAX, "static buffer is "\
@@ -1099,22 +1099,19 @@ iproto_decode_error_stack(const char **pos, const char *end)
 	uint32_t stack_sz = mp_decode_array(pos);
 	for (uint32_t i = 0; i < stack_sz; i++) {
 		uint32_t code = 0;
-		if (mp_typeof(**pos) != MP_MAP || mp_check_map(*pos, end) != 0)
+		if (mp_typeof(**pos) != MP_MAP)
 			return -1;
 		uint32_t map_sz = mp_decode_map(pos);
 		for (uint32_t key_idx = 0; key_idx < map_sz; key_idx++) {
-			if (mp_typeof(**pos) != MP_UINT ||
-			    mp_check_uint(*pos, end) != 0)
+			if (mp_typeof(**pos) != MP_UINT)
 				return -1;
 			uint8_t key = mp_decode_uint(pos);
 			if (key == IPROTO_ERROR_CODE) {
-				if (mp_typeof(**pos) != MP_UINT ||
-				    mp_check_uint(*pos, end) != 0)
+				if (mp_typeof(**pos) != MP_UINT)
 					return -1;
 				code = mp_decode_uint(pos);
 			} else if (key == IPROTO_ERROR_MESSAGE) {
-				if (mp_typeof(**pos) != MP_STR ||
-				    mp_check_strl(*pos, end) != 0)
+				if (mp_typeof(**pos) != MP_STR)
 					return -1;
 				uint32_t len;
 				const char *str = mp_decode_str(pos, &len);
@@ -1169,7 +1166,7 @@ xrow_decode_error(struct xrow_header *row)
 		} else if (key == IPROTO_ERROR_STACK ) {
 			if (mp_check_array(pos, end) != 0)
 				goto error;
-			if (iproto_decode_error_stack(&pos, end) != 0)
+			if (iproto_decode_error_stack(&pos) != 0)
 				continue;
 		} else {
 			mp_next(&pos);

> I don't see any tests on that. Are there some? Maybe at least
> unit tests, in xrow.cc? Although a proper test would try to
> send an error stack through applier. Because xrow_decode_error()
> is used only there.

I'm a bit new to replication tests. I've been working on it.
Could you please review the rest of changes so far?

> > diff --git a/test/box/iproto.result b/test/box/iproto.result
> > new file mode 100644
> > index 000000000..bb369be5a
> > --- /dev/null
> > +++ b/test/box/iproto.result
> > @@ -0,0 +1,141 @@
> > +test_run = require('test_run').new()
> 
> 4. test_run objects seems to be not necessary here.

Reworked whole iproto.test.lua according to your suggestion
to use separate function setting error stack. See diff below.
 
> > +...
> > +IPROTO_ERROR_MESSAGE  = 0x02
> > +---
> > +...
> > +IPROTO_OK             = 0x00
> > +---
> > +...
> > +IPROTO_SCHEMA_VERSION = 0x05
> > +---
> > +...
> > +IPROTO_STATUS_KEY     = 0x00
> 
> 5. The last 3 keys are unused.

For sure. Removed.
 
> > +...
> > +assert(response.body[IPROTO_ERROR] ~= nil)
> > +---
> > +- true
> > +...
> > +err = response.body[IPROTO_ERROR_STACK][1]
> > +---
> > +...
> > +assert(err[IPROTO_ERROR_MESSAGE] == body[IPROTO_ERROR])
> > +---
> > +- error: assertion failed!
> 
> 6.Shouldn't it be true? You meant response.body, not just body?

Yep, fixed. Thx.
 
> > +---
> > +- true
> > +...
> > +assert(type(err[IPROTO_ERROR_MESSAGE]) == 'string')
> 
> 7. This test covers the previous assertion. type(nil)
> is nil, so it won't be equal to 'string'.

Didn't get it. It was not nil, but string containing message.
Nevermind, test is reworked.
 
> > diff --git a/test/box/net.box.result b/test/box/net.box.result
> > index e3dabf7d9..eeefc0e3f 100644
> > --- a/test/box/net.box.result
> > +++ b/test/box/net.box.result
> > @@ -3902,6 +3902,66 @@ sock:close()
> >  ---
> >  - true
> >  ...
> > +-- gh-1148: test stacked diagnostics.
> > +--
> > +test_run:cmd("push filter \"file: .*\" to \"file: <filename>\"")
> 
> 8. I removed the filter and nothing changed. Why do you
> need it?

This test has been reworked as well.
 
> 9. Functional index is an overkill for such a simple test. You
> may prefer creating a stored function, which creates a stack of 2
> errors, and calls box.error(). That would be simpler IMO, and you can
> test more than 2 errors too. The same for the iproto.test.lua. To get
> that error you can use IPROTO_CALL. No need to create spaces, indexes,
> bother with some 'determenistic sandboxed' shit. No even need to call
> box.schema.func.create assuming you have 'universe execute' access.


diff --git a/test/box/iproto.result b/test/box/iproto.result
index d6411d71a..b6dc7ed4c 100644
--- a/test/box/iproto.result
+++ b/test/box/iproto.result
@@ -7,16 +7,19 @@ urilib = require('uri')
 msgpack = require('msgpack')
 ---
 ...
-IPROTO_REQUEST_TYPE   = 0x00
+test_run = require('test_run').new()
 ---
 ...
-IPROTO_INSERT         = 0x02
+IPROTO_REQUEST_TYPE   = 0x00
 ---
 ...
 IPROTO_SYNC           = 0x01
 ---
 ...
-IPROTO_SPACE_ID       = 0x10
+IPROTO_CALL           = 0x0A
+---
+...
+IPROTO_FUNCTION_NAME  = 0x22
 ---
 ...
 IPROTO_TUPLE          = 0x21
@@ -36,20 +39,25 @@ IPROTO_ERROR_MESSAGE  = 0x02
 ...
 -- gh-1148: test capabilities of stacked diagnostics bypassing net.box.
 --
-lua_func = [[function(tuple) local json = require('json') return json.encode(tuple) end]]
----
-...
-box.schema.func.create('f1', {body = lua_func, is_deterministic = true, is_sandboxed = true})
----
-...
-s = box.schema.space.create('s')
+test_run:cmd("setopt delimiter ';'")
 ---
+- true
 ...
-pk = s:create_index('pk')
+stack_err = function()
+    local e1 = box.error.new({code = 111, reason = "e1"})
+    local e2 = box.error.new({code = 111, reason = "e2"})
+    local e3 = box.error.new({code = 111, reason = "e3"})
+    assert(e1 ~= nil)
+    e2:set_prev(e1)
+    assert(e2.prev == e1)
+    e3:set_prev(e2)
+    box.error(e3)
+end;
 ---
 ...
-idx = s:create_index('idx', {func = box.func.f1.id, parts = {{1, 'string'}}})
+test_run:cmd("setopt delimiter ''");
 ---
+- true
 ...
 box.schema.user.grant('guest', 'read, write, execute', 'universe')
 ---
@@ -58,14 +66,14 @@ next_request_id = 16
 ---
 ...
 header = { \
-    [IPROTO_REQUEST_TYPE] = IPROTO_INSERT, \
+    [IPROTO_REQUEST_TYPE] = IPROTO_CALL, \
     [IPROTO_SYNC]         = next_request_id, \
 }
 ---
 ...
 body = { \
-    [IPROTO_SPACE_ID] = s.id, \
-    [IPROTO_TUPLE]    = box.tuple.new({1}) \
+    [IPROTO_FUNCTION_NAME] = 'stack_err', \
+    [IPROTO_TUPLE]    = box.tuple.new({nil}) \
 }
 ---
 ...
@@ -99,31 +107,36 @@ assert(err[IPROTO_ERROR_MESSAGE] == response.body[IPROTO_ERROR])
 ---
 - true
 ...
-err = response.body[IPROTO_ERROR_STACK][2]
+assert(err[IPROTO_ERROR_MESSAGE] == 'e3')
 ---
+- true
 ...
-assert(err[IPROTO_ERROR_CODE] ~= nil)
+assert(err[IPROTO_ERROR_CODE] == 111)
 ---
 - true
 ...
-assert(type(err[IPROTO_ERROR_CODE]) == 'number')
+err = response.body[IPROTO_ERROR_STACK][2]
 ---
-- true
 ...
-assert(err[IPROTO_ERROR_MESSAGE] ~= nil)
+assert(err[IPROTO_ERROR_MESSAGE] == 'e2')
 ---
 - true
 ...
-assert(type(err[IPROTO_ERROR_MESSAGE]) == 'string')
+assert(err[IPROTO_ERROR_CODE] == 111)
 ---
 - true
 ...
-box.schema.user.revoke('guest', 'read,write,execute', 'universe')
+err = response.body[IPROTO_ERROR_STACK][3]
 ---
 ...
-s:drop()
+assert(err[IPROTO_ERROR_MESSAGE] == 'e1')
 ---
+- true
 ...
-box.func.f1:drop()
+assert(err[IPROTO_ERROR_CODE] == 111)
+---
+- true
+...
+box.schema.user.revoke('guest', 'read,write,execute', 'universe')
 ---
 ...
diff --git a/test/box/iproto.test.lua b/test/box/iproto.test.lua
index d0b55467b..6402a22ba 100644
--- a/test/box/iproto.test.lua
+++ b/test/box/iproto.test.lua
@@ -1,11 +1,13 @@
 net_box = require('net.box')
 urilib = require('uri')
 msgpack = require('msgpack')
+test_run = require('test_run').new()
 
 IPROTO_REQUEST_TYPE   = 0x00
-IPROTO_INSERT         = 0x02
+
 IPROTO_SYNC           = 0x01
-IPROTO_SPACE_ID       = 0x10
+IPROTO_CALL           = 0x0A
+IPROTO_FUNCTION_NAME  = 0x22
 IPROTO_TUPLE          = 0x21
 IPROTO_ERROR          = 0x31
 IPROTO_ERROR_STACK    = 0x52
@@ -14,24 +16,29 @@ IPROTO_ERROR_MESSAGE  = 0x02
 
 -- gh-1148: test capabilities of stacked diagnostics bypassing net.box.
 --
-lua_func = [[function(tuple) local json = require('json') return json.encode(tuple) end]]
-
-box.schema.func.create('f1', {body = lua_func, is_deterministic = true, is_sandboxed = true})
-s = box.schema.space.create('s')
-pk = s:create_index('pk')
-idx = s:create_index('idx', {func = box.func.f1.id, parts = {{1, 'string'}}})
-
+test_run:cmd("setopt delimiter ';'")
+stack_err = function()
+    local e1 = box.error.new({code = 111, reason = "e1"})
+    local e2 = box.error.new({code = 111, reason = "e2"})
+    local e3 = box.error.new({code = 111, reason = "e3"})
+    assert(e1 ~= nil)
+    e2:set_prev(e1)
+    assert(e2.prev == e1)
+    e3:set_prev(e2)
+    box.error(e3)
+end;
+test_run:cmd("setopt delimiter ''");
 box.schema.user.grant('guest', 'read, write, execute', 'universe')
 
 next_request_id = 16
 header = { \
-    [IPROTO_REQUEST_TYPE] = IPROTO_INSERT, \
+    [IPROTO_REQUEST_TYPE] = IPROTO_CALL, \
     [IPROTO_SYNC]         = next_request_id, \
 }
 
 body = { \
-    [IPROTO_SPACE_ID] = s.id, \
-    [IPROTO_TUPLE]    = box.tuple.new({1}) \
+    [IPROTO_FUNCTION_NAME] = 'stack_err', \
+    [IPROTO_TUPLE]    = box.tuple.new({nil}) \
 }
 
 uri = urilib.parse(box.cfg.listen)
@@ -47,12 +54,13 @@ assert(response.body[IPROTO_ERROR] ~= nil)
 
 err = response.body[IPROTO_ERROR_STACK][1]
 assert(err[IPROTO_ERROR_MESSAGE] == response.body[IPROTO_ERROR])
+assert(err[IPROTO_ERROR_MESSAGE] == 'e3')
+assert(err[IPROTO_ERROR_CODE] == 111)
 err = response.body[IPROTO_ERROR_STACK][2]
-assert(err[IPROTO_ERROR_CODE] ~= nil)
-assert(type(err[IPROTO_ERROR_CODE]) == 'number')
-assert(err[IPROTO_ERROR_MESSAGE] ~= nil)
-assert(type(err[IPROTO_ERROR_MESSAGE]) == 'string')
+assert(err[IPROTO_ERROR_MESSAGE] == 'e2')
+assert(err[IPROTO_ERROR_CODE] == 111)
+err = response.body[IPROTO_ERROR_STACK][3]
+assert(err[IPROTO_ERROR_MESSAGE] == 'e1')
+assert(err[IPROTO_ERROR_CODE] == 111)
 
 box.schema.user.revoke('guest', 'read,write,execute', 'universe')
-s:drop()
-box.func.f1:drop()
diff --git a/test/box/net.box.result b/test/box/net.box.result
index 764eef2c4..6475707ef 100644
--- a/test/box/net.box.result
+++ b/test/box/net.box.result
@@ -3904,63 +3904,74 @@ sock:close()
 ...
 -- gh-1148: test stacked diagnostics.
 --
-test_run:cmd("push filter \"file: .*\" to \"file: <filename>\"")
+test_run:cmd("setopt delimiter ';'")
 ---
 - true
 ...
-lua_code = [[function(tuple) local json = require('json') return json.encode(tuple) end]]
+stack_err = function()
+    local e1 = box.error.new({code = 111, reason = "e1"})
+    local e2 = box.error.new({code = 111, reason = "e2"})
+    local e3 = box.error.new({code = 111, reason = "e3"})
+    assert(e1 ~= nil)
+    e2:set_prev(e1)
+    assert(e2.prev == e1)
+    e3:set_prev(e2)
+    box.error(e3)
+end;
 ---
 ...
-box.schema.func.create('f1', {body = lua_code, is_deterministic = true, is_sandboxed = true})
+test_run:cmd("setopt delimiter ''");
 ---
+- true
 ...
-s = box.schema.space.create('s')
+box.schema.user.grant('guest', 'read,write,execute', 'universe')
 ---
 ...
-pk = s:create_index('pk')
+c = net.connect(box.cfg.listen)
 ---
 ...
-idx = s:create_index('idx', {func = box.func.f1.id, parts = {{1, 'string'}}})
+f = function(...) return c:call(...) end
 ---
 ...
-box.schema.user.grant('guest', 'read,write,execute', 'universe')
+r, e3 = pcall(f, 'stack_err')
 ---
 ...
-c = net.connect(box.cfg.listen)
+assert(r == false)
 ---
+- true
 ...
-f = function(...) return c.space.s:insert(...) end
+e3
 ---
+- e3
 ...
-_, e = pcall(f, {1})
+e2 = e3.prev
 ---
 ...
-assert(e ~= nil)
+assert(e2 ~= nil)
 ---
 - true
 ...
-e:unpack().message
+e2
 ---
-- 'Failed to build a key for functional index ''idx'' of space ''s'': can''t evaluate
-  function'
+- e2
 ...
-e.prev.message
+e1 = e2.prev
 ---
-- '[string "return function(tuple) local json = require(''..."]:1: attempt to call
-  global ''require'' (a nil value)'
 ...
-box.schema.user.revoke('guest', 'read,write,execute', 'universe')
+assert(e1 ~= nil)
 ---
+- true
 ...
-s:drop()
+e1
 ---
+- e1
 ...
-box.func.f1:drop()
+assert(e1.prev == nil)
 ---
+- true
 ...
-test_run:cmd("clear filter")
+box.schema.user.revoke('guest', 'read,write,execute', 'universe')
 ---
-- true
 ...
 test_run:wait_log('default', 'Got a corrupted row.*', nil, 10)
 ---
diff --git a/test/box/net.box.test.lua b/test/box/net.box.test.lua
index 7e3571d43..72a4207ee 100644
--- a/test/box/net.box.test.lua
+++ b/test/box/net.box.test.lua
@@ -1576,26 +1576,34 @@ sock:close()
 
 -- gh-1148: test stacked diagnostics.
 --
-test_run:cmd("push filter \"file: .*\" to \"file: <filename>\"")
-lua_code = [[function(tuple) local json = require('json') return json.encode(tuple) end]]
-box.schema.func.create('f1', {body = lua_code, is_deterministic = true, is_sandboxed = true})
-s = box.schema.space.create('s')
-pk = s:create_index('pk')
-idx = s:create_index('idx', {func = box.func.f1.id, parts = {{1, 'string'}}})
+test_run:cmd("setopt delimiter ';'")
+stack_err = function()
+    local e1 = box.error.new({code = 111, reason = "e1"})
+    local e2 = box.error.new({code = 111, reason = "e2"})
+    local e3 = box.error.new({code = 111, reason = "e3"})
+    assert(e1 ~= nil)
+    e2:set_prev(e1)
+    assert(e2.prev == e1)
+    e3:set_prev(e2)
+    box.error(e3)
+end;
+test_run:cmd("setopt delimiter ''");
 
 box.schema.user.grant('guest', 'read,write,execute', 'universe')
 c = net.connect(box.cfg.listen)
-f = function(...) return c.space.s:insert(...) end
-_, e = pcall(f, {1})
-assert(e ~= nil)
-
-e:unpack().message
-e.prev.message
+f = function(...) return c:call(...) end
+r, e3 = pcall(f, 'stack_err')
+assert(r == false)
+e3
+e2 = e3.prev
+assert(e2 ~= nil)
+e2
+e1 = e2.prev
+assert(e1 ~= nil)
+e1
+assert(e1.prev == nil)
 
 box.schema.user.revoke('guest', 'read,write,execute', 'universe')
-s:drop()
-box.func.f1:drop()
-test_run:cmd("clear filter")
 
 test_run:wait_log('default', 'Got a corrupted row.*', nil, 10)
 test_run:wait_log('default', '00000000:.*', nil, 10)

> > +---
> > +...
> > +box.schema.user.grant('guest', 'read,write,execute', 'universe')
> > +---
> > +...
> > +c = net.connect(box.cfg.listen)
> > +---
> > +...
> > +f = function(...) return c.space.s:insert(...) end
> > +---
> > +...
> > +_, e = pcall(f, {1})
> > +---
> > +...
> > +assert(e ~= nil)
> > +---
> > +- true
> > +...
> > +e:unpack().message
> > +---
> > +- 'Failed to build a key for functional index ''idx'' of space ''s'': can''t evaluate
> > +  function'
> > +...
> > +e.prev.message
> > +---
> > +- error: '[string "return e.prev.message "]:1: attempt to index field ''prev'' (a
> > +    nil value)'
> 
> 10. Something is wrong here, no?

Yes. After rebase on master branch, new IPROTO_ERROR_STACK code
has changed, but I forgot to updated net.box sources. Fixed.

Re: [Tarantool-patches] [PATCH v2 10/10] iproto: support error stacked diagnostic area

[Nikita Pettik]

On 01 Apr 16:26, Nikita Pettik wrote:
> @@ -1169,7 +1166,7 @@ xrow_decode_error(struct xrow_header *row)
>  		} else if (key == IPROTO_ERROR_STACK ) {
>  			if (mp_check_array(pos, end) != 0)
>  				goto error;
> -			if (iproto_decode_error_stack(&pos, end) != 0)
> +			if (iproto_decode_error_stack(&pos) != 0)
>  				continue;
>  		} else {
>  			mp_next(&pos);
> 
> > I don't see any tests on that. Are there some? Maybe at least
> > unit tests, in xrow.cc? Although a proper test would try to
> > send an error stack through applier. Because xrow_decode_error()
> > is used only there.
> 
> I'm a bit new to replication tests. I've been working on it.
> Could you please review the rest of changes so far?

Well, I've come up with following unit tests (and a bit more
fixes of xrow_decode_error()):

diff --git a/src/box/xrow.c b/src/box/xrow.c
index 596c765b7..f33226dc5 100644
--- a/src/box/xrow.c
+++ b/src/box/xrow.c
@@ -1094,8 +1094,7 @@ iproto_decode_error_stack(const char **pos)
 	 * box_error_add() does not replace previous errors.
 	 */
 	box_error_clear();
-	if (mp_typeof(**pos) != MP_ARRAY)
-		return -1;
+	assert(mp_typeof(**pos) == MP_ARRAY);
 	uint32_t stack_sz = mp_decode_array(pos);
 	for (uint32_t i = 0; i < stack_sz; i++) {
 		uint32_t code = 0;
@@ -1163,11 +1162,11 @@ xrow_decode_error(struct xrow_header *row)
 			const char *str = mp_decode_str(&pos, &len);
 			snprintf(error, sizeof(error), "%.*s", len, str);
 			box_error_set(__FILE__, __LINE__, code, error);
-		} else if (key == IPROTO_ERROR_STACK ) {
-			if (mp_check_array(pos, end) != 0)
+		} else if (key == IPROTO_ERROR_STACK) {
+			if (mp_typeof(*pos) != MP_ARRAY)
 				goto error;
 			if (iproto_decode_error_stack(&pos) != 0)
-				continue;
+				goto error;
 		} else {
 			mp_next(&pos);
 			continue;
diff --git a/test/unit/xrow.cc b/test/unit/xrow.cc
index 68a334239..954f22f16 100644
--- a/test/unit/xrow.cc
+++ b/test/unit/xrow.cc
@@ -241,6 +241,140 @@ test_xrow_header_encode_decode()
 	check_plan();
 }
 
+static char *
+error_stack_entry_encode(char *pos, const char *err_str)
+{
+	pos = mp_encode_map(pos, 2);
+	pos = mp_encode_uint(pos, IPROTO_ERROR_CODE);
+	pos = mp_encode_uint(pos, 159);
+	pos = mp_encode_uint(pos, IPROTO_ERROR_MESSAGE);
+	pos = mp_encode_str(pos, err_str, strlen(err_str));
+	return pos;
+}
+
+void
+test_xrow_error_stack_decode()
+{
+	plan(14);
+	char buffer[2048];
+	/*
+	 * To start with, let's test the simplest and obsolete
+	 * way of setting errors.
+	 */
+	char *pos = mp_encode_map(buffer, 1);
+	pos = mp_encode_uint(pos, IPROTO_ERROR);
+	pos = mp_encode_str(pos, "e1", strlen("e1"));
+
+	struct xrow_header header;
+	header.bodycnt = 666;
+	header.type = 159 | IPROTO_TYPE_ERROR;
+	header.body[0].iov_base = buffer;
+	header.body[0].iov_len = pos - buffer;
+
+	xrow_decode_error(&header);
+	struct error *last = diag_last_error(diag_get());
+	isnt(last, NULL, "xrow_decode succeed: diag has been set");
+	is(strcmp(last->errmsg, "e1"), 0,
+	   "xrow_decode succeed: error is parsed");
+
+	pos = mp_encode_map(buffer, 1);
+	pos = mp_encode_uint(pos, IPROTO_ERROR_STACK);
+	pos = mp_encode_array(pos, 2);
+	pos = error_stack_entry_encode(pos, "e1");
+	pos = error_stack_entry_encode(pos, "e2");
+	header.body[0].iov_base = buffer;
+	header.body[0].iov_len = pos - buffer;
+
+	diag_clear(diag_get());
+	xrow_decode_error(&header);
+	last = diag_last_error(diag_get());
+	isnt(last, NULL, "xrow_decode succeed: diag has been set");
+	is(strcmp(last->errmsg, "e2"), 0, "xrow_decode error stack suceed: "
+	   "e2 at the top of stack");
+	last = last->cause;
+	isnt(last, NULL, "xrow_decode succeed: 'cause' is present in stack")
+	is(strcmp(last->errmsg, "e1"), 0, "xrow_decode succeed: "
+	   "stack has been parsed");
+	/*
+	 * Let's try decode broken stack. Variations:
+	 * 1. Stack is not encoded as an array;
+	 * 2. Stack doesn't contain maps;
+	 * 3. Stack's map keys are not uints;
+	 * 4. Stack's map values have wrong types;
+	 * In all these cases diag_last should contain emty err.
+	 */
+	/* Stack is encoded as map. */
+	pos = mp_encode_map(buffer, 1);
+	pos = mp_encode_uint(pos, IPROTO_ERROR_STACK);
+	pos = mp_encode_map(pos, 1);
+	pos = error_stack_entry_encode(pos, "e1");
+	header.body[0].iov_base = buffer;
+	header.body[0].iov_len = pos - buffer;
+
+	diag_clear(diag_get());
+	xrow_decode_error(&header);
+	last = diag_last_error(diag_get());
+	isnt(last, NULL, "xrow_decode succeed: diag has been set");
+	is(strcmp(last->errmsg, ""), 0, "xrow_decode corrupted stack: "
+	   "stack contains map instead of array");
+
+	/* Stack doesn't containt map(s) - array instead. */
+	pos = mp_encode_map(buffer, 1);
+	pos = mp_encode_uint(pos, IPROTO_ERROR_STACK);
+	pos = mp_encode_array(pos, 2);
+	pos = mp_encode_uint(pos, IPROTO_ERROR_CODE);
+	pos = mp_encode_uint(pos, IPROTO_ERROR_MESSAGE);
+	header.body[0].iov_base = buffer;
+	header.body[0].iov_len = pos - buffer;
+
+	diag_clear(diag_get());
+	xrow_decode_error(&header);
+	last = diag_last_error(diag_get());
+	isnt(last, NULL, "xrow_decode succeed: diag has been set");
+	is(strcmp(last->errmsg, ""), 0, "xrow_decode corrupted stack: "
+	   "stack contains array values instead of maps");
+
+	/* Stack's map keys are not uints. */
+	pos = mp_encode_map(buffer, 1);
+	pos = mp_encode_uint(pos, IPROTO_ERROR_STACK);
+	pos = mp_encode_array(pos, 1);
+	pos = mp_encode_map(pos, 2);
+	pos = mp_encode_str(pos, "string instead of uint",
+			    strlen("string instead of uint"));
+	pos = mp_encode_uint(pos, IPROTO_ERROR_MESSAGE);
+	pos = mp_encode_uint(pos, IPROTO_ERROR_CODE);
+	pos = mp_encode_uint(pos, 159);
+	header.body[0].iov_base = buffer;
+	header.body[0].iov_len = pos - buffer;
+
+	diag_clear(diag_get());
+	xrow_decode_error(&header);
+	last = diag_last_error(diag_get());
+	isnt(last, NULL, "xrow_decode succeed: diag has been set");
+	is(strcmp(last->errmsg, ""), 0, "xrow_decode corrupted stack: "
+	   "stack's map keys are not uints");
+
+	/* Stack's map values have wrong types. */
+	pos = mp_encode_map(buffer, 1);
+	pos = mp_encode_uint(pos, IPROTO_ERROR_STACK);
+	pos = mp_encode_array(pos, 1);
+	pos = mp_encode_map(pos, 2);
+	pos = mp_encode_uint(pos, IPROTO_ERROR_CODE);
+	pos = mp_encode_uint(pos, 159);
+	pos = mp_encode_uint(pos, IPROTO_ERROR_MESSAGE);
+	pos = mp_encode_uint(pos, 666);
+	header.body[0].iov_base = buffer;
+	header.body[0].iov_len = pos - buffer;
+
+	diag_clear(diag_get());
+	xrow_decode_error(&header);
+	last = diag_last_error(diag_get());
+	isnt(last, NULL, "xrow_decode succeed: diag has been set");
+	is(strcmp(last->errmsg, ""), 0, "xrow_decode corrupted stack: "
+	   "stack's map wrong value type");
+	check_plan();
+}
+
 void
 test_request_str()
 {
@@ -279,13 +413,14 @@ main(void)
 {
 	memory_init();
 	fiber_init(fiber_c_invoke);
-	plan(3);
+	plan(4);
 
 	random_init();
 
 	test_iproto_constants();
 	test_greeting();
 	test_xrow_header_encode_decode();
+	test_xrow_error_stack_decode();
 	test_request_str();
 
 	random_free();
diff --git a/test/unit/xrow.result b/test/unit/xrow.result
index 5ee92ad7b..1d07915a4 100644
--- a/test/unit/xrow.result
+++ b/test/unit/xrow.result
@@ -1,4 +1,4 @@
-1..3
+1..4
     1..40
     ok 1 - round trip
     ok 2 - roundtrip.version_id
@@ -53,6 +53,22 @@ ok 1 - subtests
     ok 9 - decoded sync
     ok 10 - decoded bodycnt
 ok 2 - subtests
+    1..14
+    ok 1 - xrow_decode failed: no diag has been set
+    ok 2 - xrow_decode failed: wrong error is set
+    ok 3 - xrow_decode error stack failed: no diag has been set
+    ok 4 - xrow_decode error stack failed: e2 should come first
+    ok 5 - xrow_decode error stack failed: 'cause' is absent
+    ok 6 - xrow_decode error stack failed: wrong error is set
+    ok 7 - xrow_decode error stack failed: no diag has been set
+    ok 8 - xrow_decode corrupted stack: stack contains map instead of array
+    ok 9 - xrow_decode error stack failed: no diag has been set
+    ok 10 - xrow_decode corrupted stack: stack contains array values instead of maps
+    ok 11 - xrow_decode error stack failed: no diag has been set
+    ok 12 - xrow_decode corrupted stack: stack's map keys are not uints
+    ok 13 - xrow_decode error stack failed: no diag has been set
+    ok 14 - xrow_decode corrupted stack: stack's map wrong value type
+ok 3 - subtests
     1..1
     ok 1 - request_str
-ok 3 - subtests
+ok 4 - subtests
 

Re: [Tarantool-patches] [PATCH v2 10/10] iproto: support error stacked diagnostic area

[Vladislav Shpilevoy]

Thanks for the fixes!

See 6 comments below.

> diff --git a/src/box/xrow.c b/src/box/xrow.c
> index 28adbdc1f..f33226dc5 100644
> --- a/src/box/xrow.c
> +++ b/src/box/xrow.c
> @@ -1070,19 +1083,62 @@ xrow_encode_auth(struct xrow_header *packet, const char *salt, size_t salt_len,
>  	return 0;
>  }
>  
> +static int
> +iproto_decode_error_stack(const char **pos)
> +{
> +	const char *reason = NULL;
> +	static_assert(TT_STATIC_BUF_LEN >= DIAG_ERRMSG_MAX, "static buffer is "\
> +		      "expected to be larger than error message max length");
> +	/*
> +	 * Erase previously set diag errors. It is required since
> +	 * box_error_add() does not replace previous errors.
> +	 */
> +	box_error_clear();
> +	assert(mp_typeof(**pos) == MP_ARRAY);

1. Better keep this check here instead of xrow_decode_error(). It looks
strange, when iproto_decode_error_stack() validates everything except
the top header, which is somewhy expected to be validated by the
caller.

> +	uint32_t stack_sz = mp_decode_array(pos);
> +	for (uint32_t i = 0; i < stack_sz; i++) {
> +		uint32_t code = 0;
> +		if (mp_typeof(**pos) != MP_MAP)
> +			return -1;
> +		uint32_t map_sz = mp_decode_map(pos);
> +		for (uint32_t key_idx = 0; key_idx < map_sz; key_idx++) {
> +			if (mp_typeof(**pos) != MP_UINT)
> +				return -1;
> +			uint8_t key = mp_decode_uint(pos);

2. If I will send value 0xffffff01, it will be considered valid.
Better save it into uint64_t, since mp_decode_uint() returns this
type. In that case all will be fine.

The same with code. Better get the code as uint64_t and compare
with UINT32_MAX. Because ClientError has uint32_t error code type.

> +			if (key == IPROTO_ERROR_CODE) {
> +				if (mp_typeof(**pos) != MP_UINT)
> +					return -1;
> +				code = mp_decode_uint(pos);
> +			} else if (key == IPROTO_ERROR_MESSAGE) {
> +				if (mp_typeof(**pos) != MP_STR)
> +					return -1;
> +				uint32_t len;
> +				const char *str = mp_decode_str(pos, &len);
> +				reason = tt_cstr(str, len);
> +			} else {
> +				mp_next(pos);
> +				continue;
> +			}
> +			box_error_add(__FILE__, __LINE__, code, reason);
> +		}
> +	}
> +	return 0;
> +}
> +
>  void
>  xrow_decode_error(struct xrow_header *row)
>  {
>  	uint32_t code = row->type & (IPROTO_TYPE_ERROR - 1);
>  
>  	char error[DIAG_ERRMSG_MAX] = { 0 };
> -	const char *pos;
> +	const char *pos, *end;
>  	uint32_t map_size;
>  
>  	if (row->bodycnt == 0)
>  		goto error;
>  	pos = (char *) row->body[0].iov_base;
> -	if (mp_check(&pos, pos + row->body[0].iov_len))
> +	end = pos + row->body[0].iov_len;
> +	if (mp_check(&pos, end))
>  		goto error;

3. This hunk is clearly an artifact from the old patch
version. Dropped it and nothing changed.

>  
>  	pos = (char *) row->body[0].iov_base;
> diff --git a/test/box/iproto.result b/test/box/iproto.result
> new file mode 100644
> index 000000000..b6dc7ed4c
> --- /dev/null
> +++ b/test/box/iproto.result
> @@ -0,0 +1,142 @@
> +---
> +...
> +-- gh-1148: test capabilities of stacked diagnostics bypassing net.box.
> +--
> +test_run:cmd("setopt delimiter ';'")
> +---
> +- true
> +...
> +stack_err = function()
> +    local e1 = box.error.new({code = 111, reason = "e1"})
> +    local e2 = box.error.new({code = 111, reason = "e2"})
> +    local e3 = box.error.new({code = 111, reason = "e3"})
> +    assert(e1 ~= nil)
> +    e2:set_prev(e1)
> +    assert(e2.prev == e1)
> +    e3:set_prev(e2)
> +    box.error(e3)
> +end;
> +---
> +...
> +test_run:cmd("setopt delimiter ''");
> +---
> +- true
> +...
> +box.schema.user.grant('guest', 'read, write, execute', 'universe')

4. Useful command, easy to remember:

    box.schema.user.grant/revoke('guest', 'super')

Is shorter, and has basically the same effect + a few more permissions.
In case you like me look for this 'grant' command in other test files to
copy paste it every time. Up to you what to use. JFYI.

> diff --git a/test/unit/xrow.cc b/test/unit/xrow.cc
> index 68a334239..954f22f16 100644
> --- a/test/unit/xrow.cc
> +++ b/test/unit/xrow.cc

5. This test crashes on travis and locally on my machine. Probably
a typo somewhere.

I also added a test for the uint64 vs uint8 result of mp_decode_uint().
See it below and pushed on top of the branch as a separate commit.

====================
diff --git a/test/unit/xrow.cc b/test/unit/xrow.cc
index 954f22f16..1449d49a5 100644
--- a/test/unit/xrow.cc
+++ b/test/unit/xrow.cc
@@ -255,7 +255,7 @@ error_stack_entry_encode(char *pos, const char *err_str)
 void
 test_xrow_error_stack_decode()
 {
-	plan(14);
+	plan(15);
 	char buffer[2048];
 	/*
 	 * To start with, let's test the simplest and obsolete
@@ -372,6 +372,24 @@ test_xrow_error_stack_decode()
 	isnt(last, NULL, "xrow_decode succeed: diag has been set");
 	is(strcmp(last->errmsg, ""), 0, "xrow_decode corrupted stack: "
 	   "stack's map wrong value type");
+
+	/* Bad key in the packet. */
+	diag_clear(diag_get());
+	pos = mp_encode_map(buffer, 1);
+	pos = mp_encode_uint(pos, IPROTO_ERROR_STACK);
+	pos = mp_encode_array(pos, 1);
+	pos = mp_encode_map(pos, 2);
+	pos = mp_encode_uint(pos, 0xff000000 | IPROTO_ERROR_CODE);
+	pos = mp_encode_uint(pos, 159);
+	pos = mp_encode_uint(pos, IPROTO_ERROR_MESSAGE);
+	pos = mp_encode_str(pos, "", 0);
+	header.body[0].iov_base = buffer;
+	header.body[0].iov_len = pos - buffer;
+	xrow_decode_error(&header);
+	ok(diag_is_empty(diag_get()),
+	   "xrow_decode failed: bad error object key");
+	last = diag_last_error(diag_get());
+
 	check_plan();
 }
 ====================

On 02/04/2020 00:24, Nikita Pettik wrote:
> On 01 Apr 16:26, Nikita Pettik wrote:
>> @@ -1169,7 +1166,7 @@ xrow_decode_error(struct xrow_header *row)
>>  		} else if (key == IPROTO_ERROR_STACK ) {
>>  			if (mp_check_array(pos, end) != 0)
>>  				goto error;
>> -			if (iproto_decode_error_stack(&pos, end) != 0)
>> +			if (iproto_decode_error_stack(&pos) != 0)
>>  				continue;
>>  		} else {
>>  			mp_next(&pos);
>>
>>> I don't see any tests on that. Are there some? Maybe at least
>>> unit tests, in xrow.cc? Although a proper test would try to
>>> send an error stack through applier. Because xrow_decode_error()
>>> is used only there.
>>
>> I'm a bit new to replication tests. I've been working on it.
>> Could you please review the rest of changes so far?

6. I tried to find how to send a stacked error to applier. This
should be done from relay. And seems like it is impossible to do
from relay. Simply because there are no any user defined code to
create stacked errors, and because internals of relay don't use
stacked errors so far. Probably an error injection could help to
create an artificial error, but I think the unit test should be
enough.

Re: [Tarantool-patches] [PATCH v2 10/10] iproto: support error stacked diagnostic area

[Nikita Pettik]

On 02 Apr 02:29, Vladislav Shpilevoy wrote:
> Thanks for the fixes!
> 
> See 6 comments below.
> 
> > diff --git a/src/box/xrow.c b/src/box/xrow.c
> > index 28adbdc1f..f33226dc5 100644
> > --- a/src/box/xrow.c
> > +++ b/src/box/xrow.c
> > @@ -1070,19 +1083,62 @@ xrow_encode_auth(struct xrow_header *packet, const char *salt, size_t salt_len,
> >  	return 0;
> >  }
> >  
> > +static int
> > +iproto_decode_error_stack(const char **pos)
> > +{
> > +	const char *reason = NULL;
> > +	static_assert(TT_STATIC_BUF_LEN >= DIAG_ERRMSG_MAX, "static buffer is "\
> > +		      "expected to be larger than error message max length");
> > +	/*
> > +	 * Erase previously set diag errors. It is required since
> > +	 * box_error_add() does not replace previous errors.
> > +	 */
> > +	box_error_clear();
> > +	assert(mp_typeof(**pos) == MP_ARRAY);
> 
> 1. Better keep this check here instead of xrow_decode_error(). It looks
> strange, when iproto_decode_error_stack() validates everything except
> the top header, which is somewhy expected to be validated by the
> caller.

Ok:

diff --git a/src/box/xrow.c b/src/box/xrow.c
index f33226dc5..680f9329a 100644
--- a/src/box/xrow.c
+++ b/src/box/xrow.c
@@ -1094,7 +1094,8 @@ iproto_decode_error_stack(const char **pos)
         * box_error_add() does not replace previous errors.
         */
        box_error_clear();
-       assert(mp_typeof(**pos) == MP_ARRAY);
+       if (mp_typeof(**pos) != MP_ARRAY)
+               return -1;
        uint32_t stack_sz = mp_decode_array(pos);
        for (uint32_t i = 0; i < stack_sz; i++) {
                uint32_t code = 0;
@@ -1163,8 +1164,6 @@ xrow_decode_error(struct xrow_header *row)
                        snprintf(error, sizeof(error), "%.*s", len, str);
                        box_error_set(__FILE__, __LINE__, code, error);
                } else if (key == IPROTO_ERROR_STACK) {
-                       if (mp_typeof(*pos) != MP_ARRAY)
-                               goto error;
                        if (iproto_decode_error_stack(&pos) != 0)
                                goto error;
                } else {

 
> > +	uint32_t stack_sz = mp_decode_array(pos);
> > +	for (uint32_t i = 0; i < stack_sz; i++) {
> > +		uint32_t code = 0;
> > +		if (mp_typeof(**pos) != MP_MAP)
> > +			return -1;
> > +		uint32_t map_sz = mp_decode_map(pos);
> > +		for (uint32_t key_idx = 0; key_idx < map_sz; key_idx++) {
> > +			if (mp_typeof(**pos) != MP_UINT)
> > +				return -1;
> > +			uint8_t key = mp_decode_uint(pos);
> 
> 2. If I will send value 0xffffff01, it will be considered valid.
> Better save it into uint64_t, since mp_decode_uint() returns this
> type. In that case all will be fine.
> 
> The same with code. Better get the code as uint64_t and compare
> with UINT32_MAX. Because ClientError has uint32_t error code type.

diff --git a/src/box/xrow.c b/src/box/xrow.c
index f33226dc5..323dc8f4d 100644
--- a/src/box/xrow.c
+++ b/src/box/xrow.c
@@ -1094,17 +1094,18 @@ iproto_decode_error_stack(const char **pos)
        uint32_t stack_sz = mp_decode_array(pos);
        for (uint32_t i = 0; i < stack_sz; i++) {
-               uint32_t code = 0;
+               uint64_t code = 0;
                if (mp_typeof(**pos) != MP_MAP)
                        return -1;
                uint32_t map_sz = mp_decode_map(pos);
                for (uint32_t key_idx = 0; key_idx < map_sz; key_idx++) {
                        if (mp_typeof(**pos) != MP_UINT)
                                return -1;
-                       uint8_t key = mp_decode_uint(pos);
+                       uint32_t key = mp_decode_uint(pos);
                        if (key == IPROTO_ERROR_CODE) {
                                if (mp_typeof(**pos) != MP_UINT)
                                        return -1;


> >  void
> >  xrow_decode_error(struct xrow_header *row)
> >  {
> >  	uint32_t code = row->type & (IPROTO_TYPE_ERROR - 1);
> >  
> >  	char error[DIAG_ERRMSG_MAX] = { 0 };
> > -	const char *pos;
> > +	const char *pos, *end;
> >  	uint32_t map_size;
> >  
> >  	if (row->bodycnt == 0)
> >  		goto error;
> >  	pos = (char *) row->body[0].iov_base;
> > -	if (mp_check(&pos, pos + row->body[0].iov_len))
> > +	end = pos + row->body[0].iov_len;
> > +	if (mp_check(&pos, end))
> >  		goto error;
> 
> 3. This hunk is clearly an artifact from the old patch
> version. Dropped it and nothing changed.

Yes, sorry. Reverted change:

@@ -1131,14 +1132,13 @@ xrow_decode_error(struct xrow_header *row)
        uint32_t code = row->type & (IPROTO_TYPE_ERROR - 1);
 
        char error[DIAG_ERRMSG_MAX] = { 0 };
-       const char *pos, *end;
+       const char *pos;
        uint32_t map_size;
 
        if (row->bodycnt == 0)
                goto error;
        pos = (char *) row->body[0].iov_base;
-       end = pos + row->body[0].iov_len;
-       if (mp_check(&pos, end))
+       if (mp_check(&pos, pos + row->body[0].iov_len))
                goto error;
 
        pos = (char *) row->body[0].iov_base;

> > diff --git a/test/box/iproto.result b/test/box/iproto.result
> > new file mode 100644
> > index 000000000..b6dc7ed4c
> > --- /dev/null
> > +++ b/test/box/iproto.result
> > @@ -0,0 +1,142 @@
> > +---
> > +...
> > +-- gh-1148: test capabilities of stacked diagnostics bypassing net.box.
> > +--
> > +test_run:cmd("setopt delimiter ';'")
> > +---
> > +- true
> > +...
> > +stack_err = function()
> > +    local e1 = box.error.new({code = 111, reason = "e1"})
> > +    local e2 = box.error.new({code = 111, reason = "e2"})
> > +    local e3 = box.error.new({code = 111, reason = "e3"})
> > +    assert(e1 ~= nil)
> > +    e2:set_prev(e1)
> > +    assert(e2.prev == e1)
> > +    e3:set_prev(e2)
> > +    box.error(e3)
> > +end;
> > +---
> > +...
> > +test_run:cmd("setopt delimiter ''");
> > +---
> > +- true
> > +...
> > +box.schema.user.grant('guest', 'read, write, execute', 'universe')
> 
> 4. Useful command, easy to remember:
> 
>     box.schema.user.grant/revoke('guest', 'super')
> 
> Is shorter, and has basically the same effect + a few more permissions.
> In case you like me look for this 'grant' command in other test files to
> copy paste it every time. Up to you what to use. JFYI.

Thx, will keep it in mind.
 
> > diff --git a/test/unit/xrow.cc b/test/unit/xrow.cc
> > index 68a334239..954f22f16 100644
> > --- a/test/unit/xrow.cc
> > +++ b/test/unit/xrow.cc
> 
> 5. This test crashes on travis and locally on my machine. Probably
> a typo somewhere.

Yep, accidentally forgot to update result file. Fixed.

> I also added a test for the uint64 vs uint8 result of mp_decode_uint().
> See it below and pushed on top of the branch as a separate commit.

Nice, thanks, squashed with some changes:

@@ -372,6 +375,26 @@ test_xrow_error_stack_decode()
        isnt(last, NULL, "xrow_decode succeed: diag has been set");
        is(strcmp(last->errmsg, ""), 0, "xrow_decode corrupted stack: "
           "stack's map wrong value type");
+
+       /* Bad key in the packet. */
+       pos = mp_encode_map(buffer, 1);
+       pos = mp_encode_uint(pos, IPROTO_ERROR_STACK);
+       pos = mp_encode_array(pos, 1);
+       pos = mp_encode_map(pos, 2);
+       pos = mp_encode_uint(pos, 0xff000000 | IPROTO_ERROR_CODE);
+       pos = mp_encode_uint(pos, 159);
+       pos = mp_encode_uint(pos, IPROTO_ERROR_MESSAGE);
+       pos = mp_encode_str(pos, "test msg", strlen("test msg"));
+       header.body[0].iov_base = buffer;
+       header.body[0].iov_len = pos - buffer;
+
+       diag_clear(diag_get());
+       xrow_decode_error(&header);
+       last = diag_last_error(diag_get());
+       isnt(last, NULL, "xrow_decode succeed: diag has been set");
+       is(strcmp(last->errmsg, "test msg"), 0, "xrow_decode corrupted stack: "
+          "stack's map wrong key");
+
        check_plan();
 }

Diag won't be empty since error will be set anyway - with default
(i.e. wrong) error code (0), but correct message.

I've also found one more bug: box_error_add() was called in the wrong
place. As a result, number of errors in stack might be doubled:

@@ -1119,8 +1120,8 @@ iproto_decode_error_stack(const char **pos)
                                mp_next(pos);
                                continue;
                        }
-                       box_error_add(__FILE__, __LINE__, code, reason);
                }
+               box_error_add(__FILE__, __LINE__, code, reason);
        }
        return 0;
 }

Added another one test case covering this change:

@@ -295,12 +295,15 @@ test_xrow_error_stack_decode()
        isnt(last, NULL, "xrow_decode succeed: 'cause' is present in stack")
        is(strcmp(last->errmsg, "e1"), 0, "xrow_decode succeed: "
           "stack has been parsed");
+       last = last->cause;
+       is(last, NULL, "xrow_decode succeed: only two errors in the stack")
        /*
         * Let's try decode broken stack. Variations:
 
> ====================
> diff --git a/test/unit/xrow.cc b/test/unit/xrow.cc
> index 954f22f16..1449d49a5 100644
> --- a/test/unit/xrow.cc
> +++ b/test/unit/xrow.cc
> @@ -255,7 +255,7 @@ error_stack_entry_encode(char *pos, const char *err_str)
>  void
>  test_xrow_error_stack_decode()
>  {
> -	plan(14);
> +	plan(15);
>  	char buffer[2048];
>  	/*
>  	 * To start with, let's test the simplest and obsolete
> @@ -372,6 +372,24 @@ test_xrow_error_stack_decode()
>  	isnt(last, NULL, "xrow_decode succeed: diag has been set");
>  	is(strcmp(last->errmsg, ""), 0, "xrow_decode corrupted stack: "
>  	   "stack's map wrong value type");
> +
> +	/* Bad key in the packet. */
> +	diag_clear(diag_get());
> +	pos = mp_encode_map(buffer, 1);
> +	pos = mp_encode_uint(pos, IPROTO_ERROR_STACK);
> +	pos = mp_encode_array(pos, 1);
> +	pos = mp_encode_map(pos, 2);
> +	pos = mp_encode_uint(pos, 0xff000000 | IPROTO_ERROR_CODE);
> +	pos = mp_encode_uint(pos, 159);
> +	pos = mp_encode_uint(pos, IPROTO_ERROR_MESSAGE);
> +	pos = mp_encode_str(pos, "", 0);
> +	header.body[0].iov_base = buffer;
> +	header.body[0].iov_len = pos - buffer;
> +	xrow_decode_error(&header);
> +	ok(diag_is_empty(diag_get()),
> +	   "xrow_decode failed: bad error object key");
> +	last = diag_last_error(diag_get());
> +
>  	check_plan();
>  }
>  ====================
> 
> On 02/04/2020 00:24, Nikita Pettik wrote:
> > On 01 Apr 16:26, Nikita Pettik wrote:
> >> @@ -1169,7 +1166,7 @@ xrow_decode_error(struct xrow_header *row)
> >>  		} else if (key == IPROTO_ERROR_STACK ) {
> >>  			if (mp_check_array(pos, end) != 0)
> >>  				goto error;
> >> -			if (iproto_decode_error_stack(&pos, end) != 0)
> >> +			if (iproto_decode_error_stack(&pos) != 0)
> >>  				continue;
> >>  		} else {
> >>  			mp_next(&pos);
> >>
> >>> I don't see any tests on that. Are there some? Maybe at least
> >>> unit tests, in xrow.cc? Although a proper test would try to
> >>> send an error stack through applier. Because xrow_decode_error()
> >>> is used only there.
> >>
> >> I'm a bit new to replication tests. I've been working on it.
> >> Could you please review the rest of changes so far?
> 
> 6. I tried to find how to send a stacked error to applier. This
> should be done from relay. And seems like it is impossible to do
> from relay. Simply because there are no any user defined code to
> create stacked errors, and because internals of relay don't use
> stacked errors so far. Probably an error injection could help to
> create an artificial error, but I think the unit test should be
> enough.

Re: [Tarantool-patches] [PATCH v2 10/10] iproto: support error stacked diagnostic area

[Vladislav Shpilevoy]

Thanks for the fixes!

See 2 comments below.

> diff --git a/src/box/error.cc b/src/box/error.cc
> index 8e77c2e9e..897aa9261 100644
> --- a/src/box/error.cc
> +++ b/src/box/error.cc
> @@ -102,6 +102,23 @@ box_error_new(const char *file, unsigned line, uint32_t code,
>  	return e;
>  }
>  
> +int
> +box_error_add(const char *file, unsigned line, uint32_t code,
> +	      const char *fmt, ...)
> +{
> +	struct error *e = BuildClientError(file, line, ER_UNKNOWN);
> +	ClientError *client_error = type_cast(ClientError, e);
> +	if (client_error) {
> +		client_error->m_errcode = code;
> +		va_list ap;
> +		va_start(ap, fmt);
> +		error_vformat_msg(e, fmt, ap);
> +		va_end(ap);
> +	}
> +	diag_add_error(&fiber()->diag, e);
> +	return -1;

1. Why do we return -1 instead of the new error object?
box_error_new() returns an error, and this function does not.
Seems inconsistent.

>>> diff --git a/test/unit/xrow.cc b/test/unit/xrow.cc
>>> index 68a334239..954f22f16 100644
>>> --- a/test/unit/xrow.cc
>>> +++ b/test/unit/xrow.cc
> @@ -372,6 +375,26 @@ test_xrow_error_stack_decode()
>         isnt(last, NULL, "xrow_decode succeed: diag has been set");
>         is(strcmp(last->errmsg, ""), 0, "xrow_decode corrupted stack: "
>            "stack's map wrong value type");
> +
> +       /* Bad key in the packet. */
> +       pos = mp_encode_map(buffer, 1);
> +       pos = mp_encode_uint(pos, IPROTO_ERROR_STACK);
> +       pos = mp_encode_array(pos, 1);
> +       pos = mp_encode_map(pos, 2);
> +       pos = mp_encode_uint(pos, 0xff000000 | IPROTO_ERROR_CODE);
> +       pos = mp_encode_uint(pos, 159);
> +       pos = mp_encode_uint(pos, IPROTO_ERROR_MESSAGE);
> +       pos = mp_encode_str(pos, "test msg", strlen("test msg"));
> +       header.body[0].iov_base = buffer;
> +       header.body[0].iov_len = pos - buffer;
> +
> +       diag_clear(diag_get());
> +       xrow_decode_error(&header);
> +       last = diag_last_error(diag_get());
> +       isnt(last, NULL, "xrow_decode succeed: diag has been set");
> +       is(strcmp(last->errmsg, "test msg"), 0, "xrow_decode corrupted stack: "
> +          "stack's map wrong key");
> +
>         check_plan();
>  }
> 
> Diag won't be empty since error will be set anyway - with default
> (i.e. wrong) error code (0), but correct message.

2. I added box_error_code() check to ensure this.

But more importantly that the original bug I was referring to
still is here. About overflows and integer truncation. I fixed it
and added tests. See them below and on the branch in a separate
commit.

====================
diff --git a/src/box/xrow.c b/src/box/xrow.c
index 9d30bcaf9..be026a43c 100644
--- a/src/box/xrow.c
+++ b/src/box/xrow.c
@@ -1105,11 +1105,13 @@ iproto_decode_error_stack(const char **pos)
 		for (uint32_t key_idx = 0; key_idx < map_sz; key_idx++) {
 			if (mp_typeof(**pos) != MP_UINT)
 				return -1;
-			uint32_t key = mp_decode_uint(pos);
+			uint64_t key = mp_decode_uint(pos);
 			if (key == IPROTO_ERROR_CODE) {
 				if (mp_typeof(**pos) != MP_UINT)
 					return -1;
 				code = mp_decode_uint(pos);
+				if (code > UINT32_MAX)
+					return -1;
 			} else if (key == IPROTO_ERROR_MESSAGE) {
 				if (mp_typeof(**pos) != MP_STR)
 					return -1;
diff --git a/test/unit/xrow.cc b/test/unit/xrow.cc
index ae45f18b0..718ef15a2 100644
--- a/test/unit/xrow.cc
+++ b/test/unit/xrow.cc
@@ -32,6 +32,7 @@ extern "C" {
 #include "unit.h"
 } /* extern "C" */
 #include "trivia/util.h"
+#include "box/error.h"
 #include "box/xrow.h"
 #include "box/iproto_constants.h"
 #include "uuid/tt_uuid.h"
@@ -255,7 +256,7 @@ error_stack_entry_encode(char *pos, const char *err_str)
 void
 test_xrow_error_stack_decode()
 {
-	plan(17);
+	plan(24);
 	char buffer[2048];
 	/*
 	 * To start with, let's test the simplest and obsolete
@@ -392,9 +393,51 @@ test_xrow_error_stack_decode()
 	xrow_decode_error(&header);
 	last = diag_last_error(diag_get());
 	isnt(last, NULL, "xrow_decode succeed: diag has been set");
+	is(box_error_code(last), 0, "xrow_decode last error code is default 0");
 	is(strcmp(last->errmsg, "test msg"), 0, "xrow_decode corrupted stack: "
 	   "stack's map wrong key");
 
+	/* Overflow error code. */
+	pos = mp_encode_map(buffer, 1);
+	pos = mp_encode_uint(pos, IPROTO_ERROR_STACK);
+	pos = mp_encode_array(pos, 1);
+	pos = mp_encode_map(pos, 2);
+	pos = mp_encode_uint(pos, IPROTO_ERROR_CODE);
+	pos = mp_encode_uint(pos, (uint64_t)1 << 40);
+	pos = mp_encode_uint(pos, IPROTO_ERROR_MESSAGE);
+	pos = mp_encode_str(pos, "test msg", strlen("test msg"));
+	header.body[0].iov_base = buffer;
+	header.body[0].iov_len = pos - buffer;
+
+	diag_clear(diag_get());
+	xrow_decode_error(&header);
+	last = diag_last_error(diag_get());
+	isnt(last, NULL, "xrow_decode succeed: diag has been set");
+	is(box_error_code(last), 159, "xrow_decode failed, took code from "
+	   "header");
+	is(strcmp(last->errmsg, ""), 0, "xrow_decode failed, message is not "
+	   "decoded");
+
+	/* Overflow error key. */
+	pos = mp_encode_map(buffer, 1);
+	pos = mp_encode_uint(pos, IPROTO_ERROR_STACK);
+	pos = mp_encode_array(pos, 1);
+	pos = mp_encode_map(pos, 2);
+	pos = mp_encode_uint(pos, 0xff00000000 | IPROTO_ERROR_CODE);
+	pos = mp_encode_uint(pos, 159);
+	pos = mp_encode_uint(pos, IPROTO_ERROR_MESSAGE);
+	pos = mp_encode_str(pos, "test msg", strlen("test msg"));
+	header.body[0].iov_base = buffer;
+	header.body[0].iov_len = pos - buffer;
+
+	diag_clear(diag_get());
+	xrow_decode_error(&header);
+	last = diag_last_error(diag_get());
+	isnt(last, NULL, "xrow_decode succeed: diag has been set");
+	is(box_error_code(last), 0, "xrow_decode last error code is default 0");
+	is(strcmp(last->errmsg, "test msg"), 0, "xrow_decode message is fine "
+	   "even without the code");
+
 	check_plan();
 }
 
diff --git a/test/unit/xrow.result b/test/unit/xrow.result
index d24e9ea4f..7213ab6c7 100644
--- a/test/unit/xrow.result
+++ b/test/unit/xrow.result
@@ -53,7 +53,7 @@ ok 1 - subtests
     ok 9 - decoded sync
     ok 10 - decoded bodycnt
 ok 2 - subtests
-    1..17
+    1..24
     ok 1 - xrow_decode succeed: diag has been set
     ok 2 - xrow_decode succeed: error is parsed
     ok 3 - xrow_decode succeed: diag has been set
@@ -70,7 +70,14 @@ ok 2 - subtests
     ok 14 - xrow_decode succeed: diag has been set
     ok 15 - xrow_decode corrupted stack: stack's map wrong value type
     ok 16 - xrow_decode succeed: diag has been set
-    ok 17 - xrow_decode corrupted stack: stack's map wrong key
+    ok 17 - xrow_decode last error code is default 0
+    ok 18 - xrow_decode corrupted stack: stack's map wrong key
+    ok 19 - xrow_decode succeed: diag has been set
+    ok 20 - xrow_decode failed, took code from header
+    ok 21 - xrow_decode failed, message is not decoded
+    ok 22 - xrow_decode succeed: diag has been set
+    ok 23 - xrow_decode last error code is default 0
+    ok 24 - xrow_decode message is fine even without the code
 ok 3 - subtests
     1..1
     ok 1 - request_str

Re: [Tarantool-patches] [PATCH v2 10/10] iproto: support error stacked diagnostic area

[Nikita Pettik]

On 03 Apr 00:20, Vladislav Shpilevoy wrote:
> Thanks for the fixes!
> 
> See 2 comments below.
> 
> > diff --git a/src/box/error.cc b/src/box/error.cc
> > index 8e77c2e9e..897aa9261 100644
> > --- a/src/box/error.cc
> > +++ b/src/box/error.cc
> > @@ -102,6 +102,23 @@ box_error_new(const char *file, unsigned line, uint32_t code,
> >  	return e;
> >  }
> >  
> > +int
> > +box_error_add(const char *file, unsigned line, uint32_t code,
> > +	      const char *fmt, ...)
> > +{
> > +	struct error *e = BuildClientError(file, line, ER_UNKNOWN);
> > +	ClientError *client_error = type_cast(ClientError, e);
> > +	if (client_error) {
> > +		client_error->m_errcode = code;
> > +		va_list ap;
> > +		va_start(ap, fmt);
> > +		error_vformat_msg(e, fmt, ap);
> > +		va_end(ap);
> > +	}
> > +	diag_add_error(&fiber()->diag, e);
> > +	return -1;
> 
> 1. Why do we return -1 instead of the new error object?
> box_error_new() returns an error, and this function does not.
> Seems inconsistent.

box_error_new() is supposed to return new error according to
its name (to be more precise - '_new' suffix). It allocates and
fills in error struct. On the other hand, box_error_add() creates
error and adds it to the diagnostic area. It is simply meaningless
to return error, which is already set to diag.
 
> >>> diff --git a/test/unit/xrow.cc b/test/unit/xrow.cc
> >>> index 68a334239..954f22f16 100644
> >>> --- a/test/unit/xrow.cc
> >>> +++ b/test/unit/xrow.cc
> > @@ -372,6 +375,26 @@ test_xrow_error_stack_decode()
> >         isnt(last, NULL, "xrow_decode succeed: diag has been set");
> >         is(strcmp(last->errmsg, ""), 0, "xrow_decode corrupted stack: "
> >            "stack's map wrong value type");
> > +
> > +       /* Bad key in the packet. */
> > +       pos = mp_encode_map(buffer, 1);
> > +       pos = mp_encode_uint(pos, IPROTO_ERROR_STACK);
> > +       pos = mp_encode_array(pos, 1);
> > +       pos = mp_encode_map(pos, 2);
> > +       pos = mp_encode_uint(pos, 0xff000000 | IPROTO_ERROR_CODE);
> > +       pos = mp_encode_uint(pos, 159);
> > +       pos = mp_encode_uint(pos, IPROTO_ERROR_MESSAGE);
> > +       pos = mp_encode_str(pos, "test msg", strlen("test msg"));
> > +       header.body[0].iov_base = buffer;
> > +       header.body[0].iov_len = pos - buffer;
> > +
> > +       diag_clear(diag_get());
> > +       xrow_decode_error(&header);
> > +       last = diag_last_error(diag_get());
> > +       isnt(last, NULL, "xrow_decode succeed: diag has been set");
> > +       is(strcmp(last->errmsg, "test msg"), 0, "xrow_decode corrupted stack: "
> > +          "stack's map wrong key");
> > +
> >         check_plan();
> >  }
> > 
> > Diag won't be empty since error will be set anyway - with default
> > (i.e. wrong) error code (0), but correct message.
> 
> 2. I added box_error_code() check to ensure this.
> 
> But more importantly that the original bug I was referring to
> still is here. About overflows and integer truncation. I fixed it
> and added tests. See them below and on the branch in a separate
> commit.

Oops, seems like I forgot to pop diff out from stash...

> diff --git a/src/box/xrow.c b/src/box/xrow.c
> index 9d30bcaf9..be026a43c 100644
> --- a/src/box/xrow.c
> +++ b/src/box/xrow.c
> @@ -1105,11 +1105,13 @@ iproto_decode_error_stack(const char **pos)
>  		for (uint32_t key_idx = 0; key_idx < map_sz; key_idx++) {
>  			if (mp_typeof(**pos) != MP_UINT)
>  				return -1;
> -			uint32_t key = mp_decode_uint(pos);
> +			uint64_t key = mp_decode_uint(pos);
>  			if (key == IPROTO_ERROR_CODE) {
>  				if (mp_typeof(**pos) != MP_UINT)
>  					return -1;
>  				code = mp_decode_uint(pos);
> +				if (code > UINT32_MAX)
> +					return -1;
>  			} else if (key == IPROTO_ERROR_MESSAGE) {
>  				if (mp_typeof(**pos) != MP_STR)
>  					return -1;

Applied.

> diff --git a/test/unit/xrow.cc b/test/unit/xrow.cc
> index ae45f18b0..718ef15a2 100644
> --- a/test/unit/xrow.cc
> +++ b/test/unit/xrow.cc
> @@ -32,6 +32,7 @@ extern "C" {
>  #include "unit.h"
>  } /* extern "C" */
>  #include "trivia/util.h"
> +#include "box/error.h"
>  #include "box/xrow.h"
>  #include "box/iproto_constants.h"
>  #include "uuid/tt_uuid.h"
> @@ -255,7 +256,7 @@ error_stack_entry_encode(char *pos, const char *err_str)
>  void
>  test_xrow_error_stack_decode()
>  {
> -	plan(17);
> +	plan(24);
>  	char buffer[2048];
>  	/*
>  	 * To start with, let's test the simplest and obsolete
> @@ -392,9 +393,51 @@ test_xrow_error_stack_decode()
>  	xrow_decode_error(&header);
>  	last = diag_last_error(diag_get());
>  	isnt(last, NULL, "xrow_decode succeed: diag has been set");
> +	is(box_error_code(last), 0, "xrow_decode last error code is default 0");
>  	is(strcmp(last->errmsg, "test msg"), 0, "xrow_decode corrupted stack: "
>  	   "stack's map wrong key");
>  
> +	/* Overflow error code. */
> +	pos = mp_encode_map(buffer, 1);
> +	pos = mp_encode_uint(pos, IPROTO_ERROR_STACK);
> +	pos = mp_encode_array(pos, 1);
> +	pos = mp_encode_map(pos, 2);
> +	pos = mp_encode_uint(pos, IPROTO_ERROR_CODE);
> +	pos = mp_encode_uint(pos, (uint64_t)1 << 40);
> +	pos = mp_encode_uint(pos, IPROTO_ERROR_MESSAGE);
> +	pos = mp_encode_str(pos, "test msg", strlen("test msg"));
> +	header.body[0].iov_base = buffer;
> +	header.body[0].iov_len = pos - buffer;
> +
> +	diag_clear(diag_get());
> +	xrow_decode_error(&header);
> +	last = diag_last_error(diag_get());
> +	isnt(last, NULL, "xrow_decode succeed: diag has been set");
> +	is(box_error_code(last), 159, "xrow_decode failed, took code from "
> +	   "header");
> +	is(strcmp(last->errmsg, ""), 0, "xrow_decode failed, message is not "
> +	   "decoded");
> +
> +	/* Overflow error key. */
> +	pos = mp_encode_map(buffer, 1);
> +	pos = mp_encode_uint(pos, IPROTO_ERROR_STACK);
> +	pos = mp_encode_array(pos, 1);
> +	pos = mp_encode_map(pos, 2);
> +	pos = mp_encode_uint(pos, 0xff00000000 | IPROTO_ERROR_CODE);
> +	pos = mp_encode_uint(pos, 159);
> +	pos = mp_encode_uint(pos, IPROTO_ERROR_MESSAGE);
> +	pos = mp_encode_str(pos, "test msg", strlen("test msg"));
> +	header.body[0].iov_base = buffer;
> +	header.body[0].iov_len = pos - buffer;
> +
> +	diag_clear(diag_get());
> +	xrow_decode_error(&header);
> +	last = diag_last_error(diag_get());
> +	isnt(last, NULL, "xrow_decode succeed: diag has been set");
> +	is(box_error_code(last), 0, "xrow_decode last error code is default 0");
> +	is(strcmp(last->errmsg, "test msg"), 0, "xrow_decode message is fine "
> +	   "even without the code");

It is likely to be the same as "Bad key in the packet.". At least it
seems to cover the same execution path.

Squashed both patches on the top of branch.

Re: [Tarantool-patches] [PATCH v2 10/10] iproto: support error stacked diagnostic area

[Vladislav Shpilevoy]

Thanks for the fixes!

See 3 comments below.

>     iproto: support error stacked diagnostic area
>     
>     This patch introduces support of stacked errors in IProto protocol and
>     in net.box module.
>     
>     Closes #1148
>     ```
>     MAP{IPROTO_ERROR : string, IPROTO_ERROR_STACK : ARRAY[MAP{ERROR_CODE : uint, ERROR_MESSAGE : string}, MAP{...}, ...]}
>     ```
>     where IPROTO_ERROR is 0x31 key, IPROTO_ERROR_STACK is 0x51, ERROR_CODE

1. IPROTO_ERROR_STACK is actually 0x52.

>     is 0x01 and ERROR_MESSAGE is 0x02.
On 03/04/2020 04:16, Nikita Pettik wrote:
> On 03 Apr 00:20, Vladislav Shpilevoy wrote:
>> Thanks for the fixes!
>>
>> See 2 comments below.
>>
>>> diff --git a/src/box/error.cc b/src/box/error.cc
>>> index 8e77c2e9e..897aa9261 100644
>>> --- a/src/box/error.cc
>>> +++ b/src/box/error.cc
>>> @@ -102,6 +102,23 @@ box_error_new(const char *file, unsigned line, uint32_t code,
>>>  	return e;
>>>  }
>>>  
>>> +int
>>> +box_error_add(const char *file, unsigned line, uint32_t code,
>>> +	      const char *fmt, ...)
>>> +{
>>> +	struct error *e = BuildClientError(file, line, ER_UNKNOWN);
>>> +	ClientError *client_error = type_cast(ClientError, e);
>>> +	if (client_error) {
>>> +		client_error->m_errcode = code;
>>> +		va_list ap;
>>> +		va_start(ap, fmt);
>>> +		error_vformat_msg(e, fmt, ap);
>>> +		va_end(ap);
>>> +	}
>>> +	diag_add_error(&fiber()->diag, e);
>>> +	return -1;
>>
>> 1. Why do we return -1 instead of the new error object?
>> box_error_new() returns an error, and this function does not.
>> Seems inconsistent.
> 
> box_error_new() is supposed to return new error according to
> its name (to be more precise - '_new' suffix). It allocates and
> fills in error struct. On the other hand, box_error_add() creates
> error and adds it to the diagnostic area. It is simply meaningless
> to return error, which is already set to diag.

2. Yeah, sorry. I mistakenly took box_error_new() for box_error_set().
You are right.

>> diff --git a/test/unit/xrow.cc b/test/unit/xrow.cc
>> index ae45f18b0..718ef15a2 100644
>> --- a/test/unit/xrow.cc
>> +++ b/test/unit/xrow.cc
>> @@ -392,9 +393,51 @@ test_xrow_error_stack_decode()
>> +	/* Overflow error code. */
>> +	pos = mp_encode_map(buffer, 1);
>> +	pos = mp_encode_uint(pos, IPROTO_ERROR_STACK);
>> +	pos = mp_encode_array(pos, 1);
>> +	pos = mp_encode_map(pos, 2);
>> +	pos = mp_encode_uint(pos, IPROTO_ERROR_CODE);
>> +	pos = mp_encode_uint(pos, (uint64_t)1 << 40);
>> +	pos = mp_encode_uint(pos, IPROTO_ERROR_MESSAGE);
>> +	pos = mp_encode_str(pos, "test msg", strlen("test msg"));
>> +	header.body[0].iov_base = buffer;
>> +	header.body[0].iov_len = pos - buffer;
>> +
>> +	diag_clear(diag_get());
>> +	xrow_decode_error(&header);
>> +	last = diag_last_error(diag_get());
>> +	isnt(last, NULL, "xrow_decode succeed: diag has been set");
>> +	is(box_error_code(last), 159, "xrow_decode failed, took code from "
>> +	   "header");
>> +	is(strcmp(last->errmsg, ""), 0, "xrow_decode failed, message is not "
>> +	   "decoded");
>> +
>> +	/* Overflow error key. */
>> +	pos = mp_encode_map(buffer, 1);
>> +	pos = mp_encode_uint(pos, IPROTO_ERROR_STACK);
>> +	pos = mp_encode_array(pos, 1);
>> +	pos = mp_encode_map(pos, 2);
>> +	pos = mp_encode_uint(pos, 0xff00000000 | IPROTO_ERROR_CODE);
>> +	pos = mp_encode_uint(pos, 159);
>> +	pos = mp_encode_uint(pos, IPROTO_ERROR_MESSAGE);
>> +	pos = mp_encode_str(pos, "test msg", strlen("test msg"));
>> +	header.body[0].iov_base = buffer;
>> +	header.body[0].iov_len = pos - buffer;
>> +
>> +	diag_clear(diag_get());
>> +	xrow_decode_error(&header);
>> +	last = diag_last_error(diag_get());
>> +	isnt(last, NULL, "xrow_decode succeed: diag has been set");
>> +	is(box_error_code(last), 0, "xrow_decode last error code is default 0");
>> +	is(strcmp(last->errmsg, "test msg"), 0, "xrow_decode message is fine "
>> +	   "even without the code");
> 
> It is likely to be the same as "Bad key in the packet.". At least it
> seems to cover the same execution path.

3. You can merge them if you want. But then you need to use
0xff00000000, not 0xff000000. To ensure, that it is truncated
to 0 if cast to uint32.

The patchset LGTM, including the box.error() path to set diag
with a struct error object. Except the wrong IPROTO_ERROR_STACK
code in the commit message. Please, fix it, and send the patchset
to a second reviewer, keeping me in copy just in case.

Re: [Tarantool-patches] [PATCH v2 10/10] iproto: support error stacked diagnostic area

[Nikita Pettik]

On 04 Apr 01:17, Vladislav Shpilevoy wrote:
> Thanks for the fixes!
> 
> See 3 comments below.
> 
> >     iproto: support error stacked diagnostic area
> >     
> >     This patch introduces support of stacked errors in IProto protocol and
> >     in net.box module.
> >     
> >     Closes #1148
> >     ```
> >     MAP{IPROTO_ERROR : string, IPROTO_ERROR_STACK : ARRAY[MAP{ERROR_CODE : uint, ERROR_MESSAGE : string}, MAP{...}, ...]}
> >     ```
> >     where IPROTO_ERROR is 0x31 key, IPROTO_ERROR_STACK is 0x51, ERROR_CODE
> 
> 1. IPROTO_ERROR_STACK is actually 0x52.

Fixed (artifact after rebase on master).
 
> >     is 0x01 and ERROR_MESSAGE is 0x02.
> On 03/04/2020 04:16, Nikita Pettik wrote:
> > On 03 Apr 00:20, Vladislav Shpilevoy wrote:
> >> Thanks for the fixes!
> >>
> >> diff --git a/test/unit/xrow.cc b/test/unit/xrow.cc
> >> index ae45f18b0..718ef15a2 100644
> >> --- a/test/unit/xrow.cc
> >> +++ b/test/unit/xrow.cc
> >> @@ -392,9 +393,51 @@ test_xrow_error_stack_decode()
> >> +	/* Overflow error code. */
> >> +	pos = mp_encode_map(buffer, 1);
> >> +	pos = mp_encode_uint(pos, IPROTO_ERROR_STACK);
> >> +	pos = mp_encode_array(pos, 1);
> >> +	pos = mp_encode_map(pos, 2);
> >> +	pos = mp_encode_uint(pos, IPROTO_ERROR_CODE);
> >> +	pos = mp_encode_uint(pos, (uint64_t)1 << 40);
> >> +	pos = mp_encode_uint(pos, IPROTO_ERROR_MESSAGE);
> >> +	pos = mp_encode_str(pos, "test msg", strlen("test msg"));
> >> +	header.body[0].iov_base = buffer;
> >> +	header.body[0].iov_len = pos - buffer;
> >> +
> >> +	diag_clear(diag_get());
> >> +	xrow_decode_error(&header);
> >> +	last = diag_last_error(diag_get());
> >> +	isnt(last, NULL, "xrow_decode succeed: diag has been set");
> >> +	is(box_error_code(last), 159, "xrow_decode failed, took code from "
> >> +	   "header");
> >> +	is(strcmp(last->errmsg, ""), 0, "xrow_decode failed, message is not "
> >> +	   "decoded");
> >> +
> >> +	/* Overflow error key. */
> >> +	pos = mp_encode_map(buffer, 1);
> >> +	pos = mp_encode_uint(pos, IPROTO_ERROR_STACK);
> >> +	pos = mp_encode_array(pos, 1);
> >> +	pos = mp_encode_map(pos, 2);
> >> +	pos = mp_encode_uint(pos, 0xff00000000 | IPROTO_ERROR_CODE);
> >> +	pos = mp_encode_uint(pos, 159);
> >> +	pos = mp_encode_uint(pos, IPROTO_ERROR_MESSAGE);
> >> +	pos = mp_encode_str(pos, "test msg", strlen("test msg"));
> >> +	header.body[0].iov_base = buffer;
> >> +	header.body[0].iov_len = pos - buffer;
> >> +
> >> +	diag_clear(diag_get());
> >> +	xrow_decode_error(&header);
> >> +	last = diag_last_error(diag_get());
> >> +	isnt(last, NULL, "xrow_decode succeed: diag has been set");
> >> +	is(box_error_code(last), 0, "xrow_decode last error code is default 0");
> >> +	is(strcmp(last->errmsg, "test msg"), 0, "xrow_decode message is fine "
> >> +	   "even without the code");
> > 
> > It is likely to be the same as "Bad key in the packet.". At least it
> > seems to cover the same execution path.
> 
> 3. You can merge them if you want. But then you need to use
> 0xff00000000, not 0xff000000. To ensure, that it is truncated
> to 0 if cast to uint32.

Merged:

diff --git a/test/unit/xrow.cc b/test/unit/xrow.cc
index 7b138f1bf..9b76350af 100644
--- a/test/unit/xrow.cc
+++ b/test/unit/xrow.cc
@@ -256,7 +256,7 @@ error_stack_entry_encode(char *pos, const char *err_str)
 void
 test_xrow_error_stack_decode()
 {
-       plan(24);
+       plan(21);
        char buffer[2048];
        /*
         * To start with, let's test the simplest and obsolete
@@ -306,7 +306,6 @@ test_xrow_error_stack_decode()
         * 4. Stack's map values have wrong types;
         * 5. Stack's map key is broken (doesn't fit into u8);
         * 6. Stack's map contains overflowed (> 2^32) error code;
-        * 7. Stack's map contains overflowed (> 2^32) key.
         * In all these cases diag_last should contain empty err.
         */
        /* Stack is encoded as map. */
@@ -384,7 +383,7 @@ test_xrow_error_stack_decode()
        pos = mp_encode_uint(pos, IPROTO_ERROR_STACK);
        pos = mp_encode_array(pos, 1);
        pos = mp_encode_map(pos, 2);
-       pos = mp_encode_uint(pos, 0xff000000 | IPROTO_ERROR_CODE);
+       pos = mp_encode_uint(pos, 0xff00000000 | IPROTO_ERROR_CODE);
        pos = mp_encode_uint(pos, 159);
        pos = mp_encode_uint(pos, IPROTO_ERROR_MESSAGE);
        pos = mp_encode_str(pos, "test msg", strlen("test msg"));
@@ -420,26 +419,6 @@ test_xrow_error_stack_decode()
        is(strcmp(last->errmsg, ""), 0, "xrow_decode failed, message is not "
           "decoded");
 
-       /* Overflow error key. */
-       pos = mp_encode_map(buffer, 1);
-       pos = mp_encode_uint(pos, IPROTO_ERROR_STACK);
-       pos = mp_encode_array(pos, 1);
-       pos = mp_encode_map(pos, 2);
-       pos = mp_encode_uint(pos, 0xff00000000 | IPROTO_ERROR_CODE);
-       pos = mp_encode_uint(pos, 159);
-       pos = mp_encode_uint(pos, IPROTO_ERROR_MESSAGE);
-       pos = mp_encode_str(pos, "test msg", strlen("test msg"));
-       header.body[0].iov_base = buffer;
-       header.body[0].iov_len = pos - buffer;
-
-       diag_clear(diag_get());
-       xrow_decode_error(&header);
-       last = diag_last_error(diag_get());
-       isnt(last, NULL, "xrow_decode succeed: diag has been set");
-       is(box_error_code(last), 0, "xrow_decode last error code is default 0");
-       is(strcmp(last->errmsg, "test msg"), 0, "xrow_decode message is fine "
-          "even without the code");
-
        check_plan();
 }
 
> The patchset LGTM, including the box.error() path to set diag
> with a struct error object. Except the wrong IPROTO_ERROR_STACK
> code in the commit message. Please, fix it, and send the patchset
> to a second reviewer, keeping me in copy just in case.