Tarantool development patches archive
 help / color / mirror / Atom feed
From: Sergey Kaplun via Tarantool-patches <tarantool-patches@dev.tarantool.org>
To: Sergey Ostanevich <sergos@tarantool.org>,
	Igor Munkin <imun@tarantool.org>
Cc: tarantool-patches@dev.tarantool.org
Subject: [Tarantool-patches] [PATCH luajit v3 16/29] test: adapt PUC-Rio test for %q in string.format
Date: Tue, 13 Apr 2021 16:27:16 +0300	[thread overview]
Message-ID: <03646b43769edae6838b79b88cb8ecbedb2becbe.1618320000.git.skaplun@tarantool.org> (raw)
In-Reply-To: <cover.1618320000.git.skaplun@tarantool.org>

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=UTF-8, Size: 4234 bytes --]

LuaJIT since v2.0.0-beta6 has extension from Lua 5.2:
string.format(): %q reversible.
See also https://luajit.org/extensions.html#lua52.

The q option formats a string in a form suitable to be safely read
back by the Lua interpreter [1].

In Lua 5.1, '\0' was converted to "\000" unconditionally.

In Lua 5.2, control characters are written as \nnn when needed, see
d62a21b9d379a576bae7426c80039ca1a4d2bb07 ("when formatting with '%q',
all control characters are coded as \nnn.") [1]. In this patch, %q
specifier starts writing control characters to a new string in the same
way through \d, instead of their binary representation as it is. If the
control character is followed by a digit, then for the correct work of
the parser, it is necessary to extend the escape sequence to 3
significant characters (otherwise, the transition "\0002" -> "\02"
corrupts string). For this patch, the expansion is performed
unconditionally (the check for the next symbol is not performed) if this
first symbol is a zero byte.

Consistent work with the zero byte processing omitting the custom check
was added in the commit 658ea8752b979102627e2fede7b7ddfbb67ba6c9 ("no
need to handle '\0' differently from other control chars in format
'%q'") [2]. Note, that this patch does not change the semantics of %q
specifier - the new string can still be "safely read back by the Lua
interpreter".

[1]: https://www.lua.org/manual/5.1/manual.html#pdf-string.format
[2]: https://github.com/lua/lua/commit/d62a21b9d379a576bae7426c80039ca1a4d2bb07
[3]: https://github.com/lua/lua/commit/658ea8752b979102627e2fede7b7ddfbb67ba6c9

This patch adapts test for LuaJIT and Lua 5.2 behaviour considering
the test from Lua 5.2 test suite taken from
https://www.lua.org/tests/lua-5.2.0-tests.tar.gz.

Resolves tarantool/tarantool#5710
Part of tarantool/tarantool#5845
Part of tarantool/tarantool#4473
---
 test/PUC-Rio-Lua-5.1-tests/strings.lua | 38 +++++++++++++++++++++++++-
 1 file changed, 37 insertions(+), 1 deletion(-)

diff --git a/test/PUC-Rio-Lua-5.1-tests/strings.lua b/test/PUC-Rio-Lua-5.1-tests/strings.lua
index 237dbad3..0eee4c6e 100644
--- a/test/PUC-Rio-Lua-5.1-tests/strings.lua
+++ b/test/PUC-Rio-Lua-5.1-tests/strings.lua
@@ -102,7 +102,43 @@ print('+')
 
 x = '"ílo"\n\\'
 assert(string.format('%q%s', x, x) == '"\\"ílo\\"\\\n\\\\""ílo"\n\\')
-assert(string.format('%q', "\0") == [["\000"]])
+--[[
+LuaJIT: LuaJIT since v2.0.0-beta6 has extension from Lua 5.2:
+string.format(): %q reversible.
+
+The q option formats a string in a form suitable to be safely read
+back by the Lua interpreter [1].
+
+In Lua 5.1, '\0' was converted to "\000" unconditionally.
+
+In Lua 5.2, control characters are written as \nnn when needed,
+see d62a21b9d379a576bae7426c80039ca1a4d2bb07 ("when formatting
+with '%q', all control characters are coded as \nnn.") [1].
+In this patch, %q specifier starts writing control characters to
+a new string in the same way through \d, instead of their binary
+representation as it is. If the control character is followed by
+a digit, then for the correct work of the parser, it is necessary
+to extend the escape sequence to 3 significant characters
+(otherwise, the transition "\0002" -> "\02" corrupts string).
+For this patch, the expansion is performed unconditionally
+(the check for the next symbol is not performed) if this first
+symbol is a zero byte.
+
+Consistent work with the zero byte processing omitting the custom
+check was added in the commit
+658ea8752b979102627e2fede7b7ddfbb67ba6c9 ("no need to handle '\0'
+differently from other control chars in format '%q'") [2]. Note,
+that this patch does not change the semantics of %q specifier -
+the new string can still be "safely read back by the Lua
+interpreter".
+
+[1]: https://www.lua.org/manual/5.1/manual.html#pdf-string.format
+[2]: https://github.com/lua/lua/commit/d62a21b9d379a576bae7426c80039ca1a4d2bb07
+[3]: https://github.com/lua/lua/commit/658ea8752b979102627e2fede7b7ddfbb67ba6c9
+
+Test is adapted from PUC-Rio Lua 5.2 test suite.
+--]]
+assert(string.format('%q', "\0") == [["\0"]])
 assert(string.format("\0%c\0%c%x\0", string.byte("á"), string.byte("b"), 140) ==
               "\0á\0b8c\0")
 assert(string.format('') == "")
-- 
2.31.0


  parent reply	other threads:[~2021-04-13 13:36 UTC|newest]

Thread overview: 35+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-13 13:27 [Tarantool-patches] [PATCH luajit v3 00/29] Adapt PUC-Rio Lua 5.1 test suite Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 01/29] test: add " Sergey Kaplun via Tarantool-patches
2021-04-13 20:31   ` Igor Munkin via Tarantool-patches
2021-04-14 13:54     ` Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 02/29] test: build auxiliary C libs from PUC-Rio Lua 5.1 Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 03/29] test: adapt PUC-Rio suite for out-of-source build Sergey Kaplun via Tarantool-patches
2021-04-13 21:05   ` Igor Munkin via Tarantool-patches
2021-04-14 13:54     ` Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 04/29] test: remove quotes in progname from PUC-Rio Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 05/29] test: adapt PUC-Rio test for arg presence Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 06/29] test: disable PUC-Rio tests confused by -v output Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 07/29] test: disable PUC-Rio tests for bytecode header Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 08/29] test: adapt PUC-Rio tests counting GC steps Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 09/29] test: disable PUC-Rio suite tests for line hook Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 10/29] test: adapt PUC-Rio tests with vararg functions Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 11/29] test: adapt PUC-Rio test for debug in vararg func Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 12/29] test: adapt PUC-Rio test with count hooks Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 13/29] test: disable PUC-Rio test for tail call info Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 14/29] test: adapt PUC-Rio test with activeline check Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 15/29] test: disable PUC-Rio test for per-coroutine hooks Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` Sergey Kaplun via Tarantool-patches [this message]
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 17/29] test: disable locale-dependent PUC-Rio tests Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 18/29] test: use math.fmod in " Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 19/29] test: remove string.gfind assert in PUC-Rio test Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 20/29] test: disable PUC-Rio test for getfenv in tailcall Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 21/29] test: disable PUC-Rio test for variables in error Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 22/29] test: disable PUC-Rio test for fast function name Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 23/29] test: disable PUC-Rio test for non-asci identifier Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 24/29] test: disable PUC-Rio test for syntax level error Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 25/29] test: disable PUC-RIO tests for several -l options Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 26/29] test: disable PUC-Rio test for checking arg layout Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 27/29] test: disable PUC-Rio test checking -h option Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 28/29] test: disable PUC-Rio hanging GC test Sergey Kaplun via Tarantool-patches
2021-04-13 13:27 ` [Tarantool-patches] [PATCH luajit v3 29/29] test: disable too deep recursive PUC-Rio test Sergey Kaplun via Tarantool-patches
2021-04-14 21:08 ` [Tarantool-patches] [PATCH luajit v3 00/29] Adapt PUC-Rio Lua 5.1 test suite Igor Munkin via Tarantool-patches

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=03646b43769edae6838b79b88cb8ecbedb2becbe.1618320000.git.skaplun@tarantool.org \
    --to=tarantool-patches@dev.tarantool.org \
    --cc=imun@tarantool.org \
    --cc=sergos@tarantool.org \
    --cc=skaplun@tarantool.org \
    --subject='Re: [Tarantool-patches] [PATCH luajit v3 16/29] test: adapt PUC-Rio test for %q in string.format' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox