From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 1A056BA0E96; Thu, 13 Jun 2024 14:01:08 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 1A056BA0E96 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1718276468; bh=ij+CbYn0M0S57lSnoi7jiOJtlzp1ExMYOd8unzS33CA=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=eWsi62V5gLZpvPjilCSgos28xc9ih5yBn993NshgzdsptjNevkAelXm2eobGWkyCt PegUU4jcyQJm0Gl1x/D/Ijec5mqGn5Ke0lnaz/T+yBqpUjrWPsZwzWnOGhOmBP1Zr/ C8qsSBnOzElTImM6IiE3MQlodNB2vALsq8FQPnGw= Received: from smtp56.i.mail.ru (smtp56.i.mail.ru [95.163.41.94]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 4E23ABA0E89 for ; Thu, 13 Jun 2024 14:01:07 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 4E23ABA0E89 Received: by smtp56.i.mail.ru with esmtpa (envelope-from ) id 1sHiCW-0000000AXYM-3VG5; Thu, 13 Jun 2024 14:01:05 +0300 Date: Thu, 13 Jun 2024 13:56:52 +0300 To: Sergey Bronnikov Message-ID: References: <6f8a08e1823bfceebb4057207ee2f2bdb7d2d47c.1715776117.git.skaplun@tarantool.org> <3605e667-a4e6-4ee1-abd2-412e81d76c89@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <3605e667-a4e6-4ee1-abd2-412e81d76c89@tarantool.org> X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD9AC8CA0B4439200FA2E41D1825D8172A0E0E393E23BEE614B00894C459B0CD1B99F11ABF1C5EB4F3DD1C6CC7D4FF78A0BF74F4D8FD4BF17987B66FD4AEC4B96DFB7AB0E7134D2E1A8 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7B2CE06D3E4B8AFEBEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637B34A545A74F29B438638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D870B390D93E4142C0806F25BD6C71235D0A27D7299DB58A31CC7F00164DA146DAFE8445B8C89999728AA50765F7900637F924B32C592EA89F389733CBF5DBD5E9C8A9BA7A39EFB766F5D81C698A659EA7CC7F00164DA146DA9985D098DBDEAEC8062BEEFFB5F8EA3EF6B57BC7E6449061A352F6E88A58FB86F5D81C698A659EA7E827F84554CEF5019E625A9149C048EE33AC447995A7AD181150BA43C84913FC3A03B725D353964B21E93C0F2A571C7B35872C767BF85DA227C277FBC8AE2E8B9149C560DC76099D75ECD9A6C639B01B4E70A05D1297E1BBCB5012B2E24CD356 X-C1DE0DAB: 0D63561A33F958A584E1D41A7D6AE3385002B1117B3ED6961EDBDADF5A592988FB820E9FE7BD014C823CB91A9FED034534781492E4B8EEAD5DF1C2DF01CE7211BDAD6C7F3747799A X-C8649E89: 1C3962B70DF3F0ADE00A9FD3E00BEEDF3FED46C3ACD6F73ED3581295AF09D3DF87807E0823442EA2ED31085941D9CD0AF7F820E7B07EA4CF4AADFDC0156B97B20D4F103FE2DB0FC994DDAC2DE2AA4B615F81E01FC09DA9AF77ACD0092D6ACCC804A26A3878DC13301A1455A98152D41FEEF59437730A033932CAADFA2F0BFDEE5F4332CA8FE04980913E6812662D5F2A5EAB5682573093F7837F15F2B5E4A70B33F2C28C22F508233FCF178C6DD14203 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojP/c/PTD82AlzQzPMBrufCg== X-DA7885C5: 96AEFBAAABEA4C84F255D290C0D534F9EE45BDB248FD7BDC785EE024C1E938324E8C80916E06FEEF5B1A4C17EAA7BC4BEF2421ABFA55128DAF83EF9164C44C7E X-Mailru-Sender: 689FA8AB762F7393C6D0B12EA33CAA9B1A634961E060612406ABB3C1E0D64D59C406EFF3FABD62D5E49D44BB4BD9522A059A1ED8796F048DB274557F927329BE89D5A3BC2B10C37545BD1C3CC395C826B4A721A3011E896F X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit 1/2] build: introduce LUAJIT_USE_UBSAN option X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Kaplun via Tarantool-patches Reply-To: Sergey Kaplun Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hi, Sergey! Thanks for the review! Please considered my answers below. On 07.06.24, Sergey Bronnikov wrote: > Sergey, > > thanks for the patch! Please see my comments below. Fixed your comments, see the iterative patch below. The branch is force-pushed. =================================================================== diff --git a/CMakeLists.txt b/CMakeLists.txt index e00b1536..76e0c067 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -302,7 +302,7 @@ endif() option(LUAJIT_USE_UBSAN "Build LuaJIT with UndefinedBehaviorSanitizer" OFF) if(LUAJIT_USE_UBSAN) - # Use all recommendations from the UndefinedBehaviorSanitizer + # Use all needed checks from the UndefinedBehaviorSanitizer # documentation: # https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html. string(JOIN "," UBSAN_IGNORE_OPTIONS @@ -322,7 +322,8 @@ if(LUAJIT_USE_UBSAN) # cdata arithmetic, vmevent hash calculation, etc. shift-base ) - # GCC has no "function" UB check. + # GCC has no "function" UB check. See details here: + # https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html#index-fsanitize_003dundefined if(NOT CMAKE_C_COMPILER_ID STREQUAL "GNU") string(JOIN "," UBSAN_IGNORE_OPTIONS ${UBSAN_IGNORE_OPTIONS} @@ -337,7 +338,7 @@ if(LUAJIT_USE_UBSAN) -fno-omit-frame-pointer # Enable UndefinedBehaviorSanitizer support. # This flag enables all supported options (the documentation - # on cite is not correct about that moment, unfortunately) + # on site is not correct about that moment, unfortunately) # except float-divide-by-zero. Floating point division by zero # behaviour is defined without -ffast-math and uses the # IEEE 754 standard on which all NaN tagging is based. diff --git a/cmake/SetDynASMFlags.cmake b/cmake/SetDynASMFlags.cmake index ae3c75b1..b400cf57 100644 --- a/cmake/SetDynASMFlags.cmake +++ b/cmake/SetDynASMFlags.cmake @@ -139,7 +139,7 @@ endif() if(LUAJIT_USE_UBSAN) # XXX: Skip checks for now to avoid build failures due to # sanitizer errors. - # Need to backprot commits that fix the following issues first: + # Need to backport commits that fix the following issues first: # https://github.com/LuaJIT/LuaJIT/pull/969, # https://github.com/LuaJIT/LuaJIT/pull/970, # https://github.com/LuaJIT/LuaJIT/issues/1041, =================================================================== > > On 15.05.2024 15:32, Sergey Kaplun wrote: > > This patch adds Undefined Behaviour Sanitizer [1] support. It enables > > all checks except several that are not useful for LuaJIT. Also, it > > instruments all known issues to be fixed in future patches (except > > `kfold_intop()` since cdata arithmetic relies on integer overflow). > > > > [1]:https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html > > > > Resolves tarantool/tarantool#8473 > > --- > > CMakeLists.txt | 45 ++++++++++++++++++++++++++++++++++++++ > > cmake/SetDynASMFlags.cmake | 11 ++++++++++ > > src/lj_carith.c | 5 +++++ > > src/lj_opt_fold.c | 5 +++++ > > src/lj_parse.c | 5 +++++ > > src/lj_snap.c | 7 ++++++ > > src/lj_strfmt.c | 5 +++++ > > 7 files changed, 83 insertions(+) > > patch in mail is outdated, so I'll copypaste missed part: Yes, it is mentioned in this subthread [1]. > > > diff --git a/src/lj_buf.h b/src/lj_buf.h > index a4051694..aaecc9f8 100644 > --- a/src/lj_buf.h > +++ b/src/lj_buf.h > @@ -70,6 +70,13 @@ LJ_FUNC SBuf *lj_buf_putmem(SBuf *sb, const void *q, > MSize len); >  LJ_FUNC SBuf * LJ_FASTCALL lj_buf_putchar(SBuf *sb, int c); >  LJ_FUNC SBuf * LJ_FASTCALL lj_buf_putstr(SBuf *sb, GCstr *s); > > +#if LUAJIT_USE_UBSAN > +/* The `NULL` argument with the zero length, like in the case: > +** | luajit -e 'error("x", 3)' > +*/ > +static LJ_AINLINE char *lj_buf_wmem(char *p, const void *q, MSize len) > +  __attribute__((no_sanitize("nonnull-attribute"))); > +#endif >  static LJ_AINLINE char *lj_buf_wmem(char *p, const void *q, MSize len) >  { >    return (char *)memcpy(p, q, len) + len; > > > With this reverted patch tests passed. Do we really need this patch? Should I add the corresponding test mentioned in [1]? > > > > > > diff --git a/CMakeLists.txt b/CMakeLists.txt > > index 2355ce17..edf2012f 100644 > > --- a/CMakeLists.txt > > +++ b/CMakeLists.txt > > @@ -300,6 +300,51 @@ if(LUAJIT_USE_ASAN) > > ) > > endif() > > > > +option(LUAJIT_USE_UBSAN "Build LuaJIT with UndefinedBehaviorSanitizer" OFF) > > +if(LUAJIT_USE_UBSAN) > > + # Use all recommendations from the UndefinedBehaviorSanitizer > > probably you mean "checks" [1] and not "recommendations" Fixed, thanks. > > > 1. https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html#ubsan-checks > > > + # documentation: > > + #https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html. > > + string(JOIN "," UBSAN_IGNORE_OPTIONS > > + # Misaligned pseudo-pointers are used to determine internal > > + # variable names inside the `for` cycle. > > + alignment > > + # Not interested in float cast overflow errors. > > + float-cast-overflow > > + # NULL checking is disabled because this is not a UB and > > + # raises lots of false-positive fails. > > + null > > + # Not interested in checking arithmetic with NULL. > > + pointer-overflow > > + # Shifts of negative numbers are widely used in parsing ULEB, > > + # cdata arithmetic, vmevent hash calculation, etc. > > + shift-base > > Will we report issues produced by these checks to upstream? These particular checks -- no, since they are not so interesting for us, and most probably may be considered by Mike as "white noise". For others -- yes. I've already reported the related problem with the patch [3]. > > Decision "not interested" confuses. I've given the rationale for float-cast-overflow [2]. Pointer overflow is not interesting for us since it is widely used in LuaJIT, in particular in . So, we may avoid warnings in `NULL - ptr` arithmetics. > > > + ) > > + if(NOT CMAKE_C_COMPILER_ID STREQUAL "GNU") > > please add a link to GCC documentation > > https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html#index-fsanitize_003dundefined Added, thanks. > > > + string(JOIN "," UBSAN_IGNORE_OPTIONS > > + ${UBSAN_IGNORE_OPTIONS} > > + # Not interested in function type mismatch errors. > > + function > > + ) > > + endif() > > + AppendFlags(CMAKE_C_FLAGS > > + # Enable hints for UndefinedBehaviorSanitizer. > > + -DLUAJIT_USE_UBSAN > > + # XXX: To get nicer stack traces in error messages. > > + -fno-omit-frame-pointer > > + # Enable UndefinedBehaviorSanitizer support. > > + # This flag enables all supported options (the documentation > > + # on cite is not correct about that moment, unfortunately) > typo: cite -> site Fixed, thanks. > > + # except float-divide-by-zero. Floating point division by zero > > + # behaviour is defined without -ffast-math and uses the > > + # IEEE 754 standard on which all NaN tagging is based. > > + -fsanitize=undefined > > + -fno-sanitize=${UBSAN_IGNORE_OPTIONS} > > + # Print a verbose error report and exit the program. > > + -fno-sanitize-recover=undefined > > + ) > > +endif() > > + > > # Enable code coverage support. > > option(LUAJIT_ENABLE_COVERAGE "Enable code coverage support (gcovr)" OFF) > > if(LUAJIT_ENABLE_COVERAGE) > > diff --git a/cmake/SetDynASMFlags.cmake b/cmake/SetDynASMFlags.cmake > > index 7eead6e9..ae3c75b1 100644 > > --- a/cmake/SetDynASMFlags.cmake > > +++ b/cmake/SetDynASMFlags.cmake > > @@ -136,5 +136,16 @@ if(NOT CMAKE_SYSTEM_NAME STREQUAL ${CMAKE_HOST_SYSTEM_NAME}) > > endif() > > endif() > > > > +if(LUAJIT_USE_UBSAN) > > + # XXX: Skip checks for now to avoid build failures due to > > + # sanitizer errors. > > + # Need to backprot commits that fix the following issues first: > typo: backprot -> backport Fixed, thanks! > > + #https://github.com/LuaJIT/LuaJIT/pull/969, > > + #https://github.com/LuaJIT/LuaJIT/pull/970, > > + #https://github.com/LuaJIT/LuaJIT/issues/1041, > > + #https://github.com/LuaJIT/LuaJIT/pull/1044. > > + AppendFlags(HOST_C_FLAGS -fno-sanitize=undefined) > > +endif() > > + > > unset(LUAJIT_ARCH) > > unset(TESTARCH) > > diff --git a/src/lj_carith.c b/src/lj_carith.c > With this reverted patch tests passed. Do we really need this patch? Yes, since cdata arithmetics depends on overflows of integers. So we should ignore all warnings here. > > index 4e1d450a..1d9d6fe1 100644 > > --- a/src/lj_carith.c > > +++ b/src/lj_carith.c > > @@ -159,6 +159,11 @@ static int carith_ptr(lua_State *L, CTState *cts, CDArith *ca, MMS mm) > > } > > > > /* 64 bit integer arithmetic. */ > > +#if LUAJIT_USE_UBSAN > > +/* Seehttps://github.com/LuaJIT/LuaJIT/issues/928. */ > > +static int carith_int64(lua_State *L, CTState *cts, CDArith *ca, MMS mm) > > + __attribute__((no_sanitize("signed-integer-overflow"))); > > +#endif > > static int carith_int64(lua_State *L, CTState *cts, CDArith *ca, MMS mm) > > { > > if (ctype_isnum(ca->ct[0]->info) && ca->ct[0]->size <= 8 && [1]: https://lists.tarantool.org/pipermail/tarantool-patches/2024-May/029185.html [2]: https://lists.tarantool.org/pipermail/tarantool-patches/2024-May/029195.html [3]: https://github.com/LuaJIT/LuaJIT/issues/1193 -- Best regards, Sergey Kaplun