From: Sergey Bronnikov <sergeyb@tarantool.org> To: Igor Munkin <imun@tarantool.org> Cc: tarantool-patches@dev.tarantool.org Subject: Re: [Tarantool-patches] [PATCH 1/4] test: add infrastructure for fuzzing testing and fuzzers Date: Thu, 24 Dec 2020 20:25:48 +0300 [thread overview] Message-ID: <2ab21816-244d-7fd7-48cf-4e7076fbb64f@tarantool.org> (raw) In-Reply-To: <20201224132254.GA5396@tarantool.org> Igor, On 24.12.2020 16:22, Igor Munkin wrote: > Sergey, > > Thanks for the fixes! Unfortunately, I still see 72 symbols violation in > commit message for the first and last patches on your remote branch. > Please, also consider my other notes below. Fixed them too. > On 24.12.20, Sergey Bronnikov wrote: >> Hi, >> >> On 20.12.2020 16:31, Igor Munkin wrote: >>> Sergey, >>> >>> Thanks for the changes! >>> > <snipped> > >>> Neat, now everything works fine. However, considering your comment, I >>> have a newbie question (since I'm not an expert in fuzzing testing): how >>> do we need to check whether parsing finishes right or not? >> libfuzzer has a number of settings and one of them is flag that controls >> time of single unit execution. > I asked about the check if parsing succeeds or not, but you answered > this question below. > > <snipped> > >>> I believe the testing is not OK if <calloc> yields NULL, but the code >>> returns 0. This is odd, IMHO. What about adding either assert or abort >>> to handle this branch? To make asserts work all time simply undefine >>> NDEBUG at the beginning of the test. Same for other cases. >> Igor, I think you get everything wrong ;) Let me explain. >> >> We don't write a highly reliable and safety code here. Everything we >> need is just to properly pass a junk to a function under test. >> >> The goal of fuzzing testing is to find errors like buffer-overflows, >> use-after-free and so on. > AFAIU, these tests do not check if the "passed junk" is parsed fine. Am > I right? Absolutely! These tests are not about correctness. > >> Lack of memory during testing is rare case and I think we don't need to >> catch such cases here. >> >> Because triggered assert due to lack of memory is useless information >> from test, >> >> I don't know how we can improve Tarantool with such information. >> Gracefully exit is more than enough. > OK, then. > >> Moreover I have took a look on source code of tests for other opensource >> projects that were already used in OSS-Fuzz. >> >> They don't care about return codes from calloc(), malloc() functions at >> all. See for example [1]. > "А если все пойдут с моста прыгать, ты тоже пойдешь?" Ты напомнил мне мою учительницу, она тоже так говорила. > Anyway, I get your point, thanks for clarification! > > <snipped> > >>> Why these compile flags are added under this particular condition? >> Because when OSS Fuzz is enabled compiler and link flags passed >> >> from outside. See description how to integrate project to OSS Fuzz in [2]. > Glad to see this in commit message, thanks! > > <snipped> > >>>>> 2. Do you need to specify <address> flag once more, when ASAN is >>>>> enabled? If not the hunk above looks excess, doesn't it? >>>> Agree, it was a bad idea to manage UBSan and ASAN flags in yet another >>>> place. > I guess this should be fixed in scope of the first patch, but I see you > squashed it to the last one. Why? It seems because I was triggered by OSS_FUZZ in hunk and squashed it to a commit that introduce an OSS_FUZZ support. Fixed it and also moved add_compile_options() to the first commit too (otherwise project source code is not instrumented with -fsanitize=fuzzer-no-link). In last commit now: --- a/test/fuzz/CMakeLists.txt +++ b/test/fuzz/CMakeLists.txt @@ -9,12 +9,23 @@ add_library(fuzzer_config INTERFACE) target_compile_options( fuzzer_config INTERFACE - -fsanitize=fuzzer,address + $<$<NOT:$<BOOL:${OSS_FUZZ}>>: + -fsanitize=fuzzer + > + $<$<BOOL:${OSS_FUZZ}>: + ${CXX} + ${CXXFLAGS} + > ) target_link_libraries( fuzzer_config INTERFACE - -fsanitize=fuzzer,address + $<$<NOT:$<BOOL:${OSS_FUZZ}>>: + -fsanitize=fuzzer + > + $<$<BOOL:${OSS_FUZZ}>: + $ENV{LIB_FUZZING_ENGINE} + > ) # Use PUBLIC to force 'fuzzer_config' for all dependent targets. <snipped>
next prev parent reply other threads:[~2020-12-24 17:25 UTC|newest] Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-11-30 20:24 [Tarantool-patches] [PATCH 0/4] Add fuzzing testing sergeyb 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 1/4] test: add infrastructure for fuzzing testing and fuzzers sergeyb 2020-12-07 17:24 ` Igor Munkin 2020-12-07 19:54 ` Igor Munkin 2020-12-13 18:56 ` Sergey Bronnikov 2020-12-20 13:31 ` Igor Munkin 2020-12-24 10:18 ` Sergey Bronnikov 2020-12-24 13:22 ` Igor Munkin 2020-12-24 17:25 ` Sergey Bronnikov [this message] 2020-12-24 17:50 ` Igor Munkin 2020-12-25 7:07 ` Sergey Bronnikov 2020-12-25 9:02 ` Igor Munkin 2020-12-25 10:33 ` Sergey Bronnikov 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 2/4] test: add corpus to be used with fuzzers sergeyb 2020-12-07 17:34 ` Igor Munkin 2020-12-13 18:56 ` Sergey Bronnikov 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 3/4] travis: build tarantool with ENABLE_FUZZER sergeyb 2020-12-07 17:38 ` Igor Munkin 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 4/4] test: integrate with OSS Fuzz sergeyb 2020-12-07 17:42 ` Igor Munkin 2020-12-01 10:54 ` [Tarantool-patches] [PATCH 0/4] Add fuzzing testing Serge Petrenko 2020-12-01 14:41 ` Sergey Bronnikov 2020-12-01 14:45 ` Serge Petrenko 2020-12-07 17:49 ` Igor Munkin 2020-12-25 13:08 ` Igor Munkin 2020-12-25 14:52 ` Kirill Yukhin
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=2ab21816-244d-7fd7-48cf-4e7076fbb64f@tarantool.org \ --to=sergeyb@tarantool.org \ --cc=imun@tarantool.org \ --cc=tarantool-patches@dev.tarantool.org \ --subject='Re: [Tarantool-patches] [PATCH 1/4] test: add infrastructure for fuzzing testing and fuzzers' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox