* [Tarantool-patches] [PATCH 0/4] Add fuzzing testing @ 2020-11-30 20:24 sergeyb 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 1/4] test: add infrastructure for fuzzing testing and fuzzers sergeyb ` (7 more replies) 0 siblings, 8 replies; 26+ messages in thread From: sergeyb @ 2020-11-30 20:24 UTC (permalink / raw) To: tarantool-patches, imun, sergepetrenko From: Sergey Bronnikov <sergeyb@tarantool.org> Patch series adds required support in project infrastructure for fuzzing tests, add tests for uri, csv and http_parser modules, adds integration with OSS Fuzz (infrastructure for continuous running fuzzing tests). NOTE: v2 version had LGTM from Sergey Petrenko, see [1] Changelog: v2: - introduce CMake flag to enable building fuzzers - add fuzzers based on LibFuzzer to csv, http_parser and uri modules v3: - report error on an attempt to set ENABLE_FUZZER=ON with GCC compiler - fixed return code in tests - place tests to a common dir test/fuzz/ like we did with unit tests - enable building fuzzers in Tarantool CI job - add target to build all availabe fuzzers - add integration with OSS Fuzz [2] - add corpus for uri, csv and http_parser tests 1. https://lists.tarantool.org/pipermail/tarantool-patches/2020-April/016471.html 2. https://github.com/google/oss-fuzz/pull/4723 CI: https://gitlab.com/tarantool/tarantool/-/pipelines/223112908 Issue: https://github.com/tarantool/tarantool/issues/1809 Sergey Bronnikov (4): test: add infrastructure for fuzzing testing and fuzzers test: add corpus to be used with fuzzers travis: build tarantool with ENABLE_FUZZER test: integrate with OSS Fuzz .travis.mk | 2 +- CMakeLists.txt | 2 +- cmake/profile.cmake | 13 +++++ test/CMakeLists.txt | 3 + test/fuzz/CMakeLists.txt | 58 +++++++++++++++++++ test/fuzz/csv_fuzzer.c | 23 ++++++++ test/fuzz/http_parser_fuzzer.c | 18 ++++++ test/fuzz/uri_fuzzer.c | 19 ++++++ .../0c90a17c7b7cb5163255c7b775a8b544fb62c59d | 3 + .../0e76fc3bb39bac4f774e2ce3978e9b59bc4a04b7 | 2 + .../11f7ab7d8c2d4da313081e3282ab8120f0330b90 | 5 ++ .../2ace62c1befa19e3ea37dd52be9f6d508c5163e6 | 1 + .../3816dd9c9120fe6b3d806184874cc5c3e3a68df3 | 1 + .../409c042df7f6a518972df1bc1287442e28719b6c | 1 + .../46b175b37b5c3f746fde0dd114ead9938c012cfc | 3 + .../56754bc8c7346ef9fad5c2ae9c54cef0af72629c | 2 + .../58f43ea1e6df023f3dd4be9f718e5d0382fb9697 | 3 + .../6d131d28c6e20c3a0a0b46c3aa7308d3029ab636 | 1 + .../7560cb86c281018ee2f80d840fd8e6aa73a609ef | 5 ++ .../7b97db755e18dc519fbc49e5de19b5820b9441c1 | 1 + .../7ba0c88846fbcd28ad490df47f9ce4405d795303 | 2 + .../8a14c110827e26cfdde3693bc5db6cd8673cdc8a | 1 + .../94c1799dfba59120dd6075ee60016b809a915ca8 | 1 + .../9e109f38c3948abe743bd116a34f05954f9ddd59 | 1 + .../d21ec5eaaab3c8db1f923674b7a3ee19ce35ed13 | 1 + .../d4a6d494cc2a06a0fee4873014210ec523d23750 | 1 + .../eca825c9bdb276332d524134ddf370f7d18c2444 | 1 + .../f9ba3e378e0409d66f946b31306de19edfe859bc | 1 + .../fcad5471c295084dc1ec1eaa3753420e9803d251 | 10 ++++ .../038ec38a9ae7669590f1e3401da8ffbc9b40c8ee | 1 + .../043fdf868410e66239819e0af10cbe787cfab77e | 1 + .../08abe5308ce774ad5e9e435de386748ce06521f8 | 1 + .../10b4f603cf09e21020e494e20a7f6760d5634c88 | 1 + .../172ed6e56aa94bada4135d64fc2e0c5eac779aed | 1 + .../1e42978bf6935a0f8377f6c480c315f1ae4627b8 | 1 + .../1f6f26c085c3b209e626f90c63d0fb10526de318 | 1 + .../24b458efa865a138a314f0876a11ec680b983ed5 | 1 + .../2be565c790bc352f2cbaa0f3509e0bb6133b3616 | 1 + .../30fd5e18f586b219916fb3892961a00e92db7a7f | 1 + .../38918740212c6e5aea01f2fc8772b1d4aa89be8b | 1 + .../39d4c74a90b7f5f5426b7982857b5fd7edbd8f77 | 1 + .../3d21092e1a772d792e9dd720eeae30d8eb5ac2ad | 1 + .../454dec613ca5c71fee0467061a88d2807201ac36 | 1 + .../4f13053e61966dd08d7cd2407865481ec1d2cbf9 | 1 + .../4ffa3daceb5b6ee3fc17cef3e0ae694ea5c583b5 | 1 + .../501b3a19def06b352978c03ece7ab282cdc6cbab | 1 + .../50ad023e1465919f5afbfffb33504ddfdc6a7e69 | 1 + .../5177d236e3eea297b76c19194a4cf127b313b267 | 1 + .../51e1354d16cfa4967e91206be8bd0d8c6ca577af | 1 + .../55d5f3acc62809fc87466b2048b157e50e63d644 | 1 + .../598d19d3175046ff035be228db2f1dc43957d8e2 | 1 + .../5b7444e43e5c1fd759483c76a46a5536da043022 | 1 + .../5c489184823a8a42844066fd5b4951bbad86934f | 1 + .../6455c0e761cd485f3e38c470f216560bfc46eea4 | 1 + .../6596bb363cfd0bef3f68e265cfd824c6d37ecb7e | 1 + .../66d5206b6c75b071f57684399218d144b0d337ea | 1 + .../685073a623772c1d46475b5eb451de40247d697f | 1 + .../71d4de855c1af66a6a5cea88b1eddc5032ecb26d | 1 + .../73378b09c61739b65a5f63214f0515307f187e53 | 1 + .../768abf119ce4fa6c9a9ee9f17d9dacb32c61759c | 1 + .../7ae1560be3dffbb88f3e48de3a4be99ecdc61441 | 1 + .../7c4d33785daa5c2370201ffa236b427aa37c9996 | 1 + .../80c5002c297e86198535d1574f3e47f4c6900782 | 1 + .../86908cf9f9f2c690ebd7462cacd2b28fc45db3a5 | 1 + .../89681f4bac464df0f73f3bf3fd48db40bc5796a6 | 1 + .../8c371f137d8a10fc568a3022edf6f10280c96bc0 | 1 + .../8ff59012e5ea417bf340e59a0ab6fd9e410011ea | 1 + .../927ba76d456f6c3a06769795f87c46462e16acd5 | 1 + .../95a0311cd792a22e0de733080e0580638c8eb136 | 1 + .../9a3fc50025efc8aac22e16f10621f43a0f35ddcb | 1 + .../ad1bde0f01973fc121af6f4a1efef3a1447c3a80 | 1 + .../ad50e55369c0e4459500323e0af8eb00172f0909 | 1 + .../b2af1811e30009a2b7cec55e86f26b8f4bf3c1cf | 1 + .../bcebbda5c1d2033a62c14d6bf7e2dc54b407b403 | 1 + .../cfe75970181a6e2f5ddbf9cf968a9a1fba1a660b | 1 + .../d01798aa68e31f2964a2bc2c10b203f5186a3ba1 | 1 + .../d2dc25ea74a20f35c78091cbc1e30cf2557c315f | 1 + .../d930b69281035ce3b46f2635f5a91ee11f15876c | 1 + .../dcb92ca0399b135c88170124fcdab6b6d62d115c | 1 + .../e4e52eaca9ece89c0e3fa2ef7e6e126d47f20101 | 1 + .../ebe97f523d44667ea6e11e1c8cd9e5f02cf244e7 | 1 + .../ede1894116fdc6430ba23942e12d974e3f22875b | 1 + .../f0f8a948443b28c5e344e6e43aedbb479059779c | 1 + .../f1c4ffb8ea524f69bc8d6784f86b70ac99db2ad7 | 1 + .../f774dd0b8fc355505aab48a2107804387f45c4a0 | 1 + .../016d581dcda85465d5fa22eadd5947395b61303f | 1 + .../018287fe57cf372d929b52a560910284ba406de6 | 1 + .../048b75ce15485f8daa16e516014df26f2cf1ce09 | 1 + .../048fbb93cfd1be0ccf151cf75147e391f12c2778 | 1 + .../07e1d498149de21318fe486a73b618b800f9e46c | 1 + .../093873ef42d30bee0ec042892457333ce7d6cd65 | 1 + .../0a11f3691a10feefaeda9e30b4c891148d4ffdc1 | 1 + .../14394ce809fcbd4db56a1f1814fb322103148fc6 | 1 + .../251e9a9e4e077720a71747a94e24a6e25cb314a9 | 1 + .../2a855b2e35301abb51facebb20e78801b2219e0f | 1 + .../2ab8897ce59547952f047b48d52f9d6d4c0ea3eb | 1 + .../39f3fca5b4634a9004220b389122abedfef28db7 | 1 + .../3f18809f0df987b47c236d82d687ba16231daedb | 1 + .../4143d3a341877154d6e95211464e1df1015b74bd | 1 + .../49159cda7338d9b1a355631e7d33c55564fbfcb3 | 1 + .../4b40fa57bf6479d96f1a751c68578ee64e7b6759 | 1 + .../4f1a4d9e9b5db51ca7fdf0e9b4adb52f1813918b | 1 + .../54b8c977fd54171c25da05e420b13d05dfaad747 | 1 + .../5bfff0efcc1e7f8e3550798038d6361392379dfa | 1 + .../60f481551bdf721e8d1f2c805da6a7fb2d2e4858 | 1 + .../6c03daf8e568705738f646bfcdd03946acdce123 | 1 + .../6dcaf13f83e40d73a9c5180977b5bf171b812e11 | 1 + .../7511cc73615187b17837d01a013e4d886331fa54 | 1 + .../767e882d43ffb1c822233640f7ca4e33426f7a3b | 1 + .../777b43cd6a3f75e17d9310584ac00d4bd3ee65a1 | 1 + .../77d8be75df172643a6fb1ee235d05b904aa501a0 | 1 + .../7b8fa7a85f9f6ae6f9178504d2202666fb8dc772 | 1 + .../7d850a53909012d59b379e99f1549332f01e3088 | 1 + .../84f62a3184758e8c27e63f3f27c05a50877dc389 | 1 + .../851be661349b0ceabf7cd933409f4d79ca9ac9a8 | 1 + .../8efd1c91dceac73be0780f4f3cddce62ba84ebf6 | 1 + .../8f3feb9a63afb6e100dad08136d8cbeea4a1dfed | 1 + .../9100851463d47ef66626599f78730e4da01b8562 | 1 + .../9dfe5ccca3a5ed2e574fb1dd10ec6d98bf637484 | 1 + .../a5f6b7bdd3115f7427c55830f8fc4563256d410b | 1 + .../adc83b19e793491b1c6ea0fd8b46cd9f32e592fc | 0 .../ae13a961739ce96b0cb75d1fae1bd6eddaad473d | 1 + .../b15df036caa676bffff10933845c16ce0192f726 | 1 + .../b379e6c191cce7491020b50c345debd7ee95f14c | 1 + .../b7899610b3e9299cc4248ff090827e35962a7a81 | 1 + .../bad754935cc524ecde1a46997056db302ea3baa3 | 1 + .../bbf379045a2134133a43636c5e7807d2503f6261 | 1 + .../bdaa097dc0aecd0840c0e6e46ed269ccec14f5fe | 1 + .../be403e2dc82146c9730731e8facb7a996fbfa01a | 1 + .../c358d48b194741a6ed970a730c64c18c26076778 | 1 + .../c4dff26491e4d229af0efcae747a2b78fddd7b75 | 1 + .../c76f1c7d662c14d891f04058128ce7cd83bc09bd | 1 + .../dabc3a9d30f94608b2a68a5beb470c704890e3f1 | 1 + .../e0c30dfd58621aa24062fea9cf21d976e108a2f4 | 1 + .../e400db1ce2202210e9542f9f030c6404bf5718c8 | 1 + .../e5fa44f2b31c1fb553b6021e7360d07d5d91ff5e | 1 + .../eca60ab9f2f130ab10bf4a642d00690442dd2b74 | 1 + .../f25a33be8af0a7f725531c61f228d2339dfd089e | 1 + 138 files changed, 291 insertions(+), 2 deletions(-) create mode 100644 test/fuzz/CMakeLists.txt create mode 100644 test/fuzz/csv_fuzzer.c create mode 100644 test/fuzz/http_parser_fuzzer.c create mode 100644 test/fuzz/uri_fuzzer.c create mode 100644 test/static/corpus/csv/0c90a17c7b7cb5163255c7b775a8b544fb62c59d create mode 100644 test/static/corpus/csv/0e76fc3bb39bac4f774e2ce3978e9b59bc4a04b7 create mode 100644 test/static/corpus/csv/11f7ab7d8c2d4da313081e3282ab8120f0330b90 create mode 100644 test/static/corpus/csv/2ace62c1befa19e3ea37dd52be9f6d508c5163e6 create mode 100644 test/static/corpus/csv/3816dd9c9120fe6b3d806184874cc5c3e3a68df3 create mode 100644 test/static/corpus/csv/409c042df7f6a518972df1bc1287442e28719b6c create mode 100644 test/static/corpus/csv/46b175b37b5c3f746fde0dd114ead9938c012cfc create mode 100644 test/static/corpus/csv/56754bc8c7346ef9fad5c2ae9c54cef0af72629c create mode 100644 test/static/corpus/csv/58f43ea1e6df023f3dd4be9f718e5d0382fb9697 create mode 100644 test/static/corpus/csv/6d131d28c6e20c3a0a0b46c3aa7308d3029ab636 create mode 100644 test/static/corpus/csv/7560cb86c281018ee2f80d840fd8e6aa73a609ef create mode 100644 test/static/corpus/csv/7b97db755e18dc519fbc49e5de19b5820b9441c1 create mode 100644 test/static/corpus/csv/7ba0c88846fbcd28ad490df47f9ce4405d795303 create mode 100644 test/static/corpus/csv/8a14c110827e26cfdde3693bc5db6cd8673cdc8a create mode 100644 test/static/corpus/csv/94c1799dfba59120dd6075ee60016b809a915ca8 create mode 100644 test/static/corpus/csv/9e109f38c3948abe743bd116a34f05954f9ddd59 create mode 100644 test/static/corpus/csv/d21ec5eaaab3c8db1f923674b7a3ee19ce35ed13 create mode 100644 test/static/corpus/csv/d4a6d494cc2a06a0fee4873014210ec523d23750 create mode 100644 test/static/corpus/csv/eca825c9bdb276332d524134ddf370f7d18c2444 create mode 100644 test/static/corpus/csv/f9ba3e378e0409d66f946b31306de19edfe859bc create mode 100644 test/static/corpus/csv/fcad5471c295084dc1ec1eaa3753420e9803d251 create mode 100644 test/static/corpus/http_parser/038ec38a9ae7669590f1e3401da8ffbc9b40c8ee create mode 100644 test/static/corpus/http_parser/043fdf868410e66239819e0af10cbe787cfab77e create mode 100644 test/static/corpus/http_parser/08abe5308ce774ad5e9e435de386748ce06521f8 create mode 100644 test/static/corpus/http_parser/10b4f603cf09e21020e494e20a7f6760d5634c88 create mode 100644 test/static/corpus/http_parser/172ed6e56aa94bada4135d64fc2e0c5eac779aed create mode 100644 test/static/corpus/http_parser/1e42978bf6935a0f8377f6c480c315f1ae4627b8 create mode 100644 test/static/corpus/http_parser/1f6f26c085c3b209e626f90c63d0fb10526de318 create mode 100644 test/static/corpus/http_parser/24b458efa865a138a314f0876a11ec680b983ed5 create mode 100644 test/static/corpus/http_parser/2be565c790bc352f2cbaa0f3509e0bb6133b3616 create mode 100644 test/static/corpus/http_parser/30fd5e18f586b219916fb3892961a00e92db7a7f create mode 100644 test/static/corpus/http_parser/38918740212c6e5aea01f2fc8772b1d4aa89be8b create mode 100644 test/static/corpus/http_parser/39d4c74a90b7f5f5426b7982857b5fd7edbd8f77 create mode 100644 test/static/corpus/http_parser/3d21092e1a772d792e9dd720eeae30d8eb5ac2ad create mode 100644 test/static/corpus/http_parser/454dec613ca5c71fee0467061a88d2807201ac36 create mode 100644 test/static/corpus/http_parser/4f13053e61966dd08d7cd2407865481ec1d2cbf9 create mode 100644 test/static/corpus/http_parser/4ffa3daceb5b6ee3fc17cef3e0ae694ea5c583b5 create mode 100644 test/static/corpus/http_parser/501b3a19def06b352978c03ece7ab282cdc6cbab create mode 100644 test/static/corpus/http_parser/50ad023e1465919f5afbfffb33504ddfdc6a7e69 create mode 100644 test/static/corpus/http_parser/5177d236e3eea297b76c19194a4cf127b313b267 create mode 100644 test/static/corpus/http_parser/51e1354d16cfa4967e91206be8bd0d8c6ca577af create mode 100644 test/static/corpus/http_parser/55d5f3acc62809fc87466b2048b157e50e63d644 create mode 100644 test/static/corpus/http_parser/598d19d3175046ff035be228db2f1dc43957d8e2 create mode 100644 test/static/corpus/http_parser/5b7444e43e5c1fd759483c76a46a5536da043022 create mode 100644 test/static/corpus/http_parser/5c489184823a8a42844066fd5b4951bbad86934f create mode 100644 test/static/corpus/http_parser/6455c0e761cd485f3e38c470f216560bfc46eea4 create mode 100644 test/static/corpus/http_parser/6596bb363cfd0bef3f68e265cfd824c6d37ecb7e create mode 100644 test/static/corpus/http_parser/66d5206b6c75b071f57684399218d144b0d337ea create mode 100644 test/static/corpus/http_parser/685073a623772c1d46475b5eb451de40247d697f create mode 100644 test/static/corpus/http_parser/71d4de855c1af66a6a5cea88b1eddc5032ecb26d create mode 100644 test/static/corpus/http_parser/73378b09c61739b65a5f63214f0515307f187e53 create mode 100644 test/static/corpus/http_parser/768abf119ce4fa6c9a9ee9f17d9dacb32c61759c create mode 100644 test/static/corpus/http_parser/7ae1560be3dffbb88f3e48de3a4be99ecdc61441 create mode 100644 test/static/corpus/http_parser/7c4d33785daa5c2370201ffa236b427aa37c9996 create mode 100644 test/static/corpus/http_parser/80c5002c297e86198535d1574f3e47f4c6900782 create mode 100644 test/static/corpus/http_parser/86908cf9f9f2c690ebd7462cacd2b28fc45db3a5 create mode 100644 test/static/corpus/http_parser/89681f4bac464df0f73f3bf3fd48db40bc5796a6 create mode 100644 test/static/corpus/http_parser/8c371f137d8a10fc568a3022edf6f10280c96bc0 create mode 100644 test/static/corpus/http_parser/8ff59012e5ea417bf340e59a0ab6fd9e410011ea create mode 100644 test/static/corpus/http_parser/927ba76d456f6c3a06769795f87c46462e16acd5 create mode 100644 test/static/corpus/http_parser/95a0311cd792a22e0de733080e0580638c8eb136 create mode 100644 test/static/corpus/http_parser/9a3fc50025efc8aac22e16f10621f43a0f35ddcb create mode 100644 test/static/corpus/http_parser/ad1bde0f01973fc121af6f4a1efef3a1447c3a80 create mode 100644 test/static/corpus/http_parser/ad50e55369c0e4459500323e0af8eb00172f0909 create mode 100644 test/static/corpus/http_parser/b2af1811e30009a2b7cec55e86f26b8f4bf3c1cf create mode 100644 test/static/corpus/http_parser/bcebbda5c1d2033a62c14d6bf7e2dc54b407b403 create mode 100644 test/static/corpus/http_parser/cfe75970181a6e2f5ddbf9cf968a9a1fba1a660b create mode 100644 test/static/corpus/http_parser/d01798aa68e31f2964a2bc2c10b203f5186a3ba1 create mode 100644 test/static/corpus/http_parser/d2dc25ea74a20f35c78091cbc1e30cf2557c315f create mode 100644 test/static/corpus/http_parser/d930b69281035ce3b46f2635f5a91ee11f15876c create mode 100644 test/static/corpus/http_parser/dcb92ca0399b135c88170124fcdab6b6d62d115c create mode 100644 test/static/corpus/http_parser/e4e52eaca9ece89c0e3fa2ef7e6e126d47f20101 create mode 100644 test/static/corpus/http_parser/ebe97f523d44667ea6e11e1c8cd9e5f02cf244e7 create mode 100644 test/static/corpus/http_parser/ede1894116fdc6430ba23942e12d974e3f22875b create mode 100644 test/static/corpus/http_parser/f0f8a948443b28c5e344e6e43aedbb479059779c create mode 100644 test/static/corpus/http_parser/f1c4ffb8ea524f69bc8d6784f86b70ac99db2ad7 create mode 100644 test/static/corpus/http_parser/f774dd0b8fc355505aab48a2107804387f45c4a0 create mode 100644 test/static/corpus/uri/016d581dcda85465d5fa22eadd5947395b61303f create mode 100644 test/static/corpus/uri/018287fe57cf372d929b52a560910284ba406de6 create mode 100644 test/static/corpus/uri/048b75ce15485f8daa16e516014df26f2cf1ce09 create mode 100644 test/static/corpus/uri/048fbb93cfd1be0ccf151cf75147e391f12c2778 create mode 100644 test/static/corpus/uri/07e1d498149de21318fe486a73b618b800f9e46c create mode 100644 test/static/corpus/uri/093873ef42d30bee0ec042892457333ce7d6cd65 create mode 100644 test/static/corpus/uri/0a11f3691a10feefaeda9e30b4c891148d4ffdc1 create mode 100644 test/static/corpus/uri/14394ce809fcbd4db56a1f1814fb322103148fc6 create mode 100644 test/static/corpus/uri/251e9a9e4e077720a71747a94e24a6e25cb314a9 create mode 100644 test/static/corpus/uri/2a855b2e35301abb51facebb20e78801b2219e0f create mode 100644 test/static/corpus/uri/2ab8897ce59547952f047b48d52f9d6d4c0ea3eb create mode 100644 test/static/corpus/uri/39f3fca5b4634a9004220b389122abedfef28db7 create mode 100644 test/static/corpus/uri/3f18809f0df987b47c236d82d687ba16231daedb create mode 100644 test/static/corpus/uri/4143d3a341877154d6e95211464e1df1015b74bd create mode 100644 test/static/corpus/uri/49159cda7338d9b1a355631e7d33c55564fbfcb3 create mode 100644 test/static/corpus/uri/4b40fa57bf6479d96f1a751c68578ee64e7b6759 create mode 100644 test/static/corpus/uri/4f1a4d9e9b5db51ca7fdf0e9b4adb52f1813918b create mode 100644 test/static/corpus/uri/54b8c977fd54171c25da05e420b13d05dfaad747 create mode 100644 test/static/corpus/uri/5bfff0efcc1e7f8e3550798038d6361392379dfa create mode 100644 test/static/corpus/uri/60f481551bdf721e8d1f2c805da6a7fb2d2e4858 create mode 100644 test/static/corpus/uri/6c03daf8e568705738f646bfcdd03946acdce123 create mode 100644 test/static/corpus/uri/6dcaf13f83e40d73a9c5180977b5bf171b812e11 create mode 100644 test/static/corpus/uri/7511cc73615187b17837d01a013e4d886331fa54 create mode 100644 test/static/corpus/uri/767e882d43ffb1c822233640f7ca4e33426f7a3b create mode 100644 test/static/corpus/uri/777b43cd6a3f75e17d9310584ac00d4bd3ee65a1 create mode 100644 test/static/corpus/uri/77d8be75df172643a6fb1ee235d05b904aa501a0 create mode 100644 test/static/corpus/uri/7b8fa7a85f9f6ae6f9178504d2202666fb8dc772 create mode 100644 test/static/corpus/uri/7d850a53909012d59b379e99f1549332f01e3088 create mode 100644 test/static/corpus/uri/84f62a3184758e8c27e63f3f27c05a50877dc389 create mode 100644 test/static/corpus/uri/851be661349b0ceabf7cd933409f4d79ca9ac9a8 create mode 100644 test/static/corpus/uri/8efd1c91dceac73be0780f4f3cddce62ba84ebf6 create mode 100644 test/static/corpus/uri/8f3feb9a63afb6e100dad08136d8cbeea4a1dfed create mode 100644 test/static/corpus/uri/9100851463d47ef66626599f78730e4da01b8562 create mode 100644 test/static/corpus/uri/9dfe5ccca3a5ed2e574fb1dd10ec6d98bf637484 create mode 100644 test/static/corpus/uri/a5f6b7bdd3115f7427c55830f8fc4563256d410b create mode 100644 test/static/corpus/uri/adc83b19e793491b1c6ea0fd8b46cd9f32e592fc create mode 100644 test/static/corpus/uri/ae13a961739ce96b0cb75d1fae1bd6eddaad473d create mode 100644 test/static/corpus/uri/b15df036caa676bffff10933845c16ce0192f726 create mode 100644 test/static/corpus/uri/b379e6c191cce7491020b50c345debd7ee95f14c create mode 100644 test/static/corpus/uri/b7899610b3e9299cc4248ff090827e35962a7a81 create mode 100644 test/static/corpus/uri/bad754935cc524ecde1a46997056db302ea3baa3 create mode 100644 test/static/corpus/uri/bbf379045a2134133a43636c5e7807d2503f6261 create mode 100644 test/static/corpus/uri/bdaa097dc0aecd0840c0e6e46ed269ccec14f5fe create mode 100644 test/static/corpus/uri/be403e2dc82146c9730731e8facb7a996fbfa01a create mode 100644 test/static/corpus/uri/c358d48b194741a6ed970a730c64c18c26076778 create mode 100644 test/static/corpus/uri/c4dff26491e4d229af0efcae747a2b78fddd7b75 create mode 100644 test/static/corpus/uri/c76f1c7d662c14d891f04058128ce7cd83bc09bd create mode 100644 test/static/corpus/uri/dabc3a9d30f94608b2a68a5beb470c704890e3f1 create mode 100644 test/static/corpus/uri/e0c30dfd58621aa24062fea9cf21d976e108a2f4 create mode 100644 test/static/corpus/uri/e400db1ce2202210e9542f9f030c6404bf5718c8 create mode 100644 test/static/corpus/uri/e5fa44f2b31c1fb553b6021e7360d07d5d91ff5e create mode 100644 test/static/corpus/uri/eca60ab9f2f130ab10bf4a642d00690442dd2b74 create mode 100644 test/static/corpus/uri/f25a33be8af0a7f725531c61f228d2339dfd089e -- 2.25.1 ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Tarantool-patches] [PATCH 1/4] test: add infrastructure for fuzzing testing and fuzzers 2020-11-30 20:24 [Tarantool-patches] [PATCH 0/4] Add fuzzing testing sergeyb @ 2020-11-30 20:24 ` sergeyb 2020-12-07 17:24 ` Igor Munkin 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 2/4] test: add corpus to be used with fuzzers sergeyb ` (6 subsequent siblings) 7 siblings, 1 reply; 26+ messages in thread From: sergeyb @ 2020-11-30 20:24 UTC (permalink / raw) To: tarantool-patches, imun, sergepetrenko From: Sergey Bronnikov <sergeyb@tarantool.org> There is a number of bugs related to parsing and encoding/decoding data. Examples: - csv: #2692, #4497, #2692 - uri: #585 One of the effective method to find such issues is a fuzzing testing. Patch introduce a CMake flag to enable building fuzzers (ENABLE_FUZZER) and add fuzzers based on LibFuzzer [1] to csv, http_parser and uri modules. NOTE: LibFuzzer requires Clang compiler. [1] https://llvm.org/docs/LibFuzzer.html How-To Use: $ mkdir build && cd build $ CC=clang CXX=clang++ cmake -DENABLE_FUZZER=ON -DENABLE_ASAN=ON -DCMAKE_BUILD_TYPE=Debug .. $ make fuzzers $ ./test/fuzz/csv_fuzzer -max_total_time=60*60*60 -workers=4 ../test/static/corpus/csv Part of #1809 --- CMakeLists.txt | 2 +- cmake/profile.cmake | 13 ++++++++++ test/CMakeLists.txt | 3 +++ test/fuzz/CMakeLists.txt | 45 ++++++++++++++++++++++++++++++++++ test/fuzz/csv_fuzzer.c | 23 +++++++++++++++++ test/fuzz/http_parser_fuzzer.c | 18 ++++++++++++++ test/fuzz/uri_fuzzer.c | 19 ++++++++++++++ 7 files changed, 122 insertions(+), 1 deletion(-) create mode 100644 test/fuzz/CMakeLists.txt create mode 100644 test/fuzz/csv_fuzzer.c create mode 100644 test/fuzz/http_parser_fuzzer.c create mode 100644 test/fuzz/uri_fuzzer.c diff --git a/CMakeLists.txt b/CMakeLists.txt index fa6818f8e..8bc6f22bd 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -607,7 +607,7 @@ set(PREFIX ${CMAKE_INSTALL_PREFIX}) set(options PACKAGE VERSION BUILD C_COMPILER CXX_COMPILER C_FLAGS CXX_FLAGS PREFIX ENABLE_SSE2 ENABLE_AVX - ENABLE_GCOV ENABLE_GPROF ENABLE_VALGRIND ENABLE_ASAN ENABLE_UB_SANITIZER + ENABLE_GCOV ENABLE_GPROF ENABLE_VALGRIND ENABLE_ASAN ENABLE_UB_SANITIZER ENABLE_FUZZER ENABLE_BACKTRACE ENABLE_DOC ENABLE_DIST diff --git a/cmake/profile.cmake b/cmake/profile.cmake index bc4bf67f5..45e3d112c 100644 --- a/cmake/profile.cmake +++ b/cmake/profile.cmake @@ -42,6 +42,19 @@ else() add_definitions(-DNVALGRIND=1) endif() +option(OSS_FUZZ "Set this option to use flags by oss-fuzz" OFF) +option(ENABLE_FUZZER "Enable fuzzing testing" OFF) +if(ENABLE_FUZZER) + if(CMAKE_COMPILER_IS_GNUCC) + message(FATAL_ERROR + "\n" + "Fuzzing is unsupported with GCC compiler. Use Clang:\n" + " $ git clean -xfd; git submodule foreach --recursive git clean -xfd\n" + " $ CC=clang CXX=clang++ cmake . <...> -DENABLE_FUZZER=ON && make -j\n" + "\n") + endif() +endif() + option(ENABLE_ASAN "Enable AddressSanitizer, a fast memory error detector based on compiler instrumentation" OFF) if (ENABLE_ASAN) if (CMAKE_COMPILER_IS_GNUCC) diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt index 10882c6a1..d20a4eb5d 100644 --- a/test/CMakeLists.txt +++ b/test/CMakeLists.txt @@ -75,6 +75,9 @@ add_subdirectory(app-tap) add_subdirectory(box) add_subdirectory(box-tap) add_subdirectory(unit) +if(ENABLE_FUZZER) + add_subdirectory(fuzz) +endif() add_subdirectory(${PROJECT_SOURCE_DIR}/third_party/luajit/test ${PROJECT_BINARY_DIR}/third_party/luajit/test) diff --git a/test/fuzz/CMakeLists.txt b/test/fuzz/CMakeLists.txt new file mode 100644 index 000000000..142d38f67 --- /dev/null +++ b/test/fuzz/CMakeLists.txt @@ -0,0 +1,45 @@ +include_directories(${PROJECT_SOURCE_DIR}/src) +include_directories(${PROJECT_BINARY_DIR}/src) +include_directories(${PROJECT_SOURCE_DIR}/src/box) + +# A special target with fuzzer and sanitizer flags. +add_library(fuzzer_config INTERFACE) + +target_compile_options( + fuzzer_config + INTERFACE + $<$<BOOL:${ENABLE_ASAN}>: + -fsanitize=fuzzer,address + > + $<$<BOOL:${ENABLE_UB_SANITIZER}>: + -fsanitize=fuzzer,undefined + > +) +target_link_libraries( + fuzzer_config + INTERFACE + $<$<BOOL:${ENABLE_ASAN}>: + -fsanitize=fuzzer,address + > + $<$<BOOL:${ENABLE_UB_SANITIZER}>: + -fsanitize=fuzzer,undefined + > +) + +# Use PUBLIC to force 'fuzzer_config' for all dependent targets. +add_executable(csv_fuzzer csv_fuzzer.c) +target_link_libraries(csv_fuzzer PUBLIC csv fuzzer_config) + +add_executable(uri_fuzzer uri_fuzzer.c) +target_link_libraries(uri_fuzzer PUBLIC uri fuzzer_config) + +add_executable(http_parser_fuzzer http_parser_fuzzer.c) +target_link_libraries(http_parser_fuzzer PUBLIC http_parser fuzzer_config) + +set(fuzzing_binaries csv_fuzzer + http_parser_fuzzer + uri_fuzzer) + +add_custom_target(fuzzers + DEPENDS ${fuzzing_binaries} + COMMENT "Build fuzzers") diff --git a/test/fuzz/csv_fuzzer.c b/test/fuzz/csv_fuzzer.c new file mode 100644 index 000000000..8853d6308 --- /dev/null +++ b/test/fuzz/csv_fuzzer.c @@ -0,0 +1,23 @@ +#include <stdint.h> +#include <stdlib.h> +#include <stddef.h> +#include <string.h> +#include "csv/csv.h" + +int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { + struct csv csv; + csv_create(&csv); + char *buf = calloc(size, sizeof(char*)); + if (buf == NULL) + return -1; + memcpy(buf, data, size); + buf[size] = '\0'; + char *end = buf + size; + csv_parse_chunk(&csv, buf, end); + csv_finish_parsing(&csv); + int rc = csv_get_error_status(&csv) == CSV_ER_INVALID ? 1 : 0; + csv_destroy(&csv); + free(buf); + + return rc; +} diff --git a/test/fuzz/http_parser_fuzzer.c b/test/fuzz/http_parser_fuzzer.c new file mode 100644 index 000000000..a0aaf6786 --- /dev/null +++ b/test/fuzz/http_parser_fuzzer.c @@ -0,0 +1,18 @@ +#include <stdlib.h> +#include <stdint.h> +#include <stddef.h> +#include "http_parser/http_parser.h" + +int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { + struct http_parser parser; + char *buf = (char*)data; + http_parser_create(&parser); + parser.hdr_name = (char *)calloc((int)size, sizeof(char)); + if (parser.hdr_name == NULL) + return -1; + char *end_buf = buf + size; + int rc = http_parse_header_line(&parser, &buf, end_buf, size); + free(parser.hdr_name); + + return rc; +} diff --git a/test/fuzz/uri_fuzzer.c b/test/fuzz/uri_fuzzer.c new file mode 100644 index 000000000..8397505bd --- /dev/null +++ b/test/fuzz/uri_fuzzer.c @@ -0,0 +1,19 @@ +#include <stdlib.h> +#include <stdint.h> +#include <stddef.h> +#include <string.h> +#include "uri/uri.h" + +int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) +{ + char *buf = calloc(size, sizeof(char*)); + if (!buf) + return -1; + strncpy(buf, (char*)data, size); + buf[size] = '\0'; + struct uri uri; + int rc = uri_parse(&uri, buf); + free(buf); + + return rc; +} -- 2.25.1 ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 1/4] test: add infrastructure for fuzzing testing and fuzzers 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 1/4] test: add infrastructure for fuzzing testing and fuzzers sergeyb @ 2020-12-07 17:24 ` Igor Munkin 2020-12-07 19:54 ` Igor Munkin 2020-12-13 18:56 ` Sergey Bronnikov 0 siblings, 2 replies; 26+ messages in thread From: Igor Munkin @ 2020-12-07 17:24 UTC (permalink / raw) To: sergeyb; +Cc: tarantool-patches Sergey, Thanks for the patch! Please consider the remaining comments below. On 30.11.20, sergeyb@tarantool.org wrote: > From: Sergey Bronnikov <sergeyb@tarantool.org> > > There is a number of bugs related to parsing and encoding/decoding data. > Examples: > > - csv: #2692, #4497, #2692 > - uri: #585 > > One of the effective method to find such issues is a fuzzing testing. > Patch introduce a CMake flag to enable building fuzzers (ENABLE_FUZZER) Typo: s/introduce/introduces/. > and add fuzzers based on LibFuzzer [1] to csv, http_parser and uri modules. > NOTE: LibFuzzer requires Clang compiler. > > [1] https://llvm.org/docs/LibFuzzer.html > > How-To Use: > > $ mkdir build && cd build > $ CC=clang CXX=clang++ cmake -DENABLE_FUZZER=ON -DENABLE_ASAN=ON -DCMAKE_BUILD_TYPE=Debug .. > $ make fuzzers > $ ./test/fuzz/csv_fuzzer -max_total_time=60*60*60 -workers=4 ../test/static/corpus/csv I tried your recipe for the current revision and got the following: | $ ./test/fuzz/csv_fuzzer -max_total_time=60*60*60 -workers=4 ../test/static/corpus/csv | INFO: Seed: 2899369680 | INFO: Loaded 1 modules (3 inline 8-bit counters): 3 [0x57a130, 0x57a133), | INFO: Loaded 1 PC tables (3 PCs): 3 [0x553870,0x5538a0), | No such file or directory: ../test/static/corpus/csv; exiting AFAICS, the required directory is added in the following patch, so I checkout the branch HEAD and try once more: | $ ./test/fuzz/csv_fuzzer -max_total_time=60*60*60 -workers=4 ../test/static/corpus/csv | INFO: Seed: 1838565059 | INFO: Loaded 1 modules (3 inline 8-bit counters): 3 [0x57a130, 0x57a133), | INFO: Loaded 1 PC tables (3 PCs): 3 [0x553870,0x5538a0), | INFO: 21 files found in ../test/static/corpus/csv | INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes | INFO: seed corpus: files: 21 min: 1b max: 462b total: 1336b rss: 27Mb | csv_fuzzer: /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/fuzzer/FuzzerLoop.cpp:537: void fuzzer::Fuzzer::ExecuteCallback(const uint8_t *, size_t): Assertion `Res == 0' failed. | ==15230== ERROR: libFuzzer: deadly signal | #0 0x507287 in __sanitizer_print_stack_trace /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/asan/asan_stack.cc:38:3 | #1 0x44f978 in fuzzer::PrintStackTrace() /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/fuzzer/FuzzerUtil.cpp:206:5 | #2 0x4300f3 in fuzzer::Fuzzer::CrashCallback() /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/fuzzer/FuzzerLoop.cpp:237:3 | #3 0x4300b0 in fuzzer::Fuzzer::StaticCrashSignalCallback() /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/fuzzer/FuzzerLoop.cpp:209:6 | #4 0x7f179300c8bf (/lib64/libpthread.so.0+0x148bf) | #5 0x7f1792bfdf3a in gsignal (/lib64/libc.so.6+0x38f3a) | #6 0x7f1792be7534 in abort (/lib64/libc.so.6+0x22534) | #7 0x7f1792be740e in __tls_get_addr (/lib64/libc.so.6+0x2240e) | #8 0x7f1792bf5731 in __assert_fail (/lib64/libc.so.6+0x30731) | #9 0x431d06 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/fuzzer/FuzzerLoop.cpp:537:5 | #10 0x4310d5 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/fuzzer/FuzzerLoop.cpp:455:3 | #11 0x433aad in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, fuzzer::fuzzer_allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/fuzzer/FuzzerLoop.cpp:745:7 | #12 0x434240 in fuzzer::Fuzzer::Loop(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, fuzzer::fuzzer_allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/fuzzer/FuzzerLoop.cpp:768:3 | #13 0x425e60 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/fuzzer/FuzzerDriver.cpp:760:6 | #14 0x450132 in main /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/fuzzer/FuzzerMain.cpp:20:10 | #15 0x7f1792be8eda in __libc_start_main (/lib64/libc.so.6+0x23eda) | #16 0x41e919 in _start (/tarantool/build/test/fuzz/csv_fuzzer+0x41e919) | | NOTE: libFuzzer has rudimentary signal handlers. | Combine libFuzzer with AddressSanitizer or similar for better crash reports. | SUMMARY: libFuzzer: deadly signal | MS: 0 ; base unit: 0000000000000000000000000000000000000000 | 0x22,0x61,0x62,0x63,0x22,0x2c,0x20,0x22,0x77,0x69,0x74,0x68,0x2c,0x63,0x6f,0x6d,0x6d,0x61,0x22,0x2c,0x20,0x22,0x5c,0x22,0x69,0x6e,0x20,0x71,0x75,0x6f,0x74,0x65,0x73,0x5c,0x22,0x22,0x2c,0x20,0x22,0x31,0x20,0x5c,0x22,0x20,0x71,0x75,0x6f,0x74,0x65,0x22,0xa, \"abc\", \"with,comma\", \"\\\"in quotes\\\"\", \"1 \\\" quote\"\x0a | artifact_prefix='./'; Test unit written to ./crash-6d131d28c6e20c3a0a0b46c3aa7308d3029ab636 | Base64: ImFiYyIsICJ3aXRoLGNvbW1hIiwgIlwiaW4gcXVvdGVzXCIiLCAiMSBcIiBxdW90ZSIK I have no idea whether it is OK but this does look like it's not. Maybe there are some problems with my compiler/sanitizer? JFYI, the toolchain is the following: | $ clang -v | clang version 8.0.1 (tags/RELEASE_801/final) | Target: x86_64-pc-linux-gnu | Thread model: posix | InstalledDir: /usr/lib/llvm/8/bin | Selected GCC installation: /usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0 | Candidate multilib: .;@m64 | Candidate multilib: 32;@m32 | Selected multilib: .;@m64 | $ clang++ -v | clang version 8.0.1 (tags/RELEASE_801/final) | Target: x86_64-pc-linux-gnu | Thread model: posix | InstalledDir: /usr/lib/llvm/8/bin | Selected GCC installation: /usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0 | Candidate multilib: .;@m64 | Candidate multilib: 32;@m32 | Selected multilib: .;@m64 > > Part of #1809 > --- > CMakeLists.txt | 2 +- > cmake/profile.cmake | 13 ++++++++++ > test/CMakeLists.txt | 3 +++ > test/fuzz/CMakeLists.txt | 45 ++++++++++++++++++++++++++++++++++ > test/fuzz/csv_fuzzer.c | 23 +++++++++++++++++ > test/fuzz/http_parser_fuzzer.c | 18 ++++++++++++++ > test/fuzz/uri_fuzzer.c | 19 ++++++++++++++ > 7 files changed, 122 insertions(+), 1 deletion(-) > create mode 100644 test/fuzz/CMakeLists.txt > create mode 100644 test/fuzz/csv_fuzzer.c > create mode 100644 test/fuzz/http_parser_fuzzer.c > create mode 100644 test/fuzz/uri_fuzzer.c > <snipped> > diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt > index 10882c6a1..d20a4eb5d 100644 > --- a/test/CMakeLists.txt > +++ b/test/CMakeLists.txt > @@ -75,6 +75,9 @@ add_subdirectory(app-tap) > add_subdirectory(box) > add_subdirectory(box-tap) > add_subdirectory(unit) > +if(ENABLE_FUZZER) > + add_subdirectory(fuzz) > +endif() Minor: Well, I don't get the idea *why* this change is added here: it neither takes the place in alphabetical order, nor is added to the end of the list. > add_subdirectory(${PROJECT_SOURCE_DIR}/third_party/luajit/test > ${PROJECT_BINARY_DIR}/third_party/luajit/test) > > diff --git a/test/fuzz/CMakeLists.txt b/test/fuzz/CMakeLists.txt > new file mode 100644 > index 000000000..142d38f67 > --- /dev/null > +++ b/test/fuzz/CMakeLists.txt > @@ -0,0 +1,45 @@ > +include_directories(${PROJECT_SOURCE_DIR}/src) > +include_directories(${PROJECT_BINARY_DIR}/src) Minor: It would be nice to explicitly mention the line above is added for autogenerated headers and LuaJIT ones. Feel free to ignore. > +include_directories(${PROJECT_SOURCE_DIR}/src/box) > + > +# A special target with fuzzer and sanitizer flags. > +add_library(fuzzer_config INTERFACE) > + > +target_compile_options( > + fuzzer_config > + INTERFACE > + $<$<BOOL:${ENABLE_ASAN}>: > + -fsanitize=fuzzer,address > + > > + $<$<BOOL:${ENABLE_UB_SANITIZER}>: > + -fsanitize=fuzzer,undefined > + > > +) > +target_link_libraries( > + fuzzer_config > + INTERFACE > + $<$<BOOL:${ENABLE_ASAN}>: > + -fsanitize=fuzzer,address > + > > + $<$<BOOL:${ENABLE_UB_SANITIZER}>: > + -fsanitize=fuzzer,undefined > + > > +) OK, I ran <make fuzzers> with more verbose output and have two notes regarding it. | Scanning dependencies of target csv | make[3]: Leaving directory '/tarantool/build' | make -f src/lib/csv/CMakeFiles/csv.dir/build.make src/lib/csv/CMakeFiles/csv.dir/build | make[3]: Entering directory '/tarantool/build' | [ 0%] Building C object src/lib/csv/CMakeFiles/csv.dir/csv.c.o | cd /tarantool/build/src/lib/csv && /usr/lib/llvm/9/bin/clang-9 -DCORO_ASM | -DLUAJIT_SMART_STRINGS=1 -DLUAJIT_USE_ASAN=1 -DLUA_USE_APICHECK=1 | -DLUA_USE_ASSERT=1 -DNVALGRIND=1 -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE | -D__STDC_CONSTANT_MACROS=1 -D__STDC_FORMAT_MACROS=1 -D__STDC_LIMIT_MACROS=1 | -I/tarantool/src -I/tarantool/build/src -I/tarantool/src/lib | -I/tarantool/src/lib/small -I/tarantool/src/lib/small/third_party | -I/tarantool/src/lib/core -I/tarantool -I/tarantool/third_party/zstd/lib | -I/tarantool/third_party/zstd/lib/common -I/tarantool/build/third_party | -I/tarantool/third_party -I/tarantool/third_party/coro | -I/tarantool/third_party/luajit/src -I/tarantool/third_party/libyaml/include | -I/tarantool/src/lib/msgpuck -I/tarantool/build/build/curl/dest/include | -I/tarantool/build/third_party/decNumber | -I/tarantool/third_party/libutil_freebsd -fexceptions -funwind-tables | -fno-omit-frame-pointer -fno-stack-protector -fno-common -fopenmp -msse2 | -fsanitize=address -fsanitize-blacklist=/tarantool/asan/asan.supp | -std=c11 -Wall -Wextra -Wno-strict-aliasing -Wno-char-subscripts | -Wno-gnu-alignof-expression -Werror -g -ggdb -O0 -UASAN_INTERFACE_OLD | -o CMakeFiles/csv.dir/csv.c.o -c /tarantool/src/lib/csv/csv.c | [ 0%] Linking C static library libcsv.a | cd /tarantool/build/src/lib/csv && /usr/bin/cmake -P CMakeFiles/csv.dir/cmake_clean_target.cmake | cd /tarantool/build/src/lib/csv && /usr/bin/cmake -E cmake_link_script CMakeFiles/csv.dir/link.txt --verbose=1 | /usr/bin/ar qc libcsv.a CMakeFiles/csv.dir/csv.c.o | /usr/bin/ranlib libcsv.a | make[3]: Leaving directory '/tarantool/build' | [ 0%] Built target csv | make -f test/fuzz/CMakeFiles/csv_fuzzer.dir/build.make test/fuzz/CMakeFiles/csv_fuzzer.dir/depend | make[3]: Entering directory '/tarantool/build' | cd /tarantool/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" | /tarantool /tarantool/test/fuzz /tarantool/build /tarantool/build/test/fuzz | /tarantool/build/test/fuzz/CMakeFiles/csv_fuzzer.dir/DependInfo.cmake --color= | Dependee "/tarantool/build/test/fuzz/CMakeFiles/csv_fuzzer.dir/DependInfo.cmake" | is newer than depender "/tarantool/build/test/fuzz/CMakeFiles/csv_fuzzer.dir/depend.internal". | Dependee "/tarantool/build/test/fuzz/CMakeFiles/CMakeDirectoryInformation.cmake" | is newer than depender "/tarantool/build/test/fuzz/CMakeFiles/csv_fuzzer.dir/depend.internal". | Scanning dependencies of target csv_fuzzer | make[3]: Leaving directory '/tarantool/build' | make -f test/fuzz/CMakeFiles/csv_fuzzer.dir/build.make test/fuzz/CMakeFiles/csv_fuzzer.dir/build | make[3]: Entering directory '/tarantool/build' | [ 0%] Building C object test/fuzz/CMakeFiles/csv_fuzzer.dir/csv_fuzzer.c.o | cd /tarantool/build/test/fuzz && /usr/lib/llvm/9/bin/clang-9 -DCORO_ASM | -DLUAJIT_SMART_STRINGS=1 -DLUAJIT_USE_ASAN=1 -DLUA_USE_APICHECK=1 | -DLUA_USE_ASSERT=1 -DNVALGRIND=1 -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE | -D__STDC_CONSTANT_MACROS=1 -D__STDC_FORMAT_MACROS=1 -D__STDC_LIMIT_MACROS=1 | -I/tarantool/src -I/tarantool/build/src -I/tarantool/src/lib | -I/tarantool/src/lib/small -I/tarantool/src/lib/small/third_party | -I/tarantool/src/lib/core -I/tarantool -I/tarantool/third_party/zstd/lib | -I/tarantool/third_party/zstd/lib/common -I/tarantool/third_party/luajit/src | -I/tarantool/src/lib/msgpuck -I/tarantool/src/box -fexceptions | -funwind-tables -fno-omit-frame-pointer -fno-stack-protector -fno-common -fopenmp -msse2 | -fsanitize=address -fsanitize-blacklist=/tarantool/asan/asan.supp | -std=c11 -Wall -Wextra -Wno-strict-aliasing -Wno-char-subscripts | -Wno-gnu-alignof-expression -Werror -Wno-unused-parameter -g -ggdb -O0 | -UASAN_INTERFACE_OLD -fsanitize=fuzzer,address | -o CMakeFiles/csv_fuzzer.dir/csv_fuzzer.c.o -c /tarantool/test/fuzz/csv_fuzzer.c | [ 0%] Linking C executable csv_fuzzer | cd /tarantool/build/test/fuzz && /usr/bin/cmake -E cmake_link_script CMakeFiles/csv_fuzzer.dir/link.txt --verbose=1 | /usr/lib/llvm/9/bin/clang-9 -fexceptions -funwind-tables | -fno-omit-frame-pointer -fno-stack-protector -fno-common -fopenmp -msse2 | -fsanitize=address -fsanitize-blacklist=/tarantool/asan/asan.supp | -std=c11 -Wall -Wextra -Wno-strict-aliasing -Wno-char-subscripts | -Wno-gnu-alignof-expression -Werror -Wno-unused-parameter -g -ggdb -O0 | -rdynamic CMakeFiles/csv_fuzzer.dir/csv_fuzzer.c.o -o csv_fuzzer | ../../src/lib/csv/libcsv.a -fsanitize=fuzzer,address | make[3]: Leaving directory '/tarantool/build' | [ 0%] Built target csv_fuzzer 1. I'm totally not an expert, but quite confused with the fact the libcsv is build w/o <fuzzer> flag, but csv_fuzzer is build with it. 2. Do you need to specify <address> flag once more, when ASAN is enabled? If not the hunk above looks excess, doesn't it? > + <snipped> > + > +set(fuzzing_binaries csv_fuzzer > + http_parser_fuzzer > + uri_fuzzer) Spaces are used for indentation in CMake-related sources, not tabs. Surprisingly, you made it the right way below. > + > +add_custom_target(fuzzers > + DEPENDS ${fuzzing_binaries} > + COMMENT "Build fuzzers") > diff --git a/test/fuzz/csv_fuzzer.c b/test/fuzz/csv_fuzzer.c > new file mode 100644 > index 000000000..8853d6308 > --- /dev/null > +++ b/test/fuzz/csv_fuzzer.c */me feeling myself like a parrot* Why do you violate our style guides[1] using spaces instead of tabs for indentation? IIRC I've already mentioned it here[2]... Otherwise this hunk looks fine. > @@ -0,0 +1,23 @@ <snipped> > diff --git a/test/fuzz/http_parser_fuzzer.c b/test/fuzz/http_parser_fuzzer.c > new file mode 100644 > index 000000000..a0aaf6786 > --- /dev/null > +++ b/test/fuzz/http_parser_fuzzer.c Ditto. > @@ -0,0 +1,18 @@ <snipped> > diff --git a/test/fuzz/uri_fuzzer.c b/test/fuzz/uri_fuzzer.c > new file mode 100644 > index 000000000..8397505bd > --- /dev/null > +++ b/test/fuzz/uri_fuzzer.c Ditto. > @@ -0,0 +1,19 @@ <snipped> > 2.25.1 > [1]: https://www.tarantool.io/en/doc/latest/dev_guide/c_style_guide/#chapter-1-indentation [2]: https://lists.tarantool.org/pipermail/tarantool-patches/2020-April/016409.html -- Best regards, IM ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 1/4] test: add infrastructure for fuzzing testing and fuzzers 2020-12-07 17:24 ` Igor Munkin @ 2020-12-07 19:54 ` Igor Munkin 2020-12-13 18:56 ` Sergey Bronnikov 1 sibling, 0 replies; 26+ messages in thread From: Igor Munkin @ 2020-12-07 19:54 UTC (permalink / raw) To: sergeyb; +Cc: tarantool-patches > > */me feeling myself like a parrot* Well, now I see I'm quite bad both in idioms and English grammar. Actually, I meant I am tired of repeating the same again and again, and nothing vulgar of course. Hope, nobody tried to visualize it. */me goin tu lurn ingrish* > -- Best regards, IM ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 1/4] test: add infrastructure for fuzzing testing and fuzzers 2020-12-07 17:24 ` Igor Munkin 2020-12-07 19:54 ` Igor Munkin @ 2020-12-13 18:56 ` Sergey Bronnikov 2020-12-20 13:31 ` Igor Munkin 1 sibling, 1 reply; 26+ messages in thread From: Sergey Bronnikov @ 2020-12-13 18:56 UTC (permalink / raw) To: Igor Munkin; +Cc: tarantool-patches [-- Attachment #1: Type: text/plain, Size: 25913 bytes --] Igor, many thanks for review! I've fixed patches and pushed to the branch (but left them as a separate commits with prefix [TO SQUASH]). On 07.12.2020 20:24, Igor Munkin wrote: > Sergey, > > Thanks for the patch! Please consider the remaining comments below. > > On 30.11.20, sergeyb@tarantool.org wrote: >> From: Sergey Bronnikov <sergeyb@tarantool.org> >> >> There is a number of bugs related to parsing and encoding/decoding data. >> Examples: >> >> - csv: #2692, #4497, #2692 >> - uri: #585 >> >> One of the effective method to find such issues is a fuzzing testing. >> Patch introduce a CMake flag to enable building fuzzers (ENABLE_FUZZER) > Typo: s/introduce/introduces/. Fixed. > >> and add fuzzers based on LibFuzzer [1] to csv, http_parser and uri modules. >> NOTE: LibFuzzer requires Clang compiler. >> >> [1] https://llvm.org/docs/LibFuzzer.html >> >> How-To Use: >> >> $ mkdir build && cd build >> $ CC=clang CXX=clang++ cmake -DENABLE_FUZZER=ON -DENABLE_ASAN=ON -DCMAKE_BUILD_TYPE=Debug .. >> $ make fuzzers >> $ ./test/fuzz/csv_fuzzer -max_total_time=60*60*60 -workers=4 ../test/static/corpus/csv > I tried your recipe for the current revision and got the following: > | $ ./test/fuzz/csv_fuzzer -max_total_time=60*60*60 -workers=4 ../test/static/corpus/csv > | INFO: Seed: 2899369680 > | INFO: Loaded 1 modules (3 inline 8-bit counters): 3 [0x57a130, 0x57a133), > | INFO: Loaded 1 PC tables (3 PCs): 3 [0x553870,0x5538a0), > | No such file or directory: ../test/static/corpus/csv; exiting > > AFAICS, the required directory is added in the following patch, so I > checkout the branch HEAD and try once more: > | $ ./test/fuzz/csv_fuzzer -max_total_time=60*60*60 -workers=4 ../test/static/corpus/csv > | INFO: Seed: 1838565059 > | INFO: Loaded 1 modules (3 inline 8-bit counters): 3 [0x57a130, 0x57a133), > | INFO: Loaded 1 PC tables (3 PCs): 3 [0x553870,0x5538a0), > | INFO: 21 files found in ../test/static/corpus/csv > | INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes > | INFO: seed corpus: files: 21 min: 1b max: 462b total: 1336b rss: 27Mb > | csv_fuzzer: /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/fuzzer/FuzzerLoop.cpp:537: void fuzzer::Fuzzer::ExecuteCallback(const uint8_t *, size_t): Assertion `Res == 0' failed. > | ==15230== ERROR: libFuzzer: deadly signal > | #0 0x507287 in __sanitizer_print_stack_trace /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/asan/asan_stack.cc:38:3 > | #1 0x44f978 in fuzzer::PrintStackTrace() /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/fuzzer/FuzzerUtil.cpp:206:5 > | #2 0x4300f3 in fuzzer::Fuzzer::CrashCallback() /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/fuzzer/FuzzerLoop.cpp:237:3 > | #3 0x4300b0 in fuzzer::Fuzzer::StaticCrashSignalCallback() /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/fuzzer/FuzzerLoop.cpp:209:6 > | #4 0x7f179300c8bf (/lib64/libpthread.so.0+0x148bf) > | #5 0x7f1792bfdf3a in gsignal (/lib64/libc.so.6+0x38f3a) > | #6 0x7f1792be7534 in abort (/lib64/libc.so.6+0x22534) > | #7 0x7f1792be740e in __tls_get_addr (/lib64/libc.so.6+0x2240e) > | #8 0x7f1792bf5731 in __assert_fail (/lib64/libc.so.6+0x30731) > | #9 0x431d06 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/fuzzer/FuzzerLoop.cpp:537:5 > | #10 0x4310d5 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/fuzzer/FuzzerLoop.cpp:455:3 > | #11 0x433aad in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, fuzzer::fuzzer_allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/fuzzer/FuzzerLoop.cpp:745:7 > | #12 0x434240 in fuzzer::Fuzzer::Loop(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, fuzzer::fuzzer_allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/fuzzer/FuzzerLoop.cpp:768:3 > | #13 0x425e60 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/fuzzer/FuzzerDriver.cpp:760:6 > | #14 0x450132 in main /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/fuzzer/FuzzerMain.cpp:20:10 > | #15 0x7f1792be8eda in __libc_start_main (/lib64/libc.so.6+0x23eda) > | #16 0x41e919 in _start (/tarantool/build/test/fuzz/csv_fuzzer+0x41e919) > | > | NOTE: libFuzzer has rudimentary signal handlers. > | Combine libFuzzer with AddressSanitizer or similar for better crash reports. > | SUMMARY: libFuzzer: deadly signal > | MS: 0 ; base unit: 0000000000000000000000000000000000000000 > | 0x22,0x61,0x62,0x63,0x22,0x2c,0x20,0x22,0x77,0x69,0x74,0x68,0x2c,0x63,0x6f,0x6d,0x6d,0x61,0x22,0x2c,0x20,0x22,0x5c,0x22,0x69,0x6e,0x20,0x71,0x75,0x6f,0x74,0x65,0x73,0x5c,0x22,0x22,0x2c,0x20,0x22,0x31,0x20,0x5c,0x22,0x20,0x71,0x75,0x6f,0x74,0x65,0x22,0xa, \"abc\", \"with,comma\", \"\\\"in quotes\\\"\", \"1 \\\" quote\"\x0a > | artifact_prefix='./'; Test unit written to ./crash-6d131d28c6e20c3a0a0b46c3aa7308d3029ab636 > | Base64: ImFiYyIsICJ3aXRoLGNvbW1hIiwgIlwiaW4gcXVvdGVzXCIiLCAiMSBcIiBxdW90ZSIK > > I have no idea whether it is OK but this does look like it's not. Maybe > there are some problems with my compiler/sanitizer? JFYI, the toolchain > is the following: > | $ clang -v > | clang version 8.0.1 (tags/RELEASE_801/final) > | Target: x86_64-pc-linux-gnu > | Thread model: posix > | InstalledDir: /usr/lib/llvm/8/bin > | Selected GCC installation: /usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0 > | Candidate multilib: .;@m64 > | Candidate multilib: 32;@m32 > | Selected multilib: .;@m64 > | $ clang++ -v > | clang version 8.0.1 (tags/RELEASE_801/final) > | Target: x86_64-pc-linux-gnu > | Thread model: posix > | InstalledDir: /usr/lib/llvm/8/bin > | Selected GCC installation: /usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0 > | Candidate multilib: .;@m64 > | Candidate multilib: 32;@m32 > | Selected multilib: .;@m64 Message with assert is definitely not ok. LibFuzzer documentation says that all fuzzers must return 0 only [1]. --- a/test/fuzz/csv_fuzzer.c +++ b/test/fuzz/csv_fuzzer.c @@ -9,15 +9,14 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { csv_create(&csv); char *buf = calloc(size, sizeof(char*)); if (buf == NULL) - return -1; + return 0; memcpy(buf, data, size); buf[size] = '\0'; char *end = buf + size; csv_parse_chunk(&csv, buf, end); csv_finish_parsing(&csv); - int rc = csv_get_error_status(&csv) == CSV_ER_INVALID ? 1 : 0; csv_destroy(&csv); free(buf); - return rc; + return 0; } diff --git a/test/fuzz/http_parser_fuzzer.c b/test/fuzz/http_parser_fuzzer.c index a0aaf6786..f2dd7d09a 100644 --- a/test/fuzz/http_parser_fuzzer.c +++ b/test/fuzz/http_parser_fuzzer.c @@ -9,10 +9,10 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { http_parser_create(&parser); parser.hdr_name = (char *)calloc((int)size, sizeof(char)); if (parser.hdr_name == NULL) - return -1; + return 0; char *end_buf = buf + size; - int rc = http_parse_header_line(&parser, &buf, end_buf, size); + http_parse_header_line(&parser, &buf, end_buf, size); free(parser.hdr_name); - return rc; + return 0; } --- a/test/fuzz/uri_fuzzer.c +++ b/test/fuzz/uri_fuzzer.c @@ -8,12 +8,12 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { char *buf = calloc(size, sizeof(char*)); if (!buf) - return -1; + return 0; strncpy(buf, (char*)data, size); buf[size] = '\0'; struct uri uri; - int rc = uri_parse(&uri, buf); + uri_parse(&uri, buf); free(buf); - return rc; + return 0; } For the rest I believe the reason is | NOTE: libFuzzer has rudimentary signal handlers. | Combine libFuzzer with AddressSanitizer or similar for better crash reports. > >> Part of #1809 >> --- >> CMakeLists.txt | 2 +- >> cmake/profile.cmake | 13 ++++++++++ >> test/CMakeLists.txt | 3 +++ >> test/fuzz/CMakeLists.txt | 45 ++++++++++++++++++++++++++++++++++ >> test/fuzz/csv_fuzzer.c | 23 +++++++++++++++++ >> test/fuzz/http_parser_fuzzer.c | 18 ++++++++++++++ >> test/fuzz/uri_fuzzer.c | 19 ++++++++++++++ >> 7 files changed, 122 insertions(+), 1 deletion(-) >> create mode 100644 test/fuzz/CMakeLists.txt >> create mode 100644 test/fuzz/csv_fuzzer.c >> create mode 100644 test/fuzz/http_parser_fuzzer.c >> create mode 100644 test/fuzz/uri_fuzzer.c >> > <snipped> > >> diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt >> index 10882c6a1..d20a4eb5d 100644 >> --- a/test/CMakeLists.txt >> +++ b/test/CMakeLists.txt >> @@ -75,6 +75,9 @@ add_subdirectory(app-tap) >> add_subdirectory(box) >> add_subdirectory(box-tap) >> add_subdirectory(unit) >> +if(ENABLE_FUZZER) >> + add_subdirectory(fuzz) >> +endif() > Minor: Well, I don't get the idea *why* this change is added here: it > neither takes the place in alphabetical order, nor is added to the end > of the list. Sure, sorted alphabetically now. --- a/test/CMakeLists.txt +++ b/test/CMakeLists.txt @@ -74,10 +74,10 @@ add_subdirectory(app) add_subdirectory(app-tap) add_subdirectory(box) add_subdirectory(box-tap) -add_subdirectory(unit) if(ENABLE_FUZZER) add_subdirectory(fuzz) endif() +add_subdirectory(unit) add_subdirectory(${PROJECT_SOURCE_DIR}/third_party/luajit/test ${PROJECT_BINARY_DIR}/third_party/luajit/test) >> add_subdirectory(${PROJECT_SOURCE_DIR}/third_party/luajit/test >> ${PROJECT_BINARY_DIR}/third_party/luajit/test) >> >> diff --git a/test/fuzz/CMakeLists.txt b/test/fuzz/CMakeLists.txt >> new file mode 100644 >> index 000000000..142d38f67 >> --- /dev/null >> +++ b/test/fuzz/CMakeLists.txt >> @@ -0,0 +1,45 @@ >> +include_directories(${PROJECT_SOURCE_DIR}/src) >> +include_directories(${PROJECT_BINARY_DIR}/src) > Minor: It would be nice to explicitly mention the line above is added > for autogenerated headers and LuaJIT ones. Feel free to ignore. Fixed. --- a/test/fuzz/CMakeLists.txt +++ b/test/fuzz/CMakeLists.txt @@ -1,3 +1,4 @@ +# Added for autogenerated headers and LuaJIT ones. include_directories(${PROJECT_SOURCE_DIR}/src) include_directories(${PROJECT_BINARY_DIR}/src) include_directories(${PROJECT_SOURCE_DIR}/src/box) @@ -50,8 +51,8 @@ add_executable(http_parser_fuzzer http_parser_fuzzer.c) target_link_libraries(http_parser_fuzzer PUBLIC http_parser fuzzer_config) > >> +include_directories(${PROJECT_SOURCE_DIR}/src/box) >> + >> +# A special target with fuzzer and sanitizer flags. >> +add_library(fuzzer_config INTERFACE) >> + >> +target_compile_options( >> + fuzzer_config >> + INTERFACE >> + $<$<BOOL:${ENABLE_ASAN}>: >> + -fsanitize=fuzzer,address >> + > >> + $<$<BOOL:${ENABLE_UB_SANITIZER}>: >> + -fsanitize=fuzzer,undefined >> + > >> +) >> +target_link_libraries( >> + fuzzer_config >> + INTERFACE >> + $<$<BOOL:${ENABLE_ASAN}>: >> + -fsanitize=fuzzer,address >> + > >> + $<$<BOOL:${ENABLE_UB_SANITIZER}>: >> + -fsanitize=fuzzer,undefined >> + > >> +) > OK, I ran <make fuzzers> with more verbose output and have two notes > regarding it. > | Scanning dependencies of target csv > | make[3]: Leaving directory '/tarantool/build' > | make -f src/lib/csv/CMakeFiles/csv.dir/build.make src/lib/csv/CMakeFiles/csv.dir/build > | make[3]: Entering directory '/tarantool/build' > | [ 0%] Building C object src/lib/csv/CMakeFiles/csv.dir/csv.c.o > | cd /tarantool/build/src/lib/csv && /usr/lib/llvm/9/bin/clang-9 -DCORO_ASM > | -DLUAJIT_SMART_STRINGS=1 -DLUAJIT_USE_ASAN=1 -DLUA_USE_APICHECK=1 > | -DLUA_USE_ASSERT=1 -DNVALGRIND=1 -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE > | -D__STDC_CONSTANT_MACROS=1 -D__STDC_FORMAT_MACROS=1 -D__STDC_LIMIT_MACROS=1 > | -I/tarantool/src -I/tarantool/build/src -I/tarantool/src/lib > | -I/tarantool/src/lib/small -I/tarantool/src/lib/small/third_party > | -I/tarantool/src/lib/core -I/tarantool -I/tarantool/third_party/zstd/lib > | -I/tarantool/third_party/zstd/lib/common -I/tarantool/build/third_party > | -I/tarantool/third_party -I/tarantool/third_party/coro > | -I/tarantool/third_party/luajit/src -I/tarantool/third_party/libyaml/include > | -I/tarantool/src/lib/msgpuck -I/tarantool/build/build/curl/dest/include > | -I/tarantool/build/third_party/decNumber > | -I/tarantool/third_party/libutil_freebsd -fexceptions -funwind-tables > | -fno-omit-frame-pointer -fno-stack-protector -fno-common -fopenmp -msse2 > | -fsanitize=address -fsanitize-blacklist=/tarantool/asan/asan.supp > | -std=c11 -Wall -Wextra -Wno-strict-aliasing -Wno-char-subscripts > | -Wno-gnu-alignof-expression -Werror -g -ggdb -O0 -UASAN_INTERFACE_OLD > | -o CMakeFiles/csv.dir/csv.c.o -c /tarantool/src/lib/csv/csv.c > | [ 0%] Linking C static library libcsv.a > | cd /tarantool/build/src/lib/csv && /usr/bin/cmake -P CMakeFiles/csv.dir/cmake_clean_target.cmake > | cd /tarantool/build/src/lib/csv && /usr/bin/cmake -E cmake_link_script CMakeFiles/csv.dir/link.txt --verbose=1 > | /usr/bin/ar qc libcsv.a CMakeFiles/csv.dir/csv.c.o > | /usr/bin/ranlib libcsv.a > | make[3]: Leaving directory '/tarantool/build' > | [ 0%] Built target csv > | make -f test/fuzz/CMakeFiles/csv_fuzzer.dir/build.make test/fuzz/CMakeFiles/csv_fuzzer.dir/depend > | make[3]: Entering directory '/tarantool/build' > | cd /tarantool/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" > | /tarantool /tarantool/test/fuzz /tarantool/build /tarantool/build/test/fuzz > | /tarantool/build/test/fuzz/CMakeFiles/csv_fuzzer.dir/DependInfo.cmake --color= > | Dependee "/tarantool/build/test/fuzz/CMakeFiles/csv_fuzzer.dir/DependInfo.cmake" > | is newer than depender "/tarantool/build/test/fuzz/CMakeFiles/csv_fuzzer.dir/depend.internal". > | Dependee "/tarantool/build/test/fuzz/CMakeFiles/CMakeDirectoryInformation.cmake" > | is newer than depender "/tarantool/build/test/fuzz/CMakeFiles/csv_fuzzer.dir/depend.internal". > | Scanning dependencies of target csv_fuzzer > | make[3]: Leaving directory '/tarantool/build' > | make -f test/fuzz/CMakeFiles/csv_fuzzer.dir/build.make test/fuzz/CMakeFiles/csv_fuzzer.dir/build > | make[3]: Entering directory '/tarantool/build' > | [ 0%] Building C object test/fuzz/CMakeFiles/csv_fuzzer.dir/csv_fuzzer.c.o > | cd /tarantool/build/test/fuzz && /usr/lib/llvm/9/bin/clang-9 -DCORO_ASM > | -DLUAJIT_SMART_STRINGS=1 -DLUAJIT_USE_ASAN=1 -DLUA_USE_APICHECK=1 > | -DLUA_USE_ASSERT=1 -DNVALGRIND=1 -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE > | -D__STDC_CONSTANT_MACROS=1 -D__STDC_FORMAT_MACROS=1 -D__STDC_LIMIT_MACROS=1 > | -I/tarantool/src -I/tarantool/build/src -I/tarantool/src/lib > | -I/tarantool/src/lib/small -I/tarantool/src/lib/small/third_party > | -I/tarantool/src/lib/core -I/tarantool -I/tarantool/third_party/zstd/lib > | -I/tarantool/third_party/zstd/lib/common -I/tarantool/third_party/luajit/src > | -I/tarantool/src/lib/msgpuck -I/tarantool/src/box -fexceptions > | -funwind-tables -fno-omit-frame-pointer -fno-stack-protector -fno-common -fopenmp -msse2 > | -fsanitize=address -fsanitize-blacklist=/tarantool/asan/asan.supp > | -std=c11 -Wall -Wextra -Wno-strict-aliasing -Wno-char-subscripts > | -Wno-gnu-alignof-expression -Werror -Wno-unused-parameter -g -ggdb -O0 > | -UASAN_INTERFACE_OLD -fsanitize=fuzzer,address > | -o CMakeFiles/csv_fuzzer.dir/csv_fuzzer.c.o -c /tarantool/test/fuzz/csv_fuzzer.c > | [ 0%] Linking C executable csv_fuzzer > | cd /tarantool/build/test/fuzz && /usr/bin/cmake -E cmake_link_script CMakeFiles/csv_fuzzer.dir/link.txt --verbose=1 > | /usr/lib/llvm/9/bin/clang-9 -fexceptions -funwind-tables > | -fno-omit-frame-pointer -fno-stack-protector -fno-common -fopenmp -msse2 > | -fsanitize=address -fsanitize-blacklist=/tarantool/asan/asan.supp > | -std=c11 -Wall -Wextra -Wno-strict-aliasing -Wno-char-subscripts > | -Wno-gnu-alignof-expression -Werror -Wno-unused-parameter -g -ggdb -O0 > | -rdynamic CMakeFiles/csv_fuzzer.dir/csv_fuzzer.c.o -o csv_fuzzer > | ../../src/lib/csv/libcsv.a -fsanitize=fuzzer,address > | make[3]: Leaving directory '/tarantool/build' > | [ 0%] Built target csv_fuzzer > > 1. I'm totally not an expert, but quite confused with the fact the > libcsv is build w/o <fuzzer> flag, but csv_fuzzer is build with it. You are right. Project source code should be instrumented too and I enable it: diff --git a/cmake/profile.cmake b/cmake/profile.cmake index 45e3d112c..308d1b0fb 100644 --- a/cmake/profile.cmake +++ b/cmake/profile.cmake @@ -53,6 +53,9 @@ if(ENABLE_FUZZER) " $ CC=clang CXX=clang++ cmake . <...> -DENABLE_FUZZER=ON && make -j\n" "\n") endif() + if (NOT OSS_FUZZ) + add_compile_flags("C;CXX" -fsanitize=fuzzer-no-link) + endif() endif() option(ENABLE_ASAN "Enable AddressSanitizer, a fast memory error detector based on compiler instrumentation" OFF) You can easily check that option is passed to every source file (although for us interested csv, uri and http_parser libraries) when CMAKE_EXPORT_COMPILE_COMMANDS is enabled and passed to CMake. Entry for src/lib/csv.c contains -fsanitize=fuzzer-no-link in a compile_commands.json: { "directory": "/home/sergeyb/sources/MRG/tarantool/build/src/lib/csv", "command": "/usr/bin/clang -DCORO_ASM -DLUAJIT_SMART_STRINGS=1 -DLUA_USE_APICHECK=1 -DLUA_USE_ASSERT=1 -DNVALGRIND=1 -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS=1 -D__STDC_FORMAT_MACROS=1 -D__STDC_LIMIT_MACROS=1 -I/home/sergeyb/sources/MRG/tarantool/src -I/home/sergeyb/sources/MRG/tarantool/build/src -I/home/sergeyb/sources/MRG/tarantool/src/lib -I/home/sergeyb/sources/MRG/tarantool/src/lib/small -I/home/sergeyb/sources/MRG/tarantool/src/lib/small/third_party -I/home/sergeyb/sources/MRG/tarantool/src/lib/core -I/home/sergeyb/sources/MRG/tarantool -I/home/sergeyb/sources/MRG/tarantool/third_party/zstd/lib -I/home/sergeyb/sources/MRG/tarantool/third_party/zstd/lib/common -I/home/sergeyb/sources/MRG/tarantool/build/third_party -I/home/sergeyb/sources/MRG/tarantool/third_party -I/home/sergeyb/sources/MRG/tarantool/third_party/coro -I/home/sergeyb/sources/MRG/tarantool/third_party/luajit/src -I/home/sergeyb/sources/MRG/tarantool/third_party/libyaml/include -I/home/sergeyb/sources/MRG/tarantool/src/lib/msgpuck -I/home/sergeyb/sources/MRG/tarantool/build/build/curl/dest/include -I/home/sergeyb/sources/MRG/tarantool/build/third_party/decNumber -I/home/sergeyb/sources/MRG/tarantool/third_party/libutil_freebsd -fexceptions -funwind-tables -fno-common -fopenmp -msse2 *-fsanitize=fuzzer-no-link* -std=c11 -Wall -Wextra -Wno-strict-aliasing -Wno-char-subscripts -Wno-gnu-alignof-expression -Werror -g -ggdb -O0 -o CMakeFiles/csv.dir/csv.c.o -c /home/sergeyb/sources/MRG/tarantool/src/lib/csv/csv.c", "file": "/home/sergeyb/sources/MRG/tarantool/src/lib/csv/csv.c" }, You may not understand why options with "-fsanitize=fuzzer" two times (in cmake/profile.cmake and test/fuzz/CMakeLists.txt). I'll clarify it in advance: - cmake/profile.cmake is for project source files, -fsanitize=fuzzer-no-link option allows to instrument project source files for fuzzing, but LibFuzzer will not replace main() in these files. - test/fuzz/CMakeLists.txt uses -fsanitize=fuzzer and not -fsanitize=fuzzer-no-link because we want to add automatically generated main() for each fuzzer. > 2. Do you need to specify <address> flag once more, when ASAN is > enabled? If not the hunk above looks excess, doesn't it? Agree, it was a bad idea to manage UBSan and ASAN flags in yet another place. Moreover we don't use all flags provided by UBSan. There is an explanation in [2]. I have updated compilation and link flags in test/fuzz/CMakeLists.txt: --- a/test/fuzz/CMakeLists.txt +++ b/test/fuzz/CMakeLists.txt @@ -9,14 +9,8 @@ add_library(fuzzer_config INTERFACE) target_compile_options( fuzzer_config INTERFACE - $<$<BOOL:${ENABLE_ASAN}>: - -fsanitize=fuzzer,address - > - $<$<BOOL:${ENABLE_UB_SANITIZER}>: - -fsanitize=fuzzer,undefined - > $<$<NOT:$<BOOL:${OSS_FUZZ}>>: - -fsanitize=fuzzer + -fsanitize=fuzzer > $<$<BOOL:${OSS_FUZZ}>: ${CXX} @@ -26,14 +20,8 @@ target_compile_options( target_link_libraries( fuzzer_config INTERFACE - $<$<BOOL:${ENABLE_ASAN}>: - -fsanitize=fuzzer,address - > - $<$<BOOL:${ENABLE_UB_SANITIZER}>: - -fsanitize=fuzzer,undefined - > $<$<NOT:$<BOOL:${OSS_FUZZ}>>: - -fsanitize=fuzzer + -fsanitize=fuzzer > $<$<BOOL:${OSS_FUZZ}>: $ENV{LIB_FUZZING_ENGINE} >> + > <snipped> > >> + >> +set(fuzzing_binaries csv_fuzzer >> + http_parser_fuzzer >> + uri_fuzzer) > Spaces are used for indentation in CMake-related sources, not tabs. > Surprisingly, you made it the right way below. Fixed. --- a/test/fuzz/CMakeLists.txt +++ b/test/fuzz/CMakeLists.txt @@ -1,3 +1,4 @@ +# Added for autogenerated headers and LuaJIT ones. include_directories(${PROJECT_SOURCE_DIR}/src) include_directories(${PROJECT_BINARY_DIR}/src) include_directories(${PROJECT_SOURCE_DIR}/src/box) @@ -50,8 +51,8 @@ add_executable(http_parser_fuzzer http_parser_fuzzer.c) target_link_libraries(http_parser_fuzzer PUBLIC http_parser fuzzer_config) set(fuzzing_binaries csv_fuzzer - http_parser_fuzzer - uri_fuzzer) + http_parser_fuzzer + uri_fuzzer) add_custom_target(fuzzers DEPENDS ${fuzzing_binaries} > >> + >> +add_custom_target(fuzzers >> + DEPENDS ${fuzzing_binaries} >> + COMMENT "Build fuzzers") >> diff --git a/test/fuzz/csv_fuzzer.c b/test/fuzz/csv_fuzzer.c >> new file mode 100644 >> index 000000000..8853d6308 >> --- /dev/null >> +++ b/test/fuzz/csv_fuzzer.c > */me feeling myself like a parrot* > > Why do you violate our style guides[1] using spaces instead of tabs for > indentation? IIRC I've already mentioned it here[2]... > > Otherwise this hunk looks fine. Fixed indentation and placed function type on the same line with function name. Code style also recommend to use goto(), but I believe that LibFuzzer someday will start to accept different exit codes and probably it is better to keep code as is without using goto(). > >> @@ -0,0 +1,23 @@ > <snipped> > >> diff --git a/test/fuzz/http_parser_fuzzer.c b/test/fuzz/http_parser_fuzzer.c >> new file mode 100644 >> index 000000000..a0aaf6786 >> --- /dev/null >> +++ b/test/fuzz/http_parser_fuzzer.c > Ditto. same as for csv_fuzzer.c > >> @@ -0,0 +1,18 @@ > <snipped> > >> diff --git a/test/fuzz/uri_fuzzer.c b/test/fuzz/uri_fuzzer.c >> new file mode 100644 >> index 000000000..8397505bd >> --- /dev/null >> +++ b/test/fuzz/uri_fuzzer.c > Ditto. same as for csv_fuzzer.c > >> @@ -0,0 +1,19 @@ > <snipped> > >> 2.25.1 >> > [1]: https://www.tarantool.io/en/doc/latest/dev_guide/c_style_guide/#chapter-1-indentation > [2]: https://lists.tarantool.org/pipermail/tarantool-patches/2020-April/016409.html > Also added a warning that triggered when someone use ENABLE_FUZZER and OSS_FUZZ without environment variable LIB_FUZZING_ENGINE: --- a/cmake/profile.cmake +++ b/cmake/profile.cmake @@ -53,6 +53,13 @@ if(ENABLE_FUZZER) " $ CC=clang CXX=clang++ cmake . <...> -DENABLE_FUZZER=ON && make -j\n" "\n") endif() + if(OSS_FUZZ AND NOT DEFINED ENV{LIB_FUZZING_ENGINE}) + message(SEND_ERROR + "OSS-Fuzz builds require the environment variable " + "LIB_FUZZING_ENGINE to be set. If you are seeing this " + "warning, it points to a deeper problem in the ossfuzz " + "build setup.") + endif() if (NOT OSS_FUZZ) add_compile_flags("C;CXX" -fsanitize=fuzzer-no-link) endif() 1. http://llvm.org/docs/LibFuzzer.html#id22 2. https://github.com/tarantool/tarantool/blob/master/cmake/compiler.cmake#L290-L320 [-- Attachment #2: Type: text/html, Size: 32917 bytes --] ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 1/4] test: add infrastructure for fuzzing testing and fuzzers 2020-12-13 18:56 ` Sergey Bronnikov @ 2020-12-20 13:31 ` Igor Munkin 2020-12-24 10:18 ` Sergey Bronnikov 0 siblings, 1 reply; 26+ messages in thread From: Igor Munkin @ 2020-12-20 13:31 UTC (permalink / raw) To: Sergey Bronnikov; +Cc: tarantool-patches Sergey, Thanks for the changes! On 13.12.20, Sergey Bronnikov wrote: > Igor, many thanks for review! > > I've fixed patches and pushed to the branch (but left them as a separate > commits with prefix [TO SQUASH]). Everything is fixed in scope of the other patches in the series, but I have more questions for your answers and updates for this one, please consider them below. > > On 07.12.2020 20:24, Igor Munkin wrote: > > Sergey, > > > > Thanks for the patch! Please consider the remaining comments below. > > > > On 30.11.20, sergeyb@tarantool.org wrote: <snipped> > >> and add fuzzers based on LibFuzzer [1] to csv, http_parser and uri modules. Typo: Still exceeds 72 chars... > >> NOTE: LibFuzzer requires Clang compiler. <snipped> > > Message with assert is definitely not ok. LibFuzzer documentation says > that all fuzzers must return 0 only [1]. > Neat, now everything works fine. However, considering your comment, I have a newbie question (since I'm not an expert in fuzzing testing): how do we need to check whether parsing finishes right or not? Anyway, you can simply add asserts to check rc is 0, can't you? Otherwise these tests look kinda smoke ones to me. > > --- a/test/fuzz/csv_fuzzer.c > +++ b/test/fuzz/csv_fuzzer.c > @@ -9,15 +9,14 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t > size) { > csv_create(&csv); > char *buf = calloc(size, sizeof(char*)); Typo: s/sizeof(char*)/sizeof(char)/. This is why I failed to prove the out of boundary access below for a while. Surprisingly, everything is fine in HTTP parser test. > if (buf == NULL) > - return -1; > + return 0; I believe the testing is not OK if <calloc> yields NULL, but the code returns 0. This is odd, IMHO. What about adding either assert or abort to handle this branch? To make asserts work all time simply undefine NDEBUG at the beginning of the test. Same for other cases. > memcpy(buf, data, size); > buf[size] = '\0'; This write is out of boundaries. In fact it's not, since you overallocate a chunk above (consider typo: sizeof(char *) instead of sizeof(char)). You want to allocate the <size> bytes via <calloc> and accessing <size> byte, but the "last slot" is addressed by (<size> - 1). Moreover, these manipulations are excessive: if data passed as the first argument is NUL-terminated, then this assignment is not necessary (everything is done by <memcpy>). Otherwise, you strips the last byte of the first argument and you need to allocate (<size> + 1) bytes. Same for URI parser. > char *end = buf + size; > csv_parse_chunk(&csv, buf, end); > csv_finish_parsing(&csv); > - int rc = csv_get_error_status(&csv) == CSV_ER_INVALID ? 1 : 0; > csv_destroy(&csv); > free(buf); > > - return rc; > + return 0; > } > diff --git a/test/fuzz/http_parser_fuzzer.c b/test/fuzz/http_parser_fuzzer.c > index a0aaf6786..f2dd7d09a 100644 > --- a/test/fuzz/http_parser_fuzzer.c > +++ b/test/fuzz/http_parser_fuzzer.c > @@ -9,10 +9,10 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { > http_parser_create(&parser); > parser.hdr_name = (char *)calloc((int)size, sizeof(char)); Minor: why do you explicitly cast <size> argument here, but omit such cast in other tests? > if (parser.hdr_name == NULL) > - return -1; > + return 0; > char *end_buf = buf + size; > - int rc = http_parse_header_line(&parser, &buf, end_buf, size); > + http_parse_header_line(&parser, &buf, end_buf, size); > free(parser.hdr_name); > > - return rc; > + return 0; > } <snipped> > > For the rest I believe the reason is > > | NOTE: libFuzzer has rudimentary signal handlers. > | Combine libFuzzer with AddressSanitizer or similar for better crash reports. As I mentioned in the previous reply, I tried exactly your recipe, so ASAN was enabled. > > > > > >> Part of #1809 > >> --- > >> CMakeLists.txt | 2 +- > >> cmake/profile.cmake | 13 ++++++++++ > >> test/CMakeLists.txt | 3 +++ > >> test/fuzz/CMakeLists.txt | 45 ++++++++++++++++++++++++++++++++++ > >> test/fuzz/csv_fuzzer.c | 23 +++++++++++++++++ > >> test/fuzz/http_parser_fuzzer.c | 18 ++++++++++++++ > >> test/fuzz/uri_fuzzer.c | 19 ++++++++++++++ > >> 7 files changed, 122 insertions(+), 1 deletion(-) > >> create mode 100644 test/fuzz/CMakeLists.txt > >> create mode 100644 test/fuzz/csv_fuzzer.c > >> create mode 100644 test/fuzz/http_parser_fuzzer.c > >> create mode 100644 test/fuzz/uri_fuzzer.c <snipped> > >> diff --git a/test/fuzz/CMakeLists.txt b/test/fuzz/CMakeLists.txt > >> new file mode 100644 > >> index 000000000..142d38f67 > >> --- /dev/null > >> +++ b/test/fuzz/CMakeLists.txt > >> @@ -0,0 +1,45 @@ <snipped> > > > > 1. I'm totally not an expert, but quite confused with the fact the > > libcsv is build w/o <fuzzer> flag, but csv_fuzzer is build with it. > > You are right. Project source code should be instrumented too and I > enable it: > > diff --git a/cmake/profile.cmake b/cmake/profile.cmake > index 45e3d112c..308d1b0fb 100644 > --- a/cmake/profile.cmake > +++ b/cmake/profile.cmake > @@ -53,6 +53,9 @@ if(ENABLE_FUZZER) > " $ CC=clang CXX=clang++ cmake . <...> -DENABLE_FUZZER=ON > && make -j\n" > "\n") > endif() > + if (NOT OSS_FUZZ) > + add_compile_flags("C;CXX" -fsanitize=fuzzer-no-link) > + endif() Why these compile flags are added under this particular condition? > endif() > > option(ENABLE_ASAN "Enable AddressSanitizer, a fast memory error > detector based on compiler instrumentation" OFF) <snipped> > > You may not understand why options with "-fsanitize=fuzzer" two times > (in cmake/profile.cmake and test/fuzz/CMakeLists.txt). I'll clarify it > in advance: > > - cmake/profile.cmake is for project source files, > -fsanitize=fuzzer-no-link option allows to instrument project source > files for fuzzing, but LibFuzzer will not replace main() in these files. > > - test/fuzz/CMakeLists.txt uses -fsanitize=fuzzer and not > -fsanitize=fuzzer-no-link because we want to add automatically generated > main() for each fuzzer. This is a nice wording to comment the corresponding changes. > > > 2. Do you need to specify <address> flag once more, when ASAN is > > enabled? If not the hunk above looks excess, doesn't it? > > Agree, it was a bad idea to manage UBSan and ASAN flags in yet another > place. Side note: You can oblige one to enable ASAN/UBSAN the same way, you restrict building via clang. Of course if it makes sense :) > <snipped> > > Code style also recommend to use goto(), but I believe that LibFuzzer > someday will start to accept different exit codes > > and probably it is better to keep code as is without using goto(). Agree here. By the way, there is one nit left: please remove the space between * and the parameter name in the function signatures. > <snipped> > Also added a warning that triggered when someone use ENABLE_FUZZER and > OSS_FUZZ without > > environment variable LIB_FUZZING_ENGINE: Nice. > > --- a/cmake/profile.cmake > +++ b/cmake/profile.cmake > @@ -53,6 +53,13 @@ if(ENABLE_FUZZER) > " $ CC=clang CXX=clang++ cmake . <...> -DENABLE_FUZZER=ON > && make -j\n" > "\n") > endif() > + if(OSS_FUZZ AND NOT DEFINED ENV{LIB_FUZZING_ENGINE}) > + message(SEND_ERROR Minor: Why do you use SEND_ERROR here? I guess one can't proceed with the desired testing in this case, so FATAL_ERROR prevents one from the further misuse. > + "OSS-Fuzz builds require the environment variable " > + "LIB_FUZZING_ENGINE to be set. If you are seeing this " > + "warning, it points to a deeper problem in the ossfuzz " > + "build setup.") > + endif() Side note: the mess with whitespace is only in this patch. Everything is OK on the branch in the corresponding commit. > if (NOT OSS_FUZZ) > add_compile_flags("C;CXX" -fsanitize=fuzzer-no-link) > endif() > > > 1. http://llvm.org/docs/LibFuzzer.html#id22 > > 2. > https://github.com/tarantool/tarantool/blob/master/cmake/compiler.cmake#L290-L320 > -- Best regards, IM ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 1/4] test: add infrastructure for fuzzing testing and fuzzers 2020-12-20 13:31 ` Igor Munkin @ 2020-12-24 10:18 ` Sergey Bronnikov 2020-12-24 13:22 ` Igor Munkin 0 siblings, 1 reply; 26+ messages in thread From: Sergey Bronnikov @ 2020-12-24 10:18 UTC (permalink / raw) To: Igor Munkin; +Cc: tarantool-patches Hi, On 20.12.2020 16:31, Igor Munkin wrote: > Sergey, > > Thanks for the changes! > > On 13.12.20, Sergey Bronnikov wrote: >> Igor, many thanks for review! >> >> I've fixed patches and pushed to the branch (but left them as a separate >> commits with prefix [TO SQUASH]). > Everything is fixed in scope of the other patches in the series, but I > have more questions for your answers and updates for this one, please > consider them below. > >> On 07.12.2020 20:24, Igor Munkin wrote: >>> Sergey, >>> >>> Thanks for the patch! Please consider the remaining comments below. >>> >>> On 30.11.20, sergeyb@tarantool.org wrote: > <snipped> > >>>> and add fuzzers based on LibFuzzer [1] to csv, http_parser and uri modules. > Typo: Still exceeds 72 chars... Fixed. Also splitted cmake command-line in example to fit in 72 chars too. > >>>> NOTE: LibFuzzer requires Clang compiler. > <snipped> > >> Message with assert is definitely not ok. LibFuzzer documentation says >> that all fuzzers must return 0 only [1]. >> > Neat, now everything works fine. However, considering your comment, I > have a newbie question (since I'm not an expert in fuzzing testing): how > do we need to check whether parsing finishes right or not? libfuzzer has a number of settings and one of them is flag that controls time of single unit execution. ./test/fuzz/http_parser_fuzzer -help=1 <snipped> timeout 1200 Timeout in seconds (if positive). If one unit runs more than this number of seconds the process will abort. <snipped> > > Anyway, you can simply add asserts to check rc is 0, can't you? > Otherwise these tests look kinda smoke ones to me. > >> --- a/test/fuzz/csv_fuzzer.c >> +++ b/test/fuzz/csv_fuzzer.c >> @@ -9,15 +9,14 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t >> size) { >> csv_create(&csv); >> char *buf = calloc(size, sizeof(char*)); > Typo: s/sizeof(char*)/sizeof(char)/. Fixed. > This is why I failed to prove the out of boundary access below for a > while. Surprisingly, everything is fine in HTTP parser test. > >> if (buf == NULL) >> - return -1; >> + return 0; > I believe the testing is not OK if <calloc> yields NULL, but the code > returns 0. This is odd, IMHO. What about adding either assert or abort > to handle this branch? To make asserts work all time simply undefine > NDEBUG at the beginning of the test. Same for other cases. Igor, I think you get everything wrong ;) Let me explain. We don't write a highly reliable and safety code here. Everything we need is just to properly pass a junk to a function under test. The goal of fuzzing testing is to find errors like buffer-overflows, use-after-free and so on. Lack of memory during testing is rare case and I think we don't need to catch such cases here. Because triggered assert due to lack of memory is useless information from test, I don't know how we can improve Tarantool with such information. Gracefully exit is more than enough. Moreover I have took a look on source code of tests for other opensource projects that were already used in OSS-Fuzz. They don't care about return codes from calloc(), malloc() functions at all. See for example [1]. > >> memcpy(buf, data, size); >> buf[size] = '\0'; > This write is out of boundaries. In fact it's not, since you > overallocate a chunk above (consider typo: sizeof(char *) instead of > sizeof(char)). You want to allocate the <size> bytes via <calloc> and > accessing <size> byte, but the "last slot" is addressed by (<size> - 1). > > Moreover, these manipulations are excessive: if data passed as the first > argument is NUL-terminated, then this assignment is not necessary > (everything is done by <memcpy>). Otherwise, you strips the last byte of > the first argument and you need to allocate (<size> + 1) bytes. > > Same for URI parser. Thanks for catching this. --- a/test/fuzz/csv_fuzzer.c +++ b/test/fuzz/csv_fuzzer.c @@ -8,7 +8,7 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { struct csv csv; csv_create(&csv); - char *buf = calloc(size, sizeof(char)); + char *buf = calloc(size + 1, sizeof(char)); if (buf == NULL) return 0; memcpy(buf, data, size); diff --git a/test/fuzz/uri_fuzzer.c b/test/fuzz/uri_fuzzer.c index 0060bee9b..b4661aea1 100644 --- a/test/fuzz/uri_fuzzer.c +++ b/test/fuzz/uri_fuzzer.c @@ -6,10 +6,10 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { - char *buf = calloc(size, sizeof(char *)); + char *buf = calloc(size + 1, sizeof(char)); if (!buf) return 0; - strncpy(buf, (char *)data, size); + memcpy(buf, data, size); buf[size] = '\0'; struct uri uri; uri_parse(&uri, buf); > >> char *end = buf + size; >> csv_parse_chunk(&csv, buf, end); >> csv_finish_parsing(&csv); >> - int rc = csv_get_error_status(&csv) == CSV_ER_INVALID ? 1 : 0; >> csv_destroy(&csv); >> free(buf); >> >> - return rc; >> + return 0; >> } >> diff --git a/test/fuzz/http_parser_fuzzer.c b/test/fuzz/http_parser_fuzzer.c >> index a0aaf6786..f2dd7d09a 100644 >> --- a/test/fuzz/http_parser_fuzzer.c >> +++ b/test/fuzz/http_parser_fuzzer.c >> @@ -9,10 +9,10 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { >> http_parser_create(&parser); >> parser.hdr_name = (char *)calloc((int)size, sizeof(char)); > Minor: why do you explicitly cast <size> argument here, but omit such > cast in other tests? calloc accepts size with size_t type, so removed cast here. >> if (parser.hdr_name == NULL) >> - return -1; >> + return 0; >> char *end_buf = buf + size; >> - int rc = http_parse_header_line(&parser, &buf, end_buf, size); >> + http_parse_header_line(&parser, &buf, end_buf, size); >> free(parser.hdr_name); >> >> - return rc; >> + return 0; >> } > <snipped> > >> For the rest I believe the reason is >> >> | NOTE: libFuzzer has rudimentary signal handlers. >> | Combine libFuzzer with AddressSanitizer or similar for better crash reports. > As I mentioned in the previous reply, I tried exactly your recipe, so > ASAN was enabled. Finally :) > >> >>>> Part of #1809 >>>> --- >>>> CMakeLists.txt | 2 +- >>>> cmake/profile.cmake | 13 ++++++++++ >>>> test/CMakeLists.txt | 3 +++ >>>> test/fuzz/CMakeLists.txt | 45 ++++++++++++++++++++++++++++++++++ >>>> test/fuzz/csv_fuzzer.c | 23 +++++++++++++++++ >>>> test/fuzz/http_parser_fuzzer.c | 18 ++++++++++++++ >>>> test/fuzz/uri_fuzzer.c | 19 ++++++++++++++ >>>> 7 files changed, 122 insertions(+), 1 deletion(-) >>>> create mode 100644 test/fuzz/CMakeLists.txt >>>> create mode 100644 test/fuzz/csv_fuzzer.c >>>> create mode 100644 test/fuzz/http_parser_fuzzer.c >>>> create mode 100644 test/fuzz/uri_fuzzer.c > <snipped> > >>>> diff --git a/test/fuzz/CMakeLists.txt b/test/fuzz/CMakeLists.txt >>>> new file mode 100644 >>>> index 000000000..142d38f67 >>>> --- /dev/null >>>> +++ b/test/fuzz/CMakeLists.txt >>>> @@ -0,0 +1,45 @@ > <snipped> > >>> 1. I'm totally not an expert, but quite confused with the fact the >>> libcsv is build w/o <fuzzer> flag, but csv_fuzzer is build with it. >> You are right. Project source code should be instrumented too and I >> enable it: >> >> diff --git a/cmake/profile.cmake b/cmake/profile.cmake >> index 45e3d112c..308d1b0fb 100644 >> --- a/cmake/profile.cmake >> +++ b/cmake/profile.cmake >> @@ -53,6 +53,9 @@ if(ENABLE_FUZZER) >> " $ CC=clang CXX=clang++ cmake . <...> -DENABLE_FUZZER=ON >> && make -j\n" >> "\n") >> endif() >> + if (NOT OSS_FUZZ) >> + add_compile_flags("C;CXX" -fsanitize=fuzzer-no-link) >> + endif() > Why these compile flags are added under this particular condition? Because when OSS Fuzz is enabled compiler and link flags passed from outside. See description how to integrate project to OSS Fuzz in [2]. > >> endif() >> >> option(ENABLE_ASAN "Enable AddressSanitizer, a fast memory error >> detector based on compiler instrumentation" OFF) > <snipped> > >> You may not understand why options with "-fsanitize=fuzzer" two times >> (in cmake/profile.cmake and test/fuzz/CMakeLists.txt). I'll clarify it >> in advance: >> >> - cmake/profile.cmake is for project source files, >> -fsanitize=fuzzer-no-link option allows to instrument project source >> files for fuzzing, but LibFuzzer will not replace main() in these files. >> >> - test/fuzz/CMakeLists.txt uses -fsanitize=fuzzer and not >> -fsanitize=fuzzer-no-link because we want to add automatically generated >> main() for each fuzzer. > This is a nice wording to comment the corresponding changes. Added. > >>> 2. Do you need to specify <address> flag once more, when ASAN is >>> enabled? If not the hunk above looks excess, doesn't it? >> Agree, it was a bad idea to manage UBSan and ASAN flags in yet another >> place. > Side note: You can oblige one to enable ASAN/UBSAN the same way, you > restrict building via clang. Of course if it makes sense :) Looks like it is no sense. GCC also has support of sanitizers, [3]. > <snipped> > >> Code style also recommend to use goto(), but I believe that LibFuzzer >> someday will start to accept different exit codes >> >> and probably it is better to keep code as is without using goto(). > Agree here. > > By the way, there is one nit left: please remove the space between * and > the parameter name in the function signatures. Fixed. diff --git a/test/fuzz/csv_fuzzer.c b/test/fuzz/csv_fuzzer.c index 5e470c492..ffa917ad6 100644 --- a/test/fuzz/csv_fuzzer.c +++ b/test/fuzz/csv_fuzzer.c @@ -4,11 +4,11 @@ #include <string.h> #include "csv/csv.h" -int LLVMFuzzerTestOneInput(const uint8_t * data, size_t size) +int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { struct csv csv; csv_create(&csv); - char *buf = calloc(size, sizeof(char *)); + char *buf = calloc(size, sizeof(char)); if (buf == NULL) return 0; memcpy(buf, data, size); diff --git a/test/fuzz/http_parser_fuzzer.c b/test/fuzz/http_parser_fuzzer.c index 1d78450f0..737c89617 100644 --- a/test/fuzz/http_parser_fuzzer.c +++ b/test/fuzz/http_parser_fuzzer.c @@ -3,7 +3,7 @@ #include <stddef.h> #include "http_parser/http_parser.h" -int LLVMFuzzerTestOneInput(const uint8_t * data, size_t size) +int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { struct http_parser parser; char *buf = (char *)data; diff --git a/test/fuzz/uri_fuzzer.c b/test/fuzz/uri_fuzzer.c index 6e047bde5..0060bee9b 100644 --- a/test/fuzz/uri_fuzzer.c +++ b/test/fuzz/uri_fuzzer.c @@ -4,7 +4,7 @@ #include <string.h> #include "uri/uri.h" -int LLVMFuzzerTestOneInput(const uint8_t * data, size_t size) +int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { char *buf = calloc(size, sizeof(char *)); if (!buf) > > <snipped> > >> Also added a warning that triggered when someone use ENABLE_FUZZER and >> OSS_FUZZ without >> >> environment variable LIB_FUZZING_ENGINE: > Nice. > >> --- a/cmake/profile.cmake >> +++ b/cmake/profile.cmake >> @@ -53,6 +53,13 @@ if(ENABLE_FUZZER) >> " $ CC=clang CXX=clang++ cmake . <...> -DENABLE_FUZZER=ON >> && make -j\n" >> "\n") >> endif() >> + if(OSS_FUZZ AND NOT DEFINED ENV{LIB_FUZZING_ENGINE}) >> + message(SEND_ERROR > Minor: Why do you use SEND_ERROR here? I guess one can't proceed with > the desired testing in this case, so FATAL_ERROR prevents one from the > further misuse. I don't mind, let's replace it to FATAL_ERROR. > >> + "OSS-Fuzz builds require the environment variable " >> + "LIB_FUZZING_ENGINE to be set. If you are seeing this " >> + "warning, it points to a deeper problem in the ossfuzz " >> + "build setup.") >> + endif() > Side note: the mess with whitespace is only in this patch. Everything is > OK on the branch in the corresponding commit. > >> if (NOT OSS_FUZZ) >> add_compile_flags("C;CXX" -fsanitize=fuzzer-no-link) >> endif() >> >> >> 1. http://llvm.org/docs/LibFuzzer.html#id22 >> >> 2. >> https://github.com/tarantool/tarantool/blob/master/cmake/compiler.cmake#L290-L320 >> 1. https://github.com/google/oss-fuzz/blob/master/projects/unbound/fuzz_2.c#L16 2. https://google.github.io/oss-fuzz/getting-started/new-project-guide/#buildsh 3. https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 1/4] test: add infrastructure for fuzzing testing and fuzzers 2020-12-24 10:18 ` Sergey Bronnikov @ 2020-12-24 13:22 ` Igor Munkin 2020-12-24 17:25 ` Sergey Bronnikov 0 siblings, 1 reply; 26+ messages in thread From: Igor Munkin @ 2020-12-24 13:22 UTC (permalink / raw) To: Sergey Bronnikov; +Cc: tarantool-patches Sergey, Thanks for the fixes! Unfortunately, I still see 72 symbols violation in commit message for the first and last patches on your remote branch. Please, also consider my other notes below. On 24.12.20, Sergey Bronnikov wrote: > Hi, > > On 20.12.2020 16:31, Igor Munkin wrote: > > Sergey, > > > > Thanks for the changes! > > <snipped> > >> > > Neat, now everything works fine. However, considering your comment, I > > have a newbie question (since I'm not an expert in fuzzing testing): how > > do we need to check whether parsing finishes right or not? > > libfuzzer has a number of settings and one of them is flag that controls > time of single unit execution. I asked about the check if parsing succeeds or not, but you answered this question below. > <snipped> > > I believe the testing is not OK if <calloc> yields NULL, but the code > > returns 0. This is odd, IMHO. What about adding either assert or abort > > to handle this branch? To make asserts work all time simply undefine > > NDEBUG at the beginning of the test. Same for other cases. > > Igor, I think you get everything wrong ;) Let me explain. > > We don't write a highly reliable and safety code here. Everything we > need is just to properly pass a junk to a function under test. > > The goal of fuzzing testing is to find errors like buffer-overflows, > use-after-free and so on. AFAIU, these tests do not check if the "passed junk" is parsed fine. Am I right? > > Lack of memory during testing is rare case and I think we don't need to > catch such cases here. > > Because triggered assert due to lack of memory is useless information > from test, > > I don't know how we can improve Tarantool with such information. > Gracefully exit is more than enough. OK, then. > > Moreover I have took a look on source code of tests for other opensource > projects that were already used in OSS-Fuzz. > > They don't care about return codes from calloc(), malloc() functions at > all. See for example [1]. "А если все пойдут с моста прыгать, ты тоже пойдешь?" Anyway, I get your point, thanks for clarification! > <snipped> > > Why these compile flags are added under this particular condition? > > Because when OSS Fuzz is enabled compiler and link flags passed > > from outside. See description how to integrate project to OSS Fuzz in [2]. Glad to see this in commit message, thanks! > <snipped> > > > >>> 2. Do you need to specify <address> flag once more, when ASAN is > >>> enabled? If not the hunk above looks excess, doesn't it? > >> Agree, it was a bad idea to manage UBSan and ASAN flags in yet another > >> place. I guess this should be fixed in scope of the first patch, but I see you squashed it to the last one. Why? > > Side note: You can oblige one to enable ASAN/UBSAN the same way, you > > restrict building via clang. Of course if it makes sense :) > Looks like it is no sense. GCC also has support of sanitizers, [3]. <snipped> > > 1. > https://github.com/google/oss-fuzz/blob/master/projects/unbound/fuzz_2.c#L16 > > 2. > https://google.github.io/oss-fuzz/getting-started/new-project-guide/#buildsh > > 3. https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html > -- Best regards, IM ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 1/4] test: add infrastructure for fuzzing testing and fuzzers 2020-12-24 13:22 ` Igor Munkin @ 2020-12-24 17:25 ` Sergey Bronnikov 2020-12-24 17:50 ` Igor Munkin 0 siblings, 1 reply; 26+ messages in thread From: Sergey Bronnikov @ 2020-12-24 17:25 UTC (permalink / raw) To: Igor Munkin; +Cc: tarantool-patches Igor, On 24.12.2020 16:22, Igor Munkin wrote: > Sergey, > > Thanks for the fixes! Unfortunately, I still see 72 symbols violation in > commit message for the first and last patches on your remote branch. > Please, also consider my other notes below. Fixed them too. > On 24.12.20, Sergey Bronnikov wrote: >> Hi, >> >> On 20.12.2020 16:31, Igor Munkin wrote: >>> Sergey, >>> >>> Thanks for the changes! >>> > <snipped> > >>> Neat, now everything works fine. However, considering your comment, I >>> have a newbie question (since I'm not an expert in fuzzing testing): how >>> do we need to check whether parsing finishes right or not? >> libfuzzer has a number of settings and one of them is flag that controls >> time of single unit execution. > I asked about the check if parsing succeeds or not, but you answered > this question below. > > <snipped> > >>> I believe the testing is not OK if <calloc> yields NULL, but the code >>> returns 0. This is odd, IMHO. What about adding either assert or abort >>> to handle this branch? To make asserts work all time simply undefine >>> NDEBUG at the beginning of the test. Same for other cases. >> Igor, I think you get everything wrong ;) Let me explain. >> >> We don't write a highly reliable and safety code here. Everything we >> need is just to properly pass a junk to a function under test. >> >> The goal of fuzzing testing is to find errors like buffer-overflows, >> use-after-free and so on. > AFAIU, these tests do not check if the "passed junk" is parsed fine. Am > I right? Absolutely! These tests are not about correctness. > >> Lack of memory during testing is rare case and I think we don't need to >> catch such cases here. >> >> Because triggered assert due to lack of memory is useless information >> from test, >> >> I don't know how we can improve Tarantool with such information. >> Gracefully exit is more than enough. > OK, then. > >> Moreover I have took a look on source code of tests for other opensource >> projects that were already used in OSS-Fuzz. >> >> They don't care about return codes from calloc(), malloc() functions at >> all. See for example [1]. > "А если все пойдут с моста прыгать, ты тоже пойдешь?" Ты напомнил мне мою учительницу, она тоже так говорила. > Anyway, I get your point, thanks for clarification! > > <snipped> > >>> Why these compile flags are added under this particular condition? >> Because when OSS Fuzz is enabled compiler and link flags passed >> >> from outside. See description how to integrate project to OSS Fuzz in [2]. > Glad to see this in commit message, thanks! > > <snipped> > >>>>> 2. Do you need to specify <address> flag once more, when ASAN is >>>>> enabled? If not the hunk above looks excess, doesn't it? >>>> Agree, it was a bad idea to manage UBSan and ASAN flags in yet another >>>> place. > I guess this should be fixed in scope of the first patch, but I see you > squashed it to the last one. Why? It seems because I was triggered by OSS_FUZZ in hunk and squashed it to a commit that introduce an OSS_FUZZ support. Fixed it and also moved add_compile_options() to the first commit too (otherwise project source code is not instrumented with -fsanitize=fuzzer-no-link). In last commit now: --- a/test/fuzz/CMakeLists.txt +++ b/test/fuzz/CMakeLists.txt @@ -9,12 +9,23 @@ add_library(fuzzer_config INTERFACE) target_compile_options( fuzzer_config INTERFACE - -fsanitize=fuzzer,address + $<$<NOT:$<BOOL:${OSS_FUZZ}>>: + -fsanitize=fuzzer + > + $<$<BOOL:${OSS_FUZZ}>: + ${CXX} + ${CXXFLAGS} + > ) target_link_libraries( fuzzer_config INTERFACE - -fsanitize=fuzzer,address + $<$<NOT:$<BOOL:${OSS_FUZZ}>>: + -fsanitize=fuzzer + > + $<$<BOOL:${OSS_FUZZ}>: + $ENV{LIB_FUZZING_ENGINE} + > ) # Use PUBLIC to force 'fuzzer_config' for all dependent targets. <snipped> ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 1/4] test: add infrastructure for fuzzing testing and fuzzers 2020-12-24 17:25 ` Sergey Bronnikov @ 2020-12-24 17:50 ` Igor Munkin 2020-12-25 7:07 ` Sergey Bronnikov 0 siblings, 1 reply; 26+ messages in thread From: Igor Munkin @ 2020-12-24 17:50 UTC (permalink / raw) To: Sergey Bronnikov; +Cc: tarantool-patches Sergey, On 24.12.20, Sergey Bronnikov wrote: > Igor, > <snipped> > >>>>> 2. Do you need to specify <address> flag once more, when ASAN is > >>>>> enabled? If not the hunk above looks excess, doesn't it? > >>>> Agree, it was a bad idea to manage UBSan and ASAN flags in yet another > >>>> place. > > I guess this should be fixed in scope of the first patch, but I see you > > squashed it to the last one. Why? > > It seems because I was triggered by OSS_FUZZ in hunk and squashed it to > a commit > > that introduce an OSS_FUZZ support. Fixed it and also moved > add_compile_options() to the first commit too > > (otherwise project source code is not instrumented with > -fsanitize=fuzzer-no-link). OK, but I believe we agreed the <address> options should be removed to avoid managing ASAN flags here, didn't we? > <snipped> -- Best regards, IM ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 1/4] test: add infrastructure for fuzzing testing and fuzzers 2020-12-24 17:50 ` Igor Munkin @ 2020-12-25 7:07 ` Sergey Bronnikov 2020-12-25 9:02 ` Igor Munkin 0 siblings, 1 reply; 26+ messages in thread From: Sergey Bronnikov @ 2020-12-25 7:07 UTC (permalink / raw) To: Igor Munkin; +Cc: tarantool-patches On 24.12.2020 20:50, Igor Munkin wrote: > Sergey, > > On 24.12.20, Sergey Bronnikov wrote: >> Igor, >> > <snipped> > >>>>>>> 2. Do you need to specify <address> flag once more, when ASAN is >>>>>>> enabled? If not the hunk above looks excess, doesn't it? >>>>>> Agree, it was a bad idea to manage UBSan and ASAN flags in yet another >>>>>> place. >>> I guess this should be fixed in scope of the first patch, but I see you >>> squashed it to the last one. Why? >> It seems because I was triggered by OSS_FUZZ in hunk and squashed it to >> a commit >> >> that introduce an OSS_FUZZ support. Fixed it and also moved >> add_compile_options() to the first commit too >> >> (otherwise project source code is not instrumented with >> -fsanitize=fuzzer-no-link). > OK, but I believe we agreed the <address> options should be removed to > avoid managing ASAN flags here, didn't we? Removed asan option and double space in commit message. --- a/test/fuzz/CMakeLists.txt +++ b/test/fuzz/CMakeLists.txt @@ -9,12 +9,23 @@ add_library(fuzzer_config INTERFACE) target_compile_options( fuzzer_config INTERFACE + $<$<NOT:$<BOOL:${OSS_FUZZ}>>: -fsanitize=fuzzer + > + $<$<BOOL:${OSS_FUZZ}>: + ${CXX} + ${CXXFLAGS} + > ) target_link_libraries( fuzzer_config INTERFACE + $<$<NOT:$<BOOL:${OSS_FUZZ}>>: -fsanitize=fuzzer + > + $<$<BOOL:${OSS_FUZZ}>: + $ENV{LIB_FUZZING_ENGINE} + > ) > > <snipped> > ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 1/4] test: add infrastructure for fuzzing testing and fuzzers 2020-12-25 7:07 ` Sergey Bronnikov @ 2020-12-25 9:02 ` Igor Munkin 2020-12-25 10:33 ` Sergey Bronnikov 0 siblings, 1 reply; 26+ messages in thread From: Igor Munkin @ 2020-12-25 9:02 UTC (permalink / raw) To: Sergey Bronnikov; +Cc: tarantool-patches Sergey, On 25.12.20, Sergey Bronnikov wrote: > > On 24.12.2020 20:50, Igor Munkin wrote: > > Sergey, > > > > On 24.12.20, Sergey Bronnikov wrote: > >> Igor, > >> > > <snipped> > > > >>>>>>> 2. Do you need to specify <address> flag once more, when ASAN is > >>>>>>> enabled? If not the hunk above looks excess, doesn't it? > >>>>>> Agree, it was a bad idea to manage UBSan and ASAN flags in yet another > >>>>>> place. > >>> I guess this should be fixed in scope of the first patch, but I see you > >>> squashed it to the last one. Why? > >> It seems because I was triggered by OSS_FUZZ in hunk and squashed it to > >> a commit > >> > >> that introduce an OSS_FUZZ support. Fixed it and also moved > >> add_compile_options() to the first commit too > >> > >> (otherwise project source code is not instrumented with > >> -fsanitize=fuzzer-no-link). > > OK, but I believe we agreed the <address> options should be removed to > > avoid managing ASAN flags here, didn't we? > > Removed asan option and double space in commit message. See no changes on the branch. > <snipped> -- Best regards, IM ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 1/4] test: add infrastructure for fuzzing testing and fuzzers 2020-12-25 9:02 ` Igor Munkin @ 2020-12-25 10:33 ` Sergey Bronnikov 0 siblings, 0 replies; 26+ messages in thread From: Sergey Bronnikov @ 2020-12-25 10:33 UTC (permalink / raw) To: Igor Munkin; +Cc: tarantool-patches On 25.12.2020 12:02, Igor Munkin wrote: > Sergey, > > On 25.12.20, Sergey Bronnikov wrote: >> On 24.12.2020 20:50, Igor Munkin wrote: >>> Sergey, >>> >>> On 24.12.20, Sergey Bronnikov wrote: >>>> Igor, >>>> >>> <snipped> >>> >>>>>>>>> 2. Do you need to specify <address> flag once more, when ASAN is >>>>>>>>> enabled? If not the hunk above looks excess, doesn't it? >>>>>>>> Agree, it was a bad idea to manage UBSan and ASAN flags in yet another >>>>>>>> place. >>>>> I guess this should be fixed in scope of the first patch, but I see you >>>>> squashed it to the last one. Why? >>>> It seems because I was triggered by OSS_FUZZ in hunk and squashed it to >>>> a commit >>>> >>>> that introduce an OSS_FUZZ support. Fixed it and also moved >>>> add_compile_options() to the first commit too >>>> >>>> (otherwise project source code is not instrumented with >>>> -fsanitize=fuzzer-no-link). >>> OK, but I believe we agreed the <address> options should be removed to >>> avoid managing ASAN flags here, didn't we? >> Removed asan option and double space in commit message. > See no changes on the branch. > Sorry, pushed. CI (not started yet) https://gitlab.com/tarantool/tarantool/-/pipelines/234413198 > <snipped> > > ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Tarantool-patches] [PATCH 2/4] test: add corpus to be used with fuzzers 2020-11-30 20:24 [Tarantool-patches] [PATCH 0/4] Add fuzzing testing sergeyb 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 1/4] test: add infrastructure for fuzzing testing and fuzzers sergeyb @ 2020-11-30 20:24 ` sergeyb 2020-12-07 17:34 ` Igor Munkin 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 3/4] travis: build tarantool with ENABLE_FUZZER sergeyb ` (5 subsequent siblings) 7 siblings, 1 reply; 26+ messages in thread From: sergeyb @ 2020-11-30 20:24 UTC (permalink / raw) To: tarantool-patches, imun, sergepetrenko From: Sergey Bronnikov <sergeyb@tarantool.org> Fuzzing tools uses evolutionary algorithms. Supplying seed corpus consisting of good sample inputs is one of the best ways to improve fuzz target’s coverage. Patch adds a corpuses that can be used with existed fuzzers. The name of each file in the corpus is the sha1 checksum of its contents. Corpus with http headers was added from [1] and [2]. 1. https://google.github.io/oss-fuzz/getting-started/new-project-guide/#seed-corpus 2. https://en.wikipedia.org/wiki/List_of_HTTP_header_fields 3. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers The libFuzzer allow to minimize corpus with help of `-merge` flag: when 1 is passed, any corpus inputs from the 2nd, 3rd etc. corpus directories that trigger new code coverage will be merged into the first corpus directory, when 0 is passed an existed corpus will be minimized. All provided corpuses in a patch were minimized. Part of #1809 --- .../csv/0c90a17c7b7cb5163255c7b775a8b544fb62c59d | 3 +++ .../csv/0e76fc3bb39bac4f774e2ce3978e9b59bc4a04b7 | 2 ++ .../csv/11f7ab7d8c2d4da313081e3282ab8120f0330b90 | 5 +++++ .../csv/2ace62c1befa19e3ea37dd52be9f6d508c5163e6 | 1 + .../csv/3816dd9c9120fe6b3d806184874cc5c3e3a68df3 | 1 + .../csv/409c042df7f6a518972df1bc1287442e28719b6c | 1 + .../csv/46b175b37b5c3f746fde0dd114ead9938c012cfc | 3 +++ .../csv/56754bc8c7346ef9fad5c2ae9c54cef0af72629c | 2 ++ .../csv/58f43ea1e6df023f3dd4be9f718e5d0382fb9697 | 3 +++ .../csv/6d131d28c6e20c3a0a0b46c3aa7308d3029ab636 | 1 + .../csv/7560cb86c281018ee2f80d840fd8e6aa73a609ef | 5 +++++ .../csv/7b97db755e18dc519fbc49e5de19b5820b9441c1 | 1 + .../csv/7ba0c88846fbcd28ad490df47f9ce4405d795303 | 2 ++ .../csv/8a14c110827e26cfdde3693bc5db6cd8673cdc8a | 1 + .../csv/94c1799dfba59120dd6075ee60016b809a915ca8 | 1 + .../csv/9e109f38c3948abe743bd116a34f05954f9ddd59 | 1 + .../csv/d21ec5eaaab3c8db1f923674b7a3ee19ce35ed13 | 1 + .../csv/d4a6d494cc2a06a0fee4873014210ec523d23750 | 1 + .../csv/eca825c9bdb276332d524134ddf370f7d18c2444 | 1 + .../csv/f9ba3e378e0409d66f946b31306de19edfe859bc | 1 + .../csv/fcad5471c295084dc1ec1eaa3753420e9803d251 | 10 ++++++++++ .../038ec38a9ae7669590f1e3401da8ffbc9b40c8ee | 1 + .../043fdf868410e66239819e0af10cbe787cfab77e | 1 + .../08abe5308ce774ad5e9e435de386748ce06521f8 | 1 + .../10b4f603cf09e21020e494e20a7f6760d5634c88 | 1 + .../172ed6e56aa94bada4135d64fc2e0c5eac779aed | 1 + .../1e42978bf6935a0f8377f6c480c315f1ae4627b8 | 1 + .../1f6f26c085c3b209e626f90c63d0fb10526de318 | 1 + .../24b458efa865a138a314f0876a11ec680b983ed5 | 1 + .../2be565c790bc352f2cbaa0f3509e0bb6133b3616 | 1 + .../30fd5e18f586b219916fb3892961a00e92db7a7f | 1 + .../38918740212c6e5aea01f2fc8772b1d4aa89be8b | 1 + .../39d4c74a90b7f5f5426b7982857b5fd7edbd8f77 | 1 + .../3d21092e1a772d792e9dd720eeae30d8eb5ac2ad | 1 + .../454dec613ca5c71fee0467061a88d2807201ac36 | 1 + .../4f13053e61966dd08d7cd2407865481ec1d2cbf9 | 1 + .../4ffa3daceb5b6ee3fc17cef3e0ae694ea5c583b5 | 1 + .../501b3a19def06b352978c03ece7ab282cdc6cbab | 1 + .../50ad023e1465919f5afbfffb33504ddfdc6a7e69 | 1 + .../5177d236e3eea297b76c19194a4cf127b313b267 | 1 + .../51e1354d16cfa4967e91206be8bd0d8c6ca577af | 1 + .../55d5f3acc62809fc87466b2048b157e50e63d644 | 1 + .../598d19d3175046ff035be228db2f1dc43957d8e2 | 1 + .../5b7444e43e5c1fd759483c76a46a5536da043022 | 1 + .../5c489184823a8a42844066fd5b4951bbad86934f | 1 + .../6455c0e761cd485f3e38c470f216560bfc46eea4 | 1 + .../6596bb363cfd0bef3f68e265cfd824c6d37ecb7e | 1 + .../66d5206b6c75b071f57684399218d144b0d337ea | 1 + .../685073a623772c1d46475b5eb451de40247d697f | 1 + .../71d4de855c1af66a6a5cea88b1eddc5032ecb26d | 1 + .../73378b09c61739b65a5f63214f0515307f187e53 | 1 + .../768abf119ce4fa6c9a9ee9f17d9dacb32c61759c | 1 + .../7ae1560be3dffbb88f3e48de3a4be99ecdc61441 | 1 + .../7c4d33785daa5c2370201ffa236b427aa37c9996 | 1 + .../80c5002c297e86198535d1574f3e47f4c6900782 | 1 + .../86908cf9f9f2c690ebd7462cacd2b28fc45db3a5 | 1 + .../89681f4bac464df0f73f3bf3fd48db40bc5796a6 | 1 + .../8c371f137d8a10fc568a3022edf6f10280c96bc0 | 1 + .../8ff59012e5ea417bf340e59a0ab6fd9e410011ea | 1 + .../927ba76d456f6c3a06769795f87c46462e16acd5 | 1 + .../95a0311cd792a22e0de733080e0580638c8eb136 | 1 + .../9a3fc50025efc8aac22e16f10621f43a0f35ddcb | 1 + .../ad1bde0f01973fc121af6f4a1efef3a1447c3a80 | 1 + .../ad50e55369c0e4459500323e0af8eb00172f0909 | 1 + .../b2af1811e30009a2b7cec55e86f26b8f4bf3c1cf | 1 + .../bcebbda5c1d2033a62c14d6bf7e2dc54b407b403 | 1 + .../cfe75970181a6e2f5ddbf9cf968a9a1fba1a660b | 1 + .../d01798aa68e31f2964a2bc2c10b203f5186a3ba1 | 1 + .../d2dc25ea74a20f35c78091cbc1e30cf2557c315f | 1 + .../d930b69281035ce3b46f2635f5a91ee11f15876c | 1 + .../dcb92ca0399b135c88170124fcdab6b6d62d115c | 1 + .../e4e52eaca9ece89c0e3fa2ef7e6e126d47f20101 | 1 + .../ebe97f523d44667ea6e11e1c8cd9e5f02cf244e7 | 1 + .../ede1894116fdc6430ba23942e12d974e3f22875b | 1 + .../f0f8a948443b28c5e344e6e43aedbb479059779c | 1 + .../f1c4ffb8ea524f69bc8d6784f86b70ac99db2ad7 | 1 + .../f774dd0b8fc355505aab48a2107804387f45c4a0 | 1 + .../uri/016d581dcda85465d5fa22eadd5947395b61303f | 1 + .../uri/018287fe57cf372d929b52a560910284ba406de6 | 1 + .../uri/048b75ce15485f8daa16e516014df26f2cf1ce09 | 1 + .../uri/048fbb93cfd1be0ccf151cf75147e391f12c2778 | 1 + .../uri/07e1d498149de21318fe486a73b618b800f9e46c | 1 + .../uri/093873ef42d30bee0ec042892457333ce7d6cd65 | 1 + .../uri/0a11f3691a10feefaeda9e30b4c891148d4ffdc1 | 1 + .../uri/14394ce809fcbd4db56a1f1814fb322103148fc6 | 1 + .../uri/251e9a9e4e077720a71747a94e24a6e25cb314a9 | 1 + .../uri/2a855b2e35301abb51facebb20e78801b2219e0f | 1 + .../uri/2ab8897ce59547952f047b48d52f9d6d4c0ea3eb | 1 + .../uri/39f3fca5b4634a9004220b389122abedfef28db7 | 1 + .../uri/3f18809f0df987b47c236d82d687ba16231daedb | 1 + .../uri/4143d3a341877154d6e95211464e1df1015b74bd | 1 + .../uri/49159cda7338d9b1a355631e7d33c55564fbfcb3 | 1 + .../uri/4b40fa57bf6479d96f1a751c68578ee64e7b6759 | 1 + .../uri/4f1a4d9e9b5db51ca7fdf0e9b4adb52f1813918b | 1 + .../uri/54b8c977fd54171c25da05e420b13d05dfaad747 | 1 + .../uri/5bfff0efcc1e7f8e3550798038d6361392379dfa | 1 + .../uri/60f481551bdf721e8d1f2c805da6a7fb2d2e4858 | 1 + .../uri/6c03daf8e568705738f646bfcdd03946acdce123 | 1 + .../uri/6dcaf13f83e40d73a9c5180977b5bf171b812e11 | 1 + .../uri/7511cc73615187b17837d01a013e4d886331fa54 | 1 + .../uri/767e882d43ffb1c822233640f7ca4e33426f7a3b | 1 + .../uri/777b43cd6a3f75e17d9310584ac00d4bd3ee65a1 | 1 + .../uri/77d8be75df172643a6fb1ee235d05b904aa501a0 | 1 + .../uri/7b8fa7a85f9f6ae6f9178504d2202666fb8dc772 | 1 + .../uri/7d850a53909012d59b379e99f1549332f01e3088 | 1 + .../uri/84f62a3184758e8c27e63f3f27c05a50877dc389 | 1 + .../uri/851be661349b0ceabf7cd933409f4d79ca9ac9a8 | 1 + .../uri/8efd1c91dceac73be0780f4f3cddce62ba84ebf6 | 1 + .../uri/8f3feb9a63afb6e100dad08136d8cbeea4a1dfed | 1 + .../uri/9100851463d47ef66626599f78730e4da01b8562 | 1 + .../uri/9dfe5ccca3a5ed2e574fb1dd10ec6d98bf637484 | 1 + .../uri/a5f6b7bdd3115f7427c55830f8fc4563256d410b | 1 + .../uri/adc83b19e793491b1c6ea0fd8b46cd9f32e592fc | 0 .../uri/ae13a961739ce96b0cb75d1fae1bd6eddaad473d | 1 + .../uri/b15df036caa676bffff10933845c16ce0192f726 | 1 + .../uri/b379e6c191cce7491020b50c345debd7ee95f14c | 1 + .../uri/b7899610b3e9299cc4248ff090827e35962a7a81 | 1 + .../uri/bad754935cc524ecde1a46997056db302ea3baa3 | 1 + .../uri/bbf379045a2134133a43636c5e7807d2503f6261 | 1 + .../uri/bdaa097dc0aecd0840c0e6e46ed269ccec14f5fe | 1 + .../uri/be403e2dc82146c9730731e8facb7a996fbfa01a | 1 + .../uri/c358d48b194741a6ed970a730c64c18c26076778 | 1 + .../uri/c4dff26491e4d229af0efcae747a2b78fddd7b75 | 1 + .../uri/c76f1c7d662c14d891f04058128ce7cd83bc09bd | 1 + .../uri/dabc3a9d30f94608b2a68a5beb470c704890e3f1 | 1 + .../uri/e0c30dfd58621aa24062fea9cf21d976e108a2f4 | 1 + .../uri/e400db1ce2202210e9542f9f030c6404bf5718c8 | 1 + .../uri/e5fa44f2b31c1fb553b6021e7360d07d5d91ff5e | 1 + .../uri/eca60ab9f2f130ab10bf4a642d00690442dd2b74 | 1 + .../uri/f25a33be8af0a7f725531c61f228d2339dfd089e | 1 + 130 files changed, 155 insertions(+) create mode 100644 test/static/corpus/csv/0c90a17c7b7cb5163255c7b775a8b544fb62c59d create mode 100644 test/static/corpus/csv/0e76fc3bb39bac4f774e2ce3978e9b59bc4a04b7 create mode 100644 test/static/corpus/csv/11f7ab7d8c2d4da313081e3282ab8120f0330b90 create mode 100644 test/static/corpus/csv/2ace62c1befa19e3ea37dd52be9f6d508c5163e6 create mode 100644 test/static/corpus/csv/3816dd9c9120fe6b3d806184874cc5c3e3a68df3 create mode 100644 test/static/corpus/csv/409c042df7f6a518972df1bc1287442e28719b6c create mode 100644 test/static/corpus/csv/46b175b37b5c3f746fde0dd114ead9938c012cfc create mode 100644 test/static/corpus/csv/56754bc8c7346ef9fad5c2ae9c54cef0af72629c create mode 100644 test/static/corpus/csv/58f43ea1e6df023f3dd4be9f718e5d0382fb9697 create mode 100644 test/static/corpus/csv/6d131d28c6e20c3a0a0b46c3aa7308d3029ab636 create mode 100644 test/static/corpus/csv/7560cb86c281018ee2f80d840fd8e6aa73a609ef create mode 100644 test/static/corpus/csv/7b97db755e18dc519fbc49e5de19b5820b9441c1 create mode 100644 test/static/corpus/csv/7ba0c88846fbcd28ad490df47f9ce4405d795303 create mode 100644 test/static/corpus/csv/8a14c110827e26cfdde3693bc5db6cd8673cdc8a create mode 100644 test/static/corpus/csv/94c1799dfba59120dd6075ee60016b809a915ca8 create mode 100644 test/static/corpus/csv/9e109f38c3948abe743bd116a34f05954f9ddd59 create mode 100644 test/static/corpus/csv/d21ec5eaaab3c8db1f923674b7a3ee19ce35ed13 create mode 100644 test/static/corpus/csv/d4a6d494cc2a06a0fee4873014210ec523d23750 create mode 100644 test/static/corpus/csv/eca825c9bdb276332d524134ddf370f7d18c2444 create mode 100644 test/static/corpus/csv/f9ba3e378e0409d66f946b31306de19edfe859bc create mode 100644 test/static/corpus/csv/fcad5471c295084dc1ec1eaa3753420e9803d251 create mode 100644 test/static/corpus/http_parser/038ec38a9ae7669590f1e3401da8ffbc9b40c8ee create mode 100644 test/static/corpus/http_parser/043fdf868410e66239819e0af10cbe787cfab77e create mode 100644 test/static/corpus/http_parser/08abe5308ce774ad5e9e435de386748ce06521f8 create mode 100644 test/static/corpus/http_parser/10b4f603cf09e21020e494e20a7f6760d5634c88 create mode 100644 test/static/corpus/http_parser/172ed6e56aa94bada4135d64fc2e0c5eac779aed create mode 100644 test/static/corpus/http_parser/1e42978bf6935a0f8377f6c480c315f1ae4627b8 create mode 100644 test/static/corpus/http_parser/1f6f26c085c3b209e626f90c63d0fb10526de318 create mode 100644 test/static/corpus/http_parser/24b458efa865a138a314f0876a11ec680b983ed5 create mode 100644 test/static/corpus/http_parser/2be565c790bc352f2cbaa0f3509e0bb6133b3616 create mode 100644 test/static/corpus/http_parser/30fd5e18f586b219916fb3892961a00e92db7a7f create mode 100644 test/static/corpus/http_parser/38918740212c6e5aea01f2fc8772b1d4aa89be8b create mode 100644 test/static/corpus/http_parser/39d4c74a90b7f5f5426b7982857b5fd7edbd8f77 create mode 100644 test/static/corpus/http_parser/3d21092e1a772d792e9dd720eeae30d8eb5ac2ad create mode 100644 test/static/corpus/http_parser/454dec613ca5c71fee0467061a88d2807201ac36 create mode 100644 test/static/corpus/http_parser/4f13053e61966dd08d7cd2407865481ec1d2cbf9 create mode 100644 test/static/corpus/http_parser/4ffa3daceb5b6ee3fc17cef3e0ae694ea5c583b5 create mode 100644 test/static/corpus/http_parser/501b3a19def06b352978c03ece7ab282cdc6cbab create mode 100644 test/static/corpus/http_parser/50ad023e1465919f5afbfffb33504ddfdc6a7e69 create mode 100644 test/static/corpus/http_parser/5177d236e3eea297b76c19194a4cf127b313b267 create mode 100644 test/static/corpus/http_parser/51e1354d16cfa4967e91206be8bd0d8c6ca577af create mode 100644 test/static/corpus/http_parser/55d5f3acc62809fc87466b2048b157e50e63d644 create mode 100644 test/static/corpus/http_parser/598d19d3175046ff035be228db2f1dc43957d8e2 create mode 100644 test/static/corpus/http_parser/5b7444e43e5c1fd759483c76a46a5536da043022 create mode 100644 test/static/corpus/http_parser/5c489184823a8a42844066fd5b4951bbad86934f create mode 100644 test/static/corpus/http_parser/6455c0e761cd485f3e38c470f216560bfc46eea4 create mode 100644 test/static/corpus/http_parser/6596bb363cfd0bef3f68e265cfd824c6d37ecb7e create mode 100644 test/static/corpus/http_parser/66d5206b6c75b071f57684399218d144b0d337ea create mode 100644 test/static/corpus/http_parser/685073a623772c1d46475b5eb451de40247d697f create mode 100644 test/static/corpus/http_parser/71d4de855c1af66a6a5cea88b1eddc5032ecb26d create mode 100644 test/static/corpus/http_parser/73378b09c61739b65a5f63214f0515307f187e53 create mode 100644 test/static/corpus/http_parser/768abf119ce4fa6c9a9ee9f17d9dacb32c61759c create mode 100644 test/static/corpus/http_parser/7ae1560be3dffbb88f3e48de3a4be99ecdc61441 create mode 100644 test/static/corpus/http_parser/7c4d33785daa5c2370201ffa236b427aa37c9996 create mode 100644 test/static/corpus/http_parser/80c5002c297e86198535d1574f3e47f4c6900782 create mode 100644 test/static/corpus/http_parser/86908cf9f9f2c690ebd7462cacd2b28fc45db3a5 create mode 100644 test/static/corpus/http_parser/89681f4bac464df0f73f3bf3fd48db40bc5796a6 create mode 100644 test/static/corpus/http_parser/8c371f137d8a10fc568a3022edf6f10280c96bc0 create mode 100644 test/static/corpus/http_parser/8ff59012e5ea417bf340e59a0ab6fd9e410011ea create mode 100644 test/static/corpus/http_parser/927ba76d456f6c3a06769795f87c46462e16acd5 create mode 100644 test/static/corpus/http_parser/95a0311cd792a22e0de733080e0580638c8eb136 create mode 100644 test/static/corpus/http_parser/9a3fc50025efc8aac22e16f10621f43a0f35ddcb create mode 100644 test/static/corpus/http_parser/ad1bde0f01973fc121af6f4a1efef3a1447c3a80 create mode 100644 test/static/corpus/http_parser/ad50e55369c0e4459500323e0af8eb00172f0909 create mode 100644 test/static/corpus/http_parser/b2af1811e30009a2b7cec55e86f26b8f4bf3c1cf create mode 100644 test/static/corpus/http_parser/bcebbda5c1d2033a62c14d6bf7e2dc54b407b403 create mode 100644 test/static/corpus/http_parser/cfe75970181a6e2f5ddbf9cf968a9a1fba1a660b create mode 100644 test/static/corpus/http_parser/d01798aa68e31f2964a2bc2c10b203f5186a3ba1 create mode 100644 test/static/corpus/http_parser/d2dc25ea74a20f35c78091cbc1e30cf2557c315f create mode 100644 test/static/corpus/http_parser/d930b69281035ce3b46f2635f5a91ee11f15876c create mode 100644 test/static/corpus/http_parser/dcb92ca0399b135c88170124fcdab6b6d62d115c create mode 100644 test/static/corpus/http_parser/e4e52eaca9ece89c0e3fa2ef7e6e126d47f20101 create mode 100644 test/static/corpus/http_parser/ebe97f523d44667ea6e11e1c8cd9e5f02cf244e7 create mode 100644 test/static/corpus/http_parser/ede1894116fdc6430ba23942e12d974e3f22875b create mode 100644 test/static/corpus/http_parser/f0f8a948443b28c5e344e6e43aedbb479059779c create mode 100644 test/static/corpus/http_parser/f1c4ffb8ea524f69bc8d6784f86b70ac99db2ad7 create mode 100644 test/static/corpus/http_parser/f774dd0b8fc355505aab48a2107804387f45c4a0 create mode 100644 test/static/corpus/uri/016d581dcda85465d5fa22eadd5947395b61303f create mode 100644 test/static/corpus/uri/018287fe57cf372d929b52a560910284ba406de6 create mode 100644 test/static/corpus/uri/048b75ce15485f8daa16e516014df26f2cf1ce09 create mode 100644 test/static/corpus/uri/048fbb93cfd1be0ccf151cf75147e391f12c2778 create mode 100644 test/static/corpus/uri/07e1d498149de21318fe486a73b618b800f9e46c create mode 100644 test/static/corpus/uri/093873ef42d30bee0ec042892457333ce7d6cd65 create mode 100644 test/static/corpus/uri/0a11f3691a10feefaeda9e30b4c891148d4ffdc1 create mode 100644 test/static/corpus/uri/14394ce809fcbd4db56a1f1814fb322103148fc6 create mode 100644 test/static/corpus/uri/251e9a9e4e077720a71747a94e24a6e25cb314a9 create mode 100644 test/static/corpus/uri/2a855b2e35301abb51facebb20e78801b2219e0f create mode 100644 test/static/corpus/uri/2ab8897ce59547952f047b48d52f9d6d4c0ea3eb create mode 100644 test/static/corpus/uri/39f3fca5b4634a9004220b389122abedfef28db7 create mode 100644 test/static/corpus/uri/3f18809f0df987b47c236d82d687ba16231daedb create mode 100644 test/static/corpus/uri/4143d3a341877154d6e95211464e1df1015b74bd create mode 100644 test/static/corpus/uri/49159cda7338d9b1a355631e7d33c55564fbfcb3 create mode 100644 test/static/corpus/uri/4b40fa57bf6479d96f1a751c68578ee64e7b6759 create mode 100644 test/static/corpus/uri/4f1a4d9e9b5db51ca7fdf0e9b4adb52f1813918b create mode 100644 test/static/corpus/uri/54b8c977fd54171c25da05e420b13d05dfaad747 create mode 100644 test/static/corpus/uri/5bfff0efcc1e7f8e3550798038d6361392379dfa create mode 100644 test/static/corpus/uri/60f481551bdf721e8d1f2c805da6a7fb2d2e4858 create mode 100644 test/static/corpus/uri/6c03daf8e568705738f646bfcdd03946acdce123 create mode 100644 test/static/corpus/uri/6dcaf13f83e40d73a9c5180977b5bf171b812e11 create mode 100644 test/static/corpus/uri/7511cc73615187b17837d01a013e4d886331fa54 create mode 100644 test/static/corpus/uri/767e882d43ffb1c822233640f7ca4e33426f7a3b create mode 100644 test/static/corpus/uri/777b43cd6a3f75e17d9310584ac00d4bd3ee65a1 create mode 100644 test/static/corpus/uri/77d8be75df172643a6fb1ee235d05b904aa501a0 create mode 100644 test/static/corpus/uri/7b8fa7a85f9f6ae6f9178504d2202666fb8dc772 create mode 100644 test/static/corpus/uri/7d850a53909012d59b379e99f1549332f01e3088 create mode 100644 test/static/corpus/uri/84f62a3184758e8c27e63f3f27c05a50877dc389 create mode 100644 test/static/corpus/uri/851be661349b0ceabf7cd933409f4d79ca9ac9a8 create mode 100644 test/static/corpus/uri/8efd1c91dceac73be0780f4f3cddce62ba84ebf6 create mode 100644 test/static/corpus/uri/8f3feb9a63afb6e100dad08136d8cbeea4a1dfed create mode 100644 test/static/corpus/uri/9100851463d47ef66626599f78730e4da01b8562 create mode 100644 test/static/corpus/uri/9dfe5ccca3a5ed2e574fb1dd10ec6d98bf637484 create mode 100644 test/static/corpus/uri/a5f6b7bdd3115f7427c55830f8fc4563256d410b create mode 100644 test/static/corpus/uri/adc83b19e793491b1c6ea0fd8b46cd9f32e592fc create mode 100644 test/static/corpus/uri/ae13a961739ce96b0cb75d1fae1bd6eddaad473d create mode 100644 test/static/corpus/uri/b15df036caa676bffff10933845c16ce0192f726 create mode 100644 test/static/corpus/uri/b379e6c191cce7491020b50c345debd7ee95f14c create mode 100644 test/static/corpus/uri/b7899610b3e9299cc4248ff090827e35962a7a81 create mode 100644 test/static/corpus/uri/bad754935cc524ecde1a46997056db302ea3baa3 create mode 100644 test/static/corpus/uri/bbf379045a2134133a43636c5e7807d2503f6261 create mode 100644 test/static/corpus/uri/bdaa097dc0aecd0840c0e6e46ed269ccec14f5fe create mode 100644 test/static/corpus/uri/be403e2dc82146c9730731e8facb7a996fbfa01a create mode 100644 test/static/corpus/uri/c358d48b194741a6ed970a730c64c18c26076778 create mode 100644 test/static/corpus/uri/c4dff26491e4d229af0efcae747a2b78fddd7b75 create mode 100644 test/static/corpus/uri/c76f1c7d662c14d891f04058128ce7cd83bc09bd create mode 100644 test/static/corpus/uri/dabc3a9d30f94608b2a68a5beb470c704890e3f1 create mode 100644 test/static/corpus/uri/e0c30dfd58621aa24062fea9cf21d976e108a2f4 create mode 100644 test/static/corpus/uri/e400db1ce2202210e9542f9f030c6404bf5718c8 create mode 100644 test/static/corpus/uri/e5fa44f2b31c1fb553b6021e7360d07d5d91ff5e create mode 100644 test/static/corpus/uri/eca60ab9f2f130ab10bf4a642d00690442dd2b74 create mode 100644 test/static/corpus/uri/f25a33be8af0a7f725531c61f228d2339dfd089e diff --git a/test/static/corpus/csv/0c90a17c7b7cb5163255c7b775a8b544fb62c59d b/test/static/corpus/csv/0c90a17c7b7cb5163255c7b775a8b544fb62c59d new file mode 100644 index 000000000..478457ace --- /dev/null +++ b/test/static/corpus/csv/0c90a17c7b7cb5163255c7b775a8b544fb62c59d @@ -0,0 +1,3 @@ +"a,b,c\n" +"1,\"\",\"\"\n" +"2,3,4\n" diff --git a/test/static/corpus/csv/0e76fc3bb39bac4f774e2ce3978e9b59bc4a04b7 b/test/static/corpus/csv/0e76fc3bb39bac4f774e2ce3978e9b59bc4a04b7 new file mode 100644 index 000000000..4d12fde01 --- /dev/null +++ b/test/static/corpus/csv/0e76fc3bb39bac4f774e2ce3978e9b59bc4a04b7 @@ -0,0 +1,2 @@ +"first,last,address,city,zip\n" +"John,Doe,120 any st.,\"Anytown, WW\",08123\n" diff --git a/test/static/corpus/csv/11f7ab7d8c2d4da313081e3282ab8120f0330b90 b/test/static/corpus/csv/11f7ab7d8c2d4da313081e3282ab8120f0330b90 new file mode 100644 index 000000000..ddf8cdaa1 --- /dev/null +++ b/test/static/corpus/csv/11f7ab7d8c2d4da313081e3282ab8120f0330b90 @@ -0,0 +1,5 @@ +"a,b\n" +"1,\"ha\n" +"\"\"ha\"\"\n" +"ha\"\n" +"3,4\n" diff --git a/test/static/corpus/csv/2ace62c1befa19e3ea37dd52be9f6d508c5163e6 b/test/static/corpus/csv/2ace62c1befa19e3ea37dd52be9f6d508c5163e6 new file mode 100644 index 000000000..9d68933c4 --- /dev/null +++ b/test/static/corpus/csv/2ace62c1befa19e3ea37dd52be9f6d508c5163e6 @@ -0,0 +1 @@ +" \ No newline at end of file diff --git a/test/static/corpus/csv/3816dd9c9120fe6b3d806184874cc5c3e3a68df3 b/test/static/corpus/csv/3816dd9c9120fe6b3d806184874cc5c3e3a68df3 new file mode 100644 index 000000000..662e19ef1 --- /dev/null +++ b/test/static/corpus/csv/3816dd9c9120fe6b3d806184874cc5c3e3a68df3 @@ -0,0 +1 @@ +",,\n" diff --git a/test/static/corpus/csv/409c042df7f6a518972df1bc1287442e28719b6c b/test/static/corpus/csv/409c042df7f6a518972df1bc1287442e28719b6c new file mode 100644 index 000000000..fdae850ce --- /dev/null +++ b/test/static/corpus/csv/409c042df7f6a518972df1bc1287442e28719b6c @@ -0,0 +1 @@ +"none,none,0\n" diff --git a/test/static/corpus/csv/46b175b37b5c3f746fde0dd114ead9938c012cfc b/test/static/corpus/csv/46b175b37b5c3f746fde0dd114ead9938c012cfc new file mode 100644 index 000000000..ae8df52ee --- /dev/null +++ b/test/static/corpus/csv/46b175b37b5c3f746fde0dd114ead9938c012cfc @@ -0,0 +1,3 @@ +" a,b,c\n" +"1,2,3\n" +"4,5,а нет ли ошибок?\n" diff --git a/test/static/corpus/csv/56754bc8c7346ef9fad5c2ae9c54cef0af72629c b/test/static/corpus/csv/56754bc8c7346ef9fad5c2ae9c54cef0af72629c new file mode 100644 index 000000000..c1dd6cff7 --- /dev/null +++ b/test/static/corpus/csv/56754bc8c7346ef9fad5c2ae9c54cef0af72629c @@ -0,0 +1,2 @@ +"key,val\n" +"1,\"{\"\"type\"\": \"\"Point\"\", \"\"coordinates\"\": [102.0, 0.5]}\"\n" diff --git a/test/static/corpus/csv/58f43ea1e6df023f3dd4be9f718e5d0382fb9697 b/test/static/corpus/csv/58f43ea1e6df023f3dd4be9f718e5d0382fb9697 new file mode 100644 index 000000000..54588ce72 --- /dev/null +++ b/test/static/corpus/csv/58f43ea1e6df023f3dd4be9f718e5d0382fb9697 @@ -0,0 +1,3 @@ +"a,b\n" +"1,\"ha \"\"ha\"\" ha\"\n" +"3,4\n" diff --git a/test/static/corpus/csv/6d131d28c6e20c3a0a0b46c3aa7308d3029ab636 b/test/static/corpus/csv/6d131d28c6e20c3a0a0b46c3aa7308d3029ab636 new file mode 100644 index 000000000..eb9349cb7 --- /dev/null +++ b/test/static/corpus/csv/6d131d28c6e20c3a0a0b46c3aa7308d3029ab636 @@ -0,0 +1 @@ +"abc", "with,comma", "\"in quotes\"", "1 \" quote" diff --git a/test/static/corpus/csv/7560cb86c281018ee2f80d840fd8e6aa73a609ef b/test/static/corpus/csv/7560cb86c281018ee2f80d840fd8e6aa73a609ef new file mode 100644 index 000000000..d1f7857f7 --- /dev/null +++ b/test/static/corpus/csv/7560cb86c281018ee2f80d840fd8e6aa73a609ef @@ -0,0 +1,5 @@ +"a,b,c\n" +"1,2,3\n" +"\"Once upon \n" +"a time\",5,6\n" +"7,8,9\n" diff --git a/test/static/corpus/csv/7b97db755e18dc519fbc49e5de19b5820b9441c1 b/test/static/corpus/csv/7b97db755e18dc519fbc49e5de19b5820b9441c1 new file mode 100644 index 000000000..02febfb9e --- /dev/null +++ b/test/static/corpus/csv/7b97db755e18dc519fbc49e5de19b5820b9441c1 @@ -0,0 +1 @@ +"\n \nabc\nc\"\",\"d\",de\n\nk"; diff --git a/test/static/corpus/csv/7ba0c88846fbcd28ad490df47f9ce4405d795303 b/test/static/corpus/csv/7ba0c88846fbcd28ad490df47f9ce4405d795303 new file mode 100644 index 000000000..48815e09a --- /dev/null +++ b/test/static/corpus/csv/7ba0c88846fbcd28ad490df47f9ce4405d795303 @@ -0,0 +1,2 @@ +("123 , 5 , 92 , 0, 0\n" +"1, 12 34, 56, \"quote , \", 66\nok"); diff --git a/test/static/corpus/csv/8a14c110827e26cfdde3693bc5db6cd8673cdc8a b/test/static/corpus/csv/8a14c110827e26cfdde3693bc5db6cd8673cdc8a new file mode 100644 index 000000000..0bf3ab53e --- /dev/null +++ b/test/static/corpus/csv/8a14c110827e26cfdde3693bc5db6cd8673cdc8a @@ -0,0 +1 @@ +" www , \"aa\"a , \"tt \" \n" diff --git a/test/static/corpus/csv/94c1799dfba59120dd6075ee60016b809a915ca8 b/test/static/corpus/csv/94c1799dfba59120dd6075ee60016b809a915ca8 new file mode 100644 index 000000000..ca2693d40 --- /dev/null +++ b/test/static/corpus/csv/94c1799dfba59120dd6075ee60016b809a915ca8 @@ -0,0 +1 @@ +"1,2,3\r\n", "4,5,6", "", "" diff --git a/test/static/corpus/csv/9e109f38c3948abe743bd116a34f05954f9ddd59 b/test/static/corpus/csv/9e109f38c3948abe743bd116a34f05954f9ddd59 new file mode 100644 index 000000000..1535ccc47 --- /dev/null +++ b/test/static/corpus/csv/9e109f38c3948abe743bd116a34f05954f9ddd59 @@ -0,0 +1 @@ +"abc\tlonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglonglong\t0\n" diff --git a/test/static/corpus/csv/d21ec5eaaab3c8db1f923674b7a3ee19ce35ed13 b/test/static/corpus/csv/d21ec5eaaab3c8db1f923674b7a3ee19ce35ed13 new file mode 100644 index 000000000..474d54942 --- /dev/null +++ b/test/static/corpus/csv/d21ec5eaaab3c8db1f923674b7a3ee19ce35ed13 @@ -0,0 +1 @@ +"123,456,abcac,\'multiword field 4\'\n" diff --git a/test/static/corpus/csv/d4a6d494cc2a06a0fee4873014210ec523d23750 b/test/static/corpus/csv/d4a6d494cc2a06a0fee4873014210ec523d23750 new file mode 100644 index 000000000..b1f9c2adf --- /dev/null +++ b/test/static/corpus/csv/d4a6d494cc2a06a0fee4873014210ec523d23750 @@ -0,0 +1 @@ +"123\t456\t\n" "0\t\t\n"; diff --git a/test/static/corpus/csv/eca825c9bdb276332d524134ddf370f7d18c2444 b/test/static/corpus/csv/eca825c9bdb276332d524134ddf370f7d18c2444 new file mode 100644 index 000000000..7ae2d4413 --- /dev/null +++ b/test/static/corpus/csv/eca825c9bdb276332d524134ddf370f7d18c2444 @@ -0,0 +1 @@ +",," diff --git a/test/static/corpus/csv/f9ba3e378e0409d66f946b31306de19edfe859bc b/test/static/corpus/csv/f9ba3e378e0409d66f946b31306de19edfe859bc new file mode 100644 index 000000000..7946d4912 --- /dev/null +++ b/test/static/corpus/csv/f9ba3e378e0409d66f946b31306de19edfe859bc @@ -0,0 +1 @@ +"\ne\n\n \n\" \"\n\"quote isn't closed, sorry\n \noh"; diff --git a/test/static/corpus/csv/fcad5471c295084dc1ec1eaa3753420e9803d251 b/test/static/corpus/csv/fcad5471c295084dc1ec1eaa3753420e9803d251 new file mode 100644 index 000000000..b101b84fe --- /dev/null +++ b/test/static/corpus/csv/fcad5471c295084dc1ec1eaa3753420e9803d251 @@ -0,0 +1,10 @@ +"\n\r\" ba\r a\ra, \n\"\n\"a\nb\" \raa\rb,\n" +"\r, \n\",\r\n\"\n,a, ,\"a\n\n\r \"\r ba\r,b" +" a,\n,\"\"a\n\r \"b\" \n,\",a\r,a ,\r\rc" +"\" a,b\r\n,\"b\r\"aa \nb \n\r\r\n\n,\rb\nc" +",\n\n aa\n \"\n ab\rab,\r\" b\n\", ,,\r\r" +"bab\rb\na\n\"a\ra,\"\",\n\"a\n\n \"\r \ra\n" +"a\r\raa a\" ,baab ,a \rbb ,\r \r,\rb,, b" +"\n\r\"\nb\n\nb \n,ab \raa\r\"\nb a\"ba,b, c" +"\"a\"a \"\r\n\"b \n,b\"\",\nba\n\" \n\na \r" +"\nb\rb\"bbba,\" \n\n\n,a,b,a,b,\n\n\n\nb\"\r"; diff --git a/test/static/corpus/http_parser/038ec38a9ae7669590f1e3401da8ffbc9b40c8ee b/test/static/corpus/http_parser/038ec38a9ae7669590f1e3401da8ffbc9b40c8ee new file mode 100644 index 000000000..e7768afcb --- /dev/null +++ b/test/static/corpus/http_parser/038ec38a9ae7669590f1e3401da8ffbc9b40c8ee @@ -0,0 +1 @@ +Content-Length: 43 diff --git a/test/static/corpus/http_parser/043fdf868410e66239819e0af10cbe787cfab77e b/test/static/corpus/http_parser/043fdf868410e66239819e0af10cbe787cfab77e new file mode 100644 index 000000000..a9838e1fa --- /dev/null +++ b/test/static/corpus/http_parser/043fdf868410e66239819e0af10cbe787cfab77e @@ -0,0 +1 @@ +.& \ No newline at end of file diff --git a/test/static/corpus/http_parser/08abe5308ce774ad5e9e435de386748ce06521f8 b/test/static/corpus/http_parser/08abe5308ce774ad5e9e435de386748ce06521f8 new file mode 100644 index 000000000..932361d03 --- /dev/null +++ b/test/static/corpus/http_parser/08abe5308ce774ad5e9e435de386748ce06521f8 @@ -0,0 +1 @@ +Last-Modified: Sat, 28 Nov 2009 03:50:37 GMT diff --git a/test/static/corpus/http_parser/10b4f603cf09e21020e494e20a7f6760d5634c88 b/test/static/corpus/http_parser/10b4f603cf09e21020e494e20a7f6760d5634c88 new file mode 100644 index 000000000..ecb696486 --- /dev/null +++ b/test/static/corpus/http_parser/10b4f603cf09e21020e494e20a7f6760d5634c88 @@ -0,0 +1 @@ +.-=\b \ No newline at end of file diff --git a/test/static/corpus/http_parser/172ed6e56aa94bada4135d64fc2e0c5eac779aed b/test/static/corpus/http_parser/172ed6e56aa94bada4135d64fc2e0c5eac779aed new file mode 100644 index 000000000..6dac7d673 --- /dev/null +++ b/test/static/corpus/http_parser/172ed6e56aa94bada4135d64fc2e0c5eac779aed @@ -0,0 +1 @@ +Keep-live: 300 diff --git a/test/static/corpus/http_parser/1e42978bf6935a0f8377f6c480c315f1ae4627b8 b/test/static/corpus/http_parser/1e42978bf6935a0f8377f6c480c315f1ae4627b8 new file mode 100644 index 000000000..e3bec95f8 --- /dev/null +++ b/test/static/corpus/http_parser/1e42978bf6935a0f8377f6c480c315f1ae4627b8 @@ -0,0 +1 @@ +&-3=\b \ No newline at end of file diff --git a/test/static/corpus/http_parser/1f6f26c085c3b209e626f90c63d0fb10526de318 b/test/static/corpus/http_parser/1f6f26c085c3b209e626f90c63d0fb10526de318 new file mode 100644 index 000000000..8a6546a25 --- /dev/null +++ b/test/static/corpus/http_parser/1f6f26c085c3b209e626f90c63d0fb10526de318 @@ -0,0 +1 @@ +Host: en.wikipedia.org:8080 diff --git a/test/static/corpus/http_parser/24b458efa865a138a314f0876a11ec680b983ed5 b/test/static/corpus/http_parser/24b458efa865a138a314f0876a11ec680b983ed5 new file mode 100644 index 000000000..602ec1578 --- /dev/null +++ b/test/static/corpus/http_parser/24b458efa865a138a314f0876a11ec680b983ed5 @@ -0,0 +1 @@ +Set-Cookie: TestCookie=foobar diff --git a/test/static/corpus/http_parser/2be565c790bc352f2cbaa0f3509e0bb6133b3616 b/test/static/corpus/http_parser/2be565c790bc352f2cbaa0f3509e0bb6133b3616 new file mode 100644 index 000000000..2c8a48275 --- /dev/null +++ b/test/static/corpus/http_parser/2be565c790bc352f2cbaa0f3509e0bb6133b3616 @@ -0,0 +1 @@ +Accept-Encoding: x-compress; x-zip diff --git a/test/static/corpus/http_parser/30fd5e18f586b219916fb3892961a00e92db7a7f b/test/static/corpus/http_parser/30fd5e18f586b219916fb3892961a00e92db7a7f new file mode 100644 index 000000000..80162a22a --- /dev/null +++ b/test/static/corpus/http_parser/30fd5e18f586b219916fb3892961a00e92db7a7f @@ -0,0 +1 @@ +Accept-Ranges: none diff --git a/test/static/corpus/http_parser/38918740212c6e5aea01f2fc8772b1d4aa89be8b b/test/static/corpus/http_parser/38918740212c6e5aea01f2fc8772b1d4aa89be8b new file mode 100644 index 000000000..025a01fb9 --- /dev/null +++ b/test/static/corpus/http_parser/38918740212c6e5aea01f2fc8772b1d4aa89be8b @@ -0,0 +1 @@ +Accept-Language: da, en-gb;q=0.8, en;q=0.7 diff --git a/test/static/corpus/http_parser/39d4c74a90b7f5f5426b7982857b5fd7edbd8f77 b/test/static/corpus/http_parser/39d4c74a90b7f5f5426b7982857b5fd7edbd8f77 new file mode 100644 index 000000000..ceab8d8a1 --- /dev/null +++ b/test/static/corpus/http_parser/39d4c74a90b7f5f5426b7982857b5fd7edbd8f77 @@ -0,0 +1 @@ +Range: bytes=500-999 diff --git a/test/static/corpus/http_parser/3d21092e1a772d792e9dd720eeae30d8eb5ac2ad b/test/static/corpus/http_parser/3d21092e1a772d792e9dd720eeae30d8eb5ac2ad new file mode 100644 index 000000000..0ea58ee1e --- /dev/null +++ b/test/static/corpus/http_parser/3d21092e1a772d792e9dd720eeae30d8eb5ac2ad @@ -0,0 +1 @@ +Accept-Charset: iso-8859-5, unicode-1-1;q=0.8 diff --git a/test/static/corpus/http_parser/454dec613ca5c71fee0467061a88d2807201ac36 b/test/static/corpus/http_parser/454dec613ca5c71fee0467061a88d2807201ac36 new file mode 100644 index 000000000..ede646924 --- /dev/null +++ b/test/static/corpus/http_parser/454dec613ca5c71fee0467061a88d2807201ac36 @@ -0,0 +1 @@ +Cache-Control: min-fresh=11 diff --git a/test/static/corpus/http_parser/4f13053e61966dd08d7cd2407865481ec1d2cbf9 b/test/static/corpus/http_parser/4f13053e61966dd08d7cd2407865481ec1d2cbf9 new file mode 100644 index 000000000..9a590ea35 --- /dev/null +++ b/test/static/corpus/http_parser/4f13053e61966dd08d7cd2407865481ec1d2cbf9 @@ -0,0 +1 @@ +Max-Forwards: 10 diff --git a/test/static/corpus/http_parser/4ffa3daceb5b6ee3fc17cef3e0ae694ea5c583b5 b/test/static/corpus/http_parser/4ffa3daceb5b6ee3fc17cef3e0ae694ea5c583b5 new file mode 100644 index 000000000..d3f25e825 --- /dev/null +++ b/test/static/corpus/http_parser/4ffa3daceb5b6ee3fc17cef3e0ae694ea5c583b5 @@ -0,0 +1 @@ +User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:12.0) Gecko/20100101 Firefox/12.0 diff --git a/test/static/corpus/http_parser/501b3a19def06b352978c03ece7ab282cdc6cbab b/test/static/corpus/http_parser/501b3a19def06b352978c03ece7ab282cdc6cbab new file mode 100644 index 000000000..3b90625c6 --- /dev/null +++ b/test/static/corpus/http_parser/501b3a19def06b352978c03ece7ab282cdc6cbab @@ -0,0 +1 @@ +Cache-Control: must-revalidate diff --git a/test/static/corpus/http_parser/50ad023e1465919f5afbfffb33504ddfdc6a7e69 b/test/static/corpus/http_parser/50ad023e1465919f5afbfffb33504ddfdc6a7e69 new file mode 100644 index 000000000..6451a1aa2 --- /dev/null +++ b/test/static/corpus/http_parser/50ad023e1465919f5afbfffb33504ddfdc6a7e69 @@ -0,0 +1 @@ +Etag: "pub1259380237;gz" diff --git a/test/static/corpus/http_parser/5177d236e3eea297b76c19194a4cf127b313b267 b/test/static/corpus/http_parser/5177d236e3eea297b76c19194a4cf127b313b267 new file mode 100644 index 000000000..161b3f8e6 --- /dev/null +++ b/test/static/corpus/http_parser/5177d236e3eea297b76c19194a4cf127b313b267 @@ -0,0 +1 @@ +Accept-Datetime: Thu, 31 May 2007 20:35:00 GMT diff --git a/test/static/corpus/http_parser/51e1354d16cfa4967e91206be8bd0d8c6ca577af b/test/static/corpus/http_parser/51e1354d16cfa4967e91206be8bd0d8c6ca577af new file mode 100644 index 000000000..e853d6cee --- /dev/null +++ b/test/static/corpus/http_parser/51e1354d16cfa4967e91206be8bd0d8c6ca577af @@ -0,0 +1 @@ +Content-Type: text/html; charset=UTF-8 diff --git a/test/static/corpus/http_parser/55d5f3acc62809fc87466b2048b157e50e63d644 b/test/static/corpus/http_parser/55d5f3acc62809fc87466b2048b157e50e63d644 new file mode 100644 index 000000000..587feae5d --- /dev/null +++ b/test/static/corpus/http_parser/55d5f3acc62809fc87466b2048b157e50e63d644 @@ -0,0 +1 @@ +Keep-3=\b \ No newline at end of file diff --git a/test/static/corpus/http_parser/598d19d3175046ff035be228db2f1dc43957d8e2 b/test/static/corpus/http_parser/598d19d3175046ff035be228db2f1dc43957d8e2 new file mode 100644 index 000000000..9e2c13e63 --- /dev/null +++ b/test/static/corpus/http_parser/598d19d3175046ff035be228db2f1dc43957d8e2 @@ -0,0 +1 @@ +Content-Encoding: gzip diff --git a/test/static/corpus/http_parser/5b7444e43e5c1fd759483c76a46a5536da043022 b/test/static/corpus/http_parser/5b7444e43e5c1fd759483c76a46a5536da043022 new file mode 100644 index 000000000..a3b63bba5 --- /dev/null +++ b/test/static/corpus/http_parser/5b7444e43e5c1fd759483c76a46a5536da043022 @@ -0,0 +1 @@ +Cache-Control: max-stale=45 diff --git a/test/static/corpus/http_parser/5c489184823a8a42844066fd5b4951bbad86934f b/test/static/corpus/http_parser/5c489184823a8a42844066fd5b4951bbad86934f new file mode 100644 index 000000000..8ba627a9b --- /dev/null +++ b/test/static/corpus/http_parser/5c489184823a8a42844066fd5b4951bbad86934f @@ -0,0 +1 @@ +Cache-Control: proxy-revalidate diff --git a/test/static/corpus/http_parser/6455c0e761cd485f3e38c470f216560bfc46eea4 b/test/static/corpus/http_parser/6455c0e761cd485f3e38c470f216560bfc46eea4 new file mode 100644 index 000000000..4a30a206a --- /dev/null +++ b/test/static/corpus/http_parser/6455c0e761cd485f3e38c470f216560bfc46eea4 @@ -0,0 +1 @@ +If-Unmodified-Since: Sat, 29 Oct 1994 19:43:31 GMT diff --git a/test/static/corpus/http_parser/6596bb363cfd0bef3f68e265cfd824c6d37ecb7e b/test/static/corpus/http_parser/6596bb363cfd0bef3f68e265cfd824c6d37ecb7e new file mode 100644 index 000000000..5f0cf5daa --- /dev/null +++ b/test/static/corpus/http_parser/6596bb363cfd0bef3f68e265cfd824c6d37ecb7e @@ -0,0 +1 @@ +Keep-Alive: 300 diff --git a/test/static/corpus/http_parser/66d5206b6c75b071f57684399218d144b0d337ea b/test/static/corpus/http_parser/66d5206b6c75b071f57684399218d144b0d337ea new file mode 100644 index 000000000..dc62f1540 --- /dev/null +++ b/test/static/corpus/http_parser/66d5206b6c75b071f57684399218d144b0d337ea @@ -0,0 +1 @@ +Content-Type: image/gif diff --git a/test/static/corpus/http_parser/685073a623772c1d46475b5eb451de40247d697f b/test/static/corpus/http_parser/685073a623772c1d46475b5eb451de40247d697f new file mode 100644 index 000000000..771aba7f9 --- /dev/null +++ b/test/static/corpus/http_parser/685073a623772c1d46475b5eb451de40247d697f @@ -0,0 +1 @@ +Cache-Control: max-age=300 diff --git a/test/static/corpus/http_parser/71d4de855c1af66a6a5cea88b1eddc5032ecb26d b/test/static/corpus/http_parser/71d4de855c1af66a6a5cea88b1eddc5032ecb26d new file mode 100644 index 000000000..946193452 --- /dev/null +++ b/test/static/corpus/http_parser/71d4de855c1af66a6a5cea88b1eddc5032ecb26d @@ -0,0 +1 @@ +Pragma: no-cache diff --git a/test/static/corpus/http_parser/73378b09c61739b65a5f63214f0515307f187e53 b/test/static/corpus/http_parser/73378b09c61739b65a5f63214f0515307f187e53 new file mode 100644 index 000000000..021f2dc34 --- /dev/null +++ b/test/static/corpus/http_parser/73378b09c61739b65a5f63214f0515307f187e53 @@ -0,0 +1 @@ +Accept: text/x-dvi; q=.8; mxb=100000; mxt=5.0, text/x-c diff --git a/test/static/corpus/http_parser/768abf119ce4fa6c9a9ee9f17d9dacb32c61759c b/test/static/corpus/http_parser/768abf119ce4fa6c9a9ee9f17d9dacb32c61759c new file mode 100644 index 000000000..3119cab57 --- /dev/null +++ b/test/static/corpus/http_parser/768abf119ce4fa6c9a9ee9f17d9dacb32c61759c @@ -0,0 +1 @@ +Content-Disposition: attachment; filename="download.zip" diff --git a/test/static/corpus/http_parser/7ae1560be3dffbb88f3e48de3a4be99ecdc61441 b/test/static/corpus/http_parser/7ae1560be3dffbb88f3e48de3a4be99ecdc61441 new file mode 100644 index 000000000..d0c96be5b --- /dev/null +++ b/test/static/corpus/http_parser/7ae1560be3dffbb88f3e48de3a4be99ecdc61441 @@ -0,0 +1 @@ +Accept: text/plain, text/html diff --git a/test/static/corpus/http_parser/7c4d33785daa5c2370201ffa236b427aa37c9996 b/test/static/corpus/http_parser/7c4d33785daa5c2370201ffa236b427aa37c9996 new file mode 100644 index 000000000..00b15c0a3 --- /dev/null +++ b/test/static/corpus/http_parser/7c4d33785daa5c2370201ffa236b427aa37c9996 @@ -0,0 +1 @@ +& \ No newline at end of file diff --git a/test/static/corpus/http_parser/80c5002c297e86198535d1574f3e47f4c6900782 b/test/static/corpus/http_parser/80c5002c297e86198535d1574f3e47f4c6900782 new file mode 100644 index 000000000..fc71de395 --- /dev/null +++ b/test/static/corpus/http_parser/80c5002c297e86198535d1574f3e47f4c6900782 @@ -0,0 +1 @@ +Cookie: PHPSESSID=r2t5uvjq435r4q7ib3vtdjq120; foo=bar diff --git a/test/static/corpus/http_parser/86908cf9f9f2c690ebd7462cacd2b28fc45db3a5 b/test/static/corpus/http_parser/86908cf9f9f2c690ebd7462cacd2b28fc45db3a5 new file mode 100644 index 000000000..3d9737b98 --- /dev/null +++ b/test/static/corpus/http_parser/86908cf9f9f2c690ebd7462cacd2b28fc45db3a5 @@ -0,0 +1 @@ +WWW-Authenticate: Basic realm="Restricted Area" diff --git a/test/static/corpus/http_parser/89681f4bac464df0f73f3bf3fd48db40bc5796a6 b/test/static/corpus/http_parser/89681f4bac464df0f73f3bf3fd48db40bc5796a6 new file mode 100644 index 000000000..a3a512ae4 --- /dev/null +++ b/test/static/corpus/http_parser/89681f4bac464df0f73f3bf3fd48db40bc5796a6 @@ -0,0 +1 @@ +Location: https://tarantool.io/ diff --git a/test/static/corpus/http_parser/8c371f137d8a10fc568a3022edf6f10280c96bc0 b/test/static/corpus/http_parser/8c371f137d8a10fc568a3022edf6f10280c96bc0 new file mode 100644 index 000000000..1ba97ade6 --- /dev/null +++ b/test/static/corpus/http_parser/8c371f137d8a10fc568a3022edf6f10280c96bc0 @@ -0,0 +1 @@ +Transfer-Encoding: chunked diff --git a/test/static/corpus/http_parser/8ff59012e5ea417bf340e59a0ab6fd9e410011ea b/test/static/corpus/http_parser/8ff59012e5ea417bf340e59a0ab6fd9e410011ea new file mode 100644 index 000000000..4030ea1d3 --- /dev/null +++ b/test/static/corpus/http_parser/8ff59012e5ea417bf340e59a0ab6fd9e410011ea @@ -0,0 +1 @@ +Cache-Control: no-store diff --git a/test/static/corpus/http_parser/927ba76d456f6c3a06769795f87c46462e16acd5 b/test/static/corpus/http_parser/927ba76d456f6c3a06769795f87c46462e16acd5 new file mode 100644 index 000000000..b3780da37 --- /dev/null +++ b/test/static/corpus/http_parser/927ba76d456f6c3a06769795f87c46462e16acd5 @@ -0,0 +1 @@ +If-Match: "737060cd8c284d8af7ad3082f209582d" diff --git a/test/static/corpus/http_parser/95a0311cd792a22e0de733080e0580638c8eb136 b/test/static/corpus/http_parser/95a0311cd792a22e0de733080e0580638c8eb136 new file mode 100644 index 000000000..21e63e87b --- /dev/null +++ b/test/static/corpus/http_parser/95a0311cd792a22e0de733080e0580638c8eb136 @@ -0,0 +1 @@ +Keep-300 diff --git a/test/static/corpus/http_parser/9a3fc50025efc8aac22e16f10621f43a0f35ddcb b/test/static/corpus/http_parser/9a3fc50025efc8aac22e16f10621f43a0f35ddcb new file mode 100644 index 000000000..c3aca8920 --- /dev/null +++ b/test/static/corpus/http_parser/9a3fc50025efc8aac22e16f10621f43a0f35ddcb @@ -0,0 +1 @@ +Cache-Control: only-if-cached diff --git a/test/static/corpus/http_parser/ad1bde0f01973fc121af6f4a1efef3a1447c3a80 b/test/static/corpus/http_parser/ad1bde0f01973fc121af6f4a1efef3a1447c3a80 new file mode 100644 index 000000000..f65d4c242 --- /dev/null +++ b/test/static/corpus/http_parser/ad1bde0f01973fc121af6f4a1efef3a1447c3a80 @@ -0,0 +1 @@ +age-value = delta-seconds diff --git a/test/static/corpus/http_parser/ad50e55369c0e4459500323e0af8eb00172f0909 b/test/static/corpus/http_parser/ad50e55369c0e4459500323e0af8eb00172f0909 new file mode 100644 index 000000000..c658da69b --- /dev/null +++ b/test/static/corpus/http_parser/ad50e55369c0e4459500323e0af8eb00172f0909 @@ -0,0 +1 @@ +TE: trailers, deflate diff --git a/test/static/corpus/http_parser/b2af1811e30009a2b7cec55e86f26b8f4bf3c1cf b/test/static/corpus/http_parser/b2af1811e30009a2b7cec55e86f26b8f4bf3c1cf new file mode 100644 index 000000000..7e5541b20 --- /dev/null +++ b/test/static/corpus/http_parser/b2af1811e30009a2b7cec55e86f26b8f4bf3c1cf @@ -0,0 +1 @@ +Allow: GET, HEAD, PUT diff --git a/test/static/corpus/http_parser/bcebbda5c1d2033a62c14d6bf7e2dc54b407b403 b/test/static/corpus/http_parser/bcebbda5c1d2033a62c14d6bf7e2dc54b407b403 new file mode 100644 index 000000000..6d60ea558 --- /dev/null +++ b/test/static/corpus/http_parser/bcebbda5c1d2033a62c14d6bf7e2dc54b407b403 @@ -0,0 +1 @@ +Cache-Control: no-transform diff --git a/test/static/corpus/http_parser/cfe75970181a6e2f5ddbf9cf968a9a1fba1a660b b/test/static/corpus/http_parser/cfe75970181a6e2f5ddbf9cf968a9a1fba1a660b new file mode 100644 index 000000000..702698a34 --- /dev/null +++ b/test/static/corpus/http_parser/cfe75970181a6e2f5ddbf9cf968a9a1fba1a660b @@ -0,0 +1 @@ +Trailer: Max-Forwards diff --git a/test/static/corpus/http_parser/d01798aa68e31f2964a2bc2c10b203f5186a3ba1 b/test/static/corpus/http_parser/d01798aa68e31f2964a2bc2c10b203f5186a3ba1 new file mode 100644 index 000000000..64db5fe12 --- /dev/null +++ b/test/static/corpus/http_parser/d01798aa68e31f2964a2bc2c10b203f5186a3ba1 @@ -0,0 +1 @@ +Connection: keep-alive diff --git a/test/static/corpus/http_parser/d2dc25ea74a20f35c78091cbc1e30cf2557c315f b/test/static/corpus/http_parser/d2dc25ea74a20f35c78091cbc1e30cf2557c315f new file mode 100644 index 000000000..4fbbb4bea --- /dev/null +++ b/test/static/corpus/http_parser/d2dc25ea74a20f35c78091cbc1e30cf2557c315f @@ -0,0 +1 @@ +Accept: audio/basic q=1 diff --git a/test/static/corpus/http_parser/d930b69281035ce3b46f2635f5a91ee11f15876c b/test/static/corpus/http_parser/d930b69281035ce3b46f2635f5a91ee11f15876c new file mode 100644 index 000000000..fd10f4201 --- /dev/null +++ b/test/static/corpus/http_parser/d930b69281035ce3b46f2635f5a91ee11f15876c @@ -0,0 +1 @@ +If-None-Match: "737060cd8c284d8af7ad3082f209582d" diff --git a/test/static/corpus/http_parser/dcb92ca0399b135c88170124fcdab6b6d62d115c b/test/static/corpus/http_parser/dcb92ca0399b135c88170124fcdab6b6d62d115c new file mode 100644 index 000000000..656dc4b35 --- /dev/null +++ b/test/static/corpus/http_parser/dcb92ca0399b135c88170124fcdab6b6d62d115c @@ -0,0 +1 @@ +Referer: http://www.w3.org/hypertext/DataSources/Overview.html diff --git a/test/static/corpus/http_parser/e4e52eaca9ece89c0e3fa2ef7e6e126d47f20101 b/test/static/corpus/http_parser/e4e52eaca9ece89c0e3fa2ef7e6e126d47f20101 new file mode 100644 index 000000000..bfec7f39a --- /dev/null +++ b/test/static/corpus/http_parser/e4e52eaca9ece89c0e3fa2ef7e6e126d47f20101 @@ -0,0 +1 @@ +If-Modified-Since: Sat, 29 Oct 1994 19:43:31 GMT diff --git a/test/static/corpus/http_parser/ebe97f523d44667ea6e11e1c8cd9e5f02cf244e7 b/test/static/corpus/http_parser/ebe97f523d44667ea6e11e1c8cd9e5f02cf244e7 new file mode 100644 index 000000000..1af817ae2 --- /dev/null +++ b/test/static/corpus/http_parser/ebe97f523d44667ea6e11e1c8cd9e5f02cf244e7 @@ -0,0 +1 @@ +Access-Control-Request-Method: GET diff --git a/test/static/corpus/http_parser/ede1894116fdc6430ba23942e12d974e3f22875b b/test/static/corpus/http_parser/ede1894116fdc6430ba23942e12d974e3f22875b new file mode 100644 index 000000000..fa5c59479 --- /dev/null +++ b/test/static/corpus/http_parser/ede1894116fdc6430ba23942e12d974e3f22875b @@ -0,0 +1 @@ +If-Modified-Since: Sat, 28 Nov 2009 06:38:19 GMT diff --git a/test/static/corpus/http_parser/f0f8a948443b28c5e344e6e43aedbb479059779c b/test/static/corpus/http_parser/f0f8a948443b28c5e344e6e43aedbb479059779c new file mode 100644 index 000000000..c2baf4c7b --- /dev/null +++ b/test/static/corpus/http_parser/f0f8a948443b28c5e344e6e43aedbb479059779c @@ -0,0 +1 @@ +Authorization: kerberos kerberosauthenticationsparameters diff --git a/test/static/corpus/http_parser/f1c4ffb8ea524f69bc8d6784f86b70ac99db2ad7 b/test/static/corpus/http_parser/f1c4ffb8ea524f69bc8d6784f86b70ac99db2ad7 new file mode 100644 index 000000000..d205ba72c --- /dev/null +++ b/test/static/corpus/http_parser/f1c4ffb8ea524f69bc8d6784f86b70ac99db2ad7 @@ -0,0 +1 @@ +Accept-Ranges: bytes diff --git a/test/static/corpus/http_parser/f774dd0b8fc355505aab48a2107804387f45c4a0 b/test/static/corpus/http_parser/f774dd0b8fc355505aab48a2107804387f45c4a0 new file mode 100644 index 000000000..e971af90d --- /dev/null +++ b/test/static/corpus/http_parser/f774dd0b8fc355505aab48a2107804387f45c4a0 @@ -0,0 +1 @@ +Authorization: user tarantool:mypassword diff --git a/test/static/corpus/uri/016d581dcda85465d5fa22eadd5947395b61303f b/test/static/corpus/uri/016d581dcda85465d5fa22eadd5947395b61303f new file mode 100644 index 000000000..05514f9fd --- /dev/null +++ b/test/static/corpus/uri/016d581dcda85465d5fa22eadd5947395b61303f @@ -0,0 +1 @@ +[2001::11a3:09d7::1] diff --git a/test/static/corpus/uri/018287fe57cf372d929b52a560910284ba406de6 b/test/static/corpus/uri/018287fe57cf372d929b52a560910284ba406de6 new file mode 100644 index 000000000..f7d87dee1 --- /dev/null +++ b/test/static/corpus/uri/018287fe57cf372d929b52a560910284ba406de6 @@ -0,0 +1 @@ +:// diff --git a/test/static/corpus/uri/048b75ce15485f8daa16e516014df26f2cf1ce09 b/test/static/corpus/uri/048b75ce15485f8daa16e516014df26f2cf1ce09 new file mode 100644 index 000000000..5d58e595c --- /dev/null +++ b/test/static/corpus/uri/048b75ce15485f8daa16e516014df26f2cf1ce09 @@ -0,0 +1 @@ +scheme://login:password@[::ffff:11.2.3.4]:3313 diff --git a/test/static/corpus/uri/048fbb93cfd1be0ccf151cf75147e391f12c2778 b/test/static/corpus/uri/048fbb93cfd1be0ccf151cf75147e391f12c2778 new file mode 100644 index 000000000..db0721d03 --- /dev/null +++ b/test/static/corpus/uri/048fbb93cfd1be0ccf151cf75147e391f12c2778 @@ -0,0 +1 @@ +login:password@host:service diff --git a/test/static/corpus/uri/07e1d498149de21318fe486a73b618b800f9e46c b/test/static/corpus/uri/07e1d498149de21318fe486a73b618b800f9e46c new file mode 100644 index 000000000..20cbcb7ca --- /dev/null +++ b/test/static/corpus/uri/07e1d498149de21318fe486a73b618b800f9e46c @@ -0,0 +1 @@ +login:password@unix/:/path1/path2/path3: diff --git a/test/static/corpus/uri/093873ef42d30bee0ec042892457333ce7d6cd65 b/test/static/corpus/uri/093873ef42d30bee0ec042892457333ce7d6cd65 new file mode 100644 index 000000000..dab3cf250 --- /dev/null +++ b/test/static/corpus/uri/093873ef42d30bee0ec042892457333ce7d6cd65 @@ -0,0 +1 @@ +unix/:/path1/path2/path3 diff --git a/test/static/corpus/uri/0a11f3691a10feefaeda9e30b4c891148d4ffdc1 b/test/static/corpus/uri/0a11f3691a10feefaeda9e30b4c891148d4ffdc1 new file mode 100644 index 000000000..c70dc2dfa --- /dev/null +++ b/test/static/corpus/uri/0a11f3691a10feefaeda9e30b4c891148d4ffdc1 @@ -0,0 +1 @@ +host diff --git a/test/static/corpus/uri/14394ce809fcbd4db56a1f1814fb322103148fc6 b/test/static/corpus/uri/14394ce809fcbd4db56a1f1814fb322103148fc6 new file mode 100644 index 000000000..30d5645c6 --- /dev/null +++ b/test/static/corpus/uri/14394ce809fcbd4db56a1f1814fb322103148fc6 @@ -0,0 +1 @@ +login:password@host/ diff --git a/test/static/corpus/uri/251e9a9e4e077720a71747a94e24a6e25cb314a9 b/test/static/corpus/uri/251e9a9e4e077720a71747a94e24a6e25cb314a9 new file mode 100644 index 000000000..d690dc0d5 --- /dev/null +++ b/test/static/corpus/uri/251e9a9e4e077720a71747a94e24a6e25cb314a9 @@ -0,0 +1 @@ +0.0.0.0 diff --git a/test/static/corpus/uri/2a855b2e35301abb51facebb20e78801b2219e0f b/test/static/corpus/uri/2a855b2e35301abb51facebb20e78801b2219e0f new file mode 100644 index 000000000..2c8a9e001 --- /dev/null +++ b/test/static/corpus/uri/2a855b2e35301abb51facebb20e78801b2219e0f @@ -0,0 +1 @@ +host:service diff --git a/test/static/corpus/uri/2ab8897ce59547952f047b48d52f9d6d4c0ea3eb b/test/static/corpus/uri/2ab8897ce59547952f047b48d52f9d6d4c0ea3eb new file mode 100644 index 000000000..810256634 --- /dev/null +++ b/test/static/corpus/uri/2ab8897ce59547952f047b48d52f9d6d4c0ea3eb @@ -0,0 +1 @@ +login:@host diff --git a/test/static/corpus/uri/39f3fca5b4634a9004220b389122abedfef28db7 b/test/static/corpus/uri/39f3fca5b4634a9004220b389122abedfef28db7 new file mode 100644 index 000000000..89c677974 --- /dev/null +++ b/test/static/corpus/uri/39f3fca5b4634a9004220b389122abedfef28db7 @@ -0,0 +1 @@ +login@host/path1/path2/path3 diff --git a/test/static/corpus/uri/3f18809f0df987b47c236d82d687ba16231daedb b/test/static/corpus/uri/3f18809f0df987b47c236d82d687ba16231daedb new file mode 100644 index 000000000..6b8fcdca0 --- /dev/null +++ b/test/static/corpus/uri/3f18809f0df987b47c236d82d687ba16231daedb @@ -0,0 +1 @@ +login:password@/path1/path2/path3 diff --git a/test/static/corpus/uri/4143d3a341877154d6e95211464e1df1015b74bd b/test/static/corpus/uri/4143d3a341877154d6e95211464e1df1015b74bd new file mode 100644 index 000000000..f599e28b8 --- /dev/null +++ b/test/static/corpus/uri/4143d3a341877154d6e95211464e1df1015b74bd @@ -0,0 +1 @@ +10 diff --git a/test/static/corpus/uri/49159cda7338d9b1a355631e7d33c55564fbfcb3 b/test/static/corpus/uri/49159cda7338d9b1a355631e7d33c55564fbfcb3 new file mode 100644 index 000000000..73f88e2c0 --- /dev/null +++ b/test/static/corpus/uri/49159cda7338d9b1a355631e7d33c55564fbfcb3 @@ -0,0 +1 @@ +login:password@host:service/path1/path2/path3 diff --git a/test/static/corpus/uri/4b40fa57bf6479d96f1a751c68578ee64e7b6759 b/test/static/corpus/uri/4b40fa57bf6479d96f1a751c68578ee64e7b6759 new file mode 100644 index 000000000..62f38851a --- /dev/null +++ b/test/static/corpus/uri/4b40fa57bf6479d96f1a751c68578ee64e7b6759 @@ -0,0 +1 @@ +login@host/ diff --git a/test/static/corpus/uri/4f1a4d9e9b5db51ca7fdf0e9b4adb52f1813918b b/test/static/corpus/uri/4f1a4d9e9b5db51ca7fdf0e9b4adb52f1813918b new file mode 100644 index 000000000..cd18a75d2 --- /dev/null +++ b/test/static/corpus/uri/4f1a4d9e9b5db51ca7fdf0e9b4adb52f1813918b @@ -0,0 +1 @@ +host:service/path1/path2/path3 diff --git a/test/static/corpus/uri/54b8c977fd54171c25da05e420b13d05dfaad747 b/test/static/corpus/uri/54b8c977fd54171c25da05e420b13d05dfaad747 new file mode 100644 index 000000000..d77bdec97 --- /dev/null +++ b/test/static/corpus/uri/54b8c977fd54171c25da05e420b13d05dfaad747 @@ -0,0 +1 @@ +login:password@host/path1/path2/path3?q1=v1&q2=v2#fragment diff --git a/test/static/corpus/uri/5bfff0efcc1e7f8e3550798038d6361392379dfa b/test/static/corpus/uri/5bfff0efcc1e7f8e3550798038d6361392379dfa new file mode 100644 index 000000000..97becdbe3 --- /dev/null +++ b/test/static/corpus/uri/5bfff0efcc1e7f8e3550798038d6361392379dfa @@ -0,0 +1 @@ +http://tarantool.org/dist/master/debian/pool/main/t/tarantool/tarantool_1.6.3+314+g91066ee+20140910+1434.orig.tar.gz diff --git a/test/static/corpus/uri/60f481551bdf721e8d1f2c805da6a7fb2d2e4858 b/test/static/corpus/uri/60f481551bdf721e8d1f2c805da6a7fb2d2e4858 new file mode 100644 index 000000000..1d7ba9912 --- /dev/null +++ b/test/static/corpus/uri/60f481551bdf721e8d1f2c805da6a7fb2d2e4858 @@ -0,0 +1 @@ +host/path diff --git a/test/static/corpus/uri/6c03daf8e568705738f646bfcdd03946acdce123 b/test/static/corpus/uri/6c03daf8e568705738f646bfcdd03946acdce123 new file mode 100644 index 000000000..e0095b1ad --- /dev/null +++ b/test/static/corpus/uri/6c03daf8e568705738f646bfcdd03946acdce123 @@ -0,0 +1 @@ +scheme://login:password@host:service/path1/path2/path3 diff --git a/test/static/corpus/uri/6dcaf13f83e40d73a9c5180977b5bf171b812e11 b/test/static/corpus/uri/6dcaf13f83e40d73a9c5180977b5bf171b812e11 new file mode 100644 index 000000000..a69e8cdb6 --- /dev/null +++ b/test/static/corpus/uri/6dcaf13f83e40d73a9c5180977b5bf171b812e11 @@ -0,0 +1 @@ +scheme://login:password@[2001::11a3:09d7::1]:3313 diff --git a/test/static/corpus/uri/7511cc73615187b17837d01a013e4d886331fa54 b/test/static/corpus/uri/7511cc73615187b17837d01a013e4d886331fa54 new file mode 100644 index 000000000..ee6cc4289 --- /dev/null +++ b/test/static/corpus/uri/7511cc73615187b17837d01a013e4d886331fa54 @@ -0,0 +1 @@ +unix/:/path1/path2/path3?q1=v1&q2=v2#fragment diff --git a/test/static/corpus/uri/767e882d43ffb1c822233640f7ca4e33426f7a3b b/test/static/corpus/uri/767e882d43ffb1c822233640f7ca4e33426f7a3b new file mode 100644 index 000000000..1b8e4dc37 --- /dev/null +++ b/test/static/corpus/uri/767e882d43ffb1c822233640f7ca4e33426f7a3b @@ -0,0 +1 @@ +host:service/path1/path2/path3?q1=v1&q2=v2#fragment diff --git a/test/static/corpus/uri/777b43cd6a3f75e17d9310584ac00d4bd3ee65a1 b/test/static/corpus/uri/777b43cd6a3f75e17d9310584ac00d4bd3ee65a1 new file mode 100644 index 000000000..4015742c9 --- /dev/null +++ b/test/static/corpus/uri/777b43cd6a3f75e17d9310584ac00d4bd3ee65a1 @@ -0,0 +1 @@ +host/path1/path2/path3 diff --git a/test/static/corpus/uri/77d8be75df172643a6fb1ee235d05b904aa501a0 b/test/static/corpus/uri/77d8be75df172643a6fb1ee235d05b904aa501a0 new file mode 100644 index 000000000..6ea7fb065 --- /dev/null +++ b/test/static/corpus/uri/77d8be75df172643a6fb1ee235d05b904aa501a0 @@ -0,0 +1 @@ +host/;abc?q diff --git a/test/static/corpus/uri/7b8fa7a85f9f6ae6f9178504d2202666fb8dc772 b/test/static/corpus/uri/7b8fa7a85f9f6ae6f9178504d2202666fb8dc772 new file mode 100644 index 000000000..7b9ad531d --- /dev/null +++ b/test/static/corpus/uri/7b8fa7a85f9f6ae6f9178504d2202666fb8dc772 @@ -0,0 +1 @@ +127.0.0.1 diff --git a/test/static/corpus/uri/7d850a53909012d59b379e99f1549332f01e3088 b/test/static/corpus/uri/7d850a53909012d59b379e99f1549332f01e3088 new file mode 100644 index 000000000..0c66ae1ca --- /dev/null +++ b/test/static/corpus/uri/7d850a53909012d59b379e99f1549332f01e3088 @@ -0,0 +1 @@ +login:password@host:service/ diff --git a/test/static/corpus/uri/84f62a3184758e8c27e63f3f27c05a50877dc389 b/test/static/corpus/uri/84f62a3184758e8c27e63f3f27c05a50877dc389 new file mode 100644 index 000000000..e651d30a5 --- /dev/null +++ b/test/static/corpus/uri/84f62a3184758e8c27e63f3f27c05a50877dc389 @@ -0,0 +1 @@ +host// diff --git a/test/static/corpus/uri/851be661349b0ceabf7cd933409f4d79ca9ac9a8 b/test/static/corpus/uri/851be661349b0ceabf7cd933409f4d79ca9ac9a8 new file mode 100644 index 000000000..691dd4c6b --- /dev/null +++ b/test/static/corpus/uri/851be661349b0ceabf7cd933409f4d79ca9ac9a8 @@ -0,0 +1 @@ +/path1/path2/path3 diff --git a/test/static/corpus/uri/8efd1c91dceac73be0780f4f3cddce62ba84ebf6 b/test/static/corpus/uri/8efd1c91dceac73be0780f4f3cddce62ba84ebf6 new file mode 100644 index 000000000..2bfbd4a7a --- /dev/null +++ b/test/static/corpus/uri/8efd1c91dceac73be0780f4f3cddce62ba84ebf6 @@ -0,0 +1 @@ +host/~user diff --git a/test/static/corpus/uri/8f3feb9a63afb6e100dad08136d8cbeea4a1dfed b/test/static/corpus/uri/8f3feb9a63afb6e100dad08136d8cbeea4a1dfed new file mode 100644 index 000000000..844b9ae9c --- /dev/null +++ b/test/static/corpus/uri/8f3feb9a63afb6e100dad08136d8cbeea4a1dfed @@ -0,0 +1 @@ +host/ diff --git a/test/static/corpus/uri/9100851463d47ef66626599f78730e4da01b8562 b/test/static/corpus/uri/9100851463d47ef66626599f78730e4da01b8562 new file mode 100644 index 000000000..be9f05ae2 --- /dev/null +++ b/test/static/corpus/uri/9100851463d47ef66626599f78730e4da01b8562 @@ -0,0 +1 @@ +login@host diff --git a/test/static/corpus/uri/9dfe5ccca3a5ed2e574fb1dd10ec6d98bf637484 b/test/static/corpus/uri/9dfe5ccca3a5ed2e574fb1dd10ec6d98bf637484 new file mode 100644 index 000000000..b46d0b6aa --- /dev/null +++ b/test/static/corpus/uri/9dfe5ccca3a5ed2e574fb1dd10ec6d98bf637484 @@ -0,0 +1 @@ +host:service/ diff --git a/test/static/corpus/uri/a5f6b7bdd3115f7427c55830f8fc4563256d410b b/test/static/corpus/uri/a5f6b7bdd3115f7427c55830f8fc4563256d410b new file mode 100644 index 000000000..e71248d7b --- /dev/null +++ b/test/static/corpus/uri/a5f6b7bdd3115f7427c55830f8fc4563256d410b @@ -0,0 +1 @@ +scheme://login:password@unix/:/tmp/unix.sock:/path1/path2/path3 diff --git a/test/static/corpus/uri/adc83b19e793491b1c6ea0fd8b46cd9f32e592fc b/test/static/corpus/uri/adc83b19e793491b1c6ea0fd8b46cd9f32e592fc new file mode 100644 index 000000000..e69de29bb diff --git a/test/static/corpus/uri/ae13a961739ce96b0cb75d1fae1bd6eddaad473d b/test/static/corpus/uri/ae13a961739ce96b0cb75d1fae1bd6eddaad473d new file mode 100644 index 000000000..5f5211a9d --- /dev/null +++ b/test/static/corpus/uri/ae13a961739ce96b0cb75d1fae1bd6eddaad473d @@ -0,0 +1 @@ +try.tarantool.org diff --git a/test/static/corpus/uri/b15df036caa676bffff10933845c16ce0192f726 b/test/static/corpus/uri/b15df036caa676bffff10933845c16ce0192f726 new file mode 100644 index 000000000..b4c7f2dc6 --- /dev/null +++ b/test/static/corpus/uri/b15df036caa676bffff10933845c16ce0192f726 @@ -0,0 +1 @@ +scheme://unix/:./relative/path.sock:/test diff --git a/test/static/corpus/uri/b379e6c191cce7491020b50c345debd7ee95f14c b/test/static/corpus/uri/b379e6c191cce7491020b50c345debd7ee95f14c new file mode 100644 index 000000000..cb9d26c41 --- /dev/null +++ b/test/static/corpus/uri/b379e6c191cce7491020b50c345debd7ee95f14c @@ -0,0 +1 @@ +[::ffff:11.2.3.4] diff --git a/test/static/corpus/uri/b7899610b3e9299cc4248ff090827e35962a7a81 b/test/static/corpus/uri/b7899610b3e9299cc4248ff090827e35962a7a81 new file mode 100644 index 000000000..288c9c1ab --- /dev/null +++ b/test/static/corpus/uri/b7899610b3e9299cc4248ff090827e35962a7a81 @@ -0,0 +1 @@ +scheme://login:password@host:service diff --git a/test/static/corpus/uri/bad754935cc524ecde1a46997056db302ea3baa3 b/test/static/corpus/uri/bad754935cc524ecde1a46997056db302ea3baa3 new file mode 100644 index 000000000..05af774dc --- /dev/null +++ b/test/static/corpus/uri/bad754935cc524ecde1a46997056db302ea3baa3 @@ -0,0 +1 @@ +login:password@host/path1/path2/path3 diff --git a/test/static/corpus/uri/bbf379045a2134133a43636c5e7807d2503f6261 b/test/static/corpus/uri/bbf379045a2134133a43636c5e7807d2503f6261 new file mode 100644 index 000000000..5ffa8689d --- /dev/null +++ b/test/static/corpus/uri/bbf379045a2134133a43636c5e7807d2503f6261 @@ -0,0 +1 @@ +host//path diff --git a/test/static/corpus/uri/bdaa097dc0aecd0840c0e6e46ed269ccec14f5fe b/test/static/corpus/uri/bdaa097dc0aecd0840c0e6e46ed269ccec14f5fe new file mode 100644 index 000000000..e6c1cc35e --- /dev/null +++ b/test/static/corpus/uri/bdaa097dc0aecd0840c0e6e46ed269ccec14f5fe @@ -0,0 +1 @@ +login:password@host diff --git a/test/static/corpus/uri/be403e2dc82146c9730731e8facb7a996fbfa01a b/test/static/corpus/uri/be403e2dc82146c9730731e8facb7a996fbfa01a new file mode 100644 index 000000000..43413cc3c --- /dev/null +++ b/test/static/corpus/uri/be403e2dc82146c9730731e8facb7a996fbfa01a @@ -0,0 +1 @@ +www.llanfairpwllgwyngyllgogerychwyrndrobwyll-llantysiliogogogoch.com diff --git a/test/static/corpus/uri/c358d48b194741a6ed970a730c64c18c26076778 b/test/static/corpus/uri/c358d48b194741a6ed970a730c64c18c26076778 new file mode 100644 index 000000000..44b9ef7aa --- /dev/null +++ b/test/static/corpus/uri/c358d48b194741a6ed970a730c64c18c26076778 @@ -0,0 +1 @@ +host/path1/path2/path3?q1=v1&q2=v2#fragment diff --git a/test/static/corpus/uri/c4dff26491e4d229af0efcae747a2b78fddd7b75 b/test/static/corpus/uri/c4dff26491e4d229af0efcae747a2b78fddd7b75 new file mode 100644 index 000000000..13c8c1a58 --- /dev/null +++ b/test/static/corpus/uri/c4dff26491e4d229af0efcae747a2b78fddd7b75 @@ -0,0 +1 @@ +https://www.google.com/search?safe=off&site=&tbm=isch&source=hp&biw=1918&bih=1109&q=Tarantool&oq=Tarantool&gs_l=img.3..0i24l3j0i10i24j0i24&gws_rd=ssl diff --git a/test/static/corpus/uri/c76f1c7d662c14d891f04058128ce7cd83bc09bd b/test/static/corpus/uri/c76f1c7d662c14d891f04058128ce7cd83bc09bd new file mode 100644 index 000000000..8da330f1f --- /dev/null +++ b/test/static/corpus/uri/c76f1c7d662c14d891f04058128ce7cd83bc09bd @@ -0,0 +1 @@ +login@host/path1/path2/path3?q1=v1&q2=v2#fragment diff --git a/test/static/corpus/uri/dabc3a9d30f94608b2a68a5beb470c704890e3f1 b/test/static/corpus/uri/dabc3a9d30f94608b2a68a5beb470c704890e3f1 new file mode 100644 index 000000000..093335e36 --- /dev/null +++ b/test/static/corpus/uri/dabc3a9d30f94608b2a68a5beb470c704890e3f1 @@ -0,0 +1 @@ +unix/:./relative/path.sock:/test diff --git a/test/static/corpus/uri/e0c30dfd58621aa24062fea9cf21d976e108a2f4 b/test/static/corpus/uri/e0c30dfd58621aa24062fea9cf21d976e108a2f4 new file mode 100644 index 000000000..9e4a5e863 --- /dev/null +++ b/test/static/corpus/uri/e0c30dfd58621aa24062fea9cf21d976e108a2f4 @@ -0,0 +1 @@ +scheme://[2001:0db8:11a3:09d7::1] diff --git a/test/static/corpus/uri/e400db1ce2202210e9542f9f030c6404bf5718c8 b/test/static/corpus/uri/e400db1ce2202210e9542f9f030c6404bf5718c8 new file mode 100644 index 000000000..fcbabce59 --- /dev/null +++ b/test/static/corpus/uri/e400db1ce2202210e9542f9f030c6404bf5718c8 @@ -0,0 +1 @@ +127.0.0.1:3313 diff --git a/test/static/corpus/uri/e5fa44f2b31c1fb553b6021e7360d07d5d91ff5e b/test/static/corpus/uri/e5fa44f2b31c1fb553b6021e7360d07d5d91ff5e new file mode 100644 index 000000000..d00491fd7 --- /dev/null +++ b/test/static/corpus/uri/e5fa44f2b31c1fb553b6021e7360d07d5d91ff5e @@ -0,0 +1 @@ +1 diff --git a/test/static/corpus/uri/eca60ab9f2f130ab10bf4a642d00690442dd2b74 b/test/static/corpus/uri/eca60ab9f2f130ab10bf4a642d00690442dd2b74 new file mode 100644 index 000000000..11c827418 --- /dev/null +++ b/test/static/corpus/uri/eca60ab9f2f130ab10bf4a642d00690442dd2b74 @@ -0,0 +1 @@ +scheme://login:password@127.0.0.1:3313 diff --git a/test/static/corpus/uri/f25a33be8af0a7f725531c61f228d2339dfd089e b/test/static/corpus/uri/f25a33be8af0a7f725531c61f228d2339dfd089e new file mode 100644 index 000000000..24460d4e5 --- /dev/null +++ b/test/static/corpus/uri/f25a33be8af0a7f725531c61f228d2339dfd089e @@ -0,0 +1 @@ +scheme://login:password@host:service/@path1/:path2?q1=v1&q2=v2#fragment -- 2.25.1 ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 2/4] test: add corpus to be used with fuzzers 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 2/4] test: add corpus to be used with fuzzers sergeyb @ 2020-12-07 17:34 ` Igor Munkin 2020-12-13 18:56 ` Sergey Bronnikov 0 siblings, 1 reply; 26+ messages in thread From: Igor Munkin @ 2020-12-07 17:34 UTC (permalink / raw) To: sergeyb; +Cc: tarantool-patches Sergey, Thanks for the patch! Please consider my comments below. On 30.11.20, sergeyb@tarantool.org wrote: > From: Sergey Bronnikov <sergeyb@tarantool.org> > > Fuzzing tools uses evolutionary algorithms. Supplying seed corpus consisting > of good sample inputs is one of the best ways to improve fuzz target’s > coverage. Patch adds a corpuses that can be used with existed fuzzers. > The name of each file in the corpus is the sha1 checksum of its contents. Minor: The body of the commit message should be wrapped up to 72 characters (except the links I believe) according to our guidelines[1]. > > Corpus with http headers was added from [1] and [2]. > > 1. https://google.github.io/oss-fuzz/getting-started/new-project-guide/#seed-corpus > 2. https://en.wikipedia.org/wiki/List_of_HTTP_header_fields > 3. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers > > The libFuzzer allow to minimize corpus with help of `-merge` flag: > when 1 is passed, any corpus inputs from the 2nd, 3rd etc. corpus directories > that trigger new code coverage will be merged into the first corpus directory, > when 0 is passed an existed corpus will be minimized. Ditto. > > All provided corpuses in a patch were minimized. > > Part of #1809 I have no idea, how to review this patch further. It looks OK except the testing fails the way I wrote in the previous reply. So, I stop here. > --- <snipped> > -- > 2.25.1 > [1]: https://www.tarantool.io/en/doc/latest/dev_guide/developer_guidelines/ -- Best regards, IM ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 2/4] test: add corpus to be used with fuzzers 2020-12-07 17:34 ` Igor Munkin @ 2020-12-13 18:56 ` Sergey Bronnikov 0 siblings, 0 replies; 26+ messages in thread From: Sergey Bronnikov @ 2020-12-13 18:56 UTC (permalink / raw) To: Igor Munkin; +Cc: tarantool-patches Hello! thanks for review! On 07.12.2020 20:34, Igor Munkin wrote: > Sergey, > > Thanks for the patch! Please consider my comments below. > > On 30.11.20, sergeyb@tarantool.org wrote: >> From: Sergey Bronnikov <sergeyb@tarantool.org> >> >> Fuzzing tools uses evolutionary algorithms. Supplying seed corpus consisting >> of good sample inputs is one of the best ways to improve fuzz target’s >> coverage. Patch adds a corpuses that can be used with existed fuzzers. >> The name of each file in the corpus is the sha1 checksum of its contents. > Minor: The body of the commit message should be wrapped up to 72 > characters (except the links I believe) according to our guidelines[1]. Fixed in a branch. > >> Corpus with http headers was added from [1] and [2]. >> >> 1. https://google.github.io/oss-fuzz/getting-started/new-project-guide/#seed-corpus >> 2. https://en.wikipedia.org/wiki/List_of_HTTP_header_fields >> 3. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers >> >> The libFuzzer allow to minimize corpus with help of `-merge` flag: >> when 1 is passed, any corpus inputs from the 2nd, 3rd etc. corpus directories >> that trigger new code coverage will be merged into the first corpus directory, >> when 0 is passed an existed corpus will be minimized. > Ditto. Fixed in a branch. >> All provided corpuses in a patch were minimized. >> >> Part of #1809 > I have no idea, how to review this patch further. It looks OK except the > testing fails the way I wrote in the previous reply. So, I stop here. > >> --- > <snipped> > >> -- >> 2.25.1 >> > [1]: https://www.tarantool.io/en/doc/latest/dev_guide/developer_guidelines/ > ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Tarantool-patches] [PATCH 3/4] travis: build tarantool with ENABLE_FUZZER 2020-11-30 20:24 [Tarantool-patches] [PATCH 0/4] Add fuzzing testing sergeyb 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 1/4] test: add infrastructure for fuzzing testing and fuzzers sergeyb 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 2/4] test: add corpus to be used with fuzzers sergeyb @ 2020-11-30 20:24 ` sergeyb 2020-12-07 17:38 ` Igor Munkin 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 4/4] test: integrate with OSS Fuzz sergeyb ` (4 subsequent siblings) 7 siblings, 1 reply; 26+ messages in thread From: sergeyb @ 2020-11-30 20:24 UTC (permalink / raw) To: tarantool-patches, imun, sergepetrenko From: Sergey Bronnikov <sergeyb@tarantool.org> OSS Fuzz has a limited number of runs per day and now it is a 4 runs. Option ENABLE_FUZZERS is enabled to make sure that building of fuzzers is not broken. Part of #1809 --- .travis.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.travis.mk b/.travis.mk index 238e31cda..6d843cae9 100644 --- a/.travis.mk +++ b/.travis.mk @@ -154,7 +154,7 @@ coverage_debian: deps_debian test_coverage_debian_no_deps build_asan_debian: CC=clang-11 CXX=clang++-11 cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo \ -DENABLE_WERROR=ON -DENABLE_ASAN=ON -DENABLE_UB_SANITIZER=ON \ - ${CMAKE_EXTRA_PARAMS} + -DENABLE_FUZZER=ON ${CMAKE_EXTRA_PARAMS} make -j test_asan_debian_no_deps: build_asan_debian -- 2.25.1 ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 3/4] travis: build tarantool with ENABLE_FUZZER 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 3/4] travis: build tarantool with ENABLE_FUZZER sergeyb @ 2020-12-07 17:38 ` Igor Munkin 0 siblings, 0 replies; 26+ messages in thread From: Igor Munkin @ 2020-12-07 17:38 UTC (permalink / raw) To: sergeyb; +Cc: tarantool-patches Sergey, Thanks for the patch! There is a single nit below. On 30.11.20, sergeyb@tarantool.org wrote: > From: Sergey Bronnikov <sergeyb@tarantool.org> > > OSS Fuzz has a limited number of runs per day and now it is a 4 runs. > Option ENABLE_FUZZERS is enabled to make sure that building of fuzzers is not > broken. Minor: The body of the commit message should be wrapped up to 72 characters (except the links I believe) according to our guidelines[1]. > > Part of #1809 > --- > .travis.mk | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > <snipped> > -- > 2.25.1 > [1]: https://www.tarantool.io/en/doc/latest/dev_guide/developer_guidelines/ -- Best regards, IM ^ permalink raw reply [flat|nested] 26+ messages in thread
* [Tarantool-patches] [PATCH 4/4] test: integrate with OSS Fuzz 2020-11-30 20:24 [Tarantool-patches] [PATCH 0/4] Add fuzzing testing sergeyb ` (2 preceding siblings ...) 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 3/4] travis: build tarantool with ENABLE_FUZZER sergeyb @ 2020-11-30 20:24 ` sergeyb 2020-12-07 17:42 ` Igor Munkin 2020-12-01 10:54 ` [Tarantool-patches] [PATCH 0/4] Add fuzzing testing Serge Petrenko ` (3 subsequent siblings) 7 siblings, 1 reply; 26+ messages in thread From: sergeyb @ 2020-11-30 20:24 UTC (permalink / raw) To: tarantool-patches, imun, sergepetrenko From: Sergey Bronnikov <sergeyb@tarantool.org> To run Tarantool fuzzers on OSS Fuzz infrastructure it is needed to pass library $LIB_FUZZING_ENGINE to linker and use external CFLAGS and CXXFLAGS. Full description how to integrate with OSS Fuzz is in [1] and [2]. Patch to OSS Fuzz repository [2] is ready to merge. 1. https://google.github.io/oss-fuzz/getting-started/new-project-guide/ 2. https://google.github.io/oss-fuzz/advanced-topics/ideal-integration/ 3. https://github.com/google/oss-fuzz/pull/4723 Closes #1809 --- test/fuzz/CMakeLists.txt | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/test/fuzz/CMakeLists.txt b/test/fuzz/CMakeLists.txt index 142d38f67..3abedbf08 100644 --- a/test/fuzz/CMakeLists.txt +++ b/test/fuzz/CMakeLists.txt @@ -14,6 +14,13 @@ target_compile_options( $<$<BOOL:${ENABLE_UB_SANITIZER}>: -fsanitize=fuzzer,undefined > + $<$<NOT:$<BOOL:${OSS_FUZZ}>>: + -fsanitize=fuzzer + > + $<$<BOOL:${OSS_FUZZ}>: + ${CXX} + ${CXXFLAGS} + > ) target_link_libraries( fuzzer_config @@ -24,6 +31,12 @@ target_link_libraries( $<$<BOOL:${ENABLE_UB_SANITIZER}>: -fsanitize=fuzzer,undefined > + $<$<NOT:$<BOOL:${OSS_FUZZ}>>: + -fsanitize=fuzzer + > + $<$<BOOL:${OSS_FUZZ}>: + $ENV{LIB_FUZZING_ENGINE} + > ) # Use PUBLIC to force 'fuzzer_config' for all dependent targets. -- 2.25.1 ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 4/4] test: integrate with OSS Fuzz 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 4/4] test: integrate with OSS Fuzz sergeyb @ 2020-12-07 17:42 ` Igor Munkin 0 siblings, 0 replies; 26+ messages in thread From: Igor Munkin @ 2020-12-07 17:42 UTC (permalink / raw) To: sergeyb; +Cc: tarantool-patches Sergey, Thanks for the patch! Please consider the singe nit below. On 30.11.20, sergeyb@tarantool.org wrote: > From: Sergey Bronnikov <sergeyb@tarantool.org> > > To run Tarantool fuzzers on OSS Fuzz infrastructure it is needed to pass > library $LIB_FUZZING_ENGINE to linker and use external CFLAGS and CXXFLAGS. > Full description how to integrate with OSS Fuzz is in [1] and [2]. Minor: The body of the commit message should be wrapped up to 72 characters (except the links I believe) according to our guidelines[1]. > > Patch to OSS Fuzz repository [2] is ready to merge. > > 1. https://google.github.io/oss-fuzz/getting-started/new-project-guide/ > 2. https://google.github.io/oss-fuzz/advanced-topics/ideal-integration/ > 3. https://github.com/google/oss-fuzz/pull/4723 > > Closes #1809 > --- > test/fuzz/CMakeLists.txt | 13 +++++++++++++ > 1 file changed, 13 insertions(+) > <snipped> > -- > 2.25.1 -- Best regards, IM ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 0/4] Add fuzzing testing 2020-11-30 20:24 [Tarantool-patches] [PATCH 0/4] Add fuzzing testing sergeyb ` (3 preceding siblings ...) 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 4/4] test: integrate with OSS Fuzz sergeyb @ 2020-12-01 10:54 ` Serge Petrenko 2020-12-01 14:41 ` Sergey Bronnikov 2020-12-07 17:49 ` Igor Munkin ` (2 subsequent siblings) 7 siblings, 1 reply; 26+ messages in thread From: Serge Petrenko @ 2020-12-01 10:54 UTC (permalink / raw) To: sergeyb, tarantool-patches, imun 30.11.2020 23:24, sergeyb@tarantool.org пишет: > From: Sergey Bronnikov <sergeyb@tarantool.org> > > Patch series adds required support in project infrastructure for fuzzing tests, > add tests for uri, csv and http_parser modules, adds integration with OSS Fuzz > (infrastructure for continuous running fuzzing tests). > > NOTE: v2 version had LGTM from Sergey Petrenko, see [1] > > Changelog: > > v2: > - introduce CMake flag to enable building fuzzers > - add fuzzers based on LibFuzzer to csv, http_parser and uri modules > > v3: > - report error on an attempt to set ENABLE_FUZZER=ON with GCC compiler > - fixed return code in tests > - place tests to a common dir test/fuzz/ like we did with unit tests > - enable building fuzzers in Tarantool CI job > - add target to build all availabe fuzzers > - add integration with OSS Fuzz [2] > - add corpus for uri, csv and http_parser tests > > 1. https://lists.tarantool.org/pipermail/tarantool-patches/2020-April/016471.html > 2. https://github.com/google/oss-fuzz/pull/4723 > > CI: https://gitlab.com/tarantool/tarantool/-/pipelines/223112908 > Issue: https://github.com/tarantool/tarantool/issues/1809 Hi! Thanks for the patchset! Generally LGTM with one question. > > Sergey Bronnikov (4): > test: add infrastructure for fuzzing testing and fuzzers > test: add corpus to be used with fuzzers > travis: build tarantool with ENABLE_FUZZER Why travis? Don't we use gitlab for all the QA jobs? Is travis.mk shared between gitlab and travis? > test: integrate with OSS Fuzz > -- Serge Petrenko ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 0/4] Add fuzzing testing 2020-12-01 10:54 ` [Tarantool-patches] [PATCH 0/4] Add fuzzing testing Serge Petrenko @ 2020-12-01 14:41 ` Sergey Bronnikov 2020-12-01 14:45 ` Serge Petrenko 0 siblings, 1 reply; 26+ messages in thread From: Sergey Bronnikov @ 2020-12-01 14:41 UTC (permalink / raw) To: Serge Petrenko, tarantool-patches, imun Hi! Thanks for review! On 01.12.2020 13:54, Serge Petrenko wrote: <snipped> > 30.11.2020 23:24, sergeyb@tarantool.org пишет: >> >> Sergey Bronnikov (4): >> test: add infrastructure for fuzzing testing and fuzzers >> test: add corpus to be used with fuzzers >> travis: build tarantool with ENABLE_FUZZER > > > Why travis? Don't we use gitlab for all the QA jobs? > Is travis.mk shared between gitlab and travis? It's actually not a Travis :) Historically most of our CI jobs are in .travis.mk but they run on Gitlab CI. > > >> test: integrate with OSS Fuzz >> ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 0/4] Add fuzzing testing 2020-12-01 14:41 ` Sergey Bronnikov @ 2020-12-01 14:45 ` Serge Petrenko 0 siblings, 0 replies; 26+ messages in thread From: Serge Petrenko @ 2020-12-01 14:45 UTC (permalink / raw) To: Sergey Bronnikov, tarantool-patches, imun 01.12.2020 17:41, Sergey Bronnikov пишет: > Hi! Thanks for review! > > On 01.12.2020 13:54, Serge Petrenko wrote: > > <snipped> > >> 30.11.2020 23:24, sergeyb@tarantool.org пишет: >>> >>> Sergey Bronnikov (4): >>> test: add infrastructure for fuzzing testing and fuzzers >>> test: add corpus to be used with fuzzers >>> travis: build tarantool with ENABLE_FUZZER >> >> >> Why travis? Don't we use gitlab for all the QA jobs? >> Is travis.mk shared between gitlab and travis? > > It's actually not a Travis :) Historically most of our CI jobs are in > .travis.mk > > but they run on Gitlab CI. I see, thanks for the explanation. LGTM. > >> >> >>> test: integrate with OSS Fuzz >>> -- Serge Petrenko ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 0/4] Add fuzzing testing 2020-11-30 20:24 [Tarantool-patches] [PATCH 0/4] Add fuzzing testing sergeyb ` (4 preceding siblings ...) 2020-12-01 10:54 ` [Tarantool-patches] [PATCH 0/4] Add fuzzing testing Serge Petrenko @ 2020-12-07 17:49 ` Igor Munkin 2020-12-25 13:08 ` Igor Munkin 2020-12-25 14:52 ` Kirill Yukhin 7 siblings, 0 replies; 26+ messages in thread From: Igor Munkin @ 2020-12-07 17:49 UTC (permalink / raw) To: sergeyb; +Cc: tarantool-patches Sergey, Thanks for the series! I'm totally not an expert in all this machinery but it looks OK except the major remarks in the first patch and a couple of nits in the further ones. Unfortunately, I can't check everything manually and mentioned it in the first review. I suspect there is something broken in my toolchain, but still want to figure out the root cause. On 30.11.20, sergeyb@tarantool.org wrote: > From: Sergey Bronnikov <sergeyb@tarantool.org> > > Patch series adds required support in project infrastructure for fuzzing tests, > add tests for uri, csv and http_parser modules, adds integration with OSS Fuzz > (infrastructure for continuous running fuzzing tests). > > NOTE: v2 version had LGTM from Sergey Petrenko, see [1] > > Changelog: Side note: please mind in future patchsets that changelog should be sorted in a reverse order. > > v2: > - introduce CMake flag to enable building fuzzers > - add fuzzers based on LibFuzzer to csv, http_parser and uri modules > > v3: > - report error on an attempt to set ENABLE_FUZZER=ON with GCC compiler > - fixed return code in tests > - place tests to a common dir test/fuzz/ like we did with unit tests > - enable building fuzzers in Tarantool CI job > - add target to build all availabe fuzzers > - add integration with OSS Fuzz [2] > - add corpus for uri, csv and http_parser tests > > 1. https://lists.tarantool.org/pipermail/tarantool-patches/2020-April/016471.html > 2. https://github.com/google/oss-fuzz/pull/4723 Neat. Looking forward to this merge. > > CI: https://gitlab.com/tarantool/tarantool/-/pipelines/223112908 > Issue: https://github.com/tarantool/tarantool/issues/1809 > > Sergey Bronnikov (4): > test: add infrastructure for fuzzing testing and fuzzers > test: add corpus to be used with fuzzers > travis: build tarantool with ENABLE_FUZZER > test: integrate with OSS Fuzz > > .travis.mk | 2 +- > CMakeLists.txt | 2 +- > cmake/profile.cmake | 13 +++++ > test/CMakeLists.txt | 3 + > test/fuzz/CMakeLists.txt | 58 +++++++++++++++++++ > test/fuzz/csv_fuzzer.c | 23 ++++++++ > test/fuzz/http_parser_fuzzer.c | 18 ++++++ > test/fuzz/uri_fuzzer.c | 19 ++++++ > .../0c90a17c7b7cb5163255c7b775a8b544fb62c59d | 3 + > .../0e76fc3bb39bac4f774e2ce3978e9b59bc4a04b7 | 2 + > .../11f7ab7d8c2d4da313081e3282ab8120f0330b90 | 5 ++ > .../2ace62c1befa19e3ea37dd52be9f6d508c5163e6 | 1 + > .../3816dd9c9120fe6b3d806184874cc5c3e3a68df3 | 1 + > .../409c042df7f6a518972df1bc1287442e28719b6c | 1 + > .../46b175b37b5c3f746fde0dd114ead9938c012cfc | 3 + > .../56754bc8c7346ef9fad5c2ae9c54cef0af72629c | 2 + > .../58f43ea1e6df023f3dd4be9f718e5d0382fb9697 | 3 + > .../6d131d28c6e20c3a0a0b46c3aa7308d3029ab636 | 1 + > .../7560cb86c281018ee2f80d840fd8e6aa73a609ef | 5 ++ > .../7b97db755e18dc519fbc49e5de19b5820b9441c1 | 1 + > .../7ba0c88846fbcd28ad490df47f9ce4405d795303 | 2 + > .../8a14c110827e26cfdde3693bc5db6cd8673cdc8a | 1 + > .../94c1799dfba59120dd6075ee60016b809a915ca8 | 1 + > .../9e109f38c3948abe743bd116a34f05954f9ddd59 | 1 + > .../d21ec5eaaab3c8db1f923674b7a3ee19ce35ed13 | 1 + > .../d4a6d494cc2a06a0fee4873014210ec523d23750 | 1 + > .../eca825c9bdb276332d524134ddf370f7d18c2444 | 1 + > .../f9ba3e378e0409d66f946b31306de19edfe859bc | 1 + > .../fcad5471c295084dc1ec1eaa3753420e9803d251 | 10 ++++ > .../038ec38a9ae7669590f1e3401da8ffbc9b40c8ee | 1 + > .../043fdf868410e66239819e0af10cbe787cfab77e | 1 + > .../08abe5308ce774ad5e9e435de386748ce06521f8 | 1 + > .../10b4f603cf09e21020e494e20a7f6760d5634c88 | 1 + > .../172ed6e56aa94bada4135d64fc2e0c5eac779aed | 1 + > .../1e42978bf6935a0f8377f6c480c315f1ae4627b8 | 1 + > .../1f6f26c085c3b209e626f90c63d0fb10526de318 | 1 + > .../24b458efa865a138a314f0876a11ec680b983ed5 | 1 + > .../2be565c790bc352f2cbaa0f3509e0bb6133b3616 | 1 + > .../30fd5e18f586b219916fb3892961a00e92db7a7f | 1 + > .../38918740212c6e5aea01f2fc8772b1d4aa89be8b | 1 + > .../39d4c74a90b7f5f5426b7982857b5fd7edbd8f77 | 1 + > .../3d21092e1a772d792e9dd720eeae30d8eb5ac2ad | 1 + > .../454dec613ca5c71fee0467061a88d2807201ac36 | 1 + > .../4f13053e61966dd08d7cd2407865481ec1d2cbf9 | 1 + > .../4ffa3daceb5b6ee3fc17cef3e0ae694ea5c583b5 | 1 + > .../501b3a19def06b352978c03ece7ab282cdc6cbab | 1 + > .../50ad023e1465919f5afbfffb33504ddfdc6a7e69 | 1 + > .../5177d236e3eea297b76c19194a4cf127b313b267 | 1 + > .../51e1354d16cfa4967e91206be8bd0d8c6ca577af | 1 + > .../55d5f3acc62809fc87466b2048b157e50e63d644 | 1 + > .../598d19d3175046ff035be228db2f1dc43957d8e2 | 1 + > .../5b7444e43e5c1fd759483c76a46a5536da043022 | 1 + > .../5c489184823a8a42844066fd5b4951bbad86934f | 1 + > .../6455c0e761cd485f3e38c470f216560bfc46eea4 | 1 + > .../6596bb363cfd0bef3f68e265cfd824c6d37ecb7e | 1 + > .../66d5206b6c75b071f57684399218d144b0d337ea | 1 + > .../685073a623772c1d46475b5eb451de40247d697f | 1 + > .../71d4de855c1af66a6a5cea88b1eddc5032ecb26d | 1 + > .../73378b09c61739b65a5f63214f0515307f187e53 | 1 + > .../768abf119ce4fa6c9a9ee9f17d9dacb32c61759c | 1 + > .../7ae1560be3dffbb88f3e48de3a4be99ecdc61441 | 1 + > .../7c4d33785daa5c2370201ffa236b427aa37c9996 | 1 + > .../80c5002c297e86198535d1574f3e47f4c6900782 | 1 + > .../86908cf9f9f2c690ebd7462cacd2b28fc45db3a5 | 1 + > .../89681f4bac464df0f73f3bf3fd48db40bc5796a6 | 1 + > .../8c371f137d8a10fc568a3022edf6f10280c96bc0 | 1 + > .../8ff59012e5ea417bf340e59a0ab6fd9e410011ea | 1 + > .../927ba76d456f6c3a06769795f87c46462e16acd5 | 1 + > .../95a0311cd792a22e0de733080e0580638c8eb136 | 1 + > .../9a3fc50025efc8aac22e16f10621f43a0f35ddcb | 1 + > .../ad1bde0f01973fc121af6f4a1efef3a1447c3a80 | 1 + > .../ad50e55369c0e4459500323e0af8eb00172f0909 | 1 + > .../b2af1811e30009a2b7cec55e86f26b8f4bf3c1cf | 1 + > .../bcebbda5c1d2033a62c14d6bf7e2dc54b407b403 | 1 + > .../cfe75970181a6e2f5ddbf9cf968a9a1fba1a660b | 1 + > .../d01798aa68e31f2964a2bc2c10b203f5186a3ba1 | 1 + > .../d2dc25ea74a20f35c78091cbc1e30cf2557c315f | 1 + > .../d930b69281035ce3b46f2635f5a91ee11f15876c | 1 + > .../dcb92ca0399b135c88170124fcdab6b6d62d115c | 1 + > .../e4e52eaca9ece89c0e3fa2ef7e6e126d47f20101 | 1 + > .../ebe97f523d44667ea6e11e1c8cd9e5f02cf244e7 | 1 + > .../ede1894116fdc6430ba23942e12d974e3f22875b | 1 + > .../f0f8a948443b28c5e344e6e43aedbb479059779c | 1 + > .../f1c4ffb8ea524f69bc8d6784f86b70ac99db2ad7 | 1 + > .../f774dd0b8fc355505aab48a2107804387f45c4a0 | 1 + > .../016d581dcda85465d5fa22eadd5947395b61303f | 1 + > .../018287fe57cf372d929b52a560910284ba406de6 | 1 + > .../048b75ce15485f8daa16e516014df26f2cf1ce09 | 1 + > .../048fbb93cfd1be0ccf151cf75147e391f12c2778 | 1 + > .../07e1d498149de21318fe486a73b618b800f9e46c | 1 + > .../093873ef42d30bee0ec042892457333ce7d6cd65 | 1 + > .../0a11f3691a10feefaeda9e30b4c891148d4ffdc1 | 1 + > .../14394ce809fcbd4db56a1f1814fb322103148fc6 | 1 + > .../251e9a9e4e077720a71747a94e24a6e25cb314a9 | 1 + > .../2a855b2e35301abb51facebb20e78801b2219e0f | 1 + > .../2ab8897ce59547952f047b48d52f9d6d4c0ea3eb | 1 + > .../39f3fca5b4634a9004220b389122abedfef28db7 | 1 + > .../3f18809f0df987b47c236d82d687ba16231daedb | 1 + > .../4143d3a341877154d6e95211464e1df1015b74bd | 1 + > .../49159cda7338d9b1a355631e7d33c55564fbfcb3 | 1 + > .../4b40fa57bf6479d96f1a751c68578ee64e7b6759 | 1 + > .../4f1a4d9e9b5db51ca7fdf0e9b4adb52f1813918b | 1 + > .../54b8c977fd54171c25da05e420b13d05dfaad747 | 1 + > .../5bfff0efcc1e7f8e3550798038d6361392379dfa | 1 + > .../60f481551bdf721e8d1f2c805da6a7fb2d2e4858 | 1 + > .../6c03daf8e568705738f646bfcdd03946acdce123 | 1 + > .../6dcaf13f83e40d73a9c5180977b5bf171b812e11 | 1 + > .../7511cc73615187b17837d01a013e4d886331fa54 | 1 + > .../767e882d43ffb1c822233640f7ca4e33426f7a3b | 1 + > .../777b43cd6a3f75e17d9310584ac00d4bd3ee65a1 | 1 + > .../77d8be75df172643a6fb1ee235d05b904aa501a0 | 1 + > .../7b8fa7a85f9f6ae6f9178504d2202666fb8dc772 | 1 + > .../7d850a53909012d59b379e99f1549332f01e3088 | 1 + > .../84f62a3184758e8c27e63f3f27c05a50877dc389 | 1 + > .../851be661349b0ceabf7cd933409f4d79ca9ac9a8 | 1 + > .../8efd1c91dceac73be0780f4f3cddce62ba84ebf6 | 1 + > .../8f3feb9a63afb6e100dad08136d8cbeea4a1dfed | 1 + > .../9100851463d47ef66626599f78730e4da01b8562 | 1 + > .../9dfe5ccca3a5ed2e574fb1dd10ec6d98bf637484 | 1 + > .../a5f6b7bdd3115f7427c55830f8fc4563256d410b | 1 + > .../adc83b19e793491b1c6ea0fd8b46cd9f32e592fc | 0 > .../ae13a961739ce96b0cb75d1fae1bd6eddaad473d | 1 + > .../b15df036caa676bffff10933845c16ce0192f726 | 1 + > .../b379e6c191cce7491020b50c345debd7ee95f14c | 1 + > .../b7899610b3e9299cc4248ff090827e35962a7a81 | 1 + > .../bad754935cc524ecde1a46997056db302ea3baa3 | 1 + > .../bbf379045a2134133a43636c5e7807d2503f6261 | 1 + > .../bdaa097dc0aecd0840c0e6e46ed269ccec14f5fe | 1 + > .../be403e2dc82146c9730731e8facb7a996fbfa01a | 1 + > .../c358d48b194741a6ed970a730c64c18c26076778 | 1 + > .../c4dff26491e4d229af0efcae747a2b78fddd7b75 | 1 + > .../c76f1c7d662c14d891f04058128ce7cd83bc09bd | 1 + > .../dabc3a9d30f94608b2a68a5beb470c704890e3f1 | 1 + > .../e0c30dfd58621aa24062fea9cf21d976e108a2f4 | 1 + > .../e400db1ce2202210e9542f9f030c6404bf5718c8 | 1 + > .../e5fa44f2b31c1fb553b6021e7360d07d5d91ff5e | 1 + > .../eca60ab9f2f130ab10bf4a642d00690442dd2b74 | 1 + > .../f25a33be8af0a7f725531c61f228d2339dfd089e | 1 + > 138 files changed, 291 insertions(+), 2 deletions(-) > create mode 100644 test/fuzz/CMakeLists.txt > create mode 100644 test/fuzz/csv_fuzzer.c > create mode 100644 test/fuzz/http_parser_fuzzer.c > create mode 100644 test/fuzz/uri_fuzzer.c > create mode 100644 test/static/corpus/csv/0c90a17c7b7cb5163255c7b775a8b544fb62c59d > create mode 100644 test/static/corpus/csv/0e76fc3bb39bac4f774e2ce3978e9b59bc4a04b7 > create mode 100644 test/static/corpus/csv/11f7ab7d8c2d4da313081e3282ab8120f0330b90 > create mode 100644 test/static/corpus/csv/2ace62c1befa19e3ea37dd52be9f6d508c5163e6 > create mode 100644 test/static/corpus/csv/3816dd9c9120fe6b3d806184874cc5c3e3a68df3 > create mode 100644 test/static/corpus/csv/409c042df7f6a518972df1bc1287442e28719b6c > create mode 100644 test/static/corpus/csv/46b175b37b5c3f746fde0dd114ead9938c012cfc > create mode 100644 test/static/corpus/csv/56754bc8c7346ef9fad5c2ae9c54cef0af72629c > create mode 100644 test/static/corpus/csv/58f43ea1e6df023f3dd4be9f718e5d0382fb9697 > create mode 100644 test/static/corpus/csv/6d131d28c6e20c3a0a0b46c3aa7308d3029ab636 > create mode 100644 test/static/corpus/csv/7560cb86c281018ee2f80d840fd8e6aa73a609ef > create mode 100644 test/static/corpus/csv/7b97db755e18dc519fbc49e5de19b5820b9441c1 > create mode 100644 test/static/corpus/csv/7ba0c88846fbcd28ad490df47f9ce4405d795303 > create mode 100644 test/static/corpus/csv/8a14c110827e26cfdde3693bc5db6cd8673cdc8a > create mode 100644 test/static/corpus/csv/94c1799dfba59120dd6075ee60016b809a915ca8 > create mode 100644 test/static/corpus/csv/9e109f38c3948abe743bd116a34f05954f9ddd59 > create mode 100644 test/static/corpus/csv/d21ec5eaaab3c8db1f923674b7a3ee19ce35ed13 > create mode 100644 test/static/corpus/csv/d4a6d494cc2a06a0fee4873014210ec523d23750 > create mode 100644 test/static/corpus/csv/eca825c9bdb276332d524134ddf370f7d18c2444 > create mode 100644 test/static/corpus/csv/f9ba3e378e0409d66f946b31306de19edfe859bc > create mode 100644 test/static/corpus/csv/fcad5471c295084dc1ec1eaa3753420e9803d251 > create mode 100644 test/static/corpus/http_parser/038ec38a9ae7669590f1e3401da8ffbc9b40c8ee > create mode 100644 test/static/corpus/http_parser/043fdf868410e66239819e0af10cbe787cfab77e > create mode 100644 test/static/corpus/http_parser/08abe5308ce774ad5e9e435de386748ce06521f8 > create mode 100644 test/static/corpus/http_parser/10b4f603cf09e21020e494e20a7f6760d5634c88 > create mode 100644 test/static/corpus/http_parser/172ed6e56aa94bada4135d64fc2e0c5eac779aed > create mode 100644 test/static/corpus/http_parser/1e42978bf6935a0f8377f6c480c315f1ae4627b8 > create mode 100644 test/static/corpus/http_parser/1f6f26c085c3b209e626f90c63d0fb10526de318 > create mode 100644 test/static/corpus/http_parser/24b458efa865a138a314f0876a11ec680b983ed5 > create mode 100644 test/static/corpus/http_parser/2be565c790bc352f2cbaa0f3509e0bb6133b3616 > create mode 100644 test/static/corpus/http_parser/30fd5e18f586b219916fb3892961a00e92db7a7f > create mode 100644 test/static/corpus/http_parser/38918740212c6e5aea01f2fc8772b1d4aa89be8b > create mode 100644 test/static/corpus/http_parser/39d4c74a90b7f5f5426b7982857b5fd7edbd8f77 > create mode 100644 test/static/corpus/http_parser/3d21092e1a772d792e9dd720eeae30d8eb5ac2ad > create mode 100644 test/static/corpus/http_parser/454dec613ca5c71fee0467061a88d2807201ac36 > create mode 100644 test/static/corpus/http_parser/4f13053e61966dd08d7cd2407865481ec1d2cbf9 > create mode 100644 test/static/corpus/http_parser/4ffa3daceb5b6ee3fc17cef3e0ae694ea5c583b5 > create mode 100644 test/static/corpus/http_parser/501b3a19def06b352978c03ece7ab282cdc6cbab > create mode 100644 test/static/corpus/http_parser/50ad023e1465919f5afbfffb33504ddfdc6a7e69 > create mode 100644 test/static/corpus/http_parser/5177d236e3eea297b76c19194a4cf127b313b267 > create mode 100644 test/static/corpus/http_parser/51e1354d16cfa4967e91206be8bd0d8c6ca577af > create mode 100644 test/static/corpus/http_parser/55d5f3acc62809fc87466b2048b157e50e63d644 > create mode 100644 test/static/corpus/http_parser/598d19d3175046ff035be228db2f1dc43957d8e2 > create mode 100644 test/static/corpus/http_parser/5b7444e43e5c1fd759483c76a46a5536da043022 > create mode 100644 test/static/corpus/http_parser/5c489184823a8a42844066fd5b4951bbad86934f > create mode 100644 test/static/corpus/http_parser/6455c0e761cd485f3e38c470f216560bfc46eea4 > create mode 100644 test/static/corpus/http_parser/6596bb363cfd0bef3f68e265cfd824c6d37ecb7e > create mode 100644 test/static/corpus/http_parser/66d5206b6c75b071f57684399218d144b0d337ea > create mode 100644 test/static/corpus/http_parser/685073a623772c1d46475b5eb451de40247d697f > create mode 100644 test/static/corpus/http_parser/71d4de855c1af66a6a5cea88b1eddc5032ecb26d > create mode 100644 test/static/corpus/http_parser/73378b09c61739b65a5f63214f0515307f187e53 > create mode 100644 test/static/corpus/http_parser/768abf119ce4fa6c9a9ee9f17d9dacb32c61759c > create mode 100644 test/static/corpus/http_parser/7ae1560be3dffbb88f3e48de3a4be99ecdc61441 > create mode 100644 test/static/corpus/http_parser/7c4d33785daa5c2370201ffa236b427aa37c9996 > create mode 100644 test/static/corpus/http_parser/80c5002c297e86198535d1574f3e47f4c6900782 > create mode 100644 test/static/corpus/http_parser/86908cf9f9f2c690ebd7462cacd2b28fc45db3a5 > create mode 100644 test/static/corpus/http_parser/89681f4bac464df0f73f3bf3fd48db40bc5796a6 > create mode 100644 test/static/corpus/http_parser/8c371f137d8a10fc568a3022edf6f10280c96bc0 > create mode 100644 test/static/corpus/http_parser/8ff59012e5ea417bf340e59a0ab6fd9e410011ea > create mode 100644 test/static/corpus/http_parser/927ba76d456f6c3a06769795f87c46462e16acd5 > create mode 100644 test/static/corpus/http_parser/95a0311cd792a22e0de733080e0580638c8eb136 > create mode 100644 test/static/corpus/http_parser/9a3fc50025efc8aac22e16f10621f43a0f35ddcb > create mode 100644 test/static/corpus/http_parser/ad1bde0f01973fc121af6f4a1efef3a1447c3a80 > create mode 100644 test/static/corpus/http_parser/ad50e55369c0e4459500323e0af8eb00172f0909 > create mode 100644 test/static/corpus/http_parser/b2af1811e30009a2b7cec55e86f26b8f4bf3c1cf > create mode 100644 test/static/corpus/http_parser/bcebbda5c1d2033a62c14d6bf7e2dc54b407b403 > create mode 100644 test/static/corpus/http_parser/cfe75970181a6e2f5ddbf9cf968a9a1fba1a660b > create mode 100644 test/static/corpus/http_parser/d01798aa68e31f2964a2bc2c10b203f5186a3ba1 > create mode 100644 test/static/corpus/http_parser/d2dc25ea74a20f35c78091cbc1e30cf2557c315f > create mode 100644 test/static/corpus/http_parser/d930b69281035ce3b46f2635f5a91ee11f15876c > create mode 100644 test/static/corpus/http_parser/dcb92ca0399b135c88170124fcdab6b6d62d115c > create mode 100644 test/static/corpus/http_parser/e4e52eaca9ece89c0e3fa2ef7e6e126d47f20101 > create mode 100644 test/static/corpus/http_parser/ebe97f523d44667ea6e11e1c8cd9e5f02cf244e7 > create mode 100644 test/static/corpus/http_parser/ede1894116fdc6430ba23942e12d974e3f22875b > create mode 100644 test/static/corpus/http_parser/f0f8a948443b28c5e344e6e43aedbb479059779c > create mode 100644 test/static/corpus/http_parser/f1c4ffb8ea524f69bc8d6784f86b70ac99db2ad7 > create mode 100644 test/static/corpus/http_parser/f774dd0b8fc355505aab48a2107804387f45c4a0 > create mode 100644 test/static/corpus/uri/016d581dcda85465d5fa22eadd5947395b61303f > create mode 100644 test/static/corpus/uri/018287fe57cf372d929b52a560910284ba406de6 > create mode 100644 test/static/corpus/uri/048b75ce15485f8daa16e516014df26f2cf1ce09 > create mode 100644 test/static/corpus/uri/048fbb93cfd1be0ccf151cf75147e391f12c2778 > create mode 100644 test/static/corpus/uri/07e1d498149de21318fe486a73b618b800f9e46c > create mode 100644 test/static/corpus/uri/093873ef42d30bee0ec042892457333ce7d6cd65 > create mode 100644 test/static/corpus/uri/0a11f3691a10feefaeda9e30b4c891148d4ffdc1 > create mode 100644 test/static/corpus/uri/14394ce809fcbd4db56a1f1814fb322103148fc6 > create mode 100644 test/static/corpus/uri/251e9a9e4e077720a71747a94e24a6e25cb314a9 > create mode 100644 test/static/corpus/uri/2a855b2e35301abb51facebb20e78801b2219e0f > create mode 100644 test/static/corpus/uri/2ab8897ce59547952f047b48d52f9d6d4c0ea3eb > create mode 100644 test/static/corpus/uri/39f3fca5b4634a9004220b389122abedfef28db7 > create mode 100644 test/static/corpus/uri/3f18809f0df987b47c236d82d687ba16231daedb > create mode 100644 test/static/corpus/uri/4143d3a341877154d6e95211464e1df1015b74bd > create mode 100644 test/static/corpus/uri/49159cda7338d9b1a355631e7d33c55564fbfcb3 > create mode 100644 test/static/corpus/uri/4b40fa57bf6479d96f1a751c68578ee64e7b6759 > create mode 100644 test/static/corpus/uri/4f1a4d9e9b5db51ca7fdf0e9b4adb52f1813918b > create mode 100644 test/static/corpus/uri/54b8c977fd54171c25da05e420b13d05dfaad747 > create mode 100644 test/static/corpus/uri/5bfff0efcc1e7f8e3550798038d6361392379dfa > create mode 100644 test/static/corpus/uri/60f481551bdf721e8d1f2c805da6a7fb2d2e4858 > create mode 100644 test/static/corpus/uri/6c03daf8e568705738f646bfcdd03946acdce123 > create mode 100644 test/static/corpus/uri/6dcaf13f83e40d73a9c5180977b5bf171b812e11 > create mode 100644 test/static/corpus/uri/7511cc73615187b17837d01a013e4d886331fa54 > create mode 100644 test/static/corpus/uri/767e882d43ffb1c822233640f7ca4e33426f7a3b > create mode 100644 test/static/corpus/uri/777b43cd6a3f75e17d9310584ac00d4bd3ee65a1 > create mode 100644 test/static/corpus/uri/77d8be75df172643a6fb1ee235d05b904aa501a0 > create mode 100644 test/static/corpus/uri/7b8fa7a85f9f6ae6f9178504d2202666fb8dc772 > create mode 100644 test/static/corpus/uri/7d850a53909012d59b379e99f1549332f01e3088 > create mode 100644 test/static/corpus/uri/84f62a3184758e8c27e63f3f27c05a50877dc389 > create mode 100644 test/static/corpus/uri/851be661349b0ceabf7cd933409f4d79ca9ac9a8 > create mode 100644 test/static/corpus/uri/8efd1c91dceac73be0780f4f3cddce62ba84ebf6 > create mode 100644 test/static/corpus/uri/8f3feb9a63afb6e100dad08136d8cbeea4a1dfed > create mode 100644 test/static/corpus/uri/9100851463d47ef66626599f78730e4da01b8562 > create mode 100644 test/static/corpus/uri/9dfe5ccca3a5ed2e574fb1dd10ec6d98bf637484 > create mode 100644 test/static/corpus/uri/a5f6b7bdd3115f7427c55830f8fc4563256d410b > create mode 100644 test/static/corpus/uri/adc83b19e793491b1c6ea0fd8b46cd9f32e592fc > create mode 100644 test/static/corpus/uri/ae13a961739ce96b0cb75d1fae1bd6eddaad473d > create mode 100644 test/static/corpus/uri/b15df036caa676bffff10933845c16ce0192f726 > create mode 100644 test/static/corpus/uri/b379e6c191cce7491020b50c345debd7ee95f14c > create mode 100644 test/static/corpus/uri/b7899610b3e9299cc4248ff090827e35962a7a81 > create mode 100644 test/static/corpus/uri/bad754935cc524ecde1a46997056db302ea3baa3 > create mode 100644 test/static/corpus/uri/bbf379045a2134133a43636c5e7807d2503f6261 > create mode 100644 test/static/corpus/uri/bdaa097dc0aecd0840c0e6e46ed269ccec14f5fe > create mode 100644 test/static/corpus/uri/be403e2dc82146c9730731e8facb7a996fbfa01a > create mode 100644 test/static/corpus/uri/c358d48b194741a6ed970a730c64c18c26076778 > create mode 100644 test/static/corpus/uri/c4dff26491e4d229af0efcae747a2b78fddd7b75 > create mode 100644 test/static/corpus/uri/c76f1c7d662c14d891f04058128ce7cd83bc09bd > create mode 100644 test/static/corpus/uri/dabc3a9d30f94608b2a68a5beb470c704890e3f1 > create mode 100644 test/static/corpus/uri/e0c30dfd58621aa24062fea9cf21d976e108a2f4 > create mode 100644 test/static/corpus/uri/e400db1ce2202210e9542f9f030c6404bf5718c8 > create mode 100644 test/static/corpus/uri/e5fa44f2b31c1fb553b6021e7360d07d5d91ff5e > create mode 100644 test/static/corpus/uri/eca60ab9f2f130ab10bf4a642d00690442dd2b74 > create mode 100644 test/static/corpus/uri/f25a33be8af0a7f725531c61f228d2339dfd089e > > -- > 2.25.1 > -- Best regards, IM ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 0/4] Add fuzzing testing 2020-11-30 20:24 [Tarantool-patches] [PATCH 0/4] Add fuzzing testing sergeyb ` (5 preceding siblings ...) 2020-12-07 17:49 ` Igor Munkin @ 2020-12-25 13:08 ` Igor Munkin 2020-12-25 14:52 ` Kirill Yukhin 7 siblings, 0 replies; 26+ messages in thread From: Igor Munkin @ 2020-12-25 13:08 UTC (permalink / raw) To: sergeyb; +Cc: tarantool-patches Sergey, Considering this CI job result[1] I guess nothing became worse. LGTM. [1]: https://gitlab.com/tarantool/tarantool/-/jobs/931940259 -- Best regards, IM ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Tarantool-patches] [PATCH 0/4] Add fuzzing testing 2020-11-30 20:24 [Tarantool-patches] [PATCH 0/4] Add fuzzing testing sergeyb ` (6 preceding siblings ...) 2020-12-25 13:08 ` Igor Munkin @ 2020-12-25 14:52 ` Kirill Yukhin 7 siblings, 0 replies; 26+ messages in thread From: Kirill Yukhin @ 2020-12-25 14:52 UTC (permalink / raw) To: sergeyb; +Cc: tarantool-patches Hello, On 30 ноя 23:24, Sergey Bronnikov via Tarantool-patches wrote: > From: Sergey Bronnikov <sergeyb@tarantool.org> > > Patch series adds required support in project infrastructure for fuzzing tests, > add tests for uri, csv and http_parser modules, adds integration with OSS Fuzz > (infrastructure for continuous running fuzzing tests). > > NOTE: v2 version had LGTM from Sergey Petrenko, see [1] > > Changelog: > > v2: > - introduce CMake flag to enable building fuzzers > - add fuzzers based on LibFuzzer to csv, http_parser and uri modules > > v3: > - report error on an attempt to set ENABLE_FUZZER=ON with GCC compiler > - fixed return code in tests > - place tests to a common dir test/fuzz/ like we did with unit tests > - enable building fuzzers in Tarantool CI job > - add target to build all availabe fuzzers > - add integration with OSS Fuzz [2] > - add corpus for uri, csv and http_parser tests > > 1. https://lists.tarantool.org/pipermail/tarantool-patches/2020-April/016471.html > 2. https://github.com/google/oss-fuzz/pull/4723 > > CI: https://gitlab.com/tarantool/tarantool/-/pipelines/223112908 > Issue: https://github.com/tarantool/tarantool/issues/1809 I've checked your patchset into 2.5, 2.6 and master. -- Regards, Kirill Yukhin ^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2020-12-25 14:52 UTC | newest] Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-11-30 20:24 [Tarantool-patches] [PATCH 0/4] Add fuzzing testing sergeyb 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 1/4] test: add infrastructure for fuzzing testing and fuzzers sergeyb 2020-12-07 17:24 ` Igor Munkin 2020-12-07 19:54 ` Igor Munkin 2020-12-13 18:56 ` Sergey Bronnikov 2020-12-20 13:31 ` Igor Munkin 2020-12-24 10:18 ` Sergey Bronnikov 2020-12-24 13:22 ` Igor Munkin 2020-12-24 17:25 ` Sergey Bronnikov 2020-12-24 17:50 ` Igor Munkin 2020-12-25 7:07 ` Sergey Bronnikov 2020-12-25 9:02 ` Igor Munkin 2020-12-25 10:33 ` Sergey Bronnikov 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 2/4] test: add corpus to be used with fuzzers sergeyb 2020-12-07 17:34 ` Igor Munkin 2020-12-13 18:56 ` Sergey Bronnikov 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 3/4] travis: build tarantool with ENABLE_FUZZER sergeyb 2020-12-07 17:38 ` Igor Munkin 2020-11-30 20:24 ` [Tarantool-patches] [PATCH 4/4] test: integrate with OSS Fuzz sergeyb 2020-12-07 17:42 ` Igor Munkin 2020-12-01 10:54 ` [Tarantool-patches] [PATCH 0/4] Add fuzzing testing Serge Petrenko 2020-12-01 14:41 ` Sergey Bronnikov 2020-12-01 14:45 ` Serge Petrenko 2020-12-07 17:49 ` Igor Munkin 2020-12-25 13:08 ` Igor Munkin 2020-12-25 14:52 ` Kirill Yukhin
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox