From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 916442684A for ; Sun, 21 Apr 2019 13:06:44 -0400 (EDT) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JTaKzDb2a30Y for ; Sun, 21 Apr 2019 13:06:44 -0400 (EDT) Received: from smtpng3.m.smailru.net (smtpng3.m.smailru.net [94.100.177.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id ED0802303E for ; Sun, 21 Apr 2019 13:06:43 -0400 (EDT) Subject: [tarantool-patches] Re: [PATCH] Set format for spaces with sysview engine References: <20190417081320.xwcf3ogkvrspgdu3@tarantool.org> <68ec9d10-4fa6-4997-54bd-4a80b1ca79ce@tarantool.org> <20190418081630.yfe5hi7w2m3bu5lh@tarantool.org> <77d5c2c1-8a26-990c-0fc0-dc40d56f64ae@tarantool.org> <20190418111411.hl4okehi6adv6pvi@tarantool.org> <20190418120840.667b7m5iqtjl2r42@tarantool.org> <5783e513-19e4-182b-941b-9070a7266c92@tarantool.org> <20190418132522.cfpocghcjmjtcpe3@tarantool.org> <20190418141804.GB7924@chai> <20190420223627.nqxdxx2vcngrqt3h@tkn_work_nb> From: Vladislav Shpilevoy Message-ID: Date: Sun, 21 Apr 2019 20:06:38 +0300 MIME-Version: 1.0 In-Reply-To: <20190420223627.nqxdxx2vcngrqt3h@tkn_work_nb> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-Help: List-Unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-Subscribe: List-Owner: List-post: List-Archive: To: tarantool-patches@freelists.org, Alexander Turenko , Konstantin Osipov Cc: Kirill Yukhin On 21/04/2019 01:36, Alexander Turenko wrote: > On Thu, Apr 18, 2019 at 05:18:04PM +0300, Konstantin Osipov wrote: >> * Kirill Yukhin [19/04/18 17:12]: >>> How ofter do you scan list of regression tests? I bet - never. >>> But you very often try to extract 2-3 lines of failed case. >> >> If you work with the test suite as an active maintainer you work >> with the files all the time. Otherwise your tests just rot. > > It does not matter much whether test cases spread around its own files > or grouped into categories within one file. The key point here is to > make cases independent. Even when you have independent test cases you > still can extract common preparation code that will be run before each > test case (or before a group of test cases). > > Look how test cases are organized in our projects with pytest or with > failsafe maven plugin. They are grouped into files, but I still able to > do something like `pytest test/test_foo.py -k test_bar` or `mvn verify > -Dit.test=FooIT#testBar` and concentrate on one case. > > When test cases depend on each other you need to manually find and > extract all needed parts of a test file to run a case. When you do it > several times per day, hey, it becomes annoying. > > Now even if developers highly self-disciplined and write independent > test cases, an extra work is still necessary to run a case if a harness > is unable to run just what you need. > > And while we are here there is other problem with console-input-like > tests: often you don't sure which statements results are checked > intentionally and which ones are run only for its side effects. > Sometimes you need to git-blame over several attempts to rewrite a test > to understand what the idea was. > > Maybe we are near to the point when such simple approach gives more > problems then solves. > > Yep, I stated problems here, but don't propose anything. I don't know > where is a right balance between pushing developers to write structured > tests and the freedom developers have now. > > I see that some developers like ability to test something with injecting > just one line for one case into another one. I think that it will make > tests more and more embrassing. So, I don't know. My point is that many issues are because of typos or another extra simple mistakes, and what is more - they are linked somehow to a swarm of other similar issues, already tested somewhere. For example, the test you've fixed for netbox about 'unique' index flag. It was just a typo, and we have a place, where other space/index properties are tested. It is logically to test one another index property together with others. Just imagine how many tests you would have had if we had dedicated one test file to each space and index option. In such a situation parallel test run will not help you, when number of test files will be thousands. You just do not have so many cores so as to swiftly start and shutdown new processes and threads. Even SQLite does not dedicate new test file per each single tiny typo-like issue. They have dedicated file per issue, but only for really hard ones. > > WBR, Alexander Turenko. >