Tarantool development patches archive
 help / color / mirror / Atom feed
From: Vladimir Davydov <vdavydov.dev@gmail.com>
To: Alexander Turenko <alexander.turenko@tarantool.org>
Cc: tarantool-patches@freelists.org
Subject: Re: [PATCH] test: fix flaky fails of box/iproto_stress
Date: Wed, 9 Jan 2019 13:33:41 +0300	[thread overview]
Message-ID: <20190109103341.l7t35iqmvl75hbnz@esperanza> (raw)
In-Reply-To: <20181228031139.7uvh5etkdmewdn7u@tkn_work_nb>

On Fri, Dec 28, 2018 at 06:11:40AM +0300, Alexander Turenko wrote:
> On Thu, Dec 27, 2018 at 01:26:26PM +0300, Vladimir Davydov wrote:
> > On Wed, Dec 26, 2018 at 02:37:51AM +0300, Alexander Turenko wrote:
> > > Fixes #3911.
> > > ---
> > > 
> > > https://github.com/tarantool/tarantool/issues/3911
> > > https://github.com/tarantool/tarantool/tree/Totktonada/gh-3911-fix-box-iproto-stress
> > > 
> > >  test/box/iproto_stress.result   | 1 +
> > >  test/box/iproto_stress.test.lua | 1 +
> > >  2 files changed, 2 insertions(+)
> > > 
> > > diff --git a/test/box/iproto_stress.result b/test/box/iproto_stress.result
> > > index 4239b49b8..4b7b41bd7 100644
> > > --- a/test/box/iproto_stress.result
> > > +++ b/test/box/iproto_stress.result
> > > @@ -40,6 +40,7 @@ function worker(i)
> > >              n_errors = n_errors + 1
> > >              break
> > >          end
> > > +        conn:wait_state('active', 10)
> > 
> > According to
> > 
> >   https://tarantool.io/en/doc/1.10/reference/reference_lua/net_box/#lua-function.net_box.new
> > 
> > net_box.connect() waits until connection is active unless it is passed
> > wait_connected = false.
> > 
> > A quick glance at the code confirms that:
> > 
> >   https://github.com/tarantool/tarantool/blob/911139e379666ddff6e448184d066384a085df1e/src/box/lua/net_box.lua#L958
> > 
> > The question is why it doesn't happen in the test. A bug in the
> > documentation or in net.box?
> 
> It seems I wrongly determined the reason of the flaky fails. Thank you
> for catching it up.
> 
> I found that I'm able to reproduce the issue (w/o net.box changes) when
> compiling gcc in 16 threads in background :)
> 
> I guessed it is because attempts count at the end of test is lesser than
> needed for parallel run on travis-ci and tried to increase it from 100
> to 1000. But even in this case I catched the fail (once, to be honest)

1000 means waiting for 100 seconds, which is a bit of an overkill.

> with background CPU load and 1000 iproto_stress tests in 16 threads.
> 
> It seems it is rare thing. Now I don't sure we should change attempts
> count. Maybe we should run the test in non-parallel mode, after all
> parallel tests? We must move it into separate test suite to do so for
> now.

I don't think we need to add this workaround, because the failure is
pretty rare. Besides, it still isn't quite clear what's causing it so
occasional failures will remind us that we need to investigate the
problem deeper.

> 
> Maybe we should leave it unchanged and check whether it is really rare.
> 
> (The branch was updated with attemps == 1000.)
> 
> Any suggestions?

As a matter of fact, the error message mentioned in the issue
description

  E> LuajitError: [string "function worker(i)     n_workers = n_workers ..."]:1: attempt to index field 'space' (a nil value)

looks weird to me. After all, the test does nothing criminal - it simply
creates a net.box connection and tries to access an existing space and
for some reason fails. We need to find out the real cause of the
problem. May be a bug in net.box implementation...

      reply	other threads:[~2019-01-09 10:33 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-12-25 23:37 Alexander Turenko
2018-12-27 10:26 ` Vladimir Davydov
2018-12-28  3:11   ` Alexander Turenko
2019-01-09 10:33     ` Vladimir Davydov [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190109103341.l7t35iqmvl75hbnz@esperanza \
    --to=vdavydov.dev@gmail.com \
    --cc=alexander.turenko@tarantool.org \
    --cc=tarantool-patches@freelists.org \
    --subject='Re: [PATCH] test: fix flaky fails of box/iproto_stress' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox