Re: [tarantool-patches] Re: [PATCH] test: fix unix socket conflict in socket.test.lua

[Vladimir Davydov <vdavydov.dev@gmail.com>]

On Mon, Oct 29, 2018 at 11:25:29AM +0300, Alexander Turenko wrote:
> On Thu, Oct 25, 2018 at 12:43:39PM +0300, Vladimir Davydov wrote:
> > On Thu, Oct 25, 2018 at 05:21:45AM +0300, Alexander Turenko wrote:
> > > It is needed to run the test in parallel on several test-run workers to
> > > investigate flaky failures of the test.
> > 
> > I don't think I understand. Do you mean that this test doesn't fix the
> > test flakiness and is only needed for further investigation?
> > 
> 
> Yes (but now I have added some fixes that can eliminate some flaky
> failures). It is often convenient to run something like...
> 
> ```
> TEST_RUN_TESTS=$(for i in $(seq 1 100); do echo -n "app/ "; done) make test
> ```
> 
> ...to prove the suite is ready to run in parallel mode.
> 
> > > diff --git a/test/app/socket.result b/test/app/socket.result
> > > index 2f002a37e..1a570b9fa 100644
> > > --- a/test/app/socket.result
> > > +++ b/test/app/socket.result
> > > @@ -42,6 +42,29 @@ test_run:cmd("push filter '(error: .builtin/.*[.]lua):[0-9]+' to '\\1'")
> > >  ---
> > >  - true
> > >  ...
> > > +test_run:cmd("push filter '(/tmp/tarantool-test-socket)-[0-9]+' to '\\1'")
> > > +---
> > > +- true
> > > +...
> > > +-- /tmp/tarantool-test-socket-${TEST_RUN_WORKER_ID}
> > > +test_run:cmd("setopt delimiter ';'")
> > > +---
> > > +- true
> > > +...
> > > +function get_temp_socket_path()
> > > +    local base_path = '/tmp/tarantool-test-socket'
> > > +    local worker_id = os.getenv('TEST_RUN_WORKER_ID')
> > > +    if not worker_id then
> > > +        return base_path
> > > +    end
> > > +    return ('%s-%s'):format(base_path, worker_id)
> > > +end;
> > 
> > Why don't you simply create the file in the worker directory?
> 
> Sure, I can. Rewritten the test. But socket.skipcond still uses
> /tmp/tarantool-test-socket-${WORKER_ID}, because its working directory
> is `test`.
> 
> The whole new patch is below.
> 
> commit 224fcdf2a5e8f84971253f1d585b94213d06a384
> Author: Alexander Turenko <alexander.turenko@tarantool.org>
> Date:   Thu Oct 25 04:06:46 2018 +0300
> 
>     test: fix unix socket conflict in socket.test.lua
>     
>     Increased socket.readable / socket.wait timeouts.
>     
>     Rewritten port choosing: repeat bind+listen until succeeds, exclude
>     incorrect port 65536 from the range.
>     
>     All these changes are needed to run the test in parallel on several
>     test-run workers to investigate flaky failures of the test / of the
>     suite. Some of these changes can also eliminate possible flaky failures.

./test-run -j -1 app/socket.test.lua doesn't work with this patch:

} app/socket.test.lua
} Test.run() received the following error:
} Traceback (most recent call last):
}   File "/home/vlad/src/tarantool/test-run/lib/test.py", line 174, in run
}     execfile(self.skip_cond, dict(locals(), **server.__dict__))
}   File "app/socket.skipcond", line 8, in <module>
}     worker_id = os.environ['TEST_RUN_WORKER_ID']
}   File "/usr/lib/python2.7/UserDict.py", line 40, in __getitem__
}     raise KeyError(key)
} KeyError: 'TEST_RUN_WORKER_ID'

I think you shouldn't use TEST_RUN_WORKER_ID in skipcond after all.
Instead you should make a temporary file.

Also, I've seen the following failure once. Not sure if it's related:

} [001] app/socket.test.lua                                             [ fail ]
} [001]
} [001] Test failed! Result content mismatch:
} [001] --- app/socket.result     Mon Oct 29 12:06:39 2018
} [001] +++ socket.reject Mon Oct 29 12:13:34 2018
} [001] @@ -2810,7 +2810,7 @@
} [001]  ...
} [001]  echo_fiber ~= nil
} [001]  ---
} [001] -- true
} [001] +- false
} [001]  ...
} [001]  client:write('hello')
} [001]  ---
} [001]
} [001] Last 15 lines of Tarantool Log file [Instance "app"][/home/vlad/src/tarantool/test/var/001_app/app.log]:
} [001] 2018-10-29 12:13:34.188 [20123] main/144/server/unix/:tarantool-test-soc I> started
} [001] 2018-10-29 12:13:34.189 [20123] main/142/server/unix/:tarantool-test-soc utils.c:939 E> LuajitError: builtin/socket.lua:80: attempt to use closed socket
} [001] 2018-10-29 12:13:34.189 [20123] main/146/server/unix/:tarantool-test-soc I> started
} [001] 2018-10-29 12:13:34.190 [20123] main/144/server/unix/:tarantool-test-soc utils.c:939 E> LuajitError: builtin/socket.lua:80: attempt to use closed socket
} [001] 2018-10-29 12:13:34.190 [20123] main/148/server/unix/:tarantool-test-soc I> started
} [001] 2018-10-29 12:13:34.190 [20123] main/146/server/unix/:tarantool-test-soc utils.c:939 E> LuajitError: builtin/socket.lua:80: attempt to use closed socket
} [001] 2018-10-29 12:13:34.190 [20123] main/150/server/unix/:tarantool-test-soc I> started
} [001] 2018-10-29 12:13:34.191 [20123] main/148/server/unix/:tarantool-test-soc utils.c:939 E> LuajitError: builtin/socket.lua:80: attempt to use closed socket
} [001] 2018-10-29 12:13:34.191 [20123] main/152/server/unix/:tarantool-test-soc I> started
} [001] 2018-10-29 12:13:34.191 [20123] main/150/server/unix/:tarantool-test-soc utils.c:939 E> LuajitError: builtin/socket.lua:80: attempt to use closed socket
} [001] 2018-10-29 12:13:34.191 [20123] main/154/server/unix/:tarantool-test-soc I> started
} [001] 2018-10-29 12:13:34.192 [20123] main/152/server/unix/:tarantool-test-soc utils.c:939 E> LuajitError: builtin/socket.lua:80: attempt to use closed socket
} [001] 2018-10-29 12:13:34.192 [20123] main/154/server/unix/:tarantool-test-soc utils.c:939 E> LuajitError: builtin/socket.lua:80: attempt to use closed socket
} [001] 2018-10-29 12:13:34.364 [20123] main/161/echoserv/::1:0 I> started
} [001] 2018-10-29 12:13:34.417 [20123] main/161/echoserv/::1:0 utils.c:939 E> LuajitError: builtin/socket.lua:80: attempt to use closed socket
} [Main process] Got failed test; gently terminate all workers...
} [001] Worker "001_app" got failed test; stopping the server...
} ---------------------------------------------------------------------------------
} Statistics:
} * fail: 1
} Failed tasks:
} - [app/socket.test.lua, null]
} # logfile:        /home/vlad/src/tarantool/test/var/log/001_app.log
} # reproduce file: /home/vlad/src/tarantool/test/var/reproduce/001_app.list.yaml
} ---
} - [app/socket.test.lua, null]
} ...