

<HTML><BODY><p style="margin-bottom: 0.2in; line-height: 100%;" data-mce-style="margin-bottom: 0.2in; line-height: 100%;"><br> <br></p><blockquote style="margin-right: 0.5in; margin-top: 0.1in; margin-bottom: 0.1in; border-color: currentcolor currentcolor currentcolor rgb(8, 87, 166); border-style: none none none solid; border-width: medium medium medium 1px; padding-top: 0in; padding-bottom: 0in; padding-right: 0in;">Вторник, 18 июня 2019, 14:08 +03:00 от Alexander Turenko <alexander.turenko@tarantool.org>:</blockquote><div id="style_15608560881683133890_BODY" dir="ltr"><blockquote style="margin-top: 0.1in; margin-bottom: 0.1in; border-top: none; border-bottom: none; border-left: 1px solid #0857a6; border-right: none; padding-top: 0in; padding-bottom: 0in; padding-left: 0.1in; padding-right: 0in;" data-mce-style="margin-top: 0.1in; margin-bottom: 0.1in; border-top: none; border-bottom: none; border-left: 1px solid #0857a6; border-right: none; padding-top: 0in; padding-bottom: 0in; padding-left: 0.1in; padding-right: 0in;">Thanks, Alexander!<br> <br> test_foo runs runtest_foo, which runs test-run.py. This looks confusing<br> for me. We can use names like `test_ubuntu` and `test_ubuntu_no_deps`.<br> <br> Comments below are generally re decreasing number of goals that I think<br> will simplify the configuration at whole.<br> <br> WBR, Alexander Turenko.<br> <br></blockquote></div><p>Fixed.</p><div id="Section1" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY" class="mceItemAnchor"></a><br> On Sat, Jun 15, 2019 at 07:46:23AM +0300, Alexander V. Tikhonov wrote:<br> > ---<br> > <br> > Github: <a href="https://github.com/tarantool/tarantool/tree/avtikhon/gh-4156-gitlab-ci-testing" data-mce-href="https://checklink.mail.ru/proxy?es=BgbodWBV8Xrty7vLT3kyWXaAdzq%2Bf9vKp7BquItktoU%3D&egid=lrtF4X9qYBn3yWZH%2BWyxMGX7sk6UYEQxIi6nXsMTmCE%3D&url=https%3A%2F%2Fclick.mail.ru%2Fredir%3Fu%3Dhttps%253A%252F%252Fgithub.com%252Ftarantool%252Ftarantool%252Ftree%252Favtikhon%252Fgh-4156-gitlab-ci-testing%26c%3Dswm%26r%3Dhttp%26o%3Dmail%26v%3D2%26s%3D1d8c7b14275bdfa4" target="_blank">https://github.com/tarantool/tarantool/tree/avtikhon/gh-4156-gitlab-ci-testing</a><br> > Issue: <a href="https://github.com/tarantool/tarantool/issues/4156" data-mce-href="https://checklink.mail.ru/proxy?es=BgbodWBV8Xrty7vLT3kyWXaAdzq%2Bf9vKp7BquItktoU%3D&egid=lrtF4X9qYBn3yWZH%2BWyxMGX7sk6UYEQxIi6nXsMTmCE%3D&url=https%3A%2F%2Fclick.mail.ru%2Fredir%3Fu%3Dhttps%253A%252F%252Fgithub.com%252Ftarantool%252Ftarantool%252Fissues%252F4156%26c%3Dswm%26r%3Dhttp%26o%3Dmail%26v%3D2%26s%3Dac0789e63cc74d98" target="_blank">https://github.com/tarantool/tarantool/issues/4156</a><br> > <br> > .gitlab-ci.yml | 328 +++++++++++++++++++++++++++++++++++++++<br> > .gitlab.mk | 57 +++++++<br> > .travis.mk | 101 +++++++++---<br> > .travis.yml | 3 +<br> > images/Dockerfile.debian | 4 +<br> > images/Dockerfile.lto | 5 +<br> > 6 files changed, 475 insertions(+), 23 deletions(-)<br> > create mode 100644 .gitlab-ci.yml<br> > create mode 100644 .gitlab.mk<br> > create mode 100644 images/Dockerfile.debian<br> > create mode 100644 images/Dockerfile.lto<br> > <br> > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml<br> > new file mode 100644<br> > index 000000000..01c4eacea<br> > --- /dev/null<br> > +++ b/.gitlab-ci.yml<br> > @@ -0,0 +1,328 @@<br> > +stages:<br> > + - bootstrap<br> > + - test<br> > + - perf_deploy<br> <br> Nit: Maybe perf_and_deploy?<br> <br> However I would name it just 'deploy' within this patch.<br> <br></blockquote></div><p>Fixed.</p><div id="Section2" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY1" class="mceItemAnchor"></a><br> > +<br> > +variables:<br> > + VERSION_SUFFIX: "master"<br> > + IMAGE_TEST: "${CI_REGISTRY}/${CI_PROJECT_PATH}:debian-stretch_${VERSION_SUFFIX}"<br> > + IMAGE_TEST_LTO: "${CI_REGISTRY}/${CI_PROJECT_PATH}:debian-buster_${VERSION_SUFFIX}"<br> <br> This suffix does not resolve any problems with changed dependencies. So,<br> we should either remove it for now or really resolve the problem: say,<br> use $(md5sum .travis.mk | awk '{ print $1 }') as a tag name.<br> <br></blockquote></div><p>Not possible to use scripts for variables, neither use API to remove images from registry. Not resolved.<br></p><div id="Section3" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY2" class="mceItemAnchor"></a><br> BTW, I would prefer to use tags instead of suffixes; gitlab ci supports<br> it.<br> <br></blockquote></div><p>Fixed.</p><div id="Section4" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY3" class="mceItemAnchor"></a><br> > + FLAG_LTO: "-DENABLE_LTO=ON"<br> <br> I think -DENABLE_LTO-ON is more self-descriptive then this variable<br> name. There is no need to extract it to a variable.<br> <br></blockquote></div><p>Fixed, set name to: 'DENABLE_LTO_ON'</p><div id="Section5" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY4" class="mceItemAnchor"></a><br> > + GITLAB_MAKE: "make -f .gitlab.mk"<br> > +<br> > +# Bootstraps<br> > +<br> > +debian9:<br> > + stage: bootstrap<br> > + tags:<br> > + - bootstrap<br> > + variables:<br> > + IMAGE: ${IMAGE_TEST}<br> > + DOCKERFILE_SUFFIX: "debian"<br> > + script:<br> > + - ${GITLAB_MAKE} docker_bootstrap<br> > +<br> > +debian10_lto:<br> > + only:<br> > + refs:<br> > + - master<br> > + - /.*/<br> <br> As we discusssed voicely: /.*/ should not be here in a final patch. We<br> however can provide an ability to run full testing on a branch different<br> from master. Say, match /.*-full/.<br> <br> (The same is applicable for all such patterns below.)<br> <br></blockquote></div><p>Working on.</p><div id="Section6" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY5" class="mceItemAnchor"></a><br> > + stage: bootstrap<br> > + tags:<br> > + - bootstrap<br> > + variables:<br> > + IMAGE: ${IMAGE_TEST_LTO}<br> > + DOCKERFILE_SUFFIX: "lto"<br> > + script:<br> > + - ${GITLAB_MAKE} docker_bootstrap<br> > +<br> > +# Tests<br> > +<br> > +release:<br> > + image: ${IMAGE_TEST}<br> > + stage: test<br> > + tags:<br> > + - docker_test<br> > + script:<br> > + - ${GITLAB_MAKE} runtest_ubuntu<br> > +<br> > +debug:<br> > + image: ${IMAGE_TEST}<br> > + stage: test<br> > + tags:<br> > + - docker_test<br> > + script:<br> > + - ${GITLAB_MAKE} runtest_cov_ubuntu<br> > +<br> > +release_clang:<br> > + image: ${IMAGE_TEST}<br> > + stage: test<br> > + tags:<br> > + - docker_test<br> > + variables:<br> > + CC: clang<br> > + CXX: clang++<br> > + script:<br> > + - ${GITLAB_MAKE} runtest_ubuntu<br> > +<br> > +release_lto:<br> > + only:<br> > + refs:<br> > + - master<br> > + - /.*/<br> > + image: ${IMAGE_TEST_LTO}<br> > + stage: test<br> > + tags:<br> > + - docker_test<br> > + variables:<br> > + CMAKE_EXTRA_PARAMS: ${FLAG_LTO}<br> > + script:<br> > + - ${GITLAB_MAKE} runtest_ubuntu<br> > +<br> > +release_lto_clang8:<br> > + only:<br> > + refs:<br> > + - master<br> > + - /.*/<br> > + image: ${IMAGE_TEST_LTO}<br> > + stage: test<br> > + tags:<br> > + - docker_test<br> > + variables:<br> > + CC: clang-8<br> > + CXX: clang++-8<br> > + CMAKE_EXTRA_PARAMS: ${FLAG_LTO}<br> > + script:<br> > + - ${GITLAB_MAKE} runtest_ubuntu<br> > +<br> > +osx13_release:<br> > + only:<br> > + refs:<br> > + - master<br> > + - /.*/<br> > + stage: test<br> > + tags:<br> > + - vms_sierra<br> > + variables:<br> > + VMS_NAME: 'Sierra'<br> > + VMS_USER: 'tarantool'<br> > + VMS_PORT: '2212'<br> > + before_script:<br> > + - ${GITLAB_MAKE} vms_start<br> > + script:<br> > + - export<br> <br> Forgotten debug? Below too.</blockquote></div><p>Fixed.</p><div id="Section7" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY6" class="mceItemAnchor"></a><br> <br> > + - ${GITLAB_MAKE} vms_test_osx<br> > + after_script:<br> > + - ${GITLAB_MAKE} vms_shutdown<br> > +<br> > +osx14_release:<br> > + stage: test<br> > + tags:<br> > + - vms_mojave<br> > + variables:<br> > + VMS_NAME: 'Mojave'<br> > + VMS_USER: 'tarantool'<br> > + VMS_PORT: '2222'<br> > + before_script:<br> > + - ${GITLAB_MAKE} vms_start<br> > + script:<br> > + - export<br> > + - ${GITLAB_MAKE} vms_test_osx<br> > + after_script:<br> > + - ${GITLAB_MAKE} vms_shutdown<br> > +<br> > +osx14_release_lto:<br> > + only:<br> > + refs:<br> > + - master<br> > + - /.*/<br> > + stage: test<br> > + tags:<br> > + - vms_mojave<br> > + variables:<br> > + EXTRA_ENV: "export CMAKE_EXTRA_PARAMS=${FLAG_LTO} ;"<br> > + VMS_NAME: 'Mojave'<br> > + VMS_USER: 'tarantool'<br> > + VMS_PORT: '2222'<br> > + before_script:<br> > + - ${GITLAB_MAKE} vms_start<br> > + script:<br> > + - export<br> > + - ${GITLAB_MAKE} vms_test_osx<br> > + after_script:<br> > + - ${GITLAB_MAKE} vms_shutdown<br> > +<br> > +freebsd_release:<br> > + stage: test<br> > + tags:<br> > + - vms_freebsd<br> > + variables:<br> > + VMS_NAME: 'Freebsd'<br> <br> I would include a version number into the virtual machine name (and the<br> job name too).</blockquote></div><p>Fixed.</p><div id="Section8" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY7" class="mceItemAnchor"></a><br> <br> > + VMS_USER: 'vagrant'<br> > + VMS_PORT: '2232'<br> > + TRAVIS_MAKE: 'gmake -f .travis.mk'<br> > + before_script:<br> > + - ${GITLAB_MAKE} vms_start<br> > + script:<br> > + - export<br> > + - ${GITLAB_MAKE} vms_test_freebsd<br> > + after_script:<br> > + - ${GITLAB_MAKE} vms_shutdown<br> > +<br> > +# Packs<br> > +<br> > +pack_cent6:<br> <br> Why cent, not centos?</blockquote></div><p>Fixed.</p><div id="Section9" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY8" class="mceItemAnchor"></a><br> <br> > + only:<br> > + refs:<br> > + - master<br> > + - /.*/<br> > + image: packpack/packpack:el-6<br> > + stage: perf_deploy<br> > + tags:<br> > + - deploy<br> > + script:<br> > + - ${GITLAB_MAKE} pack_run_cent6<br> > +<br> > +pack_cent7:<br> > + only:<br> > + refs:<br> > + - master<br> > + - /.*/<br> > + image: packpack/packpack:el-7<br> > + stage: perf_deploy<br> > + tags:<br> > + - deploy_test<br> > + script:<br> > + - ${GITLAB_MAKE} pack_run<br> > +<br> > +pack_fedora28:<br> > + only:<br> > + refs:<br> > + - master<br> > + - /.*/<br> > + image: packpack/packpack:fedora-28<br> > + stage: perf_deploy<br> > + tags:<br> > + - deploy_test<br> > + script:<br> > + - ${GITLAB_MAKE} pack_run<br> > +<br> > +pack_fedora29:<br> > + only:<br> > + refs:<br> > + - master<br> > + - /.*/<br> > + image: packpack/packpack:fedora-29<br> > + stage: perf_deploy<br> > + tags:<br> > + - deploy_test<br> > + script:<br> > + - ${GITLAB_MAKE} pack_run<br> > +<br> > +pack_fedora30:<br> > + only:<br> > + refs:<br> > + - master<br> > + - /.*/<br> > + image: packpack/packpack:fedora-30<br> > + stage: perf_deploy<br> > + tags:<br> > + - deploy_test<br> > + script:<br> > + - ${GITLAB_MAKE} pack_run<br> > +<br> > +pack_ubuntu14:<br> > + only:<br> > + refs:<br> > + - master<br> > + - /.*/<br> > + image: packpack/packpack:ubuntu-trusty<br> > + stage: perf_deploy<br> > + tags:<br> > + - deploy<br> > + script:<br> > + - ${GITLAB_MAKE} pack_run<br> > +<br> > +pack_ubuntu16:<br> > + only:<br> > + refs:<br> > + - master<br> > + - /.*/<br> > + image: packpack/packpack:ubuntu-xenial<br> > + stage: perf_deploy<br> > + tags:<br> > + - deploy<br> > + script:<br> > + - ${GITLAB_MAKE} pack_run<br> > +<br> > +pack_ubuntu18:<br> > + only:<br> > + refs:<br> > + - master<br> > + - /.*/<br> > + image: packpack/packpack:ubuntu-bionic<br> > + stage: perf_deploy<br> > + tags:<br> > + - deploy<br> > + script:<br> > + - ${GITLAB_MAKE} pack_run<br> > +<br> > +pack_ubuntu1810:<br> > + only:<br> > + refs:<br> > + - master<br> > + - /.*/<br> > + image: packpack/packpack:ubuntu-cosmic<br> > + stage: perf_deploy<br> > + tags:<br> > + - deploy<br> > + script:<br> > + - ${GITLAB_MAKE} pack_run<br> > +<br> > +pack_ubuntu19:<br> <br> I would name it 'package_ubuntu_19_04'.</blockquote></div><p>Fixed like 'ubuntu_19_04'.</p><div id="Section10" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY9" class="mceItemAnchor"></a><br> <br> > + only:<br> > + refs:<br> > + - master<br> > + - /.*/<br> > + image: packpack/packpack:ubuntu-disco<br> > + stage: perf_deploy<br> > + tags:<br> > + - deploy<br> > + script:<br> > + - ${GITLAB_MAKE} pack_run<br> > +<br> > +pack_debian8:<br> > + only:<br> > + refs:<br> > + - master<br> > + - /.*/<br> > + image: packpack/packpack:debian-jessie<br> > + stage: perf_deploy<br> > + tags:<br> > + - deploy<br> > + script:<br> > + - ${GITLAB_MAKE} pack_run<br> > +<br> > +pack_debian9:<br> > + only:<br> > + refs:<br> > + - master<br> > + - /.*/<br> > + image: packpack/packpack:debian-stretch<br> > + stage: perf_deploy<br> > + tags:<br> > + - deploy<br> > + script:<br> > + - ${GITLAB_MAKE} pack_run<br> > +<br> > +pack_debian10:<br> > + only:<br> > + refs:<br> > + - master<br> > + - /.*/<br> > + image: packpack/packpack:debian-buster<br> > + stage: perf_deploy<br> > + tags:<br> > + - deploy<br> > + script:<br> > + - ${GITLAB_MAKE} pack_run<br> > diff --git a/.gitlab.mk b/.gitlab.mk<br> > new file mode 100644<br> > index 000000000..00a75a389<br> > --- /dev/null<br> > +++ b/.gitlab.mk<br> > @@ -0,0 +1,57 @@<br> > +GITLAB_MAKE?=make -f .gitlab.mk<br> > +TRAVIS_MAKE?=make -f .travis.mk<br> > +GIT_SUBMODULES_UPDATE?=git submodule update --recursive --init<br> <br> Nit: I would name it GIT_SUBMODULE_UPDATE (w/o 'S'), because the command<br> is named so.<br> <br></blockquote></div><p>Fixed.</p><div id="Section11" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY10" class="mceItemAnchor"></a><br> > +DOCKER_BUILD=docker build --network=host<br> <br> I think there is no much reason to extract it from 'docker_bootstrap'<br> goal.<br> <br></blockquote></div><p>Fixed.</p><div id="Section12" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY11" class="mceItemAnchor"></a><br> > +<br> > +# docker images bootstrap<br> > +docker_login:<br> > + docker login -u ${CI_REGISTRY_USER} -p ${CI_REGISTRY_PASSWORD} ${CI_REGISTRY}<br> <br> Why do we need to extract this goal? It is used only in<br> docker_bootstrap.</blockquote></div><p>Fixed.</p><div id="Section13" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY12" class="mceItemAnchor"></a><br> <br> > +<br> > +docker_bootstrap: docker_login<br> > + cp .travis.mk images/<br> > + ${DOCKER_BUILD} -t ${IMAGE} -f images/Dockerfile.${DOCKERFILE_SUFFIX} images<br> <br> We can give . (a current directory) as a context, so we'll not need to<br> copy .travis.mk.</blockquote></div><p>Fixed.</p><div id="Section14" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY13" class="mceItemAnchor"></a><br> <br> Also we can generate a dockerfile on-the-fly: I don't see much need to<br> keep it in the repository.<br> <br> If a docker version allows we can even pass a dockerfile to stdin:<br> <a href="https://stackoverflow.com/a/42559367/1598057" data-mce-href="https://checklink.mail.ru/proxy?es=BgbodWBV8Xrty7vLT3kyWXaAdzq%2Bf9vKp7BquItktoU%3D&egid=lrtF4X9qYBn3yWZH%2BWyxMGX7sk6UYEQxIi6nXsMTmCE%3D&url=https%3A%2F%2Fclick.mail.ru%2Fredir%3Fu%3Dhttps%253A%252F%252Fstackoverflow.com%252Fa%252F42559367%252F1598057%26c%3Dswm%26r%3Dhttp%26o%3Dmail%26v%3D2%26s%3Db11a5be2e493d3f5" target="_blank">https://stackoverflow.com/a/42559367/1598057</a><br> <br></blockquote></div><p>Fixed.</p><div id="Section15" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY14" class="mceItemAnchor"></a><br> > + docker push ${IMAGE}<br> > +<br> > +# build sources with different flags and test<br> <br> It is the title for the set of goals below? Looks as a comment for one<br> goal. Also here I see that bootstrap, submodules update and runnign<br> tests are doing, but not build sources. It seems the comment is not<br> precise.<br> <br></blockquote></div><p>Right, the comment for a set of goals there below.</p><div id="Section16" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY15" class="mceItemAnchor"></a><br> > +git_submodules_update:<br> > + ${GIT_SUBMODULES_UPDATE} --force<br> <br> Nit: The same as for GIT_SUBMODULES_UPDATE -> GIT_SUBMODULE_UPDATE:<br> git_submodules_update -> git_submodule_update.<br> <br></blockquote></div><p>Fixed.</p><div id="Section17" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY16" class="mceItemAnchor"></a><br> Is there real need in this flag (some specific case when it is needed)?<br> What will going on CentOS 6 if we'll run into this case?<br> <br></blockquote></div><p>Once I got the broken repository some how, '--force' flag fixed it, on CentOS 6 will need to fix it manually.</p><div id="Section18" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY17" class="mceItemAnchor"></a><br> If it is really needed and we don't want to handle it on CentOS 6 (that<br> is strange), can we just run it like so?<br> <br> $ git submodule update --recursive --init --force || git submodule update --recursive --init<br> <br> It will allow to reduce amount of goals and so simplify the file.<br> <br></blockquote></div><p>Fixed.</p><div id="Section19" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY18" class="mceItemAnchor"></a><br> > +<br> > +git_submodules_update_cent6:<br> > + ${GIT_SUBMODULES_UPDATE}<br> > +<br> > +runtest_%: git_submodules_update<br> > + ${TRAVIS_MAKE} $@<br> > +<br> > +# build sources with different flags and test under VBox virtual machines<br> > +vms_start:<br> > + VBoxManage controlvm ${VMS_NAME} poweroff || true<br> > + VBoxManage snapshot ${VMS_NAME} restore ${VMS_NAME}<br> > + VBoxManage startvm ${VMS_NAME} --type headless<br> > +<br> > +vms_test_%:<br> > + ssh -A ${VMS_USER}@127.0.0.1 -p ${VMS_PORT} "/bin/bash -c \<br> <br> Should we really expose host keys into a virtual environment (-A<br> option)?<br> <br></blockquote></div><p>Fixed.</p><div id="Section20" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY19" class="mceItemAnchor"></a><br> > + '${EXTRA_ENV} \<br> > + export PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin ; \<br> > + cd tarantool && \<br> > + git fetch -p && \<br> > + git checkout --force ${CI_BUILD_REF} && \<br> > + ${GITLAB_MAKE} git_submodules_update && \<br> > + ${TRAVIS_MAKE} $(subst vms_,,$@)'"<br> > +<br> > +vms_shutdown:<br> > + VBoxManage controlvm ${VMS_NAME} poweroff<br> > +<br> > +# build packs and test<br> > +pack_clone:<br> > + git clone <a href="https://github.com/packpack/packpack.git" data-mce-href="https://checklink.mail.ru/proxy?es=BgbodWBV8Xrty7vLT3kyWXaAdzq%2Bf9vKp7BquItktoU%3D&egid=lrtF4X9qYBn3yWZH%2BWyxMGX7sk6UYEQxIi6nXsMTmCE%3D&url=https%3A%2F%2Fclick.mail.ru%2Fredir%3Fu%3Dhttps%253A%252F%252Fgithub.com%252Fpackpack%252Fpackpack.git%26c%3Dswm%26r%3Dhttp%26o%3Dmail%26v%3D2%26s%3D39e610d5261e9c7b" target="_blank">https://github.com/packpack/packpack.git</a> packpack<br> > +<br> > +pack_make:<br> > + # to avoid of issue with too long names build path is short<br> > + make -f packpack/pack/Makefile -C . BUILDDIR=/builds/pack -j<br> <br> Why do we need to split 'pack_build' into 'pack_clone' and 'pack_make'?<br> <br></blockquote></div><p>Fixed.</p><div id="Section21" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY20" class="mceItemAnchor"></a><br> > +<br> > +pack_build: pack_clone pack_make<br> > +<br> > +pack_run: git_submodules_update pack_build<br> > +<br> > +pack_run_cent6: git_submodules_update_cent6 pack_build<br> > +<br> <br> Nit: Extra empty line at the of the file.<br> <br></blockquote></div><p>Fixed.</p><div id="Section22" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY21" class="mceItemAnchor"></a><br> I would squash all pack goals into one w/o deps (just 'package') and<br> update submodules just with a command. I guess --force is not really<br> needed and the command can be the same on all OSes.<br> <br></blockquote></div><p>Once I got the broken repository some how, '--force' flag fixed it.<br> Fixed, but left update submodules call.</p><div id="Section23" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY22" class="mceItemAnchor"></a>Yep, we'll duplicate the command in +vms_test_%, but the structure of<br> the file will be simpler and one can read a certain sequince of commands<br> w/o look into dependent goals tree.<br> <br></blockquote></div><p>I don't think so, once I already got this issue with this wrong command and spent<br> time to check all places where it was used, better to use the standalone target.</p><div id="Section24" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY23" class="mceItemAnchor"></a><br> > diff --git a/.travis.mk b/.travis.mk<br> > index 6d0c42207..7d61c218b 100644<br> > --- a/.travis.mk<br> > +++ b/.travis.mk<br> > @@ -3,6 +3,7 @@<br> > #<br> > <br> > DOCKER_IMAGE?=packpack/packpack:debian-stretch<br> > +NO_PARALLEL_TESTING?=<br> <br> I would name it TEST_RUN_EXTRA_PARAMS (we already have<br> CMAKE_EXTRA_PARAMS).<br> <br></blockquote></div><p>Fixed.</p><div id="Section25" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY24" class="mceItemAnchor"></a><br> > <br> > all: package<br> > <br> > @@ -33,8 +34,12 @@ docker_%:<br> > ${DOCKER_IMAGE} \<br> > make -f .travis.mk $(subst docker_,,$@)<br> > <br> > +#########<br> > +# Linux #<br> > +#########<br> > +<br> > deps_ubuntu:<br> > - sudo apt-get update && apt-get install -y -f \<br> > + apt-get -y update && apt-get install -y -f \<br> > build-essential cmake coreutils sed \<br> > libreadline-dev libncurses5-dev libyaml-dev libssl-dev \<br> > libcurl4-openssl-dev libunwind-dev libicu-dev \<br> > @@ -42,34 +47,29 @@ deps_ubuntu:<br> > python-msgpack python-yaml python-argparse python-six python-gevent \<br> > lcov ruby clang llvm llvm-dev<br> > <br> > -test_ubuntu: deps_ubuntu<br> > +deps_buster_clang_8:<br> > + echo "deb <a href="http://apt.llvm.org/buster/" data-mce-href="https://checklink.mail.ru/proxy?es=BgbodWBV8Xrty7vLT3kyWXaAdzq%2Bf9vKp7BquItktoU%3D&egid=lrtF4X9qYBn3yWZH%2BWyxMGX7sk6UYEQxIi6nXsMTmCE%3D&url=https%3A%2F%2Fclick.mail.ru%2Fredir%3Fu%3Dhttp%253A%252F%252Fapt.llvm.org%252Fbuster%252F%26c%3Dswm%26r%3Dhttp%26o%3Dmail%26v%3D2%26s%3Db404efecf9ea5042" target="_blank">http://apt.llvm.org/buster/</a> llvm-toolchain-buster-8 main" > /etc/apt/sources.list.d/clang_8.list<br> > + echo "deb-src <a href="http://apt.llvm.org/buster/" data-mce-href="https://checklink.mail.ru/proxy?es=BgbodWBV8Xrty7vLT3kyWXaAdzq%2Bf9vKp7BquItktoU%3D&egid=lrtF4X9qYBn3yWZH%2BWyxMGX7sk6UYEQxIi6nXsMTmCE%3D&url=https%3A%2F%2Fclick.mail.ru%2Fredir%3Fu%3Dhttp%253A%252F%252Fapt.llvm.org%252Fbuster%252F%26c%3Dswm%26r%3Dhttp%26o%3Dmail%26v%3D2%26s%3Db404efecf9ea5042" target="_blank">http://apt.llvm.org/buster/</a> llvm-toolchain-buster-8 main" >> /etc/apt/sources.list.d/clang_8.list<br> > + wget -O - <a href="https://apt.llvm.org/llvm-snapshot.gpg.key" data-mce-href="https://checklink.mail.ru/proxy?es=BgbodWBV8Xrty7vLT3kyWXaAdzq%2Bf9vKp7BquItktoU%3D&egid=lrtF4X9qYBn3yWZH%2BWyxMGX7sk6UYEQxIi6nXsMTmCE%3D&url=https%3A%2F%2Fclick.mail.ru%2Fredir%3Fu%3Dhttps%253A%252F%252Fapt.llvm.org%252Fllvm-snapshot.gpg.key%26c%3Dswm%26r%3Dhttp%26o%3Dmail%26v%3D2%26s%3Dc84b0496edc3c617" target="_blank">https://apt.llvm.org/llvm-snapshot.gpg.key</a> | sudo apt-key add -<br> <br> You removed 'sudo' above, but added it here. Is it the typo?</blockquote></div><p>Fixed.</p><div id="Section26" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY25" class="mceItemAnchor"></a><br> <br> > + apt-get -y update<br> > + apt-get -y install clang-8 llvm-8 llvm-8-dev<br> <br> Are llvm headers really needed? BTW, the same question re llvm-dev<br> above.<br> <br></blockquote></div><p>Fixed.</p><div id="Section27" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY26" class="mceItemAnchor"></a><br> > +<br> > +build_ubuntu:<br> > cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}<br> > - make -j8<br> > - cd test && /usr/bin/python test-run.py --force -j 1<br> > + make -j<br> > <br> > -deps_osx:<br> > - brew update<br> > - brew install openssl readline curl icu4c --force<br> > - python2 -V || brew install python2 --force<br> > - curl --silent --show-error --retry 5 <a href="https://bootstrap.pypa.io/get-pip.py" data-mce-href="https://checklink.mail.ru/proxy?es=BgbodWBV8Xrty7vLT3kyWXaAdzq%2Bf9vKp7BquItktoU%3D&egid=lrtF4X9qYBn3yWZH%2BWyxMGX7sk6UYEQxIi6nXsMTmCE%3D&url=https%3A%2F%2Fclick.mail.ru%2Fredir%3Fu%3Dhttps%253A%252F%252Fbootstrap.pypa.io%252Fget-pip.py%26c%3Dswm%26r%3Dhttp%26o%3Dmail%26v%3D2%26s%3D2e0ac1293f8b25a3" target="_blank">https://bootstrap.pypa.io/get-pip.py</a> | python<br> > - pip install -r test-run/requirements.txt<br> > +runtest_ubuntu: build_ubuntu<br> > + cd test && /usr/bin/python test-run.py --force $(NO_PARALLEL_TESTING)<br> > <br> > -test_osx: deps_osx<br> > - cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}<br> > - # Increase the maximum number of open file descriptors on macOS<br> > - sudo sysctl -w kern.maxfiles=20480 || :<br> > - sudo sysctl -w kern.maxfilesperproc=20480 || :<br> > - sudo launchctl limit maxfiles 20480 || :<br> > - ulimit -S -n 20480 || :<br> > - ulimit -n<br> > - make -j8<br> > - cd test && ./test-run.py --force -j 1 unit/ app/ app-tap/ box/ box-tap/<br> > +test_ubuntu: deps_ubuntu runtest_ubuntu<br> > <br> > -coverage_ubuntu: deps_ubuntu<br> > +build_cov_ubuntu:<br> <br> I would use the word 'coverage' everywhere instead of 'coverage' and<br> 'cov' here and there.<br> <br></blockquote></div><p>Fixed.</p><div id="Section28" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY27" class="mceItemAnchor"></a><br> > cmake . -DCMAKE_BUILD_TYPE=Debug -DENABLE_GCOV=ON<br> > - make -j8<br> > + make -j<br> > +<br> > +runtest_cov_ubuntu: build_cov_ubuntu<br> > # Enable --long tests for coverage<br> > - cd test && /usr/bin/python test-run.py --force -j 1 --long<br> > + cd test && /usr/bin/python test-run.py --force $(NO_PARALLEL_TESTING) --long<br> > lcov --compat-libtool --directory src/ --capture --output-file coverage.info.tmp<br> > lcov --compat-libtool --remove coverage.info.tmp 'tests/*' 'third_party/*' '/usr/*' \<br> > --output-file coverage.info<br> > @@ -81,6 +81,61 @@ coverage_ubuntu: deps_ubuntu<br> > coveralls-lcov --service-name travis-ci --service-job-id $(TRAVIS_JOB_ID) --repo-token $(COVERALLS_TOKEN) coverage.info; \<br> > fi;<br> > <br> > +coverage_ubuntu: deps_ubuntu runtest_cov_ubuntu<br> > +<br> > +#######<br> > +# OSX #<br> > +#######<br> > +<br> > +deps_osx:<br> > + brew update<br> > + brew install openssl readline curl icu4c --force<br> > + python2 -V || brew install python2 --force<br> > + curl --silent --show-error --retry 5 <a href="https://bootstrap.pypa.io/get-pip.py" data-mce-href="https://checklink.mail.ru/proxy?es=BgbodWBV8Xrty7vLT3kyWXaAdzq%2Bf9vKp7BquItktoU%3D&egid=lrtF4X9qYBn3yWZH%2BWyxMGX7sk6UYEQxIi6nXsMTmCE%3D&url=https%3A%2F%2Fclick.mail.ru%2Fredir%3Fu%3Dhttps%253A%252F%252Fbootstrap.pypa.io%252Fget-pip.py%26c%3Dswm%26r%3Dhttp%26o%3Dmail%26v%3D2%26s%3D2e0ac1293f8b25a3" target="_blank">https://bootstrap.pypa.io/get-pip.py</a> >get-pip.py<br> > + python get-pip.py --user<br> > + pip install --user --ignore-installed -r test-run/requirements.txt<br> > +<br> > +build_osx:<br> > + cmake . -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_WERROR=ON ${CMAKE_EXTRA_PARAMS}<br> > + make -j<br> > +<br> > +runtest_osx: build_osx<br> > + # Increase the maximum number of open file descriptors on macOS<br> > + sudo sysctl -w kern.maxfiles=20480 || :<br> > + sudo sysctl -w kern.maxfilesperproc=20480 || :<br> > + sudo launchctl limit maxfiles 20480 || :<br> > + ulimit -S -n 20480 || :<br> > + ulimit -n<br> > + cd test && ./test-run.py --force $(NO_PARALLEL_TESTING) unit/ app/ app-tap/ box/ box-tap/<br> > +<br> > +test_osx: deps_osx runtest_osx<br> > +<br> > +###########<br> > +# FreeBSD #<br> > +###########<br> > +<br> > +deps_freebsd:<br> > + # don't use bad py27-msgpack-python<br> <br> Can you link exact problem description here or in the commit message?</blockquote></div><p>Removed, obvious message.</p><div id="Section29" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY28" class="mceItemAnchor"></a><br> <br> > + sudo pkg install -y sudo git cmake gmake gcc coreutils \<br> <br> > sudo pkg install -y sudo<br> <br> If sudo is not installed, then this command will fail. If it is<br> installed, then we don't need to add it to the package list.</blockquote></div><p>Fixed.</p><div id="Section30" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY29" class="mceItemAnchor"></a><br> <br> > + readline ncurses libyaml openssl curl libunwind icu \<br> > + python27 py27-pip py27-setuptools py27-daemon \<br> > + py27-yaml py27-argparse py27-six py27-gevent \<br> > + gdb bash vim<br> <br> Installing vim in a testing environment is the strange thing.</blockquote></div><p>Fixed.</p><div id="Section31" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY30" class="mceItemAnchor"></a><br> <br> > +<br> > +build_freebsd:<br> > + cmake .<br> <br> I would explicitly set a build type (I guess it worth to test<br> RelWithDebInfo), set -DENABLE_WERROR=ON and pass ${CMAKE_EXTRA_PARAMS}<br> here (even despite we don't use it right now).</blockquote></div><p>Fixed.</p><div id="Section32" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY31" class="mceItemAnchor"></a><br> <br> > + gmake<br> <br> Use -j as in other make invocations?</blockquote></div><p>Fixed.</p><div id="Section33" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY32" class="mceItemAnchor"></a><br> <br> > + sudo make install<br> <br> Why do we need to install tarantool into a system? test-run able to<br> detect in-source build.<br> <br></blockquote></div><p>Fixed.</p><div id="Section34" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY33" class="mceItemAnchor"></a><br> > +<br> > +runtest_freebsd: build_freebsd<br> > + cd test && /usr/bin/python test-run.py --force $(NO_PARALLEL_TESTING) || true<br> <br> Volkswagen mode?</blockquote></div><p>Currently FreeBSD testing is not ready and will fail pipeline, the issue in github exists,<br> but we really need the test result to check the new fails.</p><div id="Section35" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY34" class="mceItemAnchor"></a><br> <br> > +<br> > +test_freebsd: deps_freebsd runtest_freebsd<br> > +<br> > +###################<br> > +# Sources & Packs #<br> > +###################<br> <br> I would say 'Source tarballs', because we have no such separate things<br> as 'source' and 'pack'. I mean, it is possible to undestand the header<br> in a wrong way.<br> <br></blockquote></div><p>Fixed.</p><div id="Section36" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY35" class="mceItemAnchor"></a><br> > +<br> > source:<br> > git clone <a href="https://github.com/packpack/packpack.git" data-mce-href="https://checklink.mail.ru/proxy?es=BgbodWBV8Xrty7vLT3kyWXaAdzq%2Bf9vKp7BquItktoU%3D&egid=lrtF4X9qYBn3yWZH%2BWyxMGX7sk6UYEQxIi6nXsMTmCE%3D&url=https%3A%2F%2Fclick.mail.ru%2Fredir%3Fu%3Dhttps%253A%252F%252Fgithub.com%252Fpackpack%252Fpackpack.git%26c%3Dswm%26r%3Dhttp%26o%3Dmail%26v%3D2%26s%3D39e610d5261e9c7b" target="_blank">https://github.com/packpack/packpack.git</a> packpack<br> > TARBALL_COMPRESSOR=gz packpack/packpack tarball<br> > diff --git a/.travis.yml b/.travis.yml<br> > index e94d02ef5..a22197845 100644<br> > --- a/.travis.yml<br> > +++ b/.travis.yml<br> > @@ -17,6 +17,9 @@ cache:<br> > git:<br> > depth: 100500<br> > <br> > +env: ><br> > + NO_PARALLEL_TESTING="-j 1"<br> > +<br> > jobs:<br> > include:<br> > # Testing targets (just run tests on Debian Stretch or OS X).<br> > diff --git a/images/Dockerfile.debian b/images/Dockerfile.debian<br> > new file mode 100644<br> > index 000000000..eb5278343<br> > --- /dev/null<br> > +++ b/images/Dockerfile.debian<br> > @@ -0,0 +1,4 @@<br> > +FROM packpack/packpack:debian-stretch<br> > +<br> > +COPY .travis.mk .<br> > +RUN make -f .travis.mk deps_ubuntu<br> > diff --git a/images/Dockerfile.lto b/images/Dockerfile.lto<br> > new file mode 100644<br> > index 000000000..54c585acc<br> > --- /dev/null<br> > +++ b/images/Dockerfile.lto<br> > @@ -0,0 +1,5 @@<br> > +FROM packpack/packpack:debian-buster<br> > +<br> > +COPY .travis.mk .<br> > +RUN make -f .travis.mk deps_ubuntu<br> > +RUN make -f .travis.mk deps_buster_clang_8<br> <br> I guess we can remove these files, se the comment to 'docker_bootstrap'<br> goal above.<br> <br></blockquote></div><p>Fixed.</p><div id="Section37" dir="ltr"><blockquote><a name="style_15608560881683133890_BODY36" class="mceItemAnchor"></a><br> <br></blockquote></div><p><br> <br> -- <br> Alexander Tikhonov<br> <br></p></BODY></HTML>