Re: [Tarantool-patches] [PATCH v7] gitlab-ci: implement packing into MCS S3

["Alexander Tikhonov" <avtikhon@tarantool.org>]

[-- Attachment #1: Type: text/plain, Size: 4213 bytes --]


Alexander, thanks for your review, I've made the changes that you suggested.

>Суббота,  1 января 2020, 1:53 +03:00 от Alexander Turenko <alexander.turenko@tarantool.org>:
>
>I looked again over the GitLab CI rules and I have a couple of
>questions:
>
>- What will going on if one will fork the repository and push to fork's
>  master (or release branch) something?
>- What will going on if one will open a PR from a forked repository to
>  master / release branch of our repository?
>- What will going on if one will open a PR from our repository branch to
>  master / release branch of our repository?
>- Are there something like Travis-CI's 'cron jobs' in GitLab CI? What
>  will going on if we'll enable it someday?
>
>I asked this, because I more or less know how Travis-CI works and I
>guess GitLab CI is quite similar in those points. In brief:
>
>- Travis-CI cherry-pick's commits from a source branch to a target ones
>  for a PR (this is the difference between 'branch jobs' and 'PR jobs').
>  So 'PR jobs' are run like they are on a target branch.
>
>- Travis-CI does not set variables from 'Settings' (at least secret
>  ones) for jobs in other repositories (because of security reasons). So
>  This is applicable also for PRs to our repository from a fork. Don't
>  sure about a PR from a branch in our repository.
>
>All those cases can be differentiated using environment variable that
>Travis-CI sets or by conditions in .travis.yml that can be set for a
>stage. So it is natural to split packaging and deployment and set
>specific conditions for the deployment stage.
>
>I guess we extract a deployment to a stage in GitLab CI like in
>Travis-CI and also set necessary conditions when it should be run. Aside
>of resolving forks / PRs / cron jobs problems it would also allow to
>don't duplicate per-distro jobs in the config, I guess.
Right, to make the jobs working correct for it I've made the changes based on the links
you suggested:
https://docs.gitlab.com/ee/ci/merge_request_pipelines/#configuring-pipelines-for-merge-requests  
https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/#pipelines-for-external-pull-requests  

.deploy_only_template: &deploy_only_definition
 only:
 - master
 except:
 - external_pull_requests
 - merge_requests

.pack_only_template: &pack_only_definition
 only:
 - external_pull_requests
 - merge_requests
 - /^.*-full-ci$/

So the external_pull_requests and merge_requests will be blocked for deploying
and must be run for packing jobs.
>
>See also two comments below.
>
>WBR, Alexander Turenko.
>
>----
>
>> >>  package: git_submodule_update
>> >>  	git clone  https://github.com/packpack/packpack.git packpack
>> >>  	PACKPACK_EXTRA_DOCKER_RUN_PARAMS='--network=host' ./packpack/packpack
>> >> 
>> >> +deploy: package
>> >> +	for key in ${GPG_SECRET_KEY} ${GPG_PUBLIC_KEY} ; do \
>> >> +		echo $${key} | base64 -d | gpg --batch --import || true ; done
>> >
>> > I guess we need just secret key to signing.
>>
>> To use the secret key for signing it has to be imported into the user's environment.
>
>Again, why do you need a public key? Let's try to add just a secret key.
>If it works w/o the public one (I guess so), then remove the loop and
>just add the secret key. 
Right, public key was not really needed - removed.
>
>
>> >
>> >> +	./tools/update_repo.sh -o=${OS} -d=${DIST} \
>> >> +		-b="s3://tarantool_repo/live/${BUCKET}" build
>> >
>> >I would hide name of an S3 bucket under a CI variable. It is more about
>> >our infrastructure and it would be good to show less in the repo.
>>
>> S3 bucket name is opened and visible in the logs, there is no need to hide it and
>> to pay the attention for it, due to name is not secret at all, while the additional
>> secret values setup will be the dead code or additional manual steps.
>
>An S3 bucket name is part of underlying infrastructure, just like S3
>compliant service we use. We should be able to replace it w/o extra
>commits to the repository. Let's add a variable.
>
>Hiding is not critical and is not my main point. I would however hide
>the variable, but up to you.
Moved both bucket/dir path to Gitlab-CI variables.

-- 
Alexander Tikhonov

[-- Attachment #2: Type: text/html, Size: 5816 bytes --]