From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp31.i.mail.ru (smtp31.i.mail.ru [94.100.177.91]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id BF914469710 for ; Tue, 24 Nov 2020 12:56:05 +0300 (MSK) Received: by smtp31.i.mail.ru with esmtpa (envelope-from ) id 1khV3N-0001In-0I for tarantool-patches@dev.tarantool.org; Tue, 24 Nov 2020 12:56:05 +0300 References: From: Oleg Koshovetc Message-ID: Date: Tue, 24 Nov 2020 12:55:27 +0300 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------67DE4D6BC6FFF2E9EF94014C" Content-Language: en-US Subject: [Tarantool-patches] Fwd: Re: Fwd: [PATCH v1] update_repo: fix Sources meta for modules List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: tarantool-patches@dev.tarantool.org This is a multi-part message in MIME format. --------------67DE4D6BC6FFF2E9EF94014C Content-Type: text/plain; charset="utf-8"; format="flowed" Content-Transfer-Encoding: 8bit LGTM On 20.11.2020 14:54, Alexander Tikhonov wrote: > > > > -------- Пересылаемое сообщение -------- > От кого: Alexander V. Tikhonov > Кому: Kirill Yukhin > Копия: Alexander V. Tikhonov , > tarantool-patches@dev.tarantool.org > Дата: Пятница, 6 ноября 2020, 21:04 +03:00 > Тема: [PATCH v1] update_repo: fix Sources meta for modules > Found that Sources file destroys when module uploaded without sources. > Also found that it could happen for Packages file on modules uploading > without binaries. To fix it was added additional its downloading from > S3 if in modules it was not updated and routine was not used. > --- > > Github: > https://github.com/tarantool/tarantool/tree/avtikhon/update_repo_meta_sources > >  tools/update_repo.sh | 20 ++++++++++++++------ >  1 file changed, 14 insertions(+), 6 deletions(-) > > diff --git a/tools/update_repo.sh b/tools/update_repo.sh > index 99896664d..e056f8f0e 100755 > --- a/tools/update_repo.sh > +++ b/tools/update_repo.sh > @@ -244,6 +244,16 @@ function update_deb_packfile { >      fi >  } > > +function initiate_deb_metadata { > + distssuffix=$1 > + > + # get the latest Packages/Sources files from S3 either create empty > files > + mkdir -p `dirname $distssuffix` || : > + $aws ls "$bucket_path/$distssuffix" >/dev/null 2>&1 && \ > + $aws cp "$bucket_path/$distssuffix" $distssuffix.saved || \ > + touch $distssuffix.saved > +} > + >  function update_deb_metadata { >      packpath=$1 >      packtype=$2 > @@ -251,12 +261,8 @@ function update_deb_metadata { > >      file_exists='' > > - if [ ! -f $packpath.saved ] ; then > - # get the latest Sources file from S3 either create empty file > - $aws ls "$bucket_path/$packpath" >/dev/null 2>&1 && \ > - $aws cp "$bucket_path/$packpath" $packpath.saved || \ > - touch $packpath.saved > - fi > + # reinitiate if the new suffix used, not as used before > + [ -f $packpath.saved ] || initiate_deb_metadata $packpath > >      [ "$remove" == "" ] || cp $packpath.saved $packpath > > @@ -545,6 +551,7 @@ EOF >          updated_files=0 > >          # 1(binaries). use reprepro tool to generate Packages file > + initiate_deb_metadata dists/$loop_dist/$component/binary-amd64/Packages >          for deb in $ws/$debdir/$loop_dist/$component/*/*/*.deb ; do >              [ -f $deb ] || continue >              updated_deb=0 > @@ -565,6 +572,7 @@ EOF >          done > >          # 1(sources). use reprepro tool to generate Sources file > + initiate_deb_metadata dists/$loop_dist/$component/source/Sources >          for dsc in $ws/$debdir/$loop_dist/$component/*/*/*.dsc ; do >              [ -f $dsc ] || continue >              updated_dsc=0 > -- > 2.25.1 > ------------------------------------------------------------------------ > -- > Alexander Tikhonov --------------67DE4D6BC6FFF2E9EF94014C Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: 8bit

LGTM

On 20.11.2020 14:54, Alexander Tikhonov wrote:



-------- Пересылаемое сообщение --------
От кого: Alexander V. Tikhonov <avtikhon@tarantool.org>
Кому: Kirill Yukhin <kyukhin@tarantool.org>
Копия: Alexander V. Tikhonov <avtikhon@tarantool.org>, tarantool-patches@dev.tarantool.org
Дата: Пятница, 6 ноября 2020, 21:04 +03:00
Тема: [PATCH v1] update_repo: fix Sources meta for modules
 
Found that Sources file destroys when module uploaded without sources.
Also found that it could happen for Packages file on modules uploading
without binaries. To fix it was added additional its downloading from
S3 if in modules it was not updated and routine was not used.
---

Github: https://github.com/tarantool/tarantool/tree/avtikhon/update_repo_meta_sources

 tools/update_repo.sh | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)

diff --git a/tools/update_repo.sh b/tools/update_repo.sh
index 99896664d..e056f8f0e 100755
--- a/tools/update_repo.sh
+++ b/tools/update_repo.sh
@@ -244,6 +244,16 @@ function update_deb_packfile {
     fi
 }
 
+function initiate_deb_metadata {
+ distssuffix=$1
+
+ # get the latest Packages/Sources files from S3 either create empty files
+ mkdir -p `dirname $distssuffix` || :
+ $aws ls "$bucket_path/$distssuffix" >/dev/null 2>&1 && \
+ $aws cp "$bucket_path/$distssuffix" $distssuffix.saved || \
+ touch $distssuffix.saved
+}
+
 function update_deb_metadata {
     packpath=$1
     packtype=$2
@@ -251,12 +261,8 @@ function update_deb_metadata {
 
     file_exists=''
 
- if [ ! -f $packpath.saved ] ; then
- # get the latest Sources file from S3 either create empty file
- $aws ls "$bucket_path/$packpath" >/dev/null 2>&1 && \
- $aws cp "$bucket_path/$packpath" $packpath.saved || \
- touch $packpath.saved
- fi
+ # reinitiate if the new suffix used, not as used before
+ [ -f $packpath.saved ] || initiate_deb_metadata $packpath
 
     [ "$remove" == "" ] || cp $packpath.saved $packpath
 
@@ -545,6 +551,7 @@ EOF
         updated_files=0
 
         # 1(binaries). use reprepro tool to generate Packages file
+ initiate_deb_metadata dists/$loop_dist/$component/binary-amd64/Packages
         for deb in $ws/$debdir/$loop_dist/$component/*/*/*.deb ; do
             [ -f $deb ] || continue
             updated_deb=0
@@ -565,6 +572,7 @@ EOF
         done
 
         # 1(sources). use reprepro tool to generate Sources file
+ initiate_deb_metadata dists/$loop_dist/$component/source/Sources
         for dsc in $ws/$debdir/$loop_dist/$component/*/*/*.dsc ; do
             [ -f $dsc ] || continue
             updated_dsc=0
--
2.25.1
 
 
 
--
Alexander Tikhonov
 
--------------67DE4D6BC6FFF2E9EF94014C--