From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <tarantool-patches-bounces@dev.tarantool.org>
Received: from [87.239.111.99] (localhost [127.0.0.1])
	by dev.tarantool.org (Postfix) with ESMTP id 556D46BD29;
	Tue, 13 Apr 2021 15:47:36 +0300 (MSK)
DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 556D46BD29
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev;
	t=1618318056; bh=K0ldcY0UvvYvxx54sS++DQiBqrVjqKKLHA1dyPAlmsE=;
	h=In-Reply-To:Date:Cc:References:To:Subject:List-Id:
	 List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe:
	 From:Reply-To:From;
	b=jmI+yj34aIGJ5exPgziu8Ghw6uhawdAs81+cXS/3H3O/EM5qGMkZROylms3sNJ4Ir
	 tgKstxzg31JQpGfhkw7/f6TJ4cKodMI4biVi5/xwLyAfx69pA9F8+MXmrxPQYoWIS/
	 1pgl5SpzxNtzim6NQKLIdg70mIZHngRc9pmyTMbU=
Received: from smtpng2.m.smailru.net (smtpng2.m.smailru.net [94.100.179.3])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by dev.tarantool.org (Postfix) with ESMTPS id CAA426BD29
 for <tarantool-patches@dev.tarantool.org>;
 Tue, 13 Apr 2021 15:47:27 +0300 (MSK)
DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org CAA426BD29
Received: by smtpng2.m.smailru.net with esmtpa (envelope-from
 <roman.habibov@tarantool.org>)
 id 1lWIRz-00040u-4V; Tue, 13 Apr 2021 15:47:27 +0300
Content-Type: text/plain;
	charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\))
In-Reply-To: <1e6cb78e-661f-e762-a952-a057b78bb4eb@tarantool.org>
Date: Tue, 13 Apr 2021 15:47:26 +0300
Cc: tarantool-patches@dev.tarantool.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <3176607B-0CAC-4A39-AD1C-F234994A3C9D@tarantool.org>
References: <20210314153457.85909-1-roman.habibov@tarantool.org>
 <15c6bce9-c9e0-d043-7b02-e795dc4804eb@tarantool.org>
 <FF1656A1-072D-405D-BC36-F4FE948F8631@tarantool.org>
 <1e6cb78e-661f-e762-a952-a057b78bb4eb@tarantool.org>
To: Leonid Vasiliev <lvasiliev@tarantool.org>
X-Mailer: Apple Mail (2.3608.120.23.2.1)
X-7564579A: 646B95376F6C166E
X-77F55803: 4F1203BC0FB41BD92FFCB8E6708E7480D608FE24BC85426BB1B55F651FED8C70182A05F538085040870A87FAAB8A36F18ACD52446BC9F9E27388A896C930F799E73B0EB806B73508
X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE779AAD18609327F83EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F790063767A3307017EE919D8638F802B75D45FF914D58D5BE9E6BC1A93B80C6DEB9DEE97C6FB206A91F05B25F0D1F9FB5A3087EE64B0DDCE0072C7216A3AA18C56759C2D2E47CDBA5A96583C09775C1D3CA48CFE478A468B35FE767117882F4460429724CE54428C33FAD30A8DF7F3B2552694AC26CFBAC0749D213D2E47CDBA5A9658378DA827A17800CE70F3DDF2BBF19B93A9FA2833FD35BB23DF004C906525384302BEBFE083D3B9BA73A03B725D353964B0B7D0EA88DDEDAC722CA9DD8327EE4930A3850AC1BE2E7356D8C47C27EEC5E9FB5C8C57E37DE458BEDA766A37F9254B7
X-C1DE0DAB: 0D63561A33F958A5B6B1B6F719D8034AC534ED62EBA74E588364523193510F89D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA7502E6951B79FF9A3F410CA545F18667F91A7EA1CDA0B5A7A0
X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34C6EC1BD9807109D218681B9AF4409F56F2068C03BD83F31EC7F23D7899A837B4CC36BFDFEC594D3E1D7E09C32AA3244C99CFFBBC295E622DAF3FEEADC26E1398F2F5F14F68F1805B729B2BEF169E0186
X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojnA7/qPBUIXGdg/56mX4VAA==
X-Mailru-Sender: 689FA8AB762F73936BC43F508A0638223C81D1EC16CC366775D83BF684B5E1530F96A0E8EF440D4CA021B31A5B48C7CFFB559BB5D741EB9609AE011F3211FEEF240331F90058701C67EA787935ED9F1B
X-Mras: Ok
Subject: Re: [Tarantool-patches] [PATCH] box: set cfg via environment
 variables
X-BeenThere: tarantool-patches@dev.tarantool.org
X-Mailman-Version: 2.1.34
Precedence: list
List-Id: Tarantool development patches <tarantool-patches.dev.tarantool.org>
List-Unsubscribe: <https://lists.tarantool.org/mailman/options/tarantool-patches>, 
 <mailto:tarantool-patches-request@dev.tarantool.org?subject=unsubscribe>
List-Archive: <https://lists.tarantool.org/pipermail/tarantool-patches/>
List-Post: <mailto:tarantool-patches@dev.tarantool.org>
List-Help: <mailto:tarantool-patches-request@dev.tarantool.org?subject=help>
List-Subscribe: <https://lists.tarantool.org/mailman/listinfo/tarantool-patches>, 
 <mailto:tarantool-patches-request@dev.tarantool.org?subject=subscribe>
From: Roman Khabibov via Tarantool-patches
 <tarantool-patches@dev.tarantool.org>
Reply-To: Roman Khabibov <roman.habibov@tarantool.org>
Errors-To: tarantool-patches-bounces@dev.tarantool.org
Sender: "Tarantool-patches" <tarantool-patches-bounces@dev.tarantool.org>

Hi! Thanks for the review.

> On Apr 12, 2021, at 11:37, Leonid Vasiliev <lvasiliev@tarantool.org> =
wrote:
>=20
> Hi! Thank you for the patch.
> See 9 comments below:
>=20
>> commit ee6525f843a347c1df8a92a015414f4cbcceafb9
>> Author: Roman Khabibov <roman.habibov@tarantool.org>
>> Date:   Tue Mar 9 16:47:11 2021 +0300
>>     box: set cfg via environment variables
>>          Add ability to set box.cfg options via env variables. These
>>     variables should have name "TT_<OPTION>". When Tarantool instance
>>     is started under tarantoolctl utility env variables options have
>>     more priority than instance file's cfg.
>>          Closes #5602
>>          @TarantoolBot document
>>     Title: Set box.cfg options via env variables
>>          Now, it is possible to set box.cfg options via environment
>>     variables. The name of variable should correspond the following
>>     pattern: "TT_<NAME>", where <NAME> is an option name. For =
example:
>>     "TT_LISTEN", "TT_READAHEAD".
>>     If it is complex value, for example, "replication" can be set by =
a
>>     table, the corresponding environment variable value has the =
syntax
>>     of lua array (without keys):
>>          `export TT_REPLICATION=3D"{'uri1', 'uri2'}"`
>> diff --git a/changelogs/unreleased/environment-cfg.md =
b/changelogs/unreleased/environment-cfg.md
>> new file mode 100755
>> index 000000000..c5d5005ac
>> --- /dev/null
>> +++ b/changelogs/unreleased/environment-cfg.md
>> @@ -0,0 +1,5 @@
>> +## feature/core
>> +
>> +* Now, it is possibly to set box.cfg options with environment =
variables.
>> +The priority of sources of configuration options is the following =
(from low
>> +to high): default, tarantoolctl, environment, box.cfg{}. (gh-5602).
>> \ No newline at end of file
>> diff --git a/src/box/lua/load_cfg.lua b/src/box/lua/load_cfg.lua
>> index 574c8bef4..a1439ae46 100644
>> --- a/src/box/lua/load_cfg.lua
>> +++ b/src/box/lua/load_cfg.lua
>> @@ -416,6 +416,20 @@ local translate_cfg =3D {
>>      end},
>>  }
>>  +-- Configuration options from environment is taken into account
>> +-- only at the first box.cfg{} call. When box.cfg is reload, they
>> +-- are ignored.
>> +--
>> +-- The priority of sources of configuration options is the
>> +-- following (from low to high): default, tarantoolctl,
>> +-- environment, box.cfg{}.
>> +local env_cfg =3D {}
>> +-- This is auxiliary table and it needed to keep values which
>> +-- passed or typed to box.cfg{} at the first call. Then we
>> +-- distinguish values (it was from environment or not) and decide
>> +-- ignore it or not during reload.
>> +local user_cfg =3D {}
>=20
> 1) AFAIU we decided what it's not needed. In my mind, on reload, we =
have
> no difference between the options that were set using environment and =
any
> other way.
Removed.

>> +
>>  -- Upgrade old config
>>  local function upgrade_cfg(cfg, translate_cfg)
>>      if cfg =3D=3D nil then
>> @@ -515,6 +529,22 @@ local function prepare_cfg(cfg, default_cfg, =
template_cfg,
>>      return new_cfg
>>  end
>>  +---
>> +-- Transfer options from env_cfg to cfg.
>> +--
>> +local function apply_env_cfg(cfg, env_cfg)
>> +    if env_cfg =3D=3D nil then
>> +        return
>> +    end
>> +    for k, v in pairs(env_cfg) do
>> +        if cfg[k] =3D=3D nil or os.getenv('TARANTOOLCTL') =3D=3D =
'true' then
>=20
> 2) Add a comment describing the condition `os.getenv('TARANTOOLCTL') =
=3D=3D 'true'`.
+---
+-- Transfer options from env_cfg to cfg.
+--
+local function apply_env_cfg(cfg, env_cfg)
+    if env_cfg =3D=3D nil then
+        return
+    end
+
+    for k, v in pairs(env_cfg) do
+        -- If Tarantool started with tarantoolctl then we have to
+        -- ignore configuration from tarantoolctl, because they
+        -- has priority lower than environment variables.
+        if cfg[k] =3D=3D nil or os.getenv('TARANTOOLCTL') =3D=3D 'true' =
then
+            cfg[k] =3D v
+        elseif type(v) =3D=3D 'table' then
+            apply_env_cfg(cfg[k], v)
+        end
+    end
+end
+

>> +            cfg[k] =3D v
>> +        elseif type(v) =3D=3D 'table' then
>> +            apply_env_cfg(cfg[k], v)
>> +        end
>> +    end
>> +end
>> +
>>  local function apply_default_cfg(cfg, default_cfg, module_cfg)
>>      for k,v in pairs(default_cfg) do
>>          if cfg[k] =3D=3D nil then
>> @@ -549,7 +579,19 @@ local function compare_cfg(cfg1, cfg2)
>>      return true
>>  end
>>  +---
>> +-- Nullify option values from environment in old configuration.
>> +--
>> +local function reset_env_options(oldcfg)
>=20
> 3) Why should we nullify option values from environment in old
> configuration?
>=20
>> +    for k, v in pairs(env_cfg) do
>> +        if user_cfg[k] =3D=3D nil then
>> +            oldcfg[k] =3D nil
>> +        end
>> +    end
>> +end
>> +
>>  local function reload_cfg(oldcfg, cfg)
>> +    reset_env_options(oldcfg)
>>      cfg =3D upgrade_cfg(cfg, translate_cfg)
>>      local newcfg =3D prepare_cfg(cfg, default_cfg, template_cfg,
>>                                 module_cfg, modify_cfg)
>> @@ -616,6 +658,104 @@ setmetatable(box, {
>>       end
>>  })
>>  +---
>> +-- Erase quotes at the begin and the end of string.
>> +--
>> +local function dequote(str)
>> +    if string.sub(str, 1, 1) =3D=3D '\'' and string.sub(str, -1) =3D=3D=
 '\'' or
>> +       string.sub(str, 1, 1) =3D=3D '"' and string.sub(str, -1) =3D=3D=
 '"' then
>> +        return string.sub(str, 2, -2)
>> +    end
>> +    return str
>> +end
>> +
>> +---
>> +-- Convert raw option value becoming from the corresponding
>> +-- environment variable represented as string to string, number or
>> +-- boolean (depending on option).
>> +--
>> +local function process_option_value(option, raw_value)
>> +    local param_type =3D template_cfg[option]
>> +
>> +    -- May be peculiar to "feedback*" parameters.
>> +    if param_type =3D=3D nil then
>> +        return nil
>> +    end
>> +
>> +    local value =3D nil
>> +    local err_msg_fmt =3D 'Environment variable TT_%s has '..
>=20
> 4) Add a space before `..`.
Done.

>> +        'incorrect value for option \'%s\': should be %s'
>> +
>> +    -- "log_level" option should be a number.
>> +    if param_type =3D=3D 'number' or option =3D=3D 'log_level' then
>=20
> 5) As for me it's strange, that in =
a94a9b3fd629b5abbc99a23ba527cd91e5c6ebf5 Cyrill Gorcunov
> moves the type check to a module, but you (in fact) return it back
> (determine the type by the parameter name). I think you need to =
discuss
> this question with Cyrill and check types either inside the module or =
in the `load_cfg.lua`.
Done. See the first patch.

>> +        value =3D tonumber(raw_value)
>> +        if value =3D=3D nil then
>> +            error(err_msg_fmt:format(string.upper(option), option,
>> +                    "convertible to number"))
>> +        end
>> +    -- "log_nonblock" option should be boolean.
>> +    elseif param_type =3D=3D 'boolean' or option =3D=3D =
'log_nonblock' then
>> +        if raw_value =3D=3D 'true' then
>> +            value =3D true
>> +        elseif raw_value =3D=3D 'false' then
>> +            value =3D false
>> +        else
>> +            error(err_msg_fmt:format(string.upper(option), option,
>> +                    "'true' or 'false'"))
>> +        end
>> +    elseif param_type =3D=3D 'string, number' then
>=20
> 6) Since some options can have several types, I think it's more
> comfortable for such options to use a table listing all possible =
types.
> But such refactoring doesn't seem to be mandatory for the patch.
>=20
>> +        -- Firstly, try to convert to number.
>> +        value =3D tonumber(raw_value)
>> +        if value =3D=3D nil then
>> +            value =3D raw_value
>> +        end
>> +    elseif param_type =3D=3D 'string, number, table' then
>> +        if string.sub(raw_value, 1, 1) =3D=3D '{' and
>> +           string.sub(raw_value, -1) =3D=3D '}' then
>> +            value =3D string.sub(raw_value, 2, -2)
>> +            value =3D require('csv').load(value)[1]
>> +            for i, item in pairs(value) do
>> +                value[i] =3D dequote(item)
>=20
> 7) In this case, the number inside the table will be parsed as a
> "string". We don't want that.
+---
+-- Convert raw option value becoming from the corresponding
+-- environment variable represented as string to string, number or
+-- boolean (depending on option).
+--
+local function process_option_value(option, raw_value)
+    local param_type =3D template_cfg[option]
+
+    -- May be peculiar to "feedback*" parameters.
+    if param_type =3D=3D nil then
+        return nil
+    end
+
+    local value =3D nil
+    local err_msg_fmt =3D 'Environment variable TT_%s has ' ..
+        'incorrect value for option \'%s\': should be %s'
+
+    -- "log_level" option should be a number.
+    if param_type =3D=3D 'number' then
+        value =3D tonumber(raw_value)
+        if value =3D=3D nil then
+            error(err_msg_fmt:format(string.upper(option), option,
+                    "convertible to number"))
+        end
+    -- "log_nonblock" option should be boolean.
+    elseif param_type =3D=3D 'boolean' then
+        if raw_value =3D=3D 'true' then
+            value =3D true
+        elseif raw_value =3D=3D 'false' then
+            value =3D false
+        else
+            error(err_msg_fmt:format(string.upper(option), option,
+                    "'true' or 'false'"))
+        end
+    elseif param_type =3D=3D 'string, number' then
+        -- Firstly, try to convert to number.
+        value =3D tonumber(raw_value)
+        if value =3D=3D nil then
+            value =3D raw_value
+        end
+    elseif param_type =3D=3D 'string, number, table' then
+        if string.sub(raw_value, 1, 1) =3D=3D '{' and
+           string.sub(raw_value, -1) =3D=3D '}' then
+            value =3D string.sub(raw_value, 2, -2)
+            value =3D require('csv').load(value)[1]
+            for i, item in pairs(value) do
+                item =3D dequote(item)
+                value[i] =3D tonumber(item)
+                if value[i] =3D=3D nil then
+                    value[i] =3D item
+                end
+            end
+        else
+            value =3D tonumber(raw_value)
+            if value =3D=3D nil then
+                value =3D raw_value
+            end
+        end
+    else
+        value =3D raw_value
+    end
+
+    return value
+end
+

>> +            end
>> +        else
>> +            value =3D tonumber(raw_value)
>> +            if value =3D=3D nil then
>> +                value =3D raw_value
>> +            end
>> +        end
>> +    else
>> +        value =3D raw_value
>> +    end
>> +
>> +    return value
>> +end
>> +
>> +-- local function collect_env_cfg()
>> +--     local env_cfg =3D {}
>> +--     for option, _ in pairs(template_cfg) do
>> +--         if cfg[option] =3D=3D nil or os.getenv('TARANTOOLCTL') =3D=3D=
 'true' then
>> +--             local env_var_name =3D 'TT_'..string.upper(option) > =
+--             local raw_value =3D os.getenv(env_var_name)
>> +--             if raw_value ~=3D nil then
>> +--                 cfg[option] =3D process_option_value(option, =
raw_value)
>> +--             end
>> +--         end
>> +--     end
>> +-- end
>=20
> 8) Don't leave commented code after work.
Removed.

>> +
>> +---
>> +-- Pull and process configuration values from environment.
>> +--
>> +local function collect_env_cfg()
>> +    for option, _ in pairs(template_cfg) do
>> +        local env_var_name =3D 'TT_'..string.upper(option)
>=20
> 9) Add spaces around the `..`
Done.

>> +        local raw_value =3D os.getenv(env_var_name)
>> +        if raw_value ~=3D nil then
>> +            env_cfg[option] =3D process_option_value(option, =
raw_value)
>> +        end
>> +    end
>> +end
>> +
>>  -- Whether box is loaded.
>>  --
>>  -- `false` when box is not configured or when the initialization
>> @@ -627,6 +767,7 @@ setmetatable(box, {
>>  local box_is_configured =3D false
>>    local function load_cfg(cfg)
>> +    user_cfg =3D cfg
>>      -- A user may save box.cfg (this function) before box loading
>>      -- and call it afterwards. We should reconfigure box in the
>>      -- case.
>> @@ -638,6 +779,8 @@ local function load_cfg(cfg)
>>      cfg =3D upgrade_cfg(cfg, translate_cfg)
>>      cfg =3D prepare_cfg(cfg, default_cfg, template_cfg,
>>                        module_cfg, modify_cfg)
>> +    collect_env_cfg()
>> +    apply_env_cfg(cfg, env_cfg)
>>      apply_default_cfg(cfg, default_cfg, module_cfg);
>>      -- Save new box.cfg
>>      box.cfg =3D cfg
>> diff --git a/test/app-tap/tarantoolctl.test.lua =
b/test/app-tap/tarantoolctl.test.lua
>> index 3c3023d34..060316714 100755
>> --- a/test/app-tap/tarantoolctl.test.lua
>> +++ b/test/app-tap/tarantoolctl.test.lua
>> @@ -160,7 +160,7 @@ local function merge(...)
>>  end
>>    local test =3D tap.test('tarantoolctl')
>> -test:plan(8)
>> +test:plan(9)
>>    -- basic start/stop test
>>  -- must be stopped afterwards
>> @@ -632,4 +632,35 @@ test:test('filter_xlog', function(test)
>>      end
>>  end)
>>  +-- gh-5602: Check that environment cfg variables are more
>> +-- prioritized than tarantoolctl cfg.
>> +do
>> +    local dir =3D fio.tempdir()
>> +    local code =3D [[ box.cfg ]]
>> +    create_script(dir, 'show_cfg.lua', code)
>> +    local code =3D [[ box.cfg{checkpoint_interval=3D3302, =
readahead=3D10001} ]]
>> +    create_script(dir, 'cfg_script.lua', code)
>> +
>> +    local status, err =3D pcall(function()
>> +        test:test("check answers in case of call", function(test_i)
>> +            test_i:plan(4)
>> +            os.setenv('TT_CHECKPOINT_INTERVAL', '3301')
>> +            os.setenv('TT_READAHEAD', '10000')
>> +            check_ok(test_i, dir, 'start', 'cfg_script', 0)
>> +            tctl_wait_start(dir, 'cfg_script')
>> +            check_ok(test_i, dir, 'eval',  'cfg_script =
show_cfg.lua', 0,
>> +                     '---\n- 3301\n- 10000\n...', nil)
>> +            check_ok(test_i, dir, 'stop', 'cfg_script', 0)
>> +        end)
>> +    end)
>> +
>> +    cleanup_instance(dir, 'cfg_script')
>> +    recursive_rmdir(dir)
>> +
>> +    if status =3D=3D false then
>> +        print(("Error: %s"):format(err))
>> +        os.exit()
>> +    end
>> +end
>> +
>>  os.exit(test:check() =3D=3D true and 0 or -1)
>> diff --git a/test/box-tap/gh-5602-environment-vars-cfg.result =
b/test/box-tap/gh-5602-environment-vars-cfg.result
>> new file mode 100644
>> index 000000000..c90dd301b
>> --- /dev/null
>> +++ b/test/box-tap/gh-5602-environment-vars-cfg.result
>> @@ -0,0 +1,41 @@
>> +TAP version 13
>> +1..5
>> +ok - box.cfg is successful
>> +ok - listen
>> +ok - readahead
>> +ok - strip_core
>> +ok - log_format is not set
>> +TAP version 13
>> +1..7
>> +ok - box.cfg is successful
>> +ok - listen
>> +ok - replication is table
>> +ok - replication URI 1
>> +ok - replication URI 2
>> +ok - replication_connect_timeout
>> +ok - replication_synchro_quorum
>> +TAP version 13
>> +1..3
>> +ok - box.cfg is successful
>> +ok - box.cfg{} background value is prioritized
>> +ok - box.cfg{} vinyl_timeout value is prioritized
>> +TAP version 13
>> +1..2
>> +ok - box.cfg is not successful
>> +ok - bad sql_cache_size value
>> +TAP version 13
>> +1..2
>> +ok - box.cfg is not successful
>> +ok - bad strip_core value
>> +TAP version 13
>> +1..2
>> +ok - box.cfg is not successful
>> +ok - bad replication value
>> +TAP version 13
>> +1..6
>> +ok - set 1
>> +ok - set 2
>> +ok - set 3
>> +ok - set 4
>> +ok - set 5
>> +ok - set 6
>> diff --git a/test/box-tap/gh-5602-environment-vars-cfg.test.lua =
b/test/box-tap/gh-5602-environment-vars-cfg.test.lua
>> new file mode 100755
>> index 000000000..3d19fdc90
>> --- /dev/null
>> +++ b/test/box-tap/gh-5602-environment-vars-cfg.test.lua
>> @@ -0,0 +1,57 @@
>> +#!/usr/bin/env tarantool
>> +
>> +local os =3D require('os')
>> +local fio =3D require('fio')
>> +local tap =3D require('tap')
>> +
>> +local test =3D tap.test('gh-5602')
>> +
>> +-- gh-5602: Check that environment cfg variables working.
>> +local TARANTOOL_PATH =3D arg[-1]
>> +local script_name =3D 'gh-5602-test-cases-script.lua'
>> +local path_to_script =3D fio.pathjoin(
>> +        os.getenv('PWD'),
>> +        'box-tap',
>> +        script_name)
>> +
>> +---
>> +-- Generate shell command where set is sequence of environment
>> +-- variables, then path to tarantool, the first arg is path to
>> +-- test script and the second one is number of set.
>> +--
>> +local function shell_command(set, i)
>> +    return ('%s %s %s %d'):format(
>> +        set,
>> +        TARANTOOL_PATH,
>> +        path_to_script,
>> +        i)
>> +end
>> +
>> +local set_1 =3D ('%s %s %s %s'):format('TT_LISTEN=3D3301', =
'TT_READAHEAD=3D10000',
>> +    'TT_STRIP_CORE=3Dfalse', 'TT_LOG_FORMAT=3Djson')
>> +local set_2 =3D ('%s %s %s %s'):format('TT_LISTEN=3D3301',
>> +    'TT_REPLICATION=3D\"{\'0.0.0.0:12345\', \'1.1.1.1:12345\'}\"',
>> +    'TT_REPLICATION_CONNECT_TIMEOUT=3D0.01',
>> +    'TT_REPLICATION_SYNCHRO_QUORUM=3D\'4 + 1\'')
>> +local set_3 =3D 'TT_BACKGROUND=3Dtrue TT_VINYL_TIMEOUT=3D60.1'
>> +local set_4 =3D 'TT_SQL_CACHE_SIZE=3Da'
>> +local set_5 =3D 'TT_STRIP_CORE=3Da'
>> +local set_6 =3D 'TT_REPLICATION=3D\"{0.0.0.0:12345\', =
\'1.1.1.1:12345\'}\"'
>> +
>> +local sets =3D {}
>> +sets[1] =3D set_1
>> +sets[2] =3D set_2
>> +sets[3] =3D set_3
>> +sets[4] =3D set_4
>> +sets[5] =3D set_5
>> +sets[6] =3D set_6
>> +
>> +test:plan(#sets)
>> +for i, set in ipairs(sets) do
>> +    local tmpdir =3D fio.tempdir()
>> +    local new_path =3D fio.pathjoin(tmpdir, script_name)
>> +    fio.copyfile(path_to_script, new_path)
>> +    test:is(os.execute(shell_command(set, i)), 0, 'set '..i)
>> +end
>> +
>> +os.exit(test:check() and 0 or 1)
>> diff --git a/test/box-tap/gh-5602-test-cases-script.lua =
b/test/box-tap/gh-5602-test-cases-script.lua
>> new file mode 100755
>> index 000000000..a8b2fc987
>> --- /dev/null
>> +++ b/test/box-tap/gh-5602-test-cases-script.lua
>> @@ -0,0 +1,69 @@
>> +local tap =3D require('tap')
>> +
>> +local test =3D tap.test('gh-5602')
>> +
>> +local status, err =3D pcall(box.cfg, {background =3D false, =
vinyl_timeout =3D 70.1})
>> +
>> +-- Check that environment cfg values are set correctly.
>> +if arg[1] =3D=3D '1' then
>> +    test:plan(5)
>> +    test:ok(status ~=3D false, 'box.cfg is successful')
>> +    test:is(box.cfg['listen'], 3301, 'listen')
>> +    test:is(box.cfg['readahead'], 10000, 'readahead')
>> +    test:is(box.cfg['strip_core'], false, 'strip_core')
>> +    test:is(box.cfg['log_format'], 'json', 'log_format is not set')
>> +end
>> +if arg[1] =3D=3D '2' then
>> +    test:plan(7)
>> +    test:ok(status ~=3D false, 'box.cfg is successful')
>> +    test:is(box.cfg['listen'], 3301, 'listen')
>> +    local replication =3D box.cfg['replication']
>> +    test:is(type(replication), 'table', 'replication is table')
>> +    test:ok(replication[1] =3D=3D '0.0.0.0:12345' or
>> +            replication[1] =3D=3D '1.1.1.1:12345', 'replication URI =
1')
>> +    test:ok(replication[2] =3D=3D '0.0.0.0:12345' or
>> +            replication[2] =3D=3D '1.1.1.1:12345', 'replication URI =
2')
>> +    test:is(box.cfg['replication_connect_timeout'], 0.01,
>> +            'replication_connect_timeout')
>> +    test:is(box.cfg['replication_synchro_quorum'], '4 + 1',
>> +            'replication_synchro_quorum')
>> +end
>> +
>> +-- Check that box.cfg{} values are more prioritized than
>> +-- environment cfg values.
>> +if arg[1] =3D=3D '3' then
>> +    test:plan(3)
>> +    test:ok(status ~=3D false, 'box.cfg is successful')
>> +    test:is(box.cfg['background'], false,
>> +            'box.cfg{} background value is prioritized')
>> +    test:is(box.cfg['vinyl_timeout'], 70.1,
>> +            'box.cfg{} vinyl_timeout value is prioritized')
>> +end
>> +
>> +local err_msg_fmt =3D 'Environment variable TT_%s has '..
>> +    'incorrect value for option \'%s\': should be %s'
>> +
>> +-- Check bad environment cfg values.
>> +if arg[1] =3D=3D '4' then
>> +    test:plan(2)
>> +    test:ok(status =3D=3D false, 'box.cfg is not successful')
>> +    local index =3D string.find(err, =
err_msg_fmt:format('SQL_CACHE_SIZE',
>> +        'sql_cache_size', 'convertible to number'))
>> +    test:ok(index ~=3D nil, 'bad sql_cache_size value')
>> +end
>> +if arg[1] =3D=3D '5' then
>> +    test:plan(2)
>> +    test:ok(status =3D=3D false, 'box.cfg is not successful')
>> +    local index =3D string.find(err, =
err_msg_fmt:format('STRIP_CORE',
>> +        'strip_core', '\'true\' or \'false\''))
>> +    test:ok(index ~=3D nil, 'bad strip_core value')
>> +end
>> +if arg[1] =3D=3D '6' then
>> +    test:plan(2)
>> +    test:ok(status =3D=3D false, 'box.cfg is not successful')
>> +    local index =3D string.find(tostring(err), 'Incorrect value for =
option '..
>> +        '\'replication\': expected host:service or /unix.socket')
>> +    test:ok(index ~=3D nil, 'bad replication value')
>> +end
>> +
>> +os.exit(test:check() and 0 or 1)

box: set cfg via environment variables

Add ability to set box.cfg options via env variables. These
variables should have name "TT_<OPTION>". When Tarantool instance
is started under tarantoolctl utility env variables options have
more priority than instance file's cfg.

Closes #5602

@TarantoolBot document
Title: Set box.cfg options via env variables

Now, it is possible to set box.cfg options via environment
variables. The name of variable should correspond the following
pattern: "TT_<NAME>", where <NAME> is an option name. For example:
"TT_LISTEN", "TT_READAHEAD".
If it is complex value, for example, "replication" can be set by a
table, the corresponding environment variable value has the syntax
of lua array (without keys):

`export TT_REPLICATION=3D"{'uri1', 'uri2'}"`
---
 changelogs/unreleased/environment-cfg.md      |   5 +
 src/box/lua/load_cfg.lua                      | 114 ++++++++++++++++++
 test/app-tap/tarantoolctl.test.lua            |  33 ++++-
 .../gh-5602-environment-vars-cfg.result       |  41 +++++++
 .../gh-5602-environment-vars-cfg.test.lua     |  57 +++++++++
 test/box-tap/gh-5602-test-cases-script.lua    |  69 +++++++++++
 6 files changed, 318 insertions(+), 1 deletion(-)
 create mode 100755 changelogs/unreleased/environment-cfg.md
 create mode 100644 test/box-tap/gh-5602-environment-vars-cfg.result
 create mode 100755 test/box-tap/gh-5602-environment-vars-cfg.test.lua
 create mode 100755 test/box-tap/gh-5602-test-cases-script.lua

diff --git a/changelogs/unreleased/environment-cfg.md =
b/changelogs/unreleased/environment-cfg.md
new file mode 100755
index 000000000..894c40bf6
--- /dev/null
+++ b/changelogs/unreleased/environment-cfg.md
@@ -0,0 +1,5 @@
+## feature/core
+
+* Now, it is possibly to set box.cfg options with environment =
variables.
+The priority of sources of configuration options is the following (from =
low
+to high): default, tarantoolctl, environment, box.cfg{}. (gh-5602).
diff --git a/src/box/lua/load_cfg.lua b/src/box/lua/load_cfg.lua
index f90ba8a9a..df6a52e94 100644
--- a/src/box/lua/load_cfg.lua
+++ b/src/box/lua/load_cfg.lua
@@ -525,6 +525,26 @@ local function prepare_cfg(cfg, default_cfg, =
template_cfg,
     return new_cfg
 end
=20
+---
+-- Transfer options from env_cfg to cfg.
+--
+local function apply_env_cfg(cfg, env_cfg)
+    if env_cfg =3D=3D nil then
+        return
+    end
+
+    for k, v in pairs(env_cfg) do
+        -- If Tarantool started with tarantoolctl then we have to
+        -- ignore configuration from tarantoolctl, because they
+        -- has priority lower than environment variables.
+        if cfg[k] =3D=3D nil or os.getenv('TARANTOOLCTL') =3D=3D 'true' =
then
+            cfg[k] =3D v
+        elseif type(v) =3D=3D 'table' then
+            apply_env_cfg(cfg[k], v)
+        end
+    end
+end
+
 local function apply_default_cfg(cfg, default_cfg, module_cfg)
     for k,v in pairs(default_cfg) do
         if cfg[k] =3D=3D nil then
@@ -647,6 +667,98 @@ local function on_initial_config()
     end
 end
=20
+---
+-- Erase quotes at the begin and the end of string.
+--
+local function dequote(str)
+    if string.sub(str, 1, 1) =3D=3D '\'' and string.sub(str, -1) =3D=3D =
'\'' or
+       string.sub(str, 1, 1) =3D=3D '"' and string.sub(str, -1) =3D=3D =
'"' then
+        return string.sub(str, 2, -2)
+    end
+    return str
+end
+
+---
+-- Convert raw option value becoming from the corresponding
+-- environment variable represented as string to string, number or
+-- boolean (depending on option).
+--
+local function process_option_value(option, raw_value)
+    local param_type =3D template_cfg[option]
+
+    -- May be peculiar to "feedback*" parameters.
+    if param_type =3D=3D nil then
+        return nil
+    end
+
+    local value =3D nil
+    local err_msg_fmt =3D 'Environment variable TT_%s has ' ..
+        'incorrect value for option \'%s\': should be %s'
+
+    -- "log_level" option should be a number.
+    if param_type =3D=3D 'number' then
+        value =3D tonumber(raw_value)
+        if value =3D=3D nil then
+            error(err_msg_fmt:format(string.upper(option), option,
+                    "convertible to number"))
+        end
+    -- "log_nonblock" option should be boolean.
+    elseif param_type =3D=3D 'boolean' then
+        if raw_value =3D=3D 'true' then
+            value =3D true
+        elseif raw_value =3D=3D 'false' then
+            value =3D false
+        else
+            error(err_msg_fmt:format(string.upper(option), option,
+                    "'true' or 'false'"))
+        end
+    elseif param_type =3D=3D 'string, number' then
+        -- Firstly, try to convert to number.
+        value =3D tonumber(raw_value)
+        if value =3D=3D nil then
+            value =3D raw_value
+        end
+    elseif param_type =3D=3D 'string, number, table' then
+        if string.sub(raw_value, 1, 1) =3D=3D '{' and
+           string.sub(raw_value, -1) =3D=3D '}' then
+            value =3D string.sub(raw_value, 2, -2)
+            value =3D require('csv').load(value)[1]
+            for i, item in pairs(value) do
+                item =3D dequote(item)
+                value[i] =3D tonumber(item)
+                if value[i] =3D=3D nil then
+                    value[i] =3D item
+                end
+            end
+        else
+            value =3D tonumber(raw_value)
+            if value =3D=3D nil then
+                value =3D raw_value
+            end
+        end
+    else
+        value =3D raw_value
+    end
+
+    return value
+end
+
+---
+-- Pull and process configuration values from environment.
+--
+local function collect_env_cfg()
+    local env_cfg =3D {}
+    for option, _ in pairs(template_cfg) do
+        local env_var_name =3D 'TT_' .. string.upper(option)
+        local raw_value =3D os.getenv(env_var_name)
+        if raw_value ~=3D nil then
+            env_cfg[option] =3D process_option_value(option, raw_value)
+        end
+    end
+
+    return env_cfg
+end
+
 -- Whether box is loaded.
 --
 -- `false` when box is not configured or when the initialization
@@ -669,6 +781,8 @@ local function load_cfg(cfg)
     cfg =3D upgrade_cfg(cfg, translate_cfg)
     cfg =3D prepare_cfg(cfg, default_cfg, template_cfg,
                       module_cfg, modify_cfg)
+    local env_cfg =3D collect_env_cfg()
+    apply_env_cfg(cfg, env_cfg)
     apply_default_cfg(cfg, default_cfg, module_cfg);
     -- Save new box.cfg
     box.cfg =3D cfg
diff --git a/test/app-tap/tarantoolctl.test.lua =
b/test/app-tap/tarantoolctl.test.lua
index 3c3023d34..060316714 100755
--- a/test/app-tap/tarantoolctl.test.lua
+++ b/test/app-tap/tarantoolctl.test.lua
@@ -160,7 +160,7 @@ local function merge(...)
 end
=20
 local test =3D tap.test('tarantoolctl')
-test:plan(8)
+test:plan(9)
=20
 -- basic start/stop test
 -- must be stopped afterwards
@@ -632,4 +632,35 @@ test:test('filter_xlog', function(test)
     end
 end)
=20
+-- gh-5602: Check that environment cfg variables are more
+-- prioritized than tarantoolctl cfg.
+do
+    local dir =3D fio.tempdir()
+    local code =3D [[ box.cfg ]]
+    create_script(dir, 'show_cfg.lua', code)
+    local code =3D [[ box.cfg{checkpoint_interval=3D3302, =
readahead=3D10001} ]]
+    create_script(dir, 'cfg_script.lua', code)
+
+    local status, err =3D pcall(function()
+        test:test("check answers in case of call", function(test_i)
+            test_i:plan(4)
+            os.setenv('TT_CHECKPOINT_INTERVAL', '3301')
+            os.setenv('TT_READAHEAD', '10000')
+            check_ok(test_i, dir, 'start', 'cfg_script', 0)
+            tctl_wait_start(dir, 'cfg_script')
+            check_ok(test_i, dir, 'eval',  'cfg_script show_cfg.lua', =
0,
+                     '---\n- 3301\n- 10000\n...', nil)
+            check_ok(test_i, dir, 'stop', 'cfg_script', 0)
+        end)
+    end)
+
+    cleanup_instance(dir, 'cfg_script')
+    recursive_rmdir(dir)
+
+    if status =3D=3D false then
+        print(("Error: %s"):format(err))
+        os.exit()
+    end
+end
+
 os.exit(test:check() =3D=3D true and 0 or -1)
diff --git a/test/box-tap/gh-5602-environment-vars-cfg.result =
b/test/box-tap/gh-5602-environment-vars-cfg.result
new file mode 100644
index 000000000..c90dd301b
--- /dev/null
+++ b/test/box-tap/gh-5602-environment-vars-cfg.result
@@ -0,0 +1,41 @@
+TAP version 13
+1..5
+ok - box.cfg is successful
+ok - listen
+ok - readahead
+ok - strip_core
+ok - log_format is not set
+TAP version 13
+1..7
+ok - box.cfg is successful
+ok - listen
+ok - replication is table
+ok - replication URI 1
+ok - replication URI 2
+ok - replication_connect_timeout
+ok - replication_synchro_quorum
+TAP version 13
+1..3
+ok - box.cfg is successful
+ok - box.cfg{} background value is prioritized
+ok - box.cfg{} vinyl_timeout value is prioritized
+TAP version 13
+1..2
+ok - box.cfg is not successful
+ok - bad sql_cache_size value
+TAP version 13
+1..2
+ok - box.cfg is not successful
+ok - bad strip_core value
+TAP version 13
+1..2
+ok - box.cfg is not successful
+ok - bad replication value
+TAP version 13
+1..6
+ok - set 1
+ok - set 2
+ok - set 3
+ok - set 4
+ok - set 5
+ok - set 6
diff --git a/test/box-tap/gh-5602-environment-vars-cfg.test.lua =
b/test/box-tap/gh-5602-environment-vars-cfg.test.lua
new file mode 100755
index 000000000..3d19fdc90
--- /dev/null
+++ b/test/box-tap/gh-5602-environment-vars-cfg.test.lua
@@ -0,0 +1,57 @@
+#!/usr/bin/env tarantool
+
+local os =3D require('os')
+local fio =3D require('fio')
+local tap =3D require('tap')
+
+local test =3D tap.test('gh-5602')
+
+-- gh-5602: Check that environment cfg variables working.
+local TARANTOOL_PATH =3D arg[-1]
+local script_name =3D 'gh-5602-test-cases-script.lua'
+local path_to_script =3D fio.pathjoin(
+        os.getenv('PWD'),
+        'box-tap',
+        script_name)
+
+---
+-- Generate shell command where set is sequence of environment
+-- variables, then path to tarantool, the first arg is path to
+-- test script and the second one is number of set.
+--
+local function shell_command(set, i)
+    return ('%s %s %s %d'):format(
+        set,
+        TARANTOOL_PATH,
+        path_to_script,
+        i)
+end
+
+local set_1 =3D ('%s %s %s %s'):format('TT_LISTEN=3D3301', =
'TT_READAHEAD=3D10000',
+    'TT_STRIP_CORE=3Dfalse', 'TT_LOG_FORMAT=3Djson')
+local set_2 =3D ('%s %s %s %s'):format('TT_LISTEN=3D3301',
+    'TT_REPLICATION=3D\"{\'0.0.0.0:12345\', \'1.1.1.1:12345\'}\"',
+    'TT_REPLICATION_CONNECT_TIMEOUT=3D0.01',
+    'TT_REPLICATION_SYNCHRO_QUORUM=3D\'4 + 1\'')
+local set_3 =3D 'TT_BACKGROUND=3Dtrue TT_VINYL_TIMEOUT=3D60.1'
+local set_4 =3D 'TT_SQL_CACHE_SIZE=3Da'
+local set_5 =3D 'TT_STRIP_CORE=3Da'
+local set_6 =3D 'TT_REPLICATION=3D\"{0.0.0.0:12345\', =
\'1.1.1.1:12345\'}\"'
+
+local sets =3D {}
+sets[1] =3D set_1
+sets[2] =3D set_2
+sets[3] =3D set_3
+sets[4] =3D set_4
+sets[5] =3D set_5
+sets[6] =3D set_6
+
+test:plan(#sets)
+for i, set in ipairs(sets) do
+    local tmpdir =3D fio.tempdir()
+    local new_path =3D fio.pathjoin(tmpdir, script_name)
+    fio.copyfile(path_to_script, new_path)
+    test:is(os.execute(shell_command(set, i)), 0, 'set '..i)
+end
+
+os.exit(test:check() and 0 or 1)
diff --git a/test/box-tap/gh-5602-test-cases-script.lua =
b/test/box-tap/gh-5602-test-cases-script.lua
new file mode 100755
index 000000000..a8b2fc987
--- /dev/null
+++ b/test/box-tap/gh-5602-test-cases-script.lua
@@ -0,0 +1,69 @@
+local tap =3D require('tap')
+
+local test =3D tap.test('gh-5602')
+
+local status, err =3D pcall(box.cfg, {background =3D false, =
vinyl_timeout =3D 70.1})
+
+-- Check that environment cfg values are set correctly.
+if arg[1] =3D=3D '1' then
+    test:plan(5)
+    test:ok(status ~=3D false, 'box.cfg is successful')
+    test:is(box.cfg['listen'], 3301, 'listen')
+    test:is(box.cfg['readahead'], 10000, 'readahead')
+    test:is(box.cfg['strip_core'], false, 'strip_core')
+    test:is(box.cfg['log_format'], 'json', 'log_format is not set')
+end
+if arg[1] =3D=3D '2' then
+    test:plan(7)
+    test:ok(status ~=3D false, 'box.cfg is successful')
+    test:is(box.cfg['listen'], 3301, 'listen')
+    local replication =3D box.cfg['replication']
+    test:is(type(replication), 'table', 'replication is table')
+    test:ok(replication[1] =3D=3D '0.0.0.0:12345' or
+            replication[1] =3D=3D '1.1.1.1:12345', 'replication URI 1')
+    test:ok(replication[2] =3D=3D '0.0.0.0:12345' or
+            replication[2] =3D=3D '1.1.1.1:12345', 'replication URI 2')
+    test:is(box.cfg['replication_connect_timeout'], 0.01,
+            'replication_connect_timeout')
+    test:is(box.cfg['replication_synchro_quorum'], '4 + 1',
+            'replication_synchro_quorum')
+end
+
+-- Check that box.cfg{} values are more prioritized than
+-- environment cfg values.
+if arg[1] =3D=3D '3' then
+    test:plan(3)
+    test:ok(status ~=3D false, 'box.cfg is successful')
+    test:is(box.cfg['background'], false,
+            'box.cfg{} background value is prioritized')
+    test:is(box.cfg['vinyl_timeout'], 70.1,
+            'box.cfg{} vinyl_timeout value is prioritized')
+end
+
+local err_msg_fmt =3D 'Environment variable TT_%s has '..
+    'incorrect value for option \'%s\': should be %s'
+
+-- Check bad environment cfg values.
+if arg[1] =3D=3D '4' then
+    test:plan(2)
+    test:ok(status =3D=3D false, 'box.cfg is not successful')
+    local index =3D string.find(err, =
err_msg_fmt:format('SQL_CACHE_SIZE',
+        'sql_cache_size', 'convertible to number'))
+    test:ok(index ~=3D nil, 'bad sql_cache_size value')
+end
+if arg[1] =3D=3D '5' then
+    test:plan(2)
+    test:ok(status =3D=3D false, 'box.cfg is not successful')
+    local index =3D string.find(err, err_msg_fmt:format('STRIP_CORE',
+        'strip_core', '\'true\' or \'false\''))
+    test:ok(index ~=3D nil, 'bad strip_core value')
+end
+if arg[1] =3D=3D '6' then
+    test:plan(2)
+    test:ok(status =3D=3D false, 'box.cfg is not successful')
+    local index =3D string.find(tostring(err), 'Incorrect value for =
option '..
+        '\'replication\': expected host:service or /unix.socket')
+    test:ok(index ~=3D nil, 'bad replication value')
+end
+
+os.exit(test:check() and 0 or 1)
--=20
2.24.3 (Apple Git-128)