[tarantool-patches] Re: [PATCH] tarantoolctl doesn't fail when box.cfg is delayed in init script

["Maxim Melentiev" <dmarc-noreply@freelists.org> (Redacted sender "m.melentiev" for DMARC)]

> BTW, is this is enough for you in the scope of your task or we need to
> boost other tarantoolctl discussions very soon?
This should be enough to complete our task (with some limitations).

However we decided not to proceed with patching tarantoolctl right now
and provide start/stop features in rock’s binary replacing ctl for cluster apps.
Thank you for review. Please, ignore this patch request.

Best regards,
Max

From 1f53e9582ecf311323f88f9ac9b23a329072f901 Mon Sep 17 00:00:00 2001
From: Max Melentiev <m.melentiev@corp.mail.ru>
Date: Mon, 19 Aug 2019 10:35:55 +0300
Subject: [PATCH] tarantoolctl: allow to start instances with delayed box.cfg{}

`tarantoolctl start` patches box.cfg two times:
1) before the init script to set default values and enforce some others,
2) after the init script to prevent changing a pid_file in runtime.

The second patching fails if an init file does not call
box.cfg{} before it's finished. This can take a place in apps with
managed instances which receive configuration from external server.

This patch moves the second patching into the box.cfg
wrapper created during the first patching. So the second patching
is performed only after box.cfg{} was invoked, and it does not fail anymore.

However there is relatively minor flaw for applications that
invoke box.cfg{} after init script is finished:
`tarantoolctl start` goes to background only when box.cfg{} is called.
Though this is not the case for daemon management systems like systemd,
as they handle backgrounding on their side.

Fixes #4435

@TarantoolBot document
Title: tarantoolctl allows to start instances without a box.cfg{}

tarantoolctl now works for instances without box.cfg{} or
with delayed box.cfg{} call. This can be managed instances which receive
configuration from external server.

For such instances `tarantoolctl start` goes to background when
box.cfg{} is called, so it will wait until options for box.cfg are received.
However this is not the case for daemon management systems like systemd,
as they handle backgrounding on their side.
---
 extra/dist/tarantoolctl.in | 22 ++++++++++------------
 1 file changed, 10 insertions(+), 12 deletions(-)

diff --git a/extra/dist/tarantoolctl.in b/extra/dist/tarantoolctl.in
index 8adb57533..6daf866ac 100755
--- a/extra/dist/tarantoolctl.in
+++ b/extra/dist/tarantoolctl.in
@@ -483,6 +483,16 @@ local function wrapper_cfg(cfg)
         os.exit(1)
     end

+    -- Prevent overwriting pid_file in subsequent box.cfg calls.
+    local box_cfg_mt = getmetatable(box.cfg)
+    local orig_cfg_call = box_cfg_mt.__call
+    box_cfg_mt.__call = function(old_cfg, new_cfg)
+        if old_cfg.pid_file ~= nil and new_cfg ~= nil and new_cfg.pid_file ~= nil then
+            new_cfg.pid_file = old_cfg.pid_file
+        end
+        orig_cfg_call(old_cfg, new_cfg)
+    end
+
     return data
 end

@@ -547,18 +557,6 @@ local function start()
         end
         os.exit(1)
     end
-    local box_cfg_mt = getmetatable(box.cfg)
-    if box_cfg_mt == nil then
-        log.error('box.cfg() is not called in an instance file')
-        os.exit(1)
-    end
-    local old_call = box_cfg_mt.__call
-    box_cfg_mt.__call = function(old_cfg, cfg)
-        if old_cfg.pid_file ~= nil and cfg ~= nil and cfg.pid_file ~= nil then
-            cfg.pid_file = old_cfg.pid_file
-        end
-        old_call(old_cfg, cfg)
-    end
     return 0
 end

--
2.21.0

> On 19 Aug 2019, at 18:05, Alexander Turenko <alexander.turenko@tarantool.org> wrote:
> 
> So, in brief the patch can be described in the following way?
> 
> | tarantoolctl patches (wraps) box.cfg function two times: before an
> | init script to set default values from
> | /etc/{default,sysconfig}/tarantool and after the init script to
> | discard changing of a pid file during an instance work time.
> |
> | The second patching procedure fails if an instance file did not call
> | box.cfg{} before an init script ends. Other then that an instance,
> | which calls box.cfg{} in, say, a background fiber, works almost fine
> | (see below).
> |
> | The patch moves the second patching procedure into the box.cfg
> | wrapper created during the first patching. So the second patching
> | procedure is called only after box.cfg{} was invoked second or next
> | time, so it does not fails anymore as it did before.
> |
> | However there are the following relatively minor flaws for
> | applications that invoked box.cfg{} in background:
> |
> | <native tarantoolctl / systemd differences here>
> 
> I don't insist on my wording, but want to verify my understanding.
> 
> If it is so, then I don't have objections against the patch. You need
> however pass the formal process: file an issue, ask one of maintainers
> to set it to a right milestone (if it is 1.10 your business need should
> be really important), fix the issue from the commit message, help
> documentation team with documenting the new feature right when the patch
> will be merged (they maybe will ask you something, maybe don't -- but I
> suggest to monitor the tarantool/doc issue when it will be created).
> 
> BTW, is this is enough for you in the scope of your task or we need to
> boost other tarantoolctl discussions very soon?
> 
> See other comments below.
> 
> WBR, Alexander Turenko.
> 
> On Mon, Aug 19, 2019 at 12:48:22PM +0300, Maxim Melentiev wrote:
>> From cd94e49e7b23a30ed05e574eb8441b477fc1c47a Mon Sep 17 00:00:00 2001
>> From: Max Melentiev <m.melentiev@corp.mail.ru>
>> Date: Mon, 19 Aug 2019 10:35:55 +0300
>> Subject: [PATCH] tarantoolctl: allow to start instances without a box.cfg{}
> 
> It is a bit confusing: AFAIU it allows to call box.cfg{} in background
> after an init script execution. But it does not allow to miss box.cfg{}
> at all.
> 
>> 
>> Before this patch, tarantoolctl would fail if box.cfg{} was not
>> called in an instance script. It used too patch box.cfg
>> once again after init script to prevent chainging pid file.
> 
> Typo: chainging.
> 
>> 
>> This patch allows to start an instance without immediate call to
>> box.cfg{} in it. For example, managed instances which receive
>> configuration from external server.
>> 
>> @TarantoolBot document
>> Title: tarantoolctl allows to start instances without a box.cfg{}
>> 
>> tarantoolctl now works for instances without box.cfg{} or
>> with dealyed box.cfg{} call. This can be managed instances which receive
> 
> Typo: dealyed.
> 
>> configuration from external server.
>> 
>> For such instances `tarantoolctl start` goes to background when
>> box.cfg{} is called, so it will wait until options for box.cfg are received.
>> However this is not the case for daemon management systems like systemd,
>> as they handle bockgrounding on their side.
> 
> Typo: bockgrounding.
> 
>> ---
>> extra/dist/tarantoolctl.in | 21 +++++++++------------
>> 1 file changed, 9 insertions(+), 12 deletions(-)
>> 
>> diff --git a/extra/dist/tarantoolctl.in b/extra/dist/tarantoolctl.in
>> index 8adb57533..fd1abf9dc 100755
>> --- a/extra/dist/tarantoolctl.in
>> +++ b/extra/dist/tarantoolctl.in
>> @@ -483,6 +483,15 @@ local function wrapper_cfg(cfg)
>>         os.exit(1)
>>     end
>> 
>> +    local box_cfg_mt = getmetatable(box.cfg)
>> +    local orig_cfg_call = box_cfg_mt.__call
>> +    box_cfg_mt.__call = function(old_cfg, new_cfg)
>> +        if old_cfg.pid_file ~= nil and new_cfg ~= nil and new_cfg.pid_file ~= nil then
>> +            new_cfg.pid_file = old_cfg.pid_file
>> +        end
>> +        orig_cfg_call(old_cfg, new_cfg)
>> +    end
>> +
> 
> Once you are here it would be good to add a comment what this block
> does.
> 
> 
>>     return data
>> end
>> 
>> @@ -547,18 +556,6 @@ local function start()
>>         end
>>         os.exit(1)
>>     end
>> -    local box_cfg_mt = getmetatable(box.cfg)
>> -    if box_cfg_mt == nil then
>> -        log.error('box.cfg() is not called in an instance file')
>> -        os.exit(1)
>> -    end
>> -    local old_call = box_cfg_mt.__call
>> -    box_cfg_mt.__call = function(old_cfg, cfg)
>> -        if old_cfg.pid_file ~= nil and cfg ~= nil and cfg.pid_file ~= nil then
>> -            cfg.pid_file = old_cfg.pid_file
>> -        end
>> -        old_call(old_cfg, cfg)
>> -    end
>>     return 0
>> end
>> 
>> --
>> 2.21.0
>> 
>> 
>>> On 19 Aug 2019, at 10:54, Konstantin Osipov <kostja@tarantool.org> wrote:
>>> 
>>> * Max Melentiev <dmarc-noreply@freelists.org> [19/08/19 10:40]:
>>>> Before this patch tarantoolctl failed with error if box.cfg
>>>> was not called in init script because it used too patch box.cfg
>>>> once again after init script. I've changed it to patch box.cfg
>>>> second time inside wrapper_cfg.
>>> 
>>> Please explain better what this patch does, not how you achieved
>>> that.
>>> 
>>> And while you are at it, explain why we need a secondary patching
>>> of box.cfg at all. What's the point of preserving the pid file?
>>> 
>>> tarantoolctl: allow to start instances without a box.cfg{}
>>> 
>>> Before this patch, tarantoolctl would fail if box.cfg{} was not
>>> called in an instance script.
>>> 
>>> This patch allows to start an instance without box.cfg{} in it. 
>>> Such an instance:
>>> - may only be started under systemd? 
>>> - what can it do? 
>>> - ???
>>> 
>>> Please add a docboc entry, since it 's a significant behaviour
>>> change. How such an instance is used? 
>>> 
>>> Otherwise LTGM.
>>> 
>>> 
>>>> ---
>>>> extra/dist/tarantoolctl.in | 21 +++++++++------------
>>>> 1 file changed, 9 insertions(+), 12 deletions(-)
>>>> 
>>>> diff --git a/extra/dist/tarantoolctl.in b/extra/dist/tarantoolctl.in
>>>> index 8adb57533..fd1abf9dc 100755
>>>> --- a/extra/dist/tarantoolctl.in
>>>> +++ b/extra/dist/tarantoolctl.in
>>>> @@ -483,6 +483,15 @@ local function wrapper_cfg(cfg)
>>>>        os.exit(1)
>>>>    end
>>>> 
>>>> +    local box_cfg_mt = getmetatable(box.cfg)
>>>> +    local orig_cfg_call = box_cfg_mt.__call
>>>> +    box_cfg_mt.__call = function(old_cfg, new_cfg)
>>>> +        if old_cfg.pid_file ~= nil and new_cfg ~= nil and new_cfg.pid_file ~= nil then
>>>> +            new_cfg.pid_file = old_cfg.pid_file
>>>> +        end
>>>> +        orig_cfg_call(old_cfg, new_cfg)
>>>> +    end
>>>> +
>>>>    return data
>>>> end
>>>> 
>>>> @@ -547,18 +556,6 @@ local function start()
>>>>        end
>>>>        os.exit(1)
>>>>    end
>>>> -    local box_cfg_mt = getmetatable(box.cfg)
>>>> -    if box_cfg_mt == nil then
>>>> -        log.error('box.cfg() is not called in an instance file')
>>>> -        os.exit(1)
>>>> -    end
>>>> -    local old_call = box_cfg_mt.__call
>>>> -    box_cfg_mt.__call = function(old_cfg, cfg)
>>>> -        if old_cfg.pid_file ~= nil and cfg ~= nil and cfg.pid_file ~= nil then
>>>> -            cfg.pid_file = old_cfg.pid_file
>>>> -        end
>>>> -        old_call(old_cfg, cfg)
>>>> -    end
>>>>    return 0
>>>> end
>>> 
>>> -- 
>>> Konstantin Osipov, Moscow, Russia
>> 
>> 
>