From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 550716EC56; Wed, 17 Mar 2021 15:14:14 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 550716EC56 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1615983254; bh=T0MuhkiKm16mPWfzqluj6OYk8VrUUlScf93Gn/82ujk=; h=To:References:Date:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=nnMccW40AybUBDq2V24exHZnrW7yGpecfiTg2ANwfAtoQ+NLu5a+qDRba/0wixndv QHyWwE3insZve1Q8zrYSwsz9VAtZNcDytPDTm2ID5vxVYLhS9Zvsn8vlmgZCb4GtWZ feA/T89xADvU1cn2kMPcAwHlztG3gNAHsK95WU+U= Received: from smtp42.i.mail.ru (smtp42.i.mail.ru [94.100.177.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 0B5C66EC56 for ; Wed, 17 Mar 2021 15:14:12 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 0B5C66EC56 Received: by smtp42.i.mail.ru with esmtpa (envelope-from ) id 1lMV40-00056d-8O; Wed, 17 Mar 2021 15:14:12 +0300 To: Vladislav Shpilevoy , Konstantin Osipov , gorcunov@gmail.com, tarantool-patches@dev.tarantool.org References: <20210224193549.70017-1-sergepetrenko@tarantool.org> <26fde1bf-a972-fe03-fffe-818839718394@tarantool.org> <20210310081804.GA87351@starling> <54666e16-f577-c9f5-e917-f834b7e4b8d5@tarantool.org> <07f4831b-2300-2bb0-28a3-9d49f197e951@tarantool.org> <973078e7-04b7-f6fb-1c71-3972e278f0f6@tarantool.org> Message-ID: <279588ba-dafd-1eba-d0fc-78090b4ae04b@tarantool.org> Date: Wed, 17 Mar 2021 15:14:11 +0300 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: <973078e7-04b7-f6fb-1c71-3972e278f0f6@tarantool.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-7564579A: 78E4E2B564C1792B X-77F55803: 4F1203BC0FB41BD96485A7A9FC1318933D9BB3A6B6EC3969033B83865BA14179182A05F538085040848395FD087C0D2F83FC8022D0FA379B7CFAFDC668D99509216EF433808AEA16 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7B9FBA884A7C9B8BAEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637861E0AC852D5F5A18638F802B75D45FF914D58D5BE9E6BC131B5C99E7648C95CF89CA98302ED496F98D3E9F31D7AA70BD8E7FD6B47E231C3A471835C12D1D9774AD6D5ED66289B5278DA827A17800CE74A95F4E53E8DCE969FA2833FD35BB23D2EF20D2F80756B5F868A13BD56FB6657A471835C12D1D977725E5C173C3A84C3BD39A56654533F91117882F4460429728AD0CFFFB425014E868A13BD56FB6657D81D268191BDAD3DC09775C1D3CA48CFEBA5F32D0D36F5CEBA3038C0950A5D36C8A9BA7A39EFB766EC990983EF5C0329BA3038C0950A5D36D5E8D9A59859A8B63479894F2F957C8376E601842F6C81A1F004C90652538430CDED94BCBF13EF3B93EC92FD9297F6718AA50765F7900637217F7D133B16D230A7F4EDE966BC389F395957E7521B51C24C7702A67D5C33162DBA43225CD8A89FCBE33A7817EDC1EF57739F23D657EF2BB5C8C57E37DE458B4C7702A67D5C3316FA3894348FB808DB48C21F01D89DB561574AF45C6390F7469DAA53EE0834AAEE X-C1DE0DAB: 0D63561A33F958A56C4F3086EAD55F08B7C6A1781A19931C70A4A5C0B35DF55BD59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA75F04B387B5D7535DE410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D3494FB0335DF05DC3A7D98868054D7FFAC984BEF31AD832E15AB307D5E3022CE48B4C1A1CA634635651D7E09C32AA3244C61E20FAD5D10E4C6DB3F4F9BA2614B0A64EE5813BBCA3A9D927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojwV/GgY7Z4vWcAdM4Zz8NFg== X-Mailru-Sender: 583F1D7ACE8F49BDD2846D59FC20E9F849EE8FA6ACD229DB6A1EA03E22C7EABF9B7A811A0C70C8CB424AE0EB1F3D1D21E2978F233C3FAE6EE63DB1732555E4A8EE80603BA4A5B0BC112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v3] wal: introduce limits on simultaneous writes X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Serge Petrenko via Tarantool-patches Reply-To: Serge Petrenko Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" 16.03.2021 23:48, Vladislav Shpilevoy пишет: > Hi! Thanks for the fixes! Thanks for the review! > See 3 comments below. > >> Here's when the option comes in handy: >> Imagine such a situation: there are 2 servers, a master and a replica, >> and the replica is down for some period of time. While the replica is >> down, the master serves requests at a reasonable pace, possibly close to >> its WAL throughput limit. Once the replica reconnects, it has to receive >> all the data master has piled up. Now there's no limit in speed at which >> master sends the data to replica, and there's no limit at which >> replica's applier submits corresponding write requests to WAL. This >> leads to a situation when replica's WAL is never in time to serve the >> requests and the amount of pending requests is constantly growing. >> There's no limit for memory WAL write requests take, and this clogging >> of WAL write queue may even lead to replica using up all the available >> memory. >> >> Now, when `wal_queue_max_size` is set, appliers will stop reading new >> transactions once the limit is reached. This will let WAL process all the >> requests that have piled up and free all the excess memory. >> >> [tosquash] remove wal_queue_max_len > 1. You forgot something, the last line. Also, while we are here, it probably > would be easier for the doc team if the old behaviour was described using a > past tense, while the new one using the present tense. Currently you use > 'now' word both for the old and for the new behaviour. For instance, you say > > Now there's no limit in speed at which master sends the data to replica, > and there's no limit at which replica's applier submits corresponding > write requests to WAL > > But >now< there is a limit. Even if 'wal_queue_max_size' is not set, it works > with the default value. Thanks! Take a look at the revised paragraph: ===================================================== Here's when the option comes in handy: Before this option was introduced such a situation could be possible: there are 2 servers, a master and a replica, and the replica is down for some period of time. While the replica is down, master serves requests at a reasonable pace, possibly close to its WAL throughput limit. Once the replica reconnects, it has to receive all the data master has piled up and there's no limit in speed at which master sends the data to replica, and, without the option, there was no limit in speed at which replica submitted corresponding write requests to WAL. This lead to a situation when replica's WAL was never in time to serve the requests and the amount of pending requests was constantly growing. There was no limit for memory WAL write requests take, and this clogging of WAL write queue could even lead to replica using up all the available memory. Now, when `wal_queue_max_size` is set, appliers will stop reading new transactions once the limit is reached. This will let WAL process all the requests that have piled up and free all the excess memory. ===================================================== >> diff --git a/changelogs/unreleased/wal-queue-limit.md b/changelogs/unreleased/wal-queue-limit.md >> new file mode 100644 >> index 000000000..393932456 >> --- /dev/null >> +++ b/changelogs/unreleased/wal-queue-limit.md >> @@ -0,0 +1,9 @@ >> +## feature/core >> + >> +* Introduce the concept of WAL queue and 2 new configuration options: >> + `wal_queue_max_len`, measured in transactions, with 100k default and >> + `wal_queue_max_size`, measured in bytes, with 100 Mb default. > 2. There is 1 option now, not 2. Sorry for the inattention, fixed. > >> + The options help limit the pace at which replica submits new transactions >> + to WAL: the limits are checked every time a transaction from master is >> + submitted to replica's WAL, and the space taken by a transaction is >> + considered empty once it's successfully written (gh-5536).> diff --git a/src/box/journal.h b/src/box/journal.h >> index 5d8d5a726..437257728 100644 >> --- a/src/box/journal.h >> +++ b/src/box/journal.h >> @@ -124,6 +142,62 @@ struct journal { >> + >> +/** Set maximal journal queue size in bytes. */ >> +static inline void >> +journal_queue_set_max_size(int64_t size) >> +{ >> + journal_queue.max_size = size; >> + journal_queue_wakeup(); >> +} >> + >> +/** Increase queue size on a new write request. */ >> +static inline void >> +journal_queue_on_append(struct journal_entry *entry) > 3. Since you will amend the patch anyway, you could also > make the entry 'const', the same in journal_queue_on_complete(). Sure. The diff's below. > >> +{ >> + journal_queue.size += entry->approx_len; >> +} >> + >> +/** Decrease queue size once write request is complete. */ >> +static inline void >> +journal_queue_on_complete(struct journal_entry *entry) >> +{ >> + journal_queue.size -= entry->approx_len; >> + assert(journal_queue.size >= 0); >> +} =====================================================diff --git a/changelogs/unreleased/wal-queue-limit.md b/changelogs/unreleased/wal-queue-limit.md index 393932456..1708e46e6 100644 --- a/changelogs/unreleased/wal-queue-limit.md +++ b/changelogs/unreleased/wal-queue-limit.md @@ -1,9 +1,8 @@  ## feature/core -* Introduce the concept of WAL queue and 2 new configuration options: -  `wal_queue_max_len`, measured in transactions, with 100k default and +* Introduce the concept of WAL queue and a new configuration option:    `wal_queue_max_size`, measured in bytes, with 100 Mb default. -  The options help limit the pace at which replica submits new transactions -  to WAL: the limits are checked every time a transaction from master is +  The option helps limit the pace at which replica submits new transactions +  to WAL: the limit is checked every time a transaction from master is    submitted to replica's WAL, and the space taken by a transaction is    considered empty once it's successfully written (gh-5536). diff --git a/src/box/journal.h b/src/box/journal.h index 437257728..76c70c19f 100644 --- a/src/box/journal.h +++ b/src/box/journal.h @@ -185,14 +185,14 @@ journal_queue_set_max_size(int64_t size)  /** Increase queue size on a new write request. */  static inline void -journal_queue_on_append(struct journal_entry *entry) +journal_queue_on_append(const struct journal_entry *entry)  {         journal_queue.size += entry->approx_len;  }  /** Decrease queue size once write request is complete. */  static inline void -journal_queue_on_complete(struct journal_entry *entry) +journal_queue_on_complete(const struct journal_entry *entry)  {         journal_queue.size -= entry->approx_len;         assert(journal_queue.size >= 0); diff --git a/test/box-tap/cfg.test.lua b/test/box-tap/cfg.test.lua index 3276ddf64..8f21c5628 100755 --- a/test/box-tap/cfg.test.lua +++ b/test/box-tap/cfg.test.lua @@ -6,7 +6,7 @@ local socket = require('socket')  local fio = require('fio')  local uuid = require('uuid')  local msgpack = require('msgpack') -test:plan(110) +test:plan(109)  --------------------------------------------------------------------------------  -- Invalid values @@ -50,7 +50,6 @@ invalid('vinyl_run_size_ratio', 1)  invalid('vinyl_bloom_fpr', 0)  invalid('vinyl_bloom_fpr', 1.1)  invalid('wal_queue_max_size', -1) -invalid('wal_queue_max_len', -1)  local function invalid_combinations(name, val)      local status, result = pcall(box.cfg, val) -- Serge Petrenko