From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 412516ECC0; Mon, 6 Dec 2021 06:03:27 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 412516ECC0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1638759807; bh=lzeYvsFhV96l4jJlcVMQUlUlO9aTx5fAUESpDGJQrug=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=bq//0deEUrqoHL/UBeGzkMJT/qbKiNrizNiQzep1PDGTjJS/5CLYeEdQwgvsiJkwJ ezYW5YQ9aVWWsBzKD4QgAVzZytLsYP3qYUC32JK0W9mdp1Zb1Nqxrzm49quqF5PNnj FpVNtla2amuvvPpsw2Ph7qraB6LHE8xdbatOjqfk= Received: from smtp38.i.mail.ru (smtp38.i.mail.ru [94.100.177.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 377136ECC0 for ; Mon, 6 Dec 2021 06:03:26 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 377136ECC0 Received: by smtp38.i.mail.ru with esmtpa (envelope-from ) id 1mu4Hl-0007aS-0y; Mon, 06 Dec 2021 06:03:25 +0300 To: v.shpilevoy@tarantool.org, vdavydov@tarantool.org Date: Mon, 6 Dec 2021 06:03:19 +0300 Message-Id: X-Mailer: git-send-email 2.30.1 (Apple Git-130) MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD93822B471089FF64D6B5EE151AFD053303E9E903E4D999E60182A05F538085040FC9A745196AAC52E65FFC43538C66A29B18050590C763886C6FAD0D80053C2BB X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE70CB15FA6C489297DEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006378010A306A5B6F90A8638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D8E26FE4BFB49EAB5657E783FC99B83086117882F4460429724CE54428C33FAD305F5C1EE8F4F765FCAE9A1BBD95851C5BA471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F44604297287769387670735201E561CDFBCA1751FE5D25F19253116ADD2E47CDBA5A96583BA9C0B312567BB2376E601842F6C81A19E625A9149C048EE38D4DC57D478E688B78CF848AE20165DD8FC6C240DEA7642DBF02ECDB25306B2B78CF848AE20165D0A6AB1C7CE11FEE3A5ED62E35AC703CC302FCEF25BFAB345C4224003CC836476EA7A3FFF5B025636E2021AF6380DFAD1A18204E546F3947CB11811A4A51E3B096D1867E19FE1407959CC434672EE6371089D37D7C0E48F6C8AA50765F7900637ED813124F2F0737AEFF80C71ABB335746BA297DBC24807EABDAD6C7F3747799A X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8183A4AFAF3EA6BDC44C234C8B12C006B7A07FEF4BC0B92C68B7EE8819CB2EC973079AA32543325782FB1881A6453793CE9C32612AADDFBE061F05E071F081E88CB9510FB958DCE06DB6ED91DBE5ABE359ADBCB5631A0A9D21F2272C4C079A4C8AD93EDB24507CE13387DFF0A840B692CF8 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D3455049D7B43D89D64AB8E0FC6F134211F318499B60676214AE3904555057A93821B474DA3C382342F1D7E09C32AA3244C722D89C37E3435CF48FDFD659DE4D5B6D08D48398F32B4A6927AC6DF5659F194 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojbL9S8ysBdXiGu4RiEaSSztw0blZzhpKd X-Mailru-Sender: 583F1D7ACE8F49BD7B46BC6C7C9DD5A8AA235155620D99CD0C9B90B0EDDF74BA931F1E698EF251A5424AE0EB1F3D1D21E2978F233C3FAE6EE63DB1732555E4A8EE80603BA4A5B0BCB0DAF586E7D11B3E67EA787935ED9F1B X-Mras: Ok Subject: [Tarantool-patches] [PATCH 0/4] replication: introduce applier thread X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Serge Petrenko via Tarantool-patches Reply-To: Serge Petrenko Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Patches #1 and #2 rework applier and coio_read_xrow to keep parsed input. This allows us to get rid of copying row bodies in applier right after parsing them. The patches are left after a failed applier-in-thread approach. They are not needed anymore, strictly speaking, so we may drop them. Or use them. After all, they make usual applier (fiber in tx thread) better. I'm still sending them for review. Patch #3 is a small refactoring, needed for patch #4. Patch #4 is the applier in thread itself. Here's a brief idea: There's a separate thread which decodes the incoming rows. This way the applier fiber (in tx thread) may deal with already decoded requests and doesn't waste its time on decoding them. This should improve replication performance quite a bit. We introduce a new config option: `replication_num_threads`. There may be not more than `replication_num_threads` applier threads (default is 1). And when there are more appliers than threads, each thread handles multiple appliers. The thread has 2 fibers per each applier: a reader and a writer. The writer simply copies applier_writer_f behaviour. Nothing to see here. The reader reads ahead as much data as possible and decodes it. As soon as there is enough data read to decode a full transaction, the reader decodes the transaction (or multiple transactions) and sends the result to the tx thread. I wasted quite some time to make the approach with coio_read_xrow() work. (It was the first one that came to mind, and looked the simplest, but failed). Everything came together once I dumped the coio_read_xrow() approach, and started using readahead. Some of the tests are still failing in CI and local runs, so I'm sending the branch without tests now. I'm going to fix the tests now. Branch: https://github.com/tarantool/tarantool/tree/sp/gh-6329-applier-in-thread-notest Issue: https://github.com/tarantool/tarantool/issues/6329 Serge Petrenko (4): xrow: rework coio_read_xrow to keep parsed input applier: reuse input buffer to store row bodies applier: factor replication stream processing out of subscribe() Introduce applier thread .../unreleased/gh-6329-applier-in-thread.md | 5 + src/box/applier.cc | 1014 +++++++++++++++-- src/box/applier.h | 4 + src/box/box.cc | 2 +- src/box/lua/load_cfg.lua | 2 + src/box/replication.cc | 5 +- src/box/replication.h | 7 +- src/box/xrow_io.cc | 28 +- src/box/xrow_io.h | 18 +- src/lib/small | 2 +- src/lua/buffer.lua | 2 + 11 files changed, 955 insertions(+), 134 deletions(-) create mode 100644 changelogs/unreleased/gh-6329-applier-in-thread.md -- 2.30.1 (Apple Git-130)