From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 53F3E30D28 for ; Wed, 19 Jun 2019 17:23:21 -0400 (EDT) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 79FptoCp8gCt for ; Wed, 19 Jun 2019 17:23:21 -0400 (EDT) Received: from smtp63.i.mail.ru (smtp63.i.mail.ru [217.69.128.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id 5409030D00 for ; Wed, 19 Jun 2019 17:23:20 -0400 (EDT) From: Georgy Kirichenko Subject: [tarantool-patches] [PATCH v4 0/9] Parallel applier Date: Thu, 20 Jun 2019 00:23:07 +0300 Message-Id: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-Help: List-Unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-Subscribe: List-Owner: List-post: List-Archive: To: tarantool-patches@freelists.org Cc: Georgy Kirichenko This patch set introduces parallel applier processing on top of asynchronous transaction feature with corresponding fixes that are essential for the patch set. This patch set is focused on three major options: 1. To make a transaction and fiber engines independent from one another. 2. To make a transaction commit asynchronous. 3. To allow applier write transaction in parallel. Changes in v4: - Rebase against latest master - Move helper commits to the head of the patch set - Squash asynchronous wal and asynchronous transaction commits - Make txn responsible for fiber yields - Make journal asynchronous and simplify its interface - Don't implement txn_wait method - Minor fixes according to review Changes in v3: - Add a parallel applier implementation - Store transaction related data such as ddl trigger to a transaction memory region - Fix privileges defining - Minor fixes according to review Changes in v2: - Three patches are merged to master - Replicated rows are also copied to txn region - A fiber to process tx_prio endpoint - Engine commit and triggers are changed in order - Minor fixes according to review NOTE: There should be a follow up patch to remove redundant and excess code such a fiber_gc invocation but let it be out of the scope of the current patch set. Branch: https://github.com/tarantool/tarantool/tree/g.kirichenko/gh-1254-parallel-applier Issue: https://github.com/tarantool/tarantool/issues/1254 Georgy Kirichenko (9): txn: handle fiber stop event at transaction level core: latch_steal routine txn: get rid of autocommit from a txn structure txn: get rid of fiber_gc from txn_rollback wal: a dedicated wal scheduling fiber wal: introduce a journal entry finalization callback txn: introduce asynchronous txn commit applier: apply transaction in parallel test: fix flaky test src/box/alter.cc | 5 + src/box/applier.cc | 162 +++++++++++--- src/box/applier.h | 6 +- src/box/box.cc | 105 +++++++-- src/box/call.c | 22 +- src/box/index.cc | 10 +- src/box/journal.c | 12 +- src/box/journal.h | 18 +- src/box/memtx_engine.c | 16 +- src/box/memtx_space.c | 8 +- src/box/replication.cc | 7 + src/box/replication.h | 14 ++ src/box/sql.c | 2 +- src/box/txn.c | 302 ++++++++++++++++---------- src/box/txn.h | 40 ++-- src/box/vinyl.c | 8 - src/box/vy_log.c | 3 +- src/box/vy_scheduler.c | 16 +- src/box/wal.c | 92 +++++--- src/lib/core/latch.h | 10 + test/replication/sync.result | 7 +- test/replication/sync.test.lua | 4 +- test/replication/transaction.result | 16 +- test/replication/transaction.test.lua | 7 +- 24 files changed, 617 insertions(+), 275 deletions(-) -- 2.22.0