From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id D52AB315DB for ; Fri, 21 Jun 2019 17:48:26 -0400 (EDT) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 749zvRPcDqSq for ; Fri, 21 Jun 2019 17:48:26 -0400 (EDT) Received: from smtp51.i.mail.ru (smtp51.i.mail.ru [94.100.177.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id E67C1315EC for ; Fri, 21 Jun 2019 17:48:25 -0400 (EDT) From: Georgy Kirichenko Subject: [tarantool-patches] [PATCH v5 0/7] Parallel applier Date: Sat, 22 Jun 2019 00:48:14 +0300 Message-Id: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-Help: List-Unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-Subscribe: List-Owner: List-post: List-Archive: To: tarantool-patches@freelists.org Cc: Georgy Kirichenko This patch set introduces parallel applier processing on top of asynchronous transaction feature with corresponding fixes that are essential for the patch set. This patch set is focused on three major options: 1. To make a transaction and fiber engines independent from one another. 2. To make a transaction commit asynchronous. 3. To allow applier write transaction in parallel. Changes in v5: - fiber_gc after each txn_rollback - Autocommit patch fixes according to review - replicaset.applier.on_{commit/rollback} triggers to coordinate appliers - Extract refactoring into a separate patch - Minor fixes according to review Changes in v4: - Rebase against latest master - Move helper commits to the head of the patch set - Squash asynchronous wal and asynchronous transaction commits - Make txn responsible for fiber yields - Make journal asynchronous and simplify its interface - Don't implement txn_wait method - Minor fixes according to review Changes in v3: - Add a parallel applier implementation - Store transaction related data such as ddl trigger to a transaction memory region - Fix privileges defining - Minor fixes according to review Changes in v2: - Three patches are merged to master - Replicated rows are also copied to txn region - A fiber to process tx_prio endpoint - Engine commit and triggers are changed in order - Minor fixes according to review Branch: https://github.com/tarantool/tarantool/tree/g.kirichenko/gh-1254-parallel-applier Issue: https://github.com/tarantool/tarantool/issues/1254 Georgy Kirichenko (7): txn: unref statement at txn_free txn: get rid of autocommit from a txn structure txn: get rid of fiber_gc from txn_rollback wal: introduce a journal entry finalization callback txn: introduce asynchronous txn commit applier: apply transaction in parallel test: fix flaky test src/box/alter.cc | 6 + src/box/applier.cc | 236 +++++++++++++++---- src/box/applier.h | 7 + src/box/box.cc | 78 ++++--- src/box/call.c | 22 +- src/box/index.cc | 10 +- src/box/journal.c | 11 +- src/box/journal.h | 35 ++- src/box/memtx_engine.c | 11 +- src/box/memtx_space.c | 6 +- src/box/replication.cc | 7 + src/box/replication.h | 11 + src/box/sql.c | 2 +- src/box/txn.c | 311 ++++++++++++++++---------- src/box/txn.h | 39 ++-- src/box/vy_log.c | 3 +- src/box/vy_scheduler.c | 16 +- src/box/wal.c | 48 ++-- src/lib/core/latch.h | 10 + test/replication/sync.result | 7 +- test/replication/sync.test.lua | 4 +- test/replication/transaction.result | 16 +- test/replication/transaction.test.lua | 7 +- 23 files changed, 630 insertions(+), 273 deletions(-) -- 2.22.0