From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: Vladislav Shpilevoy Subject: [PATCH v4 00/12] SWIM draft Date: Thu, 31 Jan 2019 00:28:26 +0300 Message-Id: To: tarantool-patches@freelists.org Cc: kostja@tarantool.org, vdavydov.dev@gmail.com List-ID: Branch: http://github.com/tarantool/tarantool/tree/gerold103/gh-3234-swim Issue: https://github.com/tarantool/tarantool/issues/3234 Commit about anti-entropy contains a comprehensive information about SWIM which I will not duplicate here. This is only a description of the patchset. SWIM consists of two main components - dissemination and failure detection, and one additional component - anti-entropy. The patchset introduces them one by one in the first three meaning commits. Also, there are some preparational patches, including one in tarantool/small repository. Note, these commits have no tests. The goal of this review is a highlevel approval of API so as to start writing tests. Here I describe some known opened questions: 1. It is still unknown how to encrypt packets. 2. Should timestamp be added to each PING/ACK in addition to incarnation? It protects from the case when ACK is duplicated accidentally, or arrived with the same incarnation, but too late. Gitlab Lua version does it, but protocol, as I remember, does not specify it. 3. Now I decrease TTL even of such events, which did not fit into a packet. Is it ok, or we should accumulate them? My logic was that anyway not-fit events will become too old and not actual, when other prior events will be disseminated and dropped. 4. Maybe we should attach destination incarnation and status to each ping/ack message? In such a case the receiver will much faster learn about a false gossip about him. Changes in v4: - removed multi-packet support; - added some methods like probe_member, broadcast; - implemented indirect requests; - removed runtime virtuality of swim_transport; - a lot of bug fixes, refactoring. Changes in v3: - packets can carry arbitrary payload; - socket reading/writing related routines and structures are moved to swim_scheduler. Changes in v2: - new API with explicit members addition, removal; - ability to create multiple SWIM instances per one Tarantool process; - multi-packet sending of one SWIM message. V1: https://www.freelists.org/post/tarantool-patches/PATCH-05-SWIM V2: https://www.freelists.org/post/tarantool-patches/PATCH-v2-06-SWIM V3: https://www.freelists.org/post/tarantool-patches/PATCH-v3-06-SWIM-draft Vladislav Shpilevoy (12): sio: introduce sio_uri_to_addr evio: expose evio_setsockopt_server function rlist: introduce rlist_add_tail_entry_sorted [RAW] swim: introduce SWIM's anti-entropy component [RAW] swim: introduce failure detection component [RAW] swim: introduce dissemination component [RAW] swim: keep encoded round message cached [RAW] swim: introduce payload [RAW] swim: introduce routing [RAW] swim: introduce 'quit' message [RAW] swim: introduce broadcast tasks [RAW] swim: allow to use broadcast tasks to send pings src/CMakeLists.txt | 5 +- src/diag.h | 2 + src/evio.c | 3 +- src/evio.h | 4 + src/exception.cc | 23 + src/exception.h | 7 + src/lib/CMakeLists.txt | 1 + src/lib/small | 2 +- src/lib/swim/CMakeLists.txt | 6 + src/lib/swim/swim.c | 1733 +++++++++++++++++++++++++++++++ src/lib/swim/swim.h | 119 +++ src/lib/swim/swim_io.c | 412 ++++++++ src/lib/swim/swim_io.h | 282 +++++ src/lib/swim/swim_proto.c | 595 +++++++++++ src/lib/swim/swim_proto.h | 652 ++++++++++++ src/lib/swim/swim_transport.h | 73 ++ src/lua/init.c | 2 + src/lua/swim.c | 466 +++++++++ src/lua/swim.h | 47 + src/sio.c | 41 + src/sio.h | 7 + test/unit/CMakeLists.txt | 6 + test/unit/rlist.c | 40 +- test/unit/rlist.result | 6 +- test/unit/sio.c | 96 ++ test/unit/swim.c | 34 + test/unit/swim_test_transport.c | 78 ++ 27 files changed, 4735 insertions(+), 7 deletions(-) create mode 100644 src/lib/swim/CMakeLists.txt create mode 100644 src/lib/swim/swim.c create mode 100644 src/lib/swim/swim.h create mode 100644 src/lib/swim/swim_io.c create mode 100644 src/lib/swim/swim_io.h create mode 100644 src/lib/swim/swim_proto.c create mode 100644 src/lib/swim/swim_proto.h create mode 100644 src/lib/swim/swim_transport.h create mode 100644 src/lua/swim.c create mode 100644 src/lua/swim.h create mode 100644 test/unit/sio.c create mode 100644 test/unit/swim.c create mode 100644 test/unit/swim_test_transport.c -- 2.17.2 (Apple Git-113)