[PATCH v4 00/12] SWIM draft

[Vladislav Shpilevoy <v.shpilevoy@tarantool.org>]

Branch: http://github.com/tarantool/tarantool/tree/gerold103/gh-3234-swim
Issue: https://github.com/tarantool/tarantool/issues/3234

Commit about anti-entropy contains a comprehensive information about SWIM which
I will not duplicate here. This is only a description of the patchset.

SWIM consists of two main components - dissemination and failure detection, and
one additional component - anti-entropy. The patchset introduces them one by one
in the first three meaning commits. Also, there are some preparational patches,
including one in tarantool/small repository.

Note, these commits have no tests. The goal of this review is a highlevel
approval of API so as to start writing tests.

Here I describe some known opened questions:

1. It is still unknown how to encrypt packets.

2. Should timestamp be added to each PING/ACK in addition to incarnation?
   It protects from the case when ACK is duplicated accidentally, or arrived
   with the same incarnation, but too late. Gitlab Lua version does it, but
   protocol, as I remember, does not specify it.

3. Now I decrease TTL even of such events, which did not fit into a packet. Is
   it ok, or we should accumulate them? My logic was that anyway not-fit events
   will become too old and not actual, when other prior events will be
   disseminated and dropped.

4. Maybe we should attach destination incarnation and status to each ping/ack
   message? In such a case the receiver will much faster learn about a false
   gossip about him.

Changes in v4:
- removed multi-packet support;
- added some methods like probe_member, broadcast;
- implemented indirect requests;
- removed runtime virtuality of swim_transport;
- a lot of bug fixes, refactoring.

Changes in v3:
- packets can carry arbitrary payload;
- socket reading/writing related routines and structures are moved to
  swim_scheduler.

Changes in v2:
- new API with explicit members addition, removal;
- ability to create multiple SWIM instances per one Tarantool process;
- multi-packet sending of one SWIM message.

V1: https://www.freelists.org/post/tarantool-patches/PATCH-05-SWIM
V2: https://www.freelists.org/post/tarantool-patches/PATCH-v2-06-SWIM
V3: https://www.freelists.org/post/tarantool-patches/PATCH-v3-06-SWIM-draft

Vladislav Shpilevoy (12):
  sio: introduce sio_uri_to_addr
  evio: expose evio_setsockopt_server function
  rlist: introduce rlist_add_tail_entry_sorted
  [RAW] swim: introduce SWIM's anti-entropy component
  [RAW] swim: introduce failure detection component
  [RAW] swim: introduce dissemination component
  [RAW] swim: keep encoded round message cached
  [RAW] swim: introduce payload
  [RAW] swim: introduce routing
  [RAW] swim: introduce 'quit' message
  [RAW] swim: introduce broadcast tasks
  [RAW] swim: allow to use broadcast tasks to send pings

 src/CMakeLists.txt              |    5 +-
 src/diag.h                      |    2 +
 src/evio.c                      |    3 +-
 src/evio.h                      |    4 +
 src/exception.cc                |   23 +
 src/exception.h                 |    7 +
 src/lib/CMakeLists.txt          |    1 +
 src/lib/small                   |    2 +-
 src/lib/swim/CMakeLists.txt     |    6 +
 src/lib/swim/swim.c             | 1733 +++++++++++++++++++++++++++++++
 src/lib/swim/swim.h             |  119 +++
 src/lib/swim/swim_io.c          |  412 ++++++++
 src/lib/swim/swim_io.h          |  282 +++++
 src/lib/swim/swim_proto.c       |  595 +++++++++++
 src/lib/swim/swim_proto.h       |  652 ++++++++++++
 src/lib/swim/swim_transport.h   |   73 ++
 src/lua/init.c                  |    2 +
 src/lua/swim.c                  |  466 +++++++++
 src/lua/swim.h                  |   47 +
 src/sio.c                       |   41 +
 src/sio.h                       |    7 +
 test/unit/CMakeLists.txt        |    6 +
 test/unit/rlist.c               |   40 +-
 test/unit/rlist.result          |    6 +-
 test/unit/sio.c                 |   96 ++
 test/unit/swim.c                |   34 +
 test/unit/swim_test_transport.c |   78 ++
 27 files changed, 4735 insertions(+), 7 deletions(-)
 create mode 100644 src/lib/swim/CMakeLists.txt
 create mode 100644 src/lib/swim/swim.c
 create mode 100644 src/lib/swim/swim.h
 create mode 100644 src/lib/swim/swim_io.c
 create mode 100644 src/lib/swim/swim_io.h
 create mode 100644 src/lib/swim/swim_proto.c
 create mode 100644 src/lib/swim/swim_proto.h
 create mode 100644 src/lib/swim/swim_transport.h
 create mode 100644 src/lua/swim.c
 create mode 100644 src/lua/swim.h
 create mode 100644 test/unit/sio.c
 create mode 100644 test/unit/swim.c
 create mode 100644 test/unit/swim_test_transport.c

-- 
2.17.2 (Apple Git-113)