From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 2D7962B2B6 for ; Tue, 9 Apr 2019 14:12:16 -0400 (EDT) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ASxfGgmKzHpQ for ; Tue, 9 Apr 2019 14:12:15 -0400 (EDT) Received: from smtpng3.m.smailru.net (smtpng3.m.smailru.net [94.100.177.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id 838A926813 for ; Tue, 9 Apr 2019 14:12:15 -0400 (EDT) From: Vladislav Shpilevoy Subject: [tarantool-patches] [PATCH 1/5] test: allow to remove swim nodes from the cluster Date: Tue, 9 Apr 2019 21:12:08 +0300 Message-Id: <965aa21307b9b99b8a67e0922c2c15ba5ce672a8.1554833062.git.v.shpilevoy@tarantool.org> In-Reply-To: References: In-Reply-To: References: Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-Help: List-Unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-Subscribe: List-Owner: List-post: List-Archive: To: tarantool-patches@freelists.org Cc: kostja@tarantool.org Until now it was impossible in swim tests to drop a SWIM instance from the cluster. It should have been either restarted, or blocked, but a real drop led to an assertion on any attemp to use one of methods like swim_wait_timeout(). It was due to inability to get instance's UUID without the instance itself. Even if it was stored in membership tables of other instances. This patch makes swim_cluster store swim instances and UUIDs separately. This is going to be used to test swim_quit() API. Also, some cfg parameters are saved as well, like ack timeout, gc mode. They are used to restart a node with exactly same cfg as it was before restart. Even if original struct swim * is not valid already. Part of #3234 --- test/unit/swim.c | 4 +- test/unit/swim_test_utils.c | 125 ++++++++++++++++++++++++++---------- test/unit/swim_test_utils.h | 5 ++ 3 files changed, 99 insertions(+), 35 deletions(-) diff --git a/test/unit/swim.c b/test/unit/swim.c index e11f006ae..84ca01ac3 100644 --- a/test/unit/swim.c +++ b/test/unit/swim.c @@ -106,12 +106,12 @@ swim_test_uuid_update(void) struct swim *s = swim_cluster_node(cluster, 0); struct tt_uuid new_uuid = uuid_nil; new_uuid.time_low = 1000; - is(swim_cfg(s, NULL, -1, -1, -1, &new_uuid), 0, "UUID update"); + is(swim_cluster_update_uuid(cluster, 0, &new_uuid), 0, "UUID update"); is(swim_cluster_wait_fullmesh(cluster, 1), 0, "old UUID is returned back as a 'ghost' member"); is(swim_size(s), 3, "two members in each + ghost third member"); new_uuid.time_low = 2; - is(swim_cfg(s, NULL, -1, -1, -1, &new_uuid), -1, + is(swim_cluster_update_uuid(cluster, 0, &new_uuid), -1, "can not update to an existing UUID - swim_cfg fails"); ok(swim_error_check_match("exists"), "diag says 'exists'"); swim_cluster_delete(cluster); diff --git a/test/unit/swim_test_utils.c b/test/unit/swim_test_utils.c index bada8ef43..a42e50cae 100644 --- a/test/unit/swim_test_utils.c +++ b/test/unit/swim_test_utils.c @@ -36,33 +36,67 @@ #include "trivia/util.h" #include "fiber.h" +/** + * SWIM cluster node and its UUID. UUID is stored separately + * because sometimes a test wants to drop a SWIM instance, but + * still check how does it look in other membership instances. + * UUID is necessary since it is a key to lookup a view of that + * instance in the member tables. + */ +struct swim_node { + /** SWIM instance. Can be NULL. */ + struct swim *swim; + /** + * UUID. Is used when @a swim is NULL to lookup view of + * that instance. + */ + struct tt_uuid uuid; +}; + /** * Cluster is a simple array of SWIM instances assigned to * different URIs. */ struct swim_cluster { int size; - struct swim **node; + struct swim_node *node; + /** + * Saved values to restart SWIM nodes with the most actual + * configuration. + */ + double ack_timeout; + enum swim_gc_mode gc_mode; }; +/** Build URI of a SWIM instance for a given @a id. */ +static inline void +swim_cluster_id_to_uri(char *buffer, int id) +{ + sprintf(buffer, "127.0.0.1:%d", id + 1); +} + struct swim_cluster * swim_cluster_new(int size) { struct swim_cluster *res = (struct swim_cluster *) malloc(sizeof(*res)); assert(res != NULL); int bsize = sizeof(res->node[0]) * size; - res->node = (struct swim **) malloc(bsize); + res->node = (struct swim_node *) malloc(bsize); assert(res->node != NULL); res->size = size; + res->ack_timeout = -1; + res->gc_mode = SWIM_GC_DEFAULT; struct tt_uuid uuid; memset(&uuid, 0, sizeof(uuid)); char *uri = tt_static_buf(); - for (int i = 0; i < size; ++i) { - res->node[i] = swim_new(); - assert(res->node[i] != NULL); - sprintf(uri, "127.0.0.1:%d", i + 1); + struct swim_node *n = res->node; + for (int i = 0; i < size; ++i, ++n) { + n->swim = swim_new(); + assert(n->swim != NULL); + swim_cluster_id_to_uri(uri, i); uuid.time_low = i + 1; - int rc = swim_cfg(res->node[i], uri, -1, -1, -1, &uuid); + n->uuid = uuid; + int rc = swim_cfg(n->swim, uri, -1, -1, -1, &uuid); assert(rc == 0); (void) rc; } @@ -71,7 +105,7 @@ swim_cluster_new(int size) #define swim_cluster_set_cfg(cluster, ...) ({ \ for (int i = 0; i < cluster->size; ++i) { \ - int rc = swim_cfg(cluster->node[i], __VA_ARGS__); \ + int rc = swim_cfg(cluster->node[i].swim, __VA_ARGS__); \ assert(rc == 0); \ (void) rc; \ } \ @@ -81,28 +115,44 @@ void swim_cluster_set_ack_timeout(struct swim_cluster *cluster, double ack_timeout) { swim_cluster_set_cfg(cluster, NULL, -1, ack_timeout, -1, NULL); + cluster->ack_timeout = ack_timeout; } void swim_cluster_set_gc(struct swim_cluster *cluster, enum swim_gc_mode gc_mode) { swim_cluster_set_cfg(cluster, NULL, -1, -1, gc_mode, NULL); + cluster->gc_mode = gc_mode; } void swim_cluster_delete(struct swim_cluster *cluster) { - for (int i = 0; i < cluster->size; ++i) - swim_delete(cluster->node[i]); + for (int i = 0; i < cluster->size; ++i) { + if (cluster->node[i].swim != NULL) + swim_delete(cluster->node[i].swim); + } free(cluster->node); free(cluster); } +int +swim_cluster_update_uuid(struct swim_cluster *cluster, int i, + const struct tt_uuid *new_uuid) +{ + assert(i >= 0 && i < cluster->size); + struct swim_node *n = &cluster->node[i]; + if (swim_cfg(n->swim, NULL, -1, -1, -1, new_uuid) != 0) + return -1; + n->uuid = *new_uuid; + return 0; +} + int swim_cluster_add_link(struct swim_cluster *cluster, int to_id, int from_id) { - const struct swim_member *from = swim_self(cluster->node[from_id]); - return swim_add_member(cluster->node[to_id], swim_member_uri(from), + const struct swim_member *from = swim_self(cluster->node[from_id].swim); + return swim_add_member(cluster->node[to_id].swim, swim_member_uri(from), swim_member_uuid(from)); } @@ -110,10 +160,12 @@ static const struct swim_member * swim_cluster_member_view(struct swim_cluster *cluster, int node_id, int member_id) { - struct swim *node = cluster->node[node_id]; - const struct swim_member *member = swim_self(cluster->node[member_id]); - const struct tt_uuid *member_uuid = swim_member_uuid(member); - return swim_member_by_uuid(node, member_uuid); + /* + * Do not use node[member_id].swim - it can be NULL + * already, for example, in case of quit or deletion. + */ + return swim_member_by_uuid(cluster->node[node_id].swim, + &cluster->node[member_id].uuid); } enum swim_member_status @@ -142,44 +194,47 @@ struct swim * swim_cluster_node(struct swim_cluster *cluster, int i) { assert(i >= 0 && i < cluster->size); - return cluster->node[i]; + return cluster->node[i].swim; } void swim_cluster_restart_node(struct swim_cluster *cluster, int i) { assert(i >= 0 && i < cluster->size); - struct swim *s = cluster->node[i]; - const struct swim_member *self = swim_self(s); - struct tt_uuid uuid = *swim_member_uuid(self); + struct swim_node *n = &cluster->node[i]; + struct swim *s = n->swim; char uri[128]; - strcpy(uri, swim_member_uri(self)); - double ack_timeout = swim_ack_timeout(s); - swim_delete(s); + swim_cluster_id_to_uri(uri, i); + if (s != NULL) { + assert(tt_uuid_is_equal(swim_member_uuid(swim_self(s)), + &n->uuid)); + swim_delete(s); + } s = swim_new(); assert(s != NULL); - int rc = swim_cfg(s, uri, -1, ack_timeout, -1, &uuid); + int rc = swim_cfg(s, uri, -1, cluster->ack_timeout, cluster->gc_mode, + &n->uuid); assert(rc == 0); (void) rc; - cluster->node[i] = s; + n->swim = s; } void swim_cluster_block_io(struct swim_cluster *cluster, int i) { - swim_test_transport_block_fd(swim_fd(cluster->node[i])); + swim_test_transport_block_fd(swim_fd(cluster->node[i].swim)); } void swim_cluster_unblock_io(struct swim_cluster *cluster, int i) { - swim_test_transport_unblock_fd(swim_fd(cluster->node[i])); + swim_test_transport_unblock_fd(swim_fd(cluster->node[i].swim)); } void swim_cluster_set_drop(struct swim_cluster *cluster, int i, double value) { - swim_test_transport_set_drop(swim_fd(cluster->node[i]), value); + swim_test_transport_set_drop(swim_fd(cluster->node[i].swim), value); } /** Check if @a s1 knows every member of @a s2's table. */ @@ -201,11 +256,15 @@ swim1_contains_swim2(struct swim *s1, struct swim *s2) bool swim_cluster_is_fullmesh(struct swim_cluster *cluster) { - struct swim **end = cluster->node + cluster->size; - for (struct swim **s1 = cluster->node; s1 < end; ++s1) { - for (struct swim **s2 = s1 + 1; s2 < end; ++s2) { - if (! swim1_contains_swim2(*s1, *s2) || - ! swim1_contains_swim2(*s2, *s1)) + struct swim_node *end = cluster->node + cluster->size; + for (struct swim_node *s1 = cluster->node; s1 < end; ++s1) { + if (s1->swim == NULL) + continue; + for (struct swim_node *s2 = s1 + 1; s2 < end; ++s2) { + if (s2->swim == NULL) + continue; + if (! swim1_contains_swim2(s1->swim, s2->swim) || + ! swim1_contains_swim2(s2->swim, s1->swim)) return false; } } diff --git a/test/unit/swim_test_utils.h b/test/unit/swim_test_utils.h index f4397aff4..6e172fb85 100644 --- a/test/unit/swim_test_utils.h +++ b/test/unit/swim_test_utils.h @@ -59,6 +59,11 @@ swim_cluster_set_gc(struct swim_cluster *cluster, enum swim_gc_mode gc_mode); void swim_cluster_delete(struct swim_cluster *cluster); +/** Update UUID of a SWIM instance with id @a i. */ +int +swim_cluster_update_uuid(struct swim_cluster *cluster, int i, + const struct tt_uuid *new_uuid); + /** Check that an error in diag contains @a msg. */ bool swim_error_check_match(const char *msg); -- 2.17.2 (Apple Git-113)