[tarantool-patches] [PATCH 1/1] swim: optimize memory layout of struct swim_member

[tarantool-patches] [PATCH 1/1] swim: optimize memory layout of struct swim_member

[Vladislav Shpilevoy]

Struct swim_member describes attributes of one remote member of
a SWIM cluster. It is relatively often accessed. And it has two
huge structures - struct swim_packet ping/ack_task. Each is 1500
bytes. When these tasks are in the middle of the structure, they
split and spoil cache lines.

This patch moves the whole failure detection attribute family to
the bottom of the structure, and moves these two tasks to the
end of layout.
---
Branch: https://github.com/tarantool/tarantool/tree/gerold103/swim-struct-member-layout

 src/lib/swim/swim.c | 68 ++++++++++++++++++++++-----------------------
 1 file changed, 34 insertions(+), 34 deletions(-)

diff --git a/src/lib/swim/swim.c b/src/lib/swim/swim.c
index 230ec52d4..7c4685a9f 100644
--- a/src/lib/swim/swim.c
+++ b/src/lib/swim/swim.c
@@ -231,40 +231,6 @@ struct swim_member {
 	 * Position in a queue of members in the current round.
 	 */
 	struct rlist in_round_queue;
-	/**
-	 *
-	 *               Failure detection component
-	 */
-	/**
-	 * A monotonically growing number to refute old member's
-	 * state, characterized by a triplet
-	 * {incarnation, status, address}.
-	 */
-	uint64_t incarnation;
-	/**
-	 * How many recent pings did not receive an ack while the
-	 * member was in the current status. When this number
-	 * reaches a configured threshold the instance is marked
-	 * as dead. After a few more unacknowledged it is removed
-	 * from the member table. This counter is reset on each
-	 * acknowledged ping, status or incarnation change.
-	 */
-	int unacknowledged_pings;
-	/**
-	 * A deadline when we stop expecting a response to the
-	 * ping and account it as unacknowledged.
-	 */
-	double ping_deadline;
-	/** Ready at hand regular ACK task. */
-	struct swim_task ack_task;
-	/** Ready at hand regular PING task. */
-	struct swim_task ping_task;
-	/**
-	 * Position in a queue of members waiting for an ack.
-	 * A member is added to the queue when we send a ping
-	 * message to it.
-	 */
-	struct heap_node in_wait_ack_heap;
 	/**
 	 *
 	 *                 Dissemination component
@@ -344,6 +310,40 @@ struct swim_member {
 	 * time.
 	 */
 	struct rlist in_dissemination_queue;
+	/**
+	 *
+	 *               Failure detection component
+	 */
+	/**
+	 * A monotonically growing number to refute old member's
+	 * state, characterized by a triplet
+	 * {incarnation, status, address}.
+	 */
+	uint64_t incarnation;
+	/**
+	 * How many recent pings did not receive an ack while the
+	 * member was in the current status. When this number
+	 * reaches a configured threshold the instance is marked
+	 * as dead. After a few more unacknowledged it is removed
+	 * from the member table. This counter is reset on each
+	 * acknowledged ping, status or incarnation change.
+	 */
+	int unacknowledged_pings;
+	/**
+	 * A deadline when we stop expecting a response to the
+	 * ping and account it as unacknowledged.
+	 */
+	double ping_deadline;
+	/**
+	 * Position in a queue of members waiting for an ack.
+	 * A member is added to the queue when we send a ping
+	 * message to it.
+	 */
+	struct heap_node in_wait_ack_heap;
+	/** Ready at hand regular ACK task. */
+	struct swim_task ack_task;
+	/** Ready at hand regular PING task. */
+	struct swim_task ping_task;
 };
 
 #define mh_name _swim_table
-- 
2.20.1 (Apple Git-117)

[tarantool-patches] Re: [PATCH 1/1] swim: optimize memory layout of struct swim_member

[Konstantin Osipov]

* Vladislav Shpilevoy <v.shpilevoy@tarantool.org> [19/04/25 00:46]:
> Struct swim_member describes attributes of one remote member of
> a SWIM cluster. It is relatively often accessed. And it has two
> huge structures - struct swim_packet ping/ack_task. Each is 1500
> bytes. When these tasks are in the middle of the structure, they
> split and spoil cache lines.
> 
> This patch moves the whole failure detection attribute family to
> the bottom of the structure, and moves these two tasks to the
> end of layout.
> ---

OK to push.


-- 
Konstantin Osipov, Moscow, Russia, +7 903 626 22 32
http://tarantool.io - www.twitter.com/kostja_osipov

[tarantool-patches] Re: [PATCH 1/1] swim: optimize memory layout of struct swim_member

[Vladislav Shpilevoy]

Pushed into the master.

On 25/04/2019 13:53, Konstantin Osipov wrote:
> * Vladislav Shpilevoy <v.shpilevoy@tarantool.org> [19/04/25 00:46]:
>> Struct swim_member describes attributes of one remote member of
>> a SWIM cluster. It is relatively often accessed. And it has two
>> huge structures - struct swim_packet ping/ack_task. Each is 1500
>> bytes. When these tasks are in the middle of the structure, they
>> split and spoil cache lines.
>>
>> This patch moves the whole failure detection attribute family to
>> the bottom of the structure, and moves these two tasks to the
>> end of layout.
>> ---
> 
> OK to push.
> 
> 
> -- 
> Konstantin Osipov, Moscow, Russia, +7 903 626 22 32
> http://tarantool.io - www.twitter.com/kostja_osipov
>