[tarantool-patches] [PATCH 4/4] swim: do not rebuild packet meta multiple times

[Vladislav Shpilevoy <v.shpilevoy@tarantool.org>]

Before the patch there were 2 cases when an unchanged packet was
rebuilt partially on each send:

  - cached round message's meta section was rebuilt on each
    EV_WRITE event in swim_scheduler_on_output() function;

  - broadcast message's meta section was rebuilt too even though
    its content does not depend on a broadcast interface.

The third case appears with indirect pings patch which aggravates
meta building business by routing and packet forwarding. When a
packet needs to be forwarded farther, its meta is built in a
special manner preserving the route before EV_WRITE appears, and
on_output should not touch that meta.

This patch adds a check preventing unnecessary meta rebuilds.
Besides, the check and the meta building code are moved into a
dedicated function out of swim_scheduler_on_output() - it allows
to completely split logic of packing a message and sending it.
Separated logic helps a lot when indirect pings are introduced.

Part of #3234
---
 src/lib/swim/swim_io.c | 29 ++++++++++++++++++++++++++---
 1 file changed, 26 insertions(+), 3 deletions(-)

diff --git a/src/lib/swim/swim_io.c b/src/lib/swim/swim_io.c
index 03d79ae3e..7d6addf02 100644
--- a/src/lib/swim/swim_io.c
+++ b/src/lib/swim/swim_io.c
@@ -57,6 +57,31 @@ swim_packet_create(struct swim_packet *packet)
 	swim_packet_alloc_meta(packet, sizeof(struct swim_meta_header_bin));
 }
 
+/** Fill metadata prefix of a packet. */
+static inline void
+swim_packet_build_meta(struct swim_packet *packet,
+		       const struct sockaddr_in *src)
+{
+	char *meta = packet->meta;
+	char *end = packet->body;
+	/*
+	 * Meta has already been built. It happens when the same
+	 * task is resent multiple times.
+	 */
+	if (meta == end)
+		return;
+	struct swim_meta_header_bin header;
+	swim_meta_header_bin_create(&header, src);
+	assert(meta + sizeof(header) == end);
+	memcpy(meta, &header, sizeof(header));
+	/*
+	 * Once meta is built, it is consumed by the body. Used
+	 * not to rebuild the meta again if the task will be
+	 * scheduled again without changes in data.
+	 */
+	packet->body = packet->meta;
+}
+
 void
 swim_task_create(struct swim_task *task, swim_task_f complete,
 		 swim_task_f cancel, const char *desc)
@@ -298,9 +323,7 @@ swim_scheduler_on_output(struct ev_loop *loop, struct ev_io *io, int events)
 	say_verbose("SWIM %d: send %s to %s", swim_scheduler_fd(scheduler),
 		    task->desc, sio_strfaddr((struct sockaddr *) &task->dst,
 					     sizeof(task->dst)));
-	struct swim_meta_header_bin header;
-	swim_meta_header_bin_create(&header, &scheduler->transport.addr);
-	memcpy(task->packet.meta, &header, sizeof(header));
+	swim_packet_build_meta(&task->packet, &scheduler->transport.addr);
 	int rc = swim_transport_send(&scheduler->transport, task->packet.buf,
 				     task->packet.pos - task->packet.buf,
 				     (const struct sockaddr *) &task->dst,
-- 
2.17.2 (Apple Git-113)