[PATCH 1/1] iproto: protect from false-correct size in msg header

[Vladislav Shpilevoy]

Consider this packet:

    msgpack = require('msgpack')
    data = msgpack.encode(18400000000000000000)..'aaaaaaa'

Tarantool interprets 18400000000000000000 as size of a coming
iproto request, and tries with no any checks to allocate buffer
of such size. It calculates needed capacity like this:

    capacity = start_value;
    while (capacity < size)
        capacity *= 2;

Here it is possible that on i-th iteration 'capacity' < 'size',
but 'capacity * 2' overflows 64 bits and becomes < 'size' again,
so this loop never ends and occupies 100% CPU.

Strictly speaking overflow has undefined behavior. On the
original system it led to nullifying 'capacity'.

Such size is improbable as a real packet gabarits, but can appear
as a result of parsing of some invalid packet, first bytes of
which accidentally appears to be valid MessagePack uint. This is
how the bug emerged on the real system.

Lets restrict the maximal packet size to 2GB.

Closes #3464
---
Branch: https://github.com/tarantool/tarantool/tree/gerold103/gh-3464-iproto-100-cpu
Issue: https://github.com/tarantool/tarantool/issues/3464

 src/box/iproto.cc         | 15 +++++++++++++--
 test/box/net.box.result   | 22 ++++++++++++++++++++++
 test/box/net.box.test.lua | 10 ++++++++++
 3 files changed, 45 insertions(+), 2 deletions(-)

diff --git a/src/box/iproto.cc b/src/box/iproto.cc
index 0b92c316e..d3765eb1d 100644
--- a/src/box/iproto.cc
+++ b/src/box/iproto.cc
@@ -64,7 +64,10 @@
 #include "cfg.h"
 
 /* The number of iproto messages in flight */
-enum { IPROTO_MSG_MAX = 768 };
+enum {
+	IPROTO_MSG_MAX = 768,
+	IPROTO_PACKET_SIZE_MAX = 2UL * 1024 * 1024 * 1024,
+};
 
 /**
  * Network readahead. A signed integer to avoid
@@ -524,8 +527,16 @@ iproto_connection_input_buffer(struct iproto_connection *con)
 	/* The type code is checked in iproto_enqueue_batch() */
 	if (con->parse_size) {
 		const char *pos = old_ibuf->wpos - con->parse_size;
-		if (mp_check_uint(pos, old_ibuf->wpos) <= 0)
+		if (mp_check_uint(pos, old_ibuf->wpos) <= 0) {
 			to_read = mp_decode_uint(&pos);
+			if (to_read > IPROTO_PACKET_SIZE_MAX) {
+				const char *err =
+					tt_sprintf("too big packet size in "\
+						   "the header: %llu",
+						   (unsigned long long)to_read);
+				tnt_raise(ClientError, ER_INVALID_MSGPACK, err);
+			}
+		}
 	}
 
 	if (ibuf_unused(old_ibuf) >= to_read)
diff --git a/test/box/net.box.result b/test/box/net.box.result
index aa418b87d..7180f7f66 100644
--- a/test/box/net.box.result
+++ b/test/box/net.box.result
@@ -2360,6 +2360,28 @@ box.internal.collation.drop('test')
 space:drop()
 ---
 ...
+--
+-- gh-3464: iproto hangs in 100% CPU when too big packet size
+-- is received due to size_t overflow.
+--
+c = net:connect(box.cfg.listen)
+---
+...
+data = msgpack.encode(18400000000000000000)..'aaaaaaa'
+---
+...
+c._transport.perform_request(nil, nil, 'inject', nil, data)
+---
+- 77
+- Peer closed
+...
+c:close()
+---
+...
+test_run:grep_log('default', 'too big packet size in the header') ~= nil
+---
+- true
+...
 box.schema.user.revoke('guest', 'read,write,execute', 'universe')
 ---
 ...
diff --git a/test/box/net.box.test.lua b/test/box/net.box.test.lua
index 4cadd3b29..1e716ca89 100644
--- a/test/box/net.box.test.lua
+++ b/test/box/net.box.test.lua
@@ -961,4 +961,14 @@ c:close()
 box.internal.collation.drop('test')
 space:drop()
 
+--
+-- gh-3464: iproto hangs in 100% CPU when too big packet size
+-- is received due to size_t overflow.
+--
+c = net:connect(box.cfg.listen)
+data = msgpack.encode(18400000000000000000)..'aaaaaaa'
+c._transport.perform_request(nil, nil, 'inject', nil, data)
+c:close()
+test_run:grep_log('default', 'too big packet size in the header') ~= nil
+
 box.schema.user.revoke('guest', 'read,write,execute', 'universe')
-- 
2.15.1 (Apple Git-101)