From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 6C7605D740A; Mon, 4 Sep 2023 19:34:04 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 6C7605D740A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1693845244; bh=aXlAUWO6oK+F8J9QnNX3/9Oza27XoboWjCWJmJHUMFo=; h=Date:To:Cc:References:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=kxgXhtXIUrbT6M9TsItUO8IOyxNhABXrIaTaTL3G/ps9Zu0l5AZVVNIIjX0caf9wm T5yeGylRkF+fz+qNtF5UcDQD5Xwizwdnzal47dlCme2H0nVQAzGMrxynlLA/4cCs6d wscyaNHEF64evqGqV2xccWWl5l252VQlBypSdj4s= Received: from smtpng1.i.mail.ru (smtpng1.i.mail.ru [94.100.181.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id BFEDE5D6F18 for ; Mon, 4 Sep 2023 19:34:02 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org BFEDE5D6F18 Received: by smtpng1.m.smailru.net with esmtpa (envelope-from ) id 1qdCWY-0004Er-3G; Mon, 04 Sep 2023 19:34:02 +0300 Message-ID: <59ea8e26-071f-1819-50c8-c63ee582505d@tarantool.org> Date: Mon, 4 Sep 2023 19:34:01 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Content-Language: en-US To: Maxim Kokryashkin , Sergey Bronnikov Cc: max.kokryashkin@gmail.com, tarantool-patches@dev.tarantool.org References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: 78E4E2B564C1792B X-77F55803: 4F1203BC0FB41BD96E142CFC92DB15CD1C47060D0B6C1F571E4C5B5C7DEF200F182A05F5380850406FE2261BAB82BDDB27E62A853A02B209923013E48AB8AD467A661363DBE8EA67 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7F2393C4755A27B53EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F790063727C65896DA7AF7D78638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D818F9DD98A47CFB926540F0B01B5870EE117882F4460429724CE54428C33FAD305F5C1EE8F4F765FCB6FC2C91A742FF12A471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F44604297287769387670735201E561CDFBCA1751FF6B57BC7E6449061A352F6E88A58FB86F5D81C698A659EA7E827F84554CEF5019E625A9149C048EE9ECD01F8117BC8BEE2021AF6380DFAD18AA50765F790063735872C767BF85DA227C277FBC8AE2E8B0E0583254ED37F2E75ECD9A6C639B01B4E70A05D1297E1BBCB5012B2E24CD356 X-B7AD71C0: 1B70FBA5C9BEEE72C9761FC34675ADEB871C96603B655635EE9D5CB6078CC77C4270D1979CEC2865EE319BF62A11F40A X-C1DE0DAB: 0D63561A33F958A5AF83E46A03405C63340168F2258E0276ECFA93933024B4D9F87CCE6106E1FC07E67D4AC08A07B9B06FF1B927F95F3316CB5012B2E24CD356 X-C8649E89: 1C3962B70DF3F0ADBF74143AD284FC71106E36FF2641B7B8424CF958EAFF5D571004E42C50DC4CA955A7F0CF078B5EC49A30900B95165D3447DF5779098ECEE90883CE018FABE0345DB3FD9E07778EDD060B31CD2043383E65E8A8EE6C1FCCCB1D7E09C32AA3244CF65E7CCE751FACB9AFCBBE7CE148242630363D8B7DA7DD44BAD658CF5C8AB4025DA084F8E80FEBD3FFA33E6B6B2F82C47A83BD0C44CE203720ABEDE4BBDD9CDD X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojNmsqR+t6vc/kQ7yYygaoCw== X-DA7885C5: D30D80AA3AABE0BDBF53D802DB38F1598468ED0ADACCD1B4F74D08EC3F52285A262E2D401490A4A0DB037EFA58388B346E8BC1A9835FDE71 X-Mailru-Sender: 689FA8AB762F73930F533AC2B33E986BBCBD9DCBF766C937A724B47DE37E95ABDD788429FD8613638ED9BB8B05EE7B3AFB559BB5D741EB96D19CD4E7312BAA970A04DAD6CC59E3365FEEDEB644C299C0ED14614B50AE0675 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit 2/2][v2] Followup fix for embedded bytecode loader. X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Bronnikov via Tarantool-patches Reply-To: Sergey Bronnikov Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hi, Max On 9/1/23 13:05, Maxim Kokryashkin via Tarantool-patches wrote: > On Thu, Aug 31, 2023 at 02:32:14PM +0300, Sergey Bronnikov via Tarantool-patches wrote: >> From: Sergey Bronnikov >> + >> +/** >> + * Function generates a huge chunk of "bytecode" with a size bigger than >> + * LJ_MAX_BUF. Generated chunk must enable endmark in a Lex state. >> + */ >> +static const char * >> +bc_reader_with_endmark(lua_State *L, void *data, size_t *size) >> +{ >> + UNUSED(data); >> + int bc_chunk_size = (size_t)0; >> + static char *bc_chunk = NULL; >> + free(bc_chunk); > What's the point of free here? Why the buffer is static? Because callee (aka Reader) is responsible for buffer, reader initializes it once and then reuse. > The block must exist until the reader function is called again. > To signal the end of the chunk, the reader must return NULL or set size to zero. > The reader function may return pieces of any size greater than zero. 1. http://www.lua.org/manual/5.1/manual.html#lua_Reader >> + >> + bc_chunk = malloc(bc_chunk_size); > Malloc of zero size doesn't seem to be the thing you wanted to do. Right. Updated: @@ -33,26 +33,9 @@ static const char *  bc_reader_with_endmark(lua_State *L, void *data, size_t *size)  {      UNUSED(data); -    int bc_chunk_size = (size_t)0; -    static char *bc_chunk = NULL; -    free(bc_chunk); - -    bc_chunk = malloc(bc_chunk_size); -    assert(bc_chunk != NULL); - -    /** -     * `lua_load` automatically detects whether the chunk is text or binary, -     * and loads it accordingly. We need a trace for bytecode input, -     * so it is necessary to deceive a check in lj_lex_setup, that -     * makes a sanity check and detects whether input is bytecode or text -     * by the first char. Put LUA_SIGNATURE[0] at the beginning of the -     * allocated region. -     */ -    bc_chunk[0] = LUA_SIGNATURE[0]; - -    *size = bc_chunk_size; +    *size = ~(size_t)0; -    return bc_chunk; +    return NULL;  }  static int bc_loader_with_endmark(void *test_state) >> + assert(bc_chunk != NULL); >> + >> + /** >> + * `lua_load` automatically detects whether the chunk is text or binary, >> + * and loads it accordingly. We need a trace for bytecode input, >> + * so it is necessary to deceive a check in lj_lex_setup, that >> + * makes a sanity check and detects whether input is bytecode or text >> + * by the first char. Put LUA_SIGNATURE[0] at the beginning of the >> + * allocated region. >> + */ >> + bc_chunk[0] = LUA_SIGNATURE[0]; >> + >> + *size = bc_chunk_size; >> + >> + return bc_chunk; >> +} >> + >> +static int bc_loader_with_endmark(void *test_state) >> +{ >> + lua_State *L = test_state; >> + void *ud = NULL; >> + int res = lua_load(L, bc_reader_with_endmark, ud, "endmark"); >> + >> + /* >> + * Make sure we passed the condition with lj_err_mem in the function >> + * `lex_more`. >> + */ >> + assert_true(res != LUA_ERRMEM); >> + >> + return TEST_EXIT_SUCCESS; >> +} >> + >> +enum bc_emission_state { >> + EMIT_BC, >> + EMIT_EOF, >> +}; >> + >> +typedef struct { >> + enum bc_emission_state state; >> +} dt; >> + >> +/** >> + * Function returns a bytecode chunk on the first call and NULL and size equal >> + * to zero on the second call. Triggers the END_OF_STREAM flag in the function >> + * `lex_more`. >> + */ >> +static const char * >> +bc_reader_with_eof(lua_State *L, void *data, size_t *size) >> +{ >> + UNUSED(data); >> + UNUSED(L); >> + dt *test_data = (dt *)data; >> + if (test_data->state == EMIT_EOF) { > This section is unreachable, isn't it? Right, fixed it. >> + *size = 0; >> + return NULL; >> + } >> + >> + static char *bc_chunk = NULL; >> + free(bc_chunk); > Ditto. >> + >> + size_t sz = 10; > Is there any reason for it to be exactly 10? Drop a comment. Set it to 2 and dropped a comment. @@ -98,10 +81,23 @@ bc_reader_with_eof(lua_State *L, void *data, size_t *size)      static char *bc_chunk = NULL;      free(bc_chunk); -    size_t sz = 10; +    /** +     * Minimal size of a buffer with bytecode: +     * signiture (1 byte) and a bytecode itself (1 byte). +     */ +    size_t sz = 2;      bc_chunk = malloc(sz); +    /** +     * `lua_load` automatically detects whether the chunk is text or binary, +     * and loads it accordingly. We need a trace for *bytecode* input, +     * so it is necessary to deceive a check in `lj_lex_setup`, that +     * makes a sanity check and detects whether input is bytecode or text +     * by the first char. Put `LUA_SIGNATURE[0]` at the beginning of the +     * allocated region. +     */      bc_chunk[0] = LUA_SIGNATURE[0];      *size = sz; +    test_data->state = EMIT_EOF;      return bc_chunk;  }