From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id C17EB6B9F21; Tue, 19 Mar 2024 11:19:22 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org C17EB6B9F21 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1710836362; bh=q3LxWlOCz6sAM94T3POypaBKrU1VK/aOio68KceoDwk=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=UgJieWyId2a84tVX/zriXtIYf4Fttp+OriGMmlM6dFqQZUfzR/N3w7ff14z0KOZIV ilT+aAcxVN9eN9SYOmb+i+dIkgAgLeaVUmBrnk+SdhP7Ako/T1yX2oZ9wbDcX3/c98 XiR4VZ7vATBER+p3bPm1D8rwYhypV2C4sDb+nx5Q= Received: from smtp46.i.mail.ru (smtp46.i.mail.ru [95.163.41.84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 2B7D06B9F21 for ; Tue, 19 Mar 2024 11:19:21 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 2B7D06B9F21 Received: by smtp46.i.mail.ru with esmtpa (envelope-from ) id 1rmUgp-0000000GH6v-42iZ; Tue, 19 Mar 2024 11:19:20 +0300 Message-ID: Date: Tue, 19 Mar 2024 11:19:19 +0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: Maxim Kokryashkin , Sergey Bronnikov References: <9a71bf0765acd6ab019de4ae9f491a6c7bcb463d.1710416150.git.sergeyb@tarantool.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Mailru-Src: smtp X-4EC0790: 10 X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD987C0EE6E7F0A597DA4A6782E56D87B648E017E9626570F41182A05F538085040DAC2EE9B5AAB14DE479CDAE959BF64244BF5CF7237000C7212676347BA7B586231DEC98D3E96EC86 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7484B509D84968742EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006378957B77C126D2D948638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D868B7EE998538A36A908DDBF545E3E25A016C42CA7A7E2574CC7F00164DA146DAFE8445B8C89999728AA50765F7900637CAEE156C82D3D7D9389733CBF5DBD5E9C8A9BA7A39EFB766F5D81C698A659EA7CC7F00164DA146DA9985D098DBDEAEC8062BEEFFB5F8EA3EF6B57BC7E6449061A352F6E88A58FB86F5D81C698A659EA73AA81AA40904B5D9A18204E546F3947CD166953D3EA3826BAD7EC71F1DB884274AD6D5ED66289B523666184CF4C3C14F6136E347CC761E07725E5C173C3A84C32C2390917000C441BA3038C0950A5D36B5C8C57E37DE458B330BD67F2E7D9AF16D1867E19FE14079C09775C1D3CA48CF90BCC82C2C62A6D11DD303D21008E298D5E8D9A59859A8B64854413538E1713F75ECD9A6C639B01B78DA827A17800CE7CD707F342D9BDC98731C566533BA786AA5CC5B56E945C8DA X-C1DE0DAB: 0D63561A33F958A56A532CC7609C40885002B1117B3ED696E3D834786587392C5D145BB8EF0DE66B823CB91A9FED034534781492E4B8EEADC24E78AA85F86F6CBDAD6C7F3747799A X-C8649E89: 1C3962B70DF3F0ADE00A9FD3E00BEEDF3FED46C3ACD6F73ED3581295AF09D3DF87807E0823442EA2ED31085941D9CD0AF7F820E7B07EA4CF10DA5C502C07F56D5D02CD959B4A27B3B0ADC1D77C15EE780DF40D80C143DBC0D917AD9EEFBA970EE1E51F4E63A65834118FD2072D66A3CFD5BA33FD7A26A51D0AF7A86A30275A68C226CC413062362A913E6812662D5F2AB9AF64DB4688768036DF5FE9C0001AF333F2C28C22F508233FCF178C6DD14203 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojipsw8XZ5OuijYgAElz9CoA== X-Mailru-Sender: 520A125C2F17F0B1E52FEF5D219D6140B8BD7B8D3984AFC46AD21358905A8704DC4C1E57CA5A8B5D0152A3D17938EB451EB5A0BCEC6A560B3DDE9B364B0DF289BE2DA36745F2EEB5CEBA01FB949A1F1EEAB4BC95F72C04283CDA0F3B3F5B9367 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH luajit 1/2] OSX/iOS/ARM64: Fix generation of Mach-O object files. X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Sergey Bronnikov via Tarantool-patches Reply-To: Sergey Bronnikov Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Max, thanks for review! see comments below On 3/18/24 15:53, Maxim Kokryashkin wrote: > Hi, Sergey, thanks for the patch! > Please consider my comments below. > On Thu, Mar 14, 2024 at 02:39:49PM +0300, Sergey Bronnikov wrote: >> From: sergeyb@tarantool.org >> >> Thanks to Carlo Cabrera. >> >> (cherry picked from commit 3065c910ad6027031aabe2dfd3c26a3d0f014b4f) >> >> Mach-O FAT object constructed by LuaJIT had an incorrect format. The > Typo: s/Mach-O FAT/The Mach-O FAT/ Fixed. >> problem is reproduced when target hardware platform has AVX512F and > Typo:s/target/the target/ Fixed. >> LuaJIT is compiled with enabled AVX512F instructions. >> >> The problem is arise because LuaJIT FFI code for Mach-O file generation > Typo: s/is arise/arises/ Fixed. >> in `bcsave.lua` relied on undefined behavior for conversions to > Typo: s/relied/relies/ Fixed. >> `uint32_t`. AVX512F has the `vcvttsd2usi` instruction which converts > Typo: s/instruction/instruction,/ Fixed. >> `double`/`float` to `uint32_t/uint64_t`. Earlier architectures (SSE2, >> AVX2) are sorely lacking such an instruction, as they only support >> signed conversions. Unsigned conversions are done with a signed convert >> and range shifting - the exact algorithm depends on the compiler. >> A side-effect of these workarounds is that negative `double`/`float` >> often inadvertently convert 'as expected', even though this is invoking >> undefined behavior. Whereas `vcvttsd2usi` always returns 0x80000000 or >> 0x8000000000000000 for out-of-range inputs. >> >> The patch fixes the problem, however, the real issue remains unfixed. >> >> Sergey Bronnikov: >> * added the description and a test for the problem > Typo: s/a test/the test/ Fixed. >> Part of tarantool/tarantool#9595 >> --- >> .github/workflows/exotic-builds-testing.yml | 5 +- >> src/jit/bcsave.lua | 6 +- >> .../lj-366-strtab-correct-size.test.lua | 10 +- >> ...generation_of_mach-o_object_files.test.lua | 271 ++++++++++++++++++ >> test/tarantool-tests/utils/tools.lua | 8 + >> 5 files changed, 287 insertions(+), 13 deletions(-) >> create mode 100644 test/tarantool-tests/lj-865-fix_generation_of_mach-o_object_files.test.lua >> >> diff --git a/.github/workflows/exotic-builds-testing.yml b/.github/workflows/exotic-builds-testing.yml >> index a9ba5fd5..df4bc2e9 100644 >> --- a/.github/workflows/exotic-builds-testing.yml >> +++ b/.github/workflows/exotic-builds-testing.yml >> @@ -32,6 +32,7 @@ jobs: >> fail-fast: false >> matrix: >> BUILDTYPE: [Debug, Release] >> + OS: [Linux, macOS] >> ARCH: [ARM64, x86_64] >> GC64: [ON, OFF] >> FLAVOR: [checkhook, dualnum, gdbjit, nojit, nounwind] >> @@ -50,13 +51,15 @@ jobs: >> FLAVORFLAGS: -DLUAJIT_USE_GDBJIT=ON > https://github.com/tarantool/luajit/actions/runs/8279362128 >> - FLAVOR: nounwind >> FLAVORFLAGS: -DLUAJIT_NO_UNWIND=ON >> + - FLAVOR: avx512 >> + CMAKEFLAGS: -DCMAKE_C_FLAGS=skylake-avx512 -DCMAKE_C_COMPILER=gcc >> exclude: >> - ARCH: ARM64 >> GC64: OFF >> # DUALNUM is default for ARM64, no need for additional testing. >> - FLAVOR: dualnum >> ARCH: ARM64 >> - runs-on: [self-hosted, regular, Linux, '${{ matrix.ARCH }}'] >> + runs-on: [self-hosted, regular, Linux, '${{ matrix.ARCH }}', '${ matrix.OS }'] > The matrix.OS variable should be wrapped with double curly braces, > instead of singular ones. So, it should be like this: > | '${{ matrix.OS }}' Fixed, thanks! > Currently, exotic build testing fails to start because of this mistake. > https://github.com/tarantool/luajit/actions/runs/8279362128 >> name: > >> LuaJIT ${{ matrix.FLAVOR }} >> (Linux/${{ matrix.ARCH }}) >> diff --git a/src/jit/bcsave.lua b/src/jit/bcsave.lua >> index a287d675..7aec1555 100644 >> --- a/src/jit/bcsave.lua >> +++ b/src/jit/bcsave.lua >> @@ -446,18 +446,18 @@ typedef struct { >> uint32_t value; >> } mach_nlist; >> typedef struct { >> - uint32_t strx; >> + int32_t strx; >> uint8_t type, sect; >> uint16_t desc; >> uint64_t value; >> } mach_nlist_64; >> typedef struct >> { >> - uint32_t magic, nfat_arch; >> + int32_t magic, nfat_arch; >> } mach_fat_header; >> typedef struct >> { >> - uint32_t cputype, cpusubtype, offset, size, align; >> + int32_t cputype, cpusubtype, offset, size, align; >> } mach_fat_arch; >> typedef struct { >> struct { >> diff --git a/test/tarantool-tests/lj-366-strtab-correct-size.test.lua b/test/tarantool-tests/lj-366-strtab-correct-size.test.lua >> index 8a97a441..0bb92da6 100644 >> --- a/test/tarantool-tests/lj-366-strtab-correct-size.test.lua >> +++ b/test/tarantool-tests/lj-366-strtab-correct-size.test.lua > Let's move this update to a separate patch alongside with the added > utility function. Currently, this change in unrelated test file is > a bit confusing. Moved to a separate patch: commit 138bcda850ed4aad4bbbae4d30315d9a0934cb26 Author: Sergey Bronnikov Date:   Tue Mar 19 09:56:51 2024 +0300     test: introduce a helper read_file     The test `lj-366-strtab-correct-size.test.lua` has a test helper     `read_file` that read a file's content and returns it.     This helper will be useful for a test upcoming in the next commit,     so it is moved to a test tools. diff --git a/test/tarantool-tests/lj-366-strtab-correct-size.test.lua b/test/tarantool-tests/lj-366-strtab-correct-size.test.lua index 8a97a441..0bb92da6 100644 --- a/test/tarantool-tests/lj-366-strtab-correct-size.test.lua +++ b/test/tarantool-tests/lj-366-strtab-correct-size.test.lua @@ -138,14 +138,6 @@ local function create_obj_file(name)    return elf_filename  end --- Reads a file located in a specified path and returns its content. -local function read_file(path) -  local file = assert(io.open(path), 'cannot open an object file') -  local content = file:read('*a') -  file:close() -  return content -end -  -- Parses a buffer in an ELF format and returns an offset and a size of strtab  -- and symtab sections.  local function read_elf(elf_content) @@ -172,7 +164,7 @@ end  test:plan(3)  local elf_filename = create_obj_file(MODULE_NAME) -local elf_content = read_file(elf_filename) +local elf_content = require('utils').tools.read_file(elf_filename)  assert(#elf_content ~= 0, 'cannot read an object file')  local strtab, symtab = read_elf(elf_content) diff --git a/test/tarantool-tests/utils/tools.lua b/test/tarantool-tests/utils/tools.lua index f35c6922..26b8c08d 100644 --- a/test/tarantool-tests/utils/tools.lua +++ b/test/tarantool-tests/utils/tools.lua @@ -12,4 +12,12 @@ function M.profilename(name)    return (arg[0]:gsub('^(.+)/([^/]+)%.test%.lua$', replacepattern))  end +-- Reads a file located in a specified path and returns its content. +function M.read_file(path) +  local file = assert(io.open(path), 'cannot open an object file') +  local content = file:read('*a') +  file:close() +  return content +end +  return M >> @@ -138,14 +138,6 @@ local function create_obj_file(name) >> return elf_filename >> end >> >> --- Reads a file located in a specified path and returns its content. >> -local function read_file(path) >> - local file = assert(io.open(path), 'cannot open an object file') >> - local content = file:read('*a') >> - file:close() >> - return content >> -end >> - >> -- Parses a buffer in an ELF format and returns an offset and a size of strtab >> -- and symtab sections. >> local function read_elf(elf_content) >> @@ -172,7 +164,7 @@ end >> test:plan(3) >> >> local elf_filename = create_obj_file(MODULE_NAME) >> -local elf_content = read_file(elf_filename) >> +local elf_content = require('utils').tools.read_file(elf_filename) >> assert(#elf_content ~= 0, 'cannot read an object file') >> >> local strtab, symtab = read_elf(elf_content) >> diff --git a/test/tarantool-tests/lj-865-fix_generation_of_mach-o_object_files.test.lua b/test/tarantool-tests/lj-865-fix_generation_of_mach-o_object_files.test.lua >> new file mode 100644 >> index 00000000..0519e134 >> --- /dev/null >> +++ b/test/tarantool-tests/lj-865-fix_generation_of_mach-o_object_files.test.lua >> @@ -0,0 +1,271 @@ >> +local tap = require('tap') >> +local test = tap.test('lj-865-fix_generation_of_mach-o_object_files'):skipcond({ >> + -- XXX: Tarantool doesn't use default LuaJIT loaders, and Lua >> + -- bytecode can't be loaded from the shared library. For more > Typo: s/the shared/a shared/ Fixed. >> + -- info: https://github.com/tarantool/tarantool/issues/9671. >> + -- luacheck: no global >> + ['Test uses exotic type of loaders (see #9671)'] = _TARANTOOL, > Typo: s/uses exotic/uses an exotic/ Fixed. >> +}) >> + >> +test:plan(4) >> + >> +-- Test creates an object file in Mach-O format with LuaJIT bytecode > Typo: s/Test/The test/ Fixed. >> +-- and checks validness of the object file fields. > Typo: s/validness/the validity/ Fixed. >> +-- >> +-- The original problem is reproduced with LuaJIT that built with >> +-- enabled AVX512F instructions. The support of AVX512F could be > Typo: s/support of/support for/ Fixed. >> +-- checked in `/proc/cpuinfo` on Linux and >> +-- `sysctl hw.optional.avx512f` on Mac. AVX512F must be >> +-- implicitly enabled in a C compiler by passing CPU codename. >> +-- Please consult for available model architecture on GCC Online > Typo: s/for/the/ Fixed. >> +-- Documentation [1] for available CPU codenames. To detect >> +-- CPU codename execute `gcc -march=native -Q --help=target | grep march`. >> +-- >> +-- 1. https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html >> +-- >> +-- Manual steps for reproducing are the following: >> +-- >> +-- $ make CC=gcc TARGET_CFLAGS='skylake-avx512' -f Makefile.original >> +-- $ echo > test.lua >> +-- $ LUA_PATH="src/?.lua;;" luajit -b -o osx -a arm test.lua test.o >> +-- $ file test.o >> +-- empty.o: DOS executable (block device driver) >> + >> +local ffi = require('ffi') >> + >> +-- Format of the Mach-O is described in a document > Typo: s/a document/the document/ Fixed. >> +-- "OS X ABI Mach-O File Format Reference", published by Apple company. >> +-- Copy of the (now removed) official documentation in [1]. > Let's replace "in [1]" with "can be found here [1].". Fixed. >> +-- Yet another source of thruth is a XNU headers, see the definition > Typo: s/is a XNU/are XNU/ > Typo: s/the definition/definitions/ Fixed. >> +-- of C-structures in: [2] (`nlist_64`), [3] (`fat_arch` and `fat_header`). >> + >> +-- 1. https://github.com/aidansteele/osx-abi-macho-file-format-reference >> +-- 2. https://github.com/apple-oss-distributions/xnu/blob/xnu-10002.1.13/EXTERNAL_HEADERS/mach-o/nlist.h >> +-- 3. https://github.com/apple-oss-distributions/xnu/blob/xnu-10002.1.13/EXTERNAL_HEADERS/mach-o/fat.h >> +-- 4. https://developer.apple.com/documentation/apple-silicon/addressing-architectural-differences-in-your-macos-code >> +-- >> +-- Using the same declarations as defined in . >> +ffi.cdef[[ >> +typedef struct >> +{ >> + uint32_t magic, cputype, cpusubtype, filetype, ncmds, sizeofcmds, flags; >> +} mach_header; >> + >> +typedef struct >> +{ >> + mach_header; uint32_t reserved; >> +} mach_header_64; >> + >> +typedef struct { >> + uint32_t cmd, cmdsize; >> + char segname[16]; >> + uint32_t vmaddr, vmsize, fileoff, filesize; >> + uint32_t maxprot, initprot, nsects, flags; >> +} mach_segment_command; >> + >> +typedef struct { >> + uint32_t cmd, cmdsize; >> + char segname[16]; >> + uint64_t vmaddr, vmsize, fileoff, filesize; >> + uint32_t maxprot, initprot, nsects, flags; >> +} mach_segment_command_64; >> + >> +typedef struct { >> + char sectname[16], segname[16]; >> + uint32_t addr, size; >> + uint32_t offset, align, reloff, nreloc, flags; >> + uint32_t reserved1, reserved2; >> +} mach_section; >> + >> +typedef struct { >> + char sectname[16], segname[16]; >> + uint64_t addr, size; >> + uint32_t offset, align, reloff, nreloc, flags; >> + uint32_t reserved1, reserved2, reserved3; >> +} mach_section_64; >> + >> +typedef struct { >> + uint32_t cmd, cmdsize, symoff, nsyms, stroff, strsize; >> +} mach_symtab_command; >> + >> +typedef struct { >> + int32_t strx; >> + uint8_t type, sect; >> + int16_t desc; >> + uint32_t value; >> +} mach_nlist; >> + >> +typedef struct { >> + uint32_t strx; >> + uint8_t type, sect; >> + uint16_t desc; >> + uint64_t value; >> +} mach_nlist_64; >> + >> +typedef struct >> +{ >> + uint32_t magic, nfat_arch; >> +} mach_fat_header; >> + >> +typedef struct >> +{ >> + uint32_t cputype, cpusubtype, offset, size, align; >> +} mach_fat_arch; >> + >> +typedef struct { >> + mach_fat_header fat; >> + mach_fat_arch fat_arch[2]; >> + struct { >> + mach_header hdr; >> + mach_segment_command seg; >> + mach_section sec; >> + mach_symtab_command sym; >> + } arch[2]; >> + mach_nlist sym_entry; >> + uint8_t space[4096]; >> +} mach_fat_obj; >> + >> +typedef struct { >> + mach_fat_header fat; >> + mach_fat_arch fat_arch[2]; >> + struct { >> + mach_header_64 hdr; >> + mach_segment_command_64 seg; >> + mach_section_64 sec; >> + mach_symtab_command sym; >> + } arch[2]; >> + mach_nlist_64 sym_entry; >> + uint8_t space[4096]; >> +} mach_fat_obj_64; >> +]] >> + >> +local function create_obj_file(name, arch) >> + local mach_o_path = os.tmpname() .. '.o' >> + local lua_path = os.getenv('LUA_PATH') >> + local lua_bin = require('utils').exec.luacmd(arg):match('%S+') >> + local cmd_fmt = 'LUA_PATH="%s" %s -b -n "%s" -o osx -a %s -e "print()" %s' >> + local cmd = (cmd_fmt):format(lua_path, lua_bin, name, arch, mach_o_path) >> + local ret = os.execute(cmd) >> + assert(ret == 0, 'cannot create an object file') >> + return mach_o_path >> +end >> + >> +-- Parses a buffer in an Mach-O format and returns > Typo: s/in an/in the/ Fixed. >> +-- an fat magic number and nfat_arch. > Typo: s/an fat/the FAT/ > Typo: s/nfat_arch/`nfat_arch` Fixed. >> +local function read_mach_o(buf, is64) >> + local res = { >> + header = { >> + magic = 0, >> + nfat_arch = 0, >> + }, >> + fat_arch = { >> + }, > I guess, that the formatting below is a bit better here: > | fat_arch = {}, Fixed. >> + } >> + >> + -- Mach-O FAT object. >> + local mach_fat_obj_type = ffi.typeof(is64 and 'mach_fat_obj_64 *' or 'mach_fat_obj *') > The line is longer than 80 symbols. Fixed. >> + local obj = ffi.cast(mach_fat_obj_type, buf) >> + >> + -- Mach-O FAT object header. >> + local mach_fat_header_type = ffi.typeof('mach_fat_header *') >> + local mach_fat_header = ffi.cast(mach_fat_header_type, obj.fat) >> + local be32 = bit.bswap -- Mach-O FAT is BE, target arch is LE. >> + res.header.magic = be32(mach_fat_header.magic) >> + res.header.nfat_arch = be32(mach_fat_header.nfat_arch) >> + >> + -- Mach-O FAT object archs. > Typo: s/archs/arches/ Fixed. > Side note: I feel like the comments for the sections are not elaborate > enough for unprepared reader. I think you should briefly desribe the > basic structure of a FAT object (FAT header, then array of per-segment > headers, then object files) Added a short description and an ASCII scheme. >> + local mach_fat_arch_type = ffi.typeof('mach_fat_arch *') >> + for i = 0, res.header.nfat_arch - 1 do >> + local fat_arch = ffi.cast(mach_fat_arch_type, obj.fat_arch[i]) >> + arch = { >> + cputype = be32(fat_arch.cputype), >> + cpusubtype = be32(fat_arch.cpusubtype), >> + } >> + table.insert(res.fat_arch, arch) >> + end >> + >> + return res >> +end >> + >> +-- Defined in . >> +local sum_cputype = { >> + x86 = 7, >> + x64 = 0x01000007, >> + arm = 7 + 12, >> + arm64 = 0x01000007 + 0x0100000c, >> +} >> +local sum_cpusubtype = { >> + x86 = 3, >> + x64 = 3, >> + arm = 3 + 9, >> + arm64 = 3 + 0, >> +} > It would be nice to have an explanation for these magic numbers. Added: @@ -183,7 +246,13 @@ local function read_mach_o(buf, is64)    return res  end --- Defined in . +-- Universal Binary can contain executables for more than one +-- CPU architecture. For simplicity the test compares +-- sum of CPU types and CPU subtypes. +-- Numbers below are defined in . +-- Original source is XNU source code, [1]. +-- +-- 1. https://opensource.apple.com/source/xnu/xnu-4570.41.2/osfmk/mach/machine.h.auto.html  local sum_cputype = {    x86 = 7,    x64 = 0x01000007, >> + >> +-- The function builds Mach-O FAT object file and retrieves >> +-- its header fields (magic and nfat_arch) >> +-- and fields of the each arch (cputype, cpusubtype). >> +-- >> +-- Mach-O FAT object header could be retrieved with `otool` on macOS: > Typo: s/could be/can be/ Fixed. >> +-- >> +-- $ otool -f empty.o >> +-- Fat headers >> +-- fat_magic 0xcafebabe >> +-- nfat_arch 2 >> +-- >> +-- >> +-- CPU type and subtype could be retrieved with `lipo` on macOS: > Typo: s/could be/can be/ Fixed. >> +-- >> +-- $ luajit -b -o osx -a arm empty.lua empty.o >> +-- $ lipo -archs empty.o >> +-- i386 armv7 >> +-- $ luajit -b -o osx -a arm64 empty.lua empty.o >> +-- $ lipo -archs empty.o >> +-- x86_64 arm64 >> +local function build_and_check_mach_o(is64) >> + local arch = is64 and 'arm64' or 'arm' >> + >> + -- FAT_MAGIC is an integer containing the value 0xCAFEBABE in >> + -- big-endian byte order format. On a big-endian host CPU, >> + -- this can be validated using the constant FAT_MAGIC; >> + -- on a little-endian host CPU, it can be validated using >> + -- the constant FAT_CIGAM. >> + -- >> + -- FAT_NARCH is an integer specifying the number of fat_arch >> + -- data structures that follow. This is the number of >> + -- architectures contained in this binary. >> + -- >> + -- See aforementioned "OS X ABI Mach-O File Format Reference". >> + -- >> + local FAT_MAGIC = '0xffffffffcafebabe' >> + local FAT_NARCH = 2 >> + >> + local MODULE_NAME = 'lango_team' >> + >> + local mach_o_obj_path = create_obj_file(MODULE_NAME, arch) >> + local mach_o_buf = require('utils').tools.read_file(mach_o_obj_path) >> + assert(mach_o_buf == nil or #mach_o_buf ~= 0, 'cannot read an object file') >> + >> + local mach_o = read_mach_o(mach_o_buf, is64) >> + >> + -- Teardown. >> + local retcode = os.remove(mach_o_obj_path) >> + assert(retcode == true, 'remove an object file') >> + >> + local magic_str = string.format('0x%02x', mach_o.header.magic) >> + test:is(magic_str, FAT_MAGIC, 'fat_magic is correct in Mach-O, ' .. arch) >> + test:is(mach_o.header.nfat_arch, FAT_NARCH, 'nfat_arch is correct in Mach-O, ' .. arch) >> + >> + local total_cputype = 0 >> + local total_cpusubtype = 0 >> + for i = 1, mach_o.header.nfat_arch do >> + total_cputype = total_cputype + mach_o.fat_arch[i].cputype >> + total_cpusubtype = total_cpusubtype + mach_o.fat_arch[i].cpusubtype >> + end >> + test:is(total_cputype, sum_cputype[arch], 'cputype is correct in Mach-O, ' .. arch) >> + test:is(total_cpusubtype, sum_cpusubtype[arch], 'cpusubtype is correct in Mach-O, ' .. arch) >> +end >> + >> +-- ARM >> +build_and_check_mach_o(false) >> + >> +test:done(true) > Please mention that alongside with test and fix for the issue > you've added this tool. IMO, it would be even better to do that > in a separate commit, to avoid confusion because of the updates in > test files unrelated to the patch. Helper is moved to a separate patch. >> diff --git a/test/tarantool-tests/utils/tools.lua b/test/tarantool-tests/utils/tools.lua >> index f35c6922..26b8c08d 100644 >> --- a/test/tarantool-tests/utils/tools.lua >> +++ b/test/tarantool-tests/utils/tools.lua >> @@ -12,4 +12,12 @@ function M.profilename(name) >> return (arg[0]:gsub('^(.+)/([^/]+)%.test%.lua$', replacepattern)) >> end >> >> +-- Reads a file located in a specified path and returns its content. >> +function M.read_file(path) >> + local file = assert(io.open(path), 'cannot open an object file') >> + local content = file:read('*a') >> + file:close() >> + return content >> +end >> + >> return M >> -- >> 2.34.1 >>