[patches] [fio 1/1] fio: Fix race condition in fio.read

Georgy Kirichenko georgy at tarantool.org
Thu Mar 1 17:44:40 MSK 2018


Please add branch hyperlink to the message and fix one codestyle typo below.
After it will be Ok
Thanks!
On Wednesday, February 28, 2018 6:18:38 PM MSK imarkov wrote:
> When read followed by yield from multiple fibers is performed,
> ibuf shared among several fibers is corrupted and read data is
> mixed.
> 
> The fix is to create new ibuf for each fio.read call in case
> buffer is not specified.
> 
> Closes #3187
> ---
Add github branch url
>  src/lua/fio.lua       |  3 +--
>  test/app/fio.result   | 74
> +++++++++++++++++++++++++++++++++++++++++++++++++++ test/app/fio.test.lua |
> 33 +++++++++++++++++++++++
>  3 files changed, 108 insertions(+), 2 deletions(-)
> 
> diff --git a/src/lua/fio.lua b/src/lua/fio.lua
> index d48c709..73719df 100644
> --- a/src/lua/fio.lua
> +++ b/src/lua/fio.lua
> @@ -40,8 +40,7 @@ fio_methods.read = function(self, buf, size)
>      end
>      if not ffi.istype(const_char_ptr_t, buf) then
>          size = buf or size
> -        tmpbuf = buffer.IBUF_SHARED
> -        tmpbuf:reset()
> +        tmpbuf = buffer.ibuf()
>          buf = tmpbuf:reserve(size)
>      end
>      local res, err = internal.read(self.fh, buf, size)
> diff --git a/test/app/fio.result b/test/app/fio.result
> index 0fc8582..1f12384 100644
> --- a/test/app/fio.result
> +++ b/test/app/fio.result
> @@ -1067,6 +1067,80 @@ fio.unlink(tmpfile)
>  ---
>  - true
>  ...
> +tmp1 = fio.pathjoin(tmpdir, "tmp1")
> +---
> +...
> +tmp2= fio.pathjoin(tmpdir, "tmp2")
> +---
> +...
> +test_run:cmd("setopt delimiter ';'")
> +---
> +- true
> +...
> +function write_big_file(name, odd)
> +    local fh = fio.open(name, { 'O_RDWR', 'O_TRUNC', 'O_CREAT' }, 0777)
> +    if odd then
> +        for i = 1, 100000, 2 do
> +            fh:write(i)
> +        end
> +    else
> +        for i = 2, 100000,2 do
add a space after comma
> +            fh:write(i)
> +        end
> +    end
> +    fh:write(name)
> +    fh:seek(0)
> +    return fh
> +end;
> +---
> +...
> +test_run:cmd("setopt delimiter ''");
> +---
> +- true
> +...
> +fh1 = write_big_file(tmp1)
> +---
> +...
> +fh2 = write_big_file(tmp2)
> +---
> +...
> +fiber = require('fiber')
> +---
> +...
> +digest = require('digest')
> +---
> +...
> +str = fh1:read()
> +---
> +...
> +fh1:seek(0)
> +---
> +- 0
> +...
> +hash = digest.crc32(str)
> +---
> +...
> +ch = fiber.channel(1)
> +---
> +...
> +f1 = fiber.create(function() str = fh1:read() ch:put(digest.crc32(str))
> end) +---
> +...
> +f2 = fiber.create(function() str = fh2:read() end)
> +---
> +...
> +ch:get() == hash
> +---
> +- true
> +...
> +fio.unlink(tmp1)
> +---
> +- true
> +...
> +fio.unlink(tmp2)
> +---
> +- true
> +...
>  fio.rmdir(tmpdir)
>  ---
>  - true
> diff --git a/test/app/fio.test.lua b/test/app/fio.test.lua
> index cd0bc16..8c37baa 100644
> --- a/test/app/fio.test.lua
> +++ b/test/app/fio.test.lua
> @@ -342,4 +342,37 @@ fio.path.lexists(link)
> 
>  fio.unlink(link)
>  fio.unlink(tmpfile)
> +tmp1 = fio.pathjoin(tmpdir, "tmp1")
> +tmp2= fio.pathjoin(tmpdir, "tmp2")
> +test_run:cmd("setopt delimiter ';'")
> +function write_big_file(name, odd)
> +    local fh = fio.open(name, { 'O_RDWR', 'O_TRUNC', 'O_CREAT' }, 0777)
> +    if odd then
> +        for i = 1, 100000, 2 do
> +            fh:write(i)
> +        end
> +    else
> +        for i = 2, 100000,2 do
> +            fh:write(i)
> +        end
> +    end
> +    fh:write(name)
> +    fh:seek(0)
> +    return fh
> +end;
> +test_run:cmd("setopt delimiter ''");
> +fh1 = write_big_file(tmp1)
> +fh2 = write_big_file(tmp2)
> +fiber = require('fiber')
> +digest = require('digest')
> +str = fh1:read()
> +fh1:seek(0)
> +hash = digest.crc32(str)
> +ch = fiber.channel(1)
> +f1 = fiber.create(function() str = fh1:read() ch:put(digest.crc32(str))
> end) +f2 = fiber.create(function() str = fh2:read() end)
> +ch:get() == hash
> +
> +fio.unlink(tmp1)
> +fio.unlink(tmp2)
>  fio.rmdir(tmpdir)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part.
URL: <https://lists.tarantool.org/pipermail/tarantool-patches/attachments/20180301/7ee3152b/attachment.sig>


More information about the Tarantool-patches mailing list