From: Cyrill Gorcunov <gorcunov@gmail.com> To: tml <tarantool-patches@freelists.org> Cc: Vladimir Davydov <vdavydov.dev@gmail.com>, Cyrill Gorcunov <gorcunov@gmail.com> Subject: [PATCH v2] core/coio_file: Use eio_sendfile_sync instead of a chunk mode Date: Tue, 16 Apr 2019 00:14:10 +0300 [thread overview] Message-ID: <20190415211410.20562-1-gorcunov@gmail.com> (raw) eio library provides a portable version of sendfile syscall which works more efficient than explicit copying fileby 4K chunks. Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com> --- v2: - Use sendfile in a cycle to address files more than 2G size - Proper testing of large files remains opened: current CI engine is hardly capable of managing it. I tested 200M files with manual splitting (to make sure the offsets do really work) but for longterm we still might need to invent something - Another question which remains -- what to do with partially copied files, neither code before the patch or after do not clean up parts of copied data. Should not we clean it up on error path? If yes then I'll prepare another patch on top. src/lib/core/coio_file.c | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/src/lib/core/coio_file.c b/src/lib/core/coio_file.c index c5b2db781..541e0f05a 100644 --- a/src/lib/core/coio_file.c +++ b/src/lib/core/coio_file.c @@ -571,7 +571,7 @@ static void coio_do_copyfile(eio_req *req) { struct coio_file_task *eio = (struct coio_file_task *)req->data; - + ssize_t pos, ret, left; struct stat st; if (stat(eio->copyfile.source, &st) < 0) { goto error; @@ -588,22 +588,18 @@ coio_do_copyfile(eio_req *req) goto error_dest; } - enum { COPY_FILE_BUF_SIZE = 4096 }; - - char buf[COPY_FILE_BUF_SIZE]; - - while (true) { - ssize_t nread = fio_read(source_fd, buf, sizeof(buf)); - if (nread < 0) - goto error_copy; - - if (nread == 0) - break; /* eof */ - - ssize_t nwritten = fio_writen(dest_fd, buf, nread); - if (nwritten < 0) + for (left = st.st_size, pos = 0; left > 0;) { + ret = eio_sendfile_sync(dest_fd, source_fd, pos, left); + if (ret < 0) { + say_syserror("sendfile, [%s -> %s]", + fio_filename(source_fd), + fio_filename(dest_fd)); goto error_copy; + } + pos += ret; + left -= ret; } + req->result = 0; close(source_fd); close(dest_fd); -- 2.20.1
next reply other threads:[~2019-04-15 21:14 UTC|newest] Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-04-15 21:14 Cyrill Gorcunov [this message] 2019-04-16 17:35 ` Vladimir Davydov 2019-04-16 17:51 ` Cyrill Gorcunov 2019-04-17 8:47 ` Vladimir Davydov
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190415211410.20562-1-gorcunov@gmail.com \ --to=gorcunov@gmail.com \ --cc=tarantool-patches@freelists.org \ --cc=vdavydov.dev@gmail.com \ --subject='Re: [PATCH v2] core/coio_file: Use eio_sendfile_sync instead of a chunk mode' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox