[PATCH v2] core: Non-blocking io.popen
Stanislav Zudin
szudin at tarantool.org
Wed May 29 10:08:09 MSK 2019
Adds nonblocking implementation of popen.
The method is available in namespace fio.
fio.popen() returns an object providing facilities
for dealing with standard input/output, getting
status of the child process.
Closes #4031
@TarantoolBot document
Title: Nonblocking fio.popen
handle, err = fio.popen(parameters)
fio.popen starts a process and redirects its input/output.
parameters - a table containing arguments to run a process.
The following arguments are expected:
argv - [mandatory] is a table of argument strings passed to
the new program. By convention, the first of these strings should
contain the filename associated with the file being executed.
environment - [optional] is a table of strings, conventionally of the
form key=value, which are passed as environment to the new program.
By default stdin, stdout,stderr of the associated process are
available for writing/reading using object's methods
handle:write() and handle:read().
One can override default behavior to redirect streams to/from file
or to input/output of another process or to the default input/output
of the parent process.
stdin - [optional] overrides the child process's standard input.
stdout - [optional] overrides the child process's standard output.
stderr - [optional] overrides the the child process's standard
error output.
May accept one of the following values:
Handle of the file open with fio.open()
Handle of the standard input/output of another process,
open by fio.popen
A constant defining the parent's STDIN, STDOUT or STDERR.
A constants:
fio.PIPE - Opens a file descriptor available for reading/writing
or redirection to/from another process.
fio.DEVNULL - Makes fio.popen redirect the output to /dev/null.
On success returns an object providing methods for
communication with the running process.
Returns nil if underlying functions calls fail;
in this case the second return value, err, contains a error message.
The object created by fio.popen provides the following methods:
read()
read2()
write()
kill()
wait()
get_status()
get_stdin()
get_stdout()
get_stderr()
number handle:get_stdin()
Returns handle of the child process's standard input.
The handle is available only if it wasn't redirected earlier.
Use this handle to setup a redirection
from file or other process to the input of the associated process.
If handle is unavailable the method returns nil.
number handle:get_stdout()
number handle:get_stderr()
Return STDOUT and STDIN of the associated process accordingly.
See handle:get_stdin() for details.
rc,err = handle:wait(timeout)
The wait() waits for the associated process to terminate
and returns the exit status of the command.
timeout - an integer specifies number of seconds to wait.
If the requested time has elapsed the method returns nil,
the second return value, err, contains a error message.
To distinguish timeout from the the other errors use errno.
If timeout is nil, the method waits infinitely until
the associated process is terminated.
On success function returns an exit code as a positive number
or signal id as a negative number.
If failed, rc is nul and err contains a error message.
If the associated process is terminated, one can use the following
methods get the exit status:
rc = handle:get_status()
returns nil if process is still running
>= 0 if process exited normally
< 0 if process was terminated by a signal
rc, err = handle:kill(sig)
The kill() sends a specified signal to the associated process.
On success the method returns true, if failed - nil and error message.
If the sig is nil the default "SIGTERM" is being sent to the process.
If the signal is unknown then the method fails (with error EINVAL).
The following signals are acceptable:
"SIGINT"
"SIGILL"
"SIGABRT"
"SIGFPE"
"SIGSEGV"
"SIGTERM"
"SIGHUP"
"SIGQUIT"
"SIGTRAP"
"SIGKILL"
"SIGBUS"
"SIGSYS"
"SIGPIPE"
"SIGALRM"
"SIGURG"
"SIGSTOP"
"SIGTSTP"
"SIGCONT"
"SIGCHLD"
"SIGTTIN"
"SIGTTOU"
"SIGPOLL"
"SIGXCPU"
"SIGXFSZ"
"SIGVTALRM"
"SIGPROF"
"SIGUSR1"
"SIGUSR2"
rc,src,err = handle:read(buffer,size)
rc,src,err = handle:read2(buffer,size,seconds)
read stdout & stderr of the process started by fio.popen
read() -> str, source
read(buf) -> len, source
read(size) -> str, source
read(buf, size) -> len, source
read2(seconds) -> str, source
read2(buf,seconds) -> len, source
read2(size,seconds) -> str, source
read2(buf, size,seconds) -> len, source
seconds - an integer specifies number of seconds to wait.
If the requested time has elapsed the method returns nil.
src contains id of the stream: fio.STDOUT or fio.STDERR.
If method failed the rc and src are nil and err contains error message.
rc, err = handle:write(data, length)
Writes specified number of bytes
On success returns number of written bytes.
If failed the rc is nil and err contains an error message.
---
Branch: https://github.com/tarantool/tarantool/tree/stanztt/gh-4031-nonblocking-popen
Issue: https://github.com/tarantool/tarantool/issues/4031
src/CMakeLists.txt | 1 +
src/lib/core/CMakeLists.txt | 1 +
src/lib/core/coio_file.c | 243 +++++++++++
src/lib/core/coio_file.h | 30 ++
src/lib/core/coio_popen.c | 661 ++++++++++++++++++++++++++++++
src/lib/core/coio_popen.h | 243 +++++++++++
src/lib/core/coio_task.c | 2 +
src/lib/core/fiber.h | 4 +-
src/lua/fio.c | 289 +++++++++++++
src/lua/fio.lua | 262 ++++++++++++
src/lua/init.c | 2 +
src/lua/lua_signal.c | 99 +++++
src/lua/lua_signal.h | 45 ++
src/main.cc | 21 +-
test/box-tap/fio_popen.sample.txt | 5 +
test/box-tap/fio_popen.test.lua | 189 +++++++++
test/box-tap/fio_popen_test1.sh | 7 +
third_party/libeio/eio.c | 19 +-
third_party/libeio/etp.c | 28 ++
third_party/libev/ev.c | 2 +
20 files changed, 2149 insertions(+), 4 deletions(-)
create mode 100644 src/lib/core/coio_popen.c
create mode 100644 src/lib/core/coio_popen.h
create mode 100644 src/lua/lua_signal.c
create mode 100644 src/lua/lua_signal.h
create mode 100644 test/box-tap/fio_popen.sample.txt
create mode 100755 test/box-tap/fio_popen.test.lua
create mode 100755 test/box-tap/fio_popen_test1.sh
diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index 68674d06a..cfe46dfe3 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -117,6 +117,7 @@ set (server_sources
lua/info.c
lua/string.c
lua/buffer.c
+ lua/lua_signal.c
${lua_sources}
${PROJECT_SOURCE_DIR}/third_party/lua-yaml/lyaml.cc
${PROJECT_SOURCE_DIR}/third_party/lua-yaml/b64.c
diff --git a/src/lib/core/CMakeLists.txt b/src/lib/core/CMakeLists.txt
index eb10b11c3..8b1f8d32e 100644
--- a/src/lib/core/CMakeLists.txt
+++ b/src/lib/core/CMakeLists.txt
@@ -26,6 +26,7 @@ set(core_sources
trigger.cc
mpstream.c
port.c
+ coio_popen.c
)
if (TARGET_OS_NETBSD)
diff --git a/src/lib/core/coio_file.c b/src/lib/core/coio_file.c
index 3359f42bc..93956e533 100644
--- a/src/lib/core/coio_file.c
+++ b/src/lib/core/coio_file.c
@@ -34,6 +34,7 @@
#include "say.h"
#include "fio.h"
#include "errinj.h"
+#include "coio_popen.h"
#include <stdio.h>
#include <stdlib.h>
#include <dirent.h>
@@ -103,6 +104,36 @@ struct coio_file_task {
const char *source;
const char *dest;
} copyfile;
+
+ struct {
+ char** argv;
+ int argc;
+ char** environment;
+ int environment_size;
+ int stdin_fd;
+ int stdout_fd;
+ int stderr_fd;
+ void *handle;
+ } popen_params;
+
+ struct {
+ void *handle;
+ } pclose_params;
+
+ struct {
+ void *handle;
+ const void* buf;
+ size_t count;
+ size_t *written;
+ } popen_write;
+
+ struct {
+ void *handle;
+ void *buf;
+ size_t count;
+ size_t *read_bytes;
+ int *output_number;
+ } popen_read;
};
};
@@ -631,3 +662,215 @@ coio_copyfile(const char *source, const char *dest)
eio_req *req = eio_custom(coio_do_copyfile, 0, coio_complete, &eio);
return coio_wait_done(req, &eio);
}
+
+static void
+coio_do_popen(eio_req *req)
+{
+ struct coio_file_task *eio = (struct coio_file_task *)req->data;
+ eio->popen_params.handle = coio_popen_impl(eio->popen_params.argv,
+ eio->popen_params.argc,
+ eio->popen_params.environment,
+ eio->popen_params.environment_size,
+ eio->popen_params.stdin_fd,
+ eio->popen_params.stdout_fd,
+ eio->popen_params.stderr_fd);
+
+ eio->result = 0;
+ eio->errorno = errno;
+}
+
+void *
+coio_popen(char** argv, int argc,
+ char** environment, int environment_size,
+ int stdin_fd, int stdout_fd, int stderr_fd)
+{
+ INIT_COEIO_FILE(eio);
+ eio.popen_params.argv = argv;
+ eio.popen_params.argc = argc;
+ eio.popen_params.environment = environment;
+ eio.popen_params.environment_size = environment_size;
+ eio.popen_params.stdin_fd = stdin_fd;
+ eio.popen_params.stdout_fd = stdout_fd;
+ eio.popen_params.stderr_fd = stderr_fd;
+
+ eio_req *req = eio_custom(coio_do_popen, 0,
+ coio_complete, &eio);
+ coio_wait_done(req, &eio);
+ return eio.popen_params.handle;
+}
+
+static void
+coio_do_popen_read(eio_req *req)
+{
+ struct coio_file_task *eio = (struct coio_file_task *)req->data;
+
+ int rc = coio_popen_try_to_read(eio->popen_read.handle,
+ eio->popen_read.buf,
+ eio->popen_read.count,
+ eio->popen_read.read_bytes,
+ eio->popen_read.output_number);
+
+ req->result = rc;
+ req->errorno = errno;
+}
+
+static int
+coio_do_nonblock_popen_read(void *fh, void *buf, size_t count,
+ size_t *read_bytes, int *source_id)
+{
+ INIT_COEIO_FILE(eio);
+ eio.popen_read.buf = buf;
+ eio.popen_read.count = count;
+ eio.popen_read.handle = fh;
+ eio.popen_read.read_bytes = read_bytes;
+ eio.popen_read.output_number = source_id;
+ eio_req *req = eio_custom(coio_do_popen_read, 0,
+ coio_complete, &eio);
+ return coio_wait_done(req, &eio);
+}
+
+ssize_t
+coio_popen_read(void *fh, void *buf, size_t count,
+ int *output_number, int timeout)
+{
+ size_t received = 0;
+ int rc = coio_popen_try_to_read(fh, buf, count,
+ &received, output_number);
+ if (rc == 0) /* The reading's succeeded */
+ return (ssize_t)received;
+ else if (rc == -1 && errno != EAGAIN) /* Failed */
+ return -1;
+
+ /* A blocking operation is expected */
+
+ time_t start_tt;
+ time(&start_tt);
+
+ bool in_progress;
+ do {
+ if (fiber_is_cancelled()) {
+ errno = EINTR;
+ return -1;
+ }
+
+ if (timeout > 0) {
+ time_t tt;
+ time(&tt);
+ if ((tt - start_tt) > timeout) {
+ errno = ETIMEDOUT;
+ return -1;
+ }
+ }
+
+ rc = coio_do_nonblock_popen_read(fh, buf, count,
+ &received, output_number);
+ in_progress = (rc == -1 && errno == EAGAIN);
+ } while (in_progress);
+
+ return (rc == 0) ? (ssize_t)received
+ : -1;
+}
+
+static void
+coio_do_popen_write(eio_req *req)
+{
+ struct coio_file_task *eio = (struct coio_file_task *)req->data;
+
+ int rc = coio_popen_try_to_write(eio->popen_write.handle,
+ eio->popen_write.buf,
+ eio->popen_write.count,
+ eio->popen_write.written);
+
+ req->result = rc;
+ req->errorno = errno;
+}
+
+static int
+coio_do_nonblock_popen_write(void *fh, const void *buf, size_t count,
+ size_t *written)
+{
+ INIT_COEIO_FILE(eio);
+ eio.popen_write.buf = buf;
+ eio.popen_write.count = count;
+ eio.popen_write.handle = fh;
+ eio.popen_write.written = written;
+ eio_req *req = eio_custom(coio_do_popen_write, 0,
+ coio_complete, &eio);
+ return coio_wait_done(req, &eio);
+}
+
+ssize_t
+coio_popen_write(void *fh, const void *buf, size_t count)
+{
+ ssize_t total = 0;
+ size_t written = 0;
+ int rc = coio_popen_try_to_write(fh, buf, count,
+ &written);
+ if (rc == 0 && written == count) /* The writing's succeeded */
+ return (ssize_t)written;
+ else if (rc == -1 && errno != EAGAIN) /* Failed */
+ return -1;
+
+ /* A blocking operation is expected */
+ bool in_progress;
+
+ do {
+ if (fiber_is_cancelled()) {
+ errno = EINTR;
+ return -1;
+ }
+
+ buf += written; /* advance writing position */
+ total += (ssize_t)written;
+ count -= written;
+
+ if (count == 0)
+ return total;
+
+ written = 0;
+ rc = coio_do_nonblock_popen_write(fh, buf, count,
+ &written);
+ in_progress = (rc == 0 && written < count) ||
+ (rc == -1 && errno == EAGAIN);
+ } while (in_progress);
+
+ return (rc == 0) ? total
+ : -1;
+}
+
+int
+coio_popen_wait(void *fh, int timeout, int *exit_code)
+{
+ time_t start_tt;
+ time(&start_tt);
+
+ do {
+ /* Wait for SIGCHLD */
+ int sig = 0;
+ int code = 0;
+
+ int rc = coio_popen_get_status(fh, &sig, &code);
+ if (rc != POPEN_RUNNING) {
+ *exit_code = (rc == POPEN_EXITED) ? code
+ : -sig;
+ return 0;
+ }
+
+ /* Check for timeout */
+ if (timeout > 0) {
+ time_t tt;
+ time(&tt);
+ if ((tt - start_tt) > timeout) {
+ errno = ETIMEDOUT;
+ return -1;
+ }
+ }
+
+ fiber_yield_timeout(0);
+
+ } while(!fiber_is_cancelled());
+
+ errno = EINTR;
+ return -1;
+}
+
diff --git a/src/lib/core/coio_file.h b/src/lib/core/coio_file.h
index f2112ceed..4181796eb 100644
--- a/src/lib/core/coio_file.h
+++ b/src/lib/core/coio_file.h
@@ -84,6 +84,36 @@ int coio_tempdir(char *path, size_t path_len);
int coio_readdir(const char *path, char **buf);
int coio_copyfile(const char *source, const char *dest);
+
+void *
+coio_popen(char** argv, int argc,
+ char** environment, int environment_size,
+ int stdin_fd, int stdout_fd, int stderr_fd);
+
+ssize_t
+coio_popen_read(void *fh, void *buf, size_t count,
+ int *output_number, int timeout);
+
+ssize_t
+coio_popen_write(void *fh, const void *buf, size_t count);
+
+/**
+ * Wait for the associated process to terminate.
+ * The function doesn't release the allocated resources.
+ *
+ * @param fd handle returned by fio.popen.
+ *
+ * @param timeout number of second to wait before function exit with error.
+ * If function exited due to timeout the errno equals to ETIMEDOUT.
+ *
+ * @exit_code On success contains the exit code as a positive number
+ * or signal id as a negative number.
+
+ * @return On success function returns 0, and -1 on error.
+ */
+int
+coio_popen_wait(void *fh, int timeout, int *exit_code);
+
#if defined(__cplusplus)
} /* extern "C" */
#endif /* defined(__cplusplus) */
diff --git a/src/lib/core/coio_popen.c b/src/lib/core/coio_popen.c
new file mode 100644
index 000000000..b44cbf60b
--- /dev/null
+++ b/src/lib/core/coio_popen.c
@@ -0,0 +1,661 @@
+/*
+ * Copyright 2010-2019, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the
+ * following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <stddef.h>
+#include <signal.h>
+#include "coio_popen.h"
+#include "coio_task.h"
+#include "fiber.h"
+#include "say.h"
+#include "fio.h"
+#include <stdio.h>
+#include <stdlib.h>
+#include <dirent.h>
+#include <sys/wait.h>
+#include <paths.h>
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <bits/types/siginfo_t.h>
+#include <pthread.h>
+
+/*
+ * On OSX this global variable is not declared
+ * in <unistd.h>
+ */
+extern char **environ;
+
+
+struct popen_data {
+ /* process id */
+ pid_t pid;
+ int fh[3];
+ /*
+ * Three handles:
+ * [0] write to stdin of the child process
+ * [1] read from stdout of the child process
+ * [2] read from stderr of the child process
+ */
+
+ /* Handle to /dev/null */
+ int devnull_fd;
+
+ /* The ID of socket was read recently
+ * (STDERR_FILENO or STDOUT_FILENO */
+ int prev_source;
+
+ /*
+ * Current process status.
+ * The SIGCHLD handler changes this status.
+ */
+ enum popen_status status;
+
+ /* exit status of the associated process. */
+ int exit_code;
+
+ /*
+ * Number of the signal that caused the
+ * assosiated process to terminate
+ */
+ int signal_id;
+
+ /*
+ * The next entry in the global list
+ */
+ struct popen_data* next;
+};
+
+static pthread_mutex_t mutex;
+static struct popen_data* popen_data_list = NULL;
+
+void
+popen_initialize()
+{
+ pthread_mutexattr_t errorcheck;
+ pthread_mutexattr_init(&errorcheck);
+ pthread_mutexattr_settype(&errorcheck,
+ PTHREAD_MUTEX_ERRORCHECK);
+ pthread_mutex_init(&mutex, &errorcheck);
+ pthread_mutexattr_destroy(&errorcheck);
+}
+
+static void
+popen_lock_data_list()
+{
+ pthread_mutex_lock(&mutex);
+}
+
+static void
+popen_unlock_data_list()
+{
+ pthread_mutex_unlock(&mutex);
+}
+
+static void
+popen_append_to_list(struct popen_data *data)
+{
+ /* Insert into head of the list */
+ data->next = popen_data_list;
+ popen_data_list = data;
+}
+
+static struct popen_data *
+popen_lookup_data_by_pid(pid_t pid)
+{
+ struct popen_data *cur = popen_data_list;
+ for(; cur; cur = cur->next) {
+ if (cur->pid == pid)
+ return cur;
+ }
+
+ return NULL;
+}
+
+static void
+popen_exclude_from_list(struct popen_data *data)
+{
+ if (popen_data_list == data) {
+ popen_data_list = data->next;
+ return;
+ }
+
+ /* Find the previous entry */
+ struct popen_data *prev = popen_data_list;
+ for( ; prev && prev->next != data; prev = prev->next) {
+ /* empty */;
+ }
+
+ if (!prev)
+ return ;
+ assert(prev->next == data);
+ prev->next = data->next;
+}
+
+/*
+ * Returns next socket to read.
+ * Use this function when both STDOUT and STDERR outputs
+ * are ready for reading.
+ * */
+static inline int
+get_handle_in_order(struct popen_data *data)
+{
+ /*
+ * Invert the order of handles to be read
+ */
+ const int mask = STDERR_FILENO | STDOUT_FILENO;
+ data->prev_source ^= mask;
+
+ /*
+ * If handle is not available, invert it back
+ */
+ if (data->fh[data->prev_source] < 0)
+ data->prev_source ^= mask;
+ /*
+ * if both reading handles are invalid return -1
+ */
+ return data->fh[data->prev_source];
+}
+
+static struct popen_data *
+popen_data_new()
+{
+ struct popen_data *data =
+ (struct popen_data *)calloc(1, sizeof(*data));
+ data->fh[0] = -1;
+ data->fh[1] = -1;
+ data->fh[2] = -1;
+ data->devnull_fd = -1;
+
+ data->prev_source = STDERR_FILENO;
+ /*
+ * if both streams are ready then
+ * start reading from STDOUT
+ */
+
+ data->status = POPEN_RUNNING;
+ return data;
+}
+
+void *
+coio_popen_impl(char** argv, int argc,
+ char** environment, int environment_size,
+ int stdin_fd, int stdout_fd, int stderr_fd)
+{
+ pid_t pid;
+ int socket_rd[2] = {-1,-1};
+ int socket_wr[2] = {-1,-1};
+ int socket_er[2] = {-1,-1};
+ errno = 0;
+
+ struct popen_data *data = popen_data_new();
+ if (data == NULL)
+ return NULL;
+
+ /* Setup a /dev/null if necessary */
+ bool read_devnull = (stdin_fd == FIO_DEVNULL);
+ bool write_devnull = (stdout_fd == FIO_DEVNULL) ||
+ (stderr_fd == FIO_DEVNULL);
+ int devnull_flags = O_RDWR | O_CREAT;
+ if (!read_devnull)
+ devnull_flags = O_WRONLY | O_CREAT;
+ else if (!write_devnull)
+ devnull_flags = O_RDONLY | O_CREAT;
+
+ if (read_devnull || write_devnull) {
+ data->devnull_fd = open("/dev/null", devnull_flags, 0666);
+ if (data->devnull_fd < 0)
+ goto on_error;
+ else {
+ if (stdin_fd == FIO_DEVNULL)
+ stdin_fd = data->devnull_fd;
+ if (stdout_fd == FIO_DEVNULL)
+ stdout_fd = data->devnull_fd;
+ if (stderr_fd == FIO_DEVNULL)
+ stderr_fd = data->devnull_fd;
+ }
+ }
+
+ if (stdin_fd == FIO_PIPE) {
+ /*
+ * Enable non-blocking for the parent side
+ * and close-on-exec on the child's side.
+ * The socketpair on OSX doesn't support
+ * SOCK_NONBLOCK & SOCK_CLOEXEC flags.
+ */
+ if (socketpair(AF_UNIX, SOCK_STREAM, 0, socket_rd) < 0 ||
+ fcntl(socket_rd[0], F_SETFL, O_NONBLOCK) < 0 ||
+ fcntl(socket_rd[1], F_SETFD, FD_CLOEXEC) < 0) {
+ goto on_error;
+ }
+ }
+
+ if (stdout_fd == FIO_PIPE) {
+ if (socketpair(AF_UNIX, SOCK_STREAM, 0, socket_wr) < 0 ||
+ fcntl(socket_wr[0], F_SETFL, O_NONBLOCK) < 0 ||
+ fcntl(socket_wr[1], F_SETFD, FD_CLOEXEC) < 0) {
+ goto on_error;
+ }
+ }
+
+ if (stderr_fd == FIO_PIPE) {
+ if (socketpair(AF_UNIX, SOCK_STREAM, 0, socket_er) < 0 ||
+ fcntl(socket_er[0], F_SETFL, O_NONBLOCK) < 0 ||
+ fcntl(socket_er[1], F_SETFD, FD_CLOEXEC) < 0) {
+ goto on_error;
+ }
+ }
+
+ pid = fork();
+
+ if (pid < 0)
+ goto on_error;
+ else if (pid == 0) /* child */ {
+ /* Setup stdin/stdout */
+ if (stdin_fd == FIO_PIPE) {
+ close(socket_rd[0]); /* close parent's side */
+ stdin_fd = socket_rd[1];
+ }
+ if (stdin_fd != STDIN_FILENO) {
+ dup2(stdin_fd, STDIN_FILENO);
+ close(stdin_fd);
+ }
+
+ if (stdout_fd == FIO_PIPE) {
+ close(socket_wr[0]); /* close parent's side */
+ stdout_fd = socket_wr[1];
+ }
+ if (stdout_fd != STDOUT_FILENO) {
+ dup2(stdout_fd, STDOUT_FILENO);
+ if (stdout_fd != STDERR_FILENO)
+ close(stdout_fd);
+ }
+
+ if (stderr_fd == FIO_PIPE) {
+ close(socket_er[0]); /* close parent's side */
+ stderr_fd = socket_er[1];
+ }
+ if (stderr_fd != STDERR_FILENO) {
+ dup2(stderr_fd, STDERR_FILENO);
+ if (stderr_fd != STDOUT_FILENO)
+ close(stderr_fd);
+ }
+
+ execve( argv[0], argv,
+ environment ? environment : environ);
+ _exit(127);
+ unreachable();
+ }
+
+ /* parent process */
+ if (stdin_fd == FIO_PIPE) {
+ close(socket_rd[1]); /* close child's side */
+ data->fh[STDIN_FILENO] = socket_rd[0];
+ }
+
+ if (stdout_fd == FIO_PIPE) {
+ close(socket_wr[1]); /* close child's side */
+ data->fh[STDOUT_FILENO] = socket_wr[0];
+ }
+
+ if (stderr_fd == FIO_PIPE) {
+ close(socket_er[1]); /* close child's side */
+ data->fh[STDERR_FILENO] = socket_er[0];
+ }
+
+ data->pid = pid;
+
+on_cleanup:
+ if (argv){
+ for(int i = 0; i < argc; ++i)
+ free(argv[i]);
+ free(argv);
+ }
+ if (environment) {
+ for(int i = 0; i < environment_size; ++i)
+ free(environment[i]);
+ free(environment);
+ }
+
+ if (data){
+ popen_lock_data_list();
+ popen_append_to_list(data);
+ popen_unlock_data_list();
+ }
+
+ return data;
+
+on_error:
+ if (socket_rd[0] >= 0) {
+ close(socket_rd[0]);
+ close(socket_rd[1]);
+ }
+ if (socket_wr[0] >= 0) {
+ close(socket_wr[0]);
+ close(socket_wr[1]);
+ }
+ if (socket_er[0] >= 0) {
+ close(socket_er[0]);
+ close(socket_er[1]);
+ }
+
+ if (data) {
+ if (data->devnull_fd >= 0)
+ close(data->devnull_fd);
+ free(data);
+ }
+ data = NULL;
+
+ goto on_cleanup;
+ unreachable();
+}
+
+static void
+popen_close_handles(struct popen_data *data)
+{
+ for(int i = 0; i < 3; ++i) {
+ if (data->fh[i] >= 0) {
+ close(data->fh[i]);
+ data->fh[i] = -1;
+ }
+ }
+ if (data->devnull_fd >= 0) {
+ close(data->devnull_fd);
+ data->devnull_fd = -1;
+ }
+}
+
+int
+coio_popen_destroy(void *fh)
+{
+ struct popen_data *data = (struct popen_data *)fh;
+
+ if (data == NULL){
+ errno = EBADF;
+ return -1;
+ }
+
+ popen_lock_data_list();
+ popen_close_handles(data);
+ popen_exclude_from_list(data);
+ popen_unlock_data_list();
+
+ free(data);
+ return 0;
+}
+
+int
+coio_popen_try_to_read(void *fh, void *buf, size_t count,
+ size_t *read_bytes, int *source_id)
+{
+ struct popen_data *data = (struct popen_data *)fh;
+
+ if (data == NULL){
+ errno = EBADF;
+ return -1;
+ }
+
+ fd_set rfds;
+ FD_ZERO(&rfds);
+ ssize_t received = 0;
+ int num = 0;
+
+ if (data->fh[STDOUT_FILENO] >= 0) {
+ FD_SET(data->fh[STDOUT_FILENO], &rfds);
+ ++num;
+ }
+ if (data->fh[STDERR_FILENO] >= 0) {
+ FD_SET(data->fh[STDERR_FILENO], &rfds);
+ ++num;
+ }
+
+ if (num == 0) {
+ /*
+ * There are no open handles for reading
+ */
+ errno = EBADF;
+ return -1;
+ }
+
+ struct timeval tv = {0,0};
+ int max_h = MAX(data->fh[STDOUT_FILENO],
+ data->fh[STDERR_FILENO]);
+
+ errno = 0;
+ int retv = select(max_h + 1, &rfds, NULL, NULL, &tv);
+ switch (retv) {
+ case -1: /* Error */
+ return -1;
+ case 0: /* Not ready yet */
+ errno = EAGAIN;
+ return -1;
+ case 1: { /* One socket is ready */
+
+ /* Choose the socket */
+ int fno = STDOUT_FILENO;
+ if (!FD_ISSET(data->fh[fno], &rfds))
+ fno = STDERR_FILENO;
+ if (!FD_ISSET(data->fh[fno], &rfds)) {
+ unreachable();
+ return -1;
+ }
+
+ received = read(data->fh[fno], buf, count);
+ if (received < 0)
+ goto on_error;
+ data->prev_source = fno;
+ *read_bytes = received;
+ *source_id = fno;
+ return 0;
+ }
+ case 2: { /* Both sockets are ready */
+ int attempt = 2;
+second_attempt:
+ --attempt;
+ received = read(get_handle_in_order(data), buf, count);
+
+ if (received < 0)
+ goto on_error;
+ if (received == 0 && attempt > 0)
+ goto second_attempt;
+
+ *read_bytes = received;
+ *source_id = (int)data->prev_source;
+ return 0;
+ }
+ }
+
+ unreachable();
+
+on_error:
+ if (errno == EINTR) {
+ *read_bytes = 0;
+ errno = EAGAIN; /* Repeat */
+ }
+
+ return -1;
+}
+
+int
+coio_popen_try_to_write(void *fh, const void *buf, size_t count,
+ size_t *written)
+{
+ if (count == 0) {
+ *written = 0;
+ return 0;
+ }
+
+ struct popen_data *data = (struct popen_data *)fh;
+
+ if (data == NULL){
+ errno = EBADF;
+ return -1;
+ }
+
+ if (data->fh[STDIN_FILENO] < 0) {
+ /*
+ * There are no open handles for writing
+ */
+ errno = EBADF;
+ return -1;
+ }
+
+ fd_set wfds;
+ FD_ZERO(&wfds);
+
+ int wh = data->fh[STDIN_FILENO];
+ FD_SET(wh, &wfds);
+
+ struct timeval tv = {0,0};
+
+ int retv = select(wh + 1, NULL, &wfds, NULL, &tv);
+ if (retv < 0)
+ goto on_error;
+ else if (retv == 0) {
+ errno = EAGAIN;
+ return -1; /* Not ready yet */
+ }
+
+ assert(retv == 1); /* The socket is ready */
+
+ if (FD_ISSET(wh, &wfds)) {
+ ssize_t rc = write(wh, buf, count);
+ if (rc < 0)
+ goto on_error;
+ *written = rc;
+ return 0;
+ }
+
+ unreachable();
+ return -1;
+
+on_error:
+ if (errno == EINTR) {
+ *written = 0;
+ errno = EAGAIN; /* Repeat */
+ }
+
+ return -1;
+}
+
+int
+coio_popen_kill(void *fh, int signal_id)
+{
+ struct popen_data *data = (struct popen_data *)fh;
+
+ if (data == NULL){
+ errno = EBADF;
+ return -1;
+ }
+
+ return kill(data->pid, signal_id);
+}
+
+int
+coio_popen_get_std_file_handle(void *fh, int file_no)
+{
+ if (file_no < STDIN_FILENO || STDERR_FILENO < file_no){
+ errno = EINVAL;
+ return -1;
+ }
+
+ struct popen_data *data = (struct popen_data *)fh;
+
+ if (data == NULL){
+ errno = EBADF;
+ return -1;
+ }
+
+ errno = 0;
+ return data->fh[file_no];
+}
+
+int
+coio_popen_get_status(void *fh, int *signal_id, int *exit_code)
+{
+ struct popen_data *data = (struct popen_data *)fh;
+
+ if (data == NULL){
+ errno = EBADF;
+ return -1;
+ }
+
+ errno = 0;
+
+ if (signal_id)
+ *signal_id = data->signal_id;
+ if (exit_code)
+ *exit_code = data->exit_code;
+
+ return data->status;
+}
+
+void
+coio_popen_child_is_dead(int sig, siginfo_t *si, void *context)
+{
+ (void)context; /* UNUSED */
+
+ if (sig != SIGCHLD)
+ return;
+
+ /*
+ * The sigaction is called with SA_NOCLDWAIT,
+ * so no need to call waitpid()
+ */
+
+ popen_lock_data_list();
+
+ struct popen_data *data = popen_lookup_data_by_pid(si->si_pid);
+ if (data) {
+ switch (si->si_code) {
+ case CLD_EXITED:
+ data->exit_code = si->si_status;
+ data->status = POPEN_EXITED;
+ break;
+ case CLD_KILLED:
+ data->signal_id = si->si_status;
+ data->status = POPEN_KILLED;
+ break;
+ case CLD_DUMPED:
+ /* exit_status makes no sense */
+ data->status = POPEN_DUMPED;
+ break;
+ }
+
+ /*
+ * We shouldn't close file descriptors here.
+ * The child process may exit earlier than
+ * the parent process finishes reading data.
+ * In this case the reading fails.
+ */
+ }
+
+ popen_unlock_data_list();
+}
diff --git a/src/lib/core/coio_popen.h b/src/lib/core/coio_popen.h
new file mode 100644
index 000000000..e90ce9b25
--- /dev/null
+++ b/src/lib/core/coio_popen.h
@@ -0,0 +1,243 @@
+#ifndef TARANTOOL_LIB_CORE_COIO_POPEN_H_INCLUDED
+#define TARANTOOL_LIB_CORE_COIO_POPEN_H_INCLUDED
+/*
+ * Copyright 2010-2019, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the
+ * following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#if defined(__cplusplus)
+extern "C" {
+#endif /* defined(__cplusplus) */
+
+/**
+ * Special values of the file descriptors passed to fio.popen
+ * */
+enum {
+ FIO_PIPE = -2,
+ /*
+ * Tells fio.popen to open a handle for
+ * direct reading/writing
+ */
+
+ FIO_DEVNULL = -3
+ /*
+ * Tells fio.popen to redirect the given standard
+ * stream into /dev/null
+ */
+};
+
+/**
+ * Possible status of the process started via fio.popen
+ **/
+enum popen_status {
+ POPEN_UNKNOWN = -1,
+
+ POPEN_RUNNING = 1,
+ /*the process is alive and well*/
+
+ POPEN_EXITED = 2,
+ /*the process exited*/
+
+ POPEN_KILLED = 3,
+ /*the process terminated by a signal*/
+
+ POPEN_DUMPED = 4
+ /* the process terminated abnormally */
+};
+
+/**
+ * Initializes inner data of fio.popen
+ * */
+void
+popen_initialize();
+
+/**
+ * Implementation of fio.popen.
+ * The function opens a process by creating a pipe
+ * forking.
+ *
+ * @param argv - is an array of character pointers
+ * to the arguments terminated by a null pointer.
+ * The null pointer terminating the argv array is
+ * not counted in argc.
+ *
+ * @param argc - is the argument count.
+ *
+ * @param environment - is the pointer to an array
+ * of character pointers to the environment strings.
+ *
+ * @param environment_size - the environment strings count.
+ * The null pointer terminating the environment array is
+ * not counted in environment_size.
+ *
+ * @param stdin_fd - the file handle to be redirected to the
+ * child process's STDIN.
+ *
+ * @param stdout_fd - the file handle receiving the STDOUT
+ * output of the child process.
+ *
+ * @param stderr_fd - the file handle receiving the STDERR
+ * output of the child process.
+ *
+ * The stdin_fd, stdout_fd & stderr_fd accept file descriptors
+ * from open() or the following values:
+ *
+ * FIO_PIPE - opens a pipe, binds it with child's
+ * input/output. The pipe is available for reading/writing.
+ *
+ * FIO_DEVNULL - redirects output from process to /dev/null.
+ *
+ * @return handle of the pipe for reading or writing
+ * (depends on value of type).
+ * In a case of error returns NULL.
+ */
+void *
+coio_popen_impl(char** argv, int argc,
+ char** environment, int environment_size,
+ int stdin_fd, int stdout_fd, int stderr_fd);
+
+/**
+ * The function releases allocated resources.
+ * The function doesn't wait for the associated process
+ * to terminate.
+ *
+ * @param fh handle returned by fio.popen.
+ *
+ * @return 0 if the process is terminated
+ * @return -1 for an error
+ */
+int
+coio_popen_destroy(void *fh);
+
+/**
+ * The function reads up to count bytes from the handle
+ * associated with the child process.
+ * Returns immediately
+ *
+ * @param fd handle returned by fio.popen.
+ * @param buf a buffer to be read into
+ * @param count size of buffer in bytes
+ * @param read_bytes A pointer to the
+ * variable that receives the number of bytes read.
+ * @param source_id A pointer to the variable that receives a
+ * source stream id, 1 - for STDOUT, 2 - for STDERR.
+ *
+ * @return 0 data were successfully read
+ * @return -1 an error occurred, see errno for error code
+ *
+ * If there is nothing to read yet function returns -1
+ * and errno set no EAGAIN.
+ */
+int
+coio_popen_try_to_read(void *fh, void *buf, size_t count,
+ size_t *read_bytes, int *source_id);
+
+/**
+ * The function writes up to count bytes to the handle
+ * associated with the child process.
+ * Tries to write as much as possible without blocking
+ * and immediately returns.
+ *
+ * @param fd handle returned by fio.popen.
+ * @param buf a buffer to be written from
+ * @param count size of buffer in bytes
+ * @param written A pointer to the
+ * variable that receives the number of bytes actually written.
+ * If function fails the number of written bytes is undefined.
+ *
+ * @return 0 data were successfully written
+ * Compare values of <count> and <written> to check
+ * whether all data were written or not.
+ * @return -1 an error occurred, see errno for error code
+ *
+ * If the writing can block, function returns -1
+ * and errno set no EAGAIN.
+ */
+int
+coio_popen_try_to_write(void *fh, const void *buf, size_t count,
+ size_t *written);
+
+
+/**
+ * The function send the specified signal
+ * to the associated process.
+ *
+ * @param fd - handle returned by fio.popen.
+ *
+ * @return 0 on success
+ * @return -1 an error occurred, see errno for error code
+ */
+int
+coio_popen_kill(void *fh, int signal_id);
+
+/**
+ * Returns descriptor of the specified file.
+ *
+ * @param fd - handle returned by fio.popen.
+ * @param file_no accepts one of the
+ * following values:
+ * STDIN_FILENO,
+ * STDOUT_FILENO,
+ * STDERR_FILENO
+ *
+ * @return file descriptor or -1 if not available
+ */
+int
+coio_popen_get_std_file_handle(void *fh, int file_no);
+
+
+/**
+ * Returns status of the associated process.
+ *
+ * @param fd - handle returned by fio.popen.
+ * @param signal_id - if not NULL accepts the signal
+ * sent to terminate the process
+ * @param exit_code - if not NULL accepts the exit code
+ * if the process terminated normally.
+ *
+ * @return one of the following values:
+ * POPEN_RUNNING if the process is alive
+ * POPEN_EXITED if the process was terminated normally
+ * POPEN_KILLED if the process was terminated by a signal
+ * POPEN_UNKNOWN an error's occurred
+ */
+int
+coio_popen_get_status(void *fh, int *signal_id, int *exit_code);
+
+/**
+ * Handle SIGCHLD signal
+ */
+void
+coio_popen_child_is_dead(int sig, siginfo_t *si, void *);
+
+#if defined(__cplusplus)
+} /* extern "C" */
+#endif /* defined(__cplusplus) */
+
+#endif /* TARANTOOL_LIB_CORE_COIO_POPEN_H_INCLUDED */
diff --git a/src/lib/core/coio_task.c b/src/lib/core/coio_task.c
index 908b336ed..e6a1b327f 100644
--- a/src/lib/core/coio_task.c
+++ b/src/lib/core/coio_task.c
@@ -40,6 +40,7 @@
#include "fiber.h"
#include "third_party/tarantool_ev.h"
+#include "coio_popen.h"
/*
* Asynchronous IO Tasks (libeio wrapper).
@@ -129,6 +130,7 @@ coio_on_stop(void *data)
void
coio_init(void)
{
+ popen_initialize();
eio_set_thread_on_start(coio_on_start, NULL);
eio_set_thread_on_stop(coio_on_stop, NULL);
}
diff --git a/src/lib/core/fiber.h b/src/lib/core/fiber.h
index fb168e25e..614b8fefe 100644
--- a/src/lib/core/fiber.h
+++ b/src/lib/core/fiber.h
@@ -494,8 +494,8 @@ struct cord {
extern __thread struct cord *cord_ptr;
#define cord() cord_ptr
-#define fiber() cord()->fiber
-#define loop() (cord()->loop)
+#define fiber() (cord() ? cord()->fiber : NULL)
+#define loop() (cord() ? cord()->loop : NULL)
void
cord_create(struct cord *cord, const char *name);
diff --git a/src/lua/fio.c b/src/lua/fio.c
index 806f4256b..519be90d7 100644
--- a/src/lua/fio.c
+++ b/src/lua/fio.c
@@ -46,6 +46,7 @@
#include "lua/utils.h"
#include "coio_file.h"
+#include "coio_popen.h"
static inline void
lbox_fio_pushsyserror(struct lua_State *L)
@@ -703,11 +704,292 @@ lbox_fio_copyfile(struct lua_State *L)
return lbox_fio_pushbool(L, coio_copyfile(source, dest) == 0);
}
+static bool
+popen_verify_argv(struct lua_State *L)
+{
+ if (!lua_istable(L, 1))
+ return false;
+ int num = (int)lua_objlen(L, 1); /*luaL_getn(L,1);*/
+ return num >= 1;
+}
+
+static char**
+popen_extract_strarray(struct lua_State *L, int index, int* array_size)
+{
+ if (lua_type(L, index) != LUA_TTABLE) {
+ if (array_size)
+ *array_size = 0;
+ return NULL;
+ }
+
+ size_t num = lua_objlen(L, index); /*luaL_getn(L,index);*/
+
+ char** array = calloc(num+1, sizeof(char*));
+ /*
+ * The last item in the array must be NULL
+ */
+
+ if (array == NULL)
+ return NULL;
+
+ for(size_t i = 0; i < num; ++i) {
+ lua_rawgeti(L, index, i+1);
+ size_t slen = 0;
+ const char* str = lua_tolstring(L, -1, &slen);
+ if (!str)
+ str = "";
+ array[i] = strdup(str);
+ lua_pop(L, 1);
+ }
+
+ if (array_size)
+ *array_size = num;
+ /*
+ * The number of elements doesn't include
+ * the trailing NULL pointer
+ */
+ return array;
+}
+
+/**
+ * A wrapper applicable for using with lua's GC.
+ * */
+struct fio_popen_wrap {
+ void* handle;
+};
+static const char* fio_popen_typename = "fio.popen.data";
+
+static struct fio_popen_wrap *
+fio_popen_get_handle_wrap(lua_State *L, int index)
+{
+ luaL_checktype(L, index, LUA_TUSERDATA);
+ struct fio_popen_wrap *wrap = (struct fio_popen_wrap *)
+ luaL_checkudata(L, index, fio_popen_typename);
+ if (wrap == NULL)
+ luaL_typerror(L, index, fio_popen_typename);
+ return wrap;
+}
+
+static void *
+fio_popen_get_handle(lua_State *L, int index)
+{
+ struct fio_popen_wrap *wrap = fio_popen_get_handle_wrap(L, index);
+ void *fh = wrap->handle;
+ if (!fh)
+ luaL_error(L, "fio.popen failed");
+ return fh;
+}
+
+static int
+lbox_fio_popen_gc(lua_State *L)
+{
+ struct fio_popen_wrap *wrap = (struct fio_popen_wrap *)
+ luaL_checkudata(L, -1, fio_popen_typename);
+ if (wrap->handle)
+ coio_popen_destroy(wrap->handle);
+ wrap->handle = NULL;
+ return 0;
+}
+
+static int
+lbox_fio_popen(struct lua_State *L)
+{
+ if (lua_gettop(L) < 1) {
+ usage:
+ luaL_error(L, "fio.popen: Invalid arguments");
+ }
+
+ /*
+ * handle* popen(argv, env, hstdin, hstdout, hstderr)
+ */
+ if (!popen_verify_argv(L))
+ goto usage;
+
+ int stdin_fd = FIO_PIPE;
+ int stdout_fd = FIO_PIPE;
+ int stderr_fd = FIO_PIPE;
+
+ int argc = 0;
+ char** argv = popen_extract_strarray(L, 1, &argc);
+ int env_num = 0;
+ char** env = popen_extract_strarray(L, 2, &env_num);
+
+ if (lua_isnumber(L, 3))
+ stdin_fd = lua_tonumber(L, 3);
+ if (lua_isnumber(L, 4))
+ stdout_fd = lua_tonumber(L, 4);
+ if (lua_isnumber(L, 5))
+ stderr_fd = lua_tonumber(L, 5);
+
+ struct fio_popen_wrap *wrap =
+ (struct fio_popen_wrap*)lua_newuserdata(L,
+ sizeof(struct fio_popen_wrap));
+
+ luaL_getmetatable(L, fio_popen_typename);
+ lua_setmetatable(L, -2);
+
+ wrap->handle = coio_popen(argv, argc, env, env_num,
+ stdin_fd, stdout_fd, stderr_fd);
+
+ if (wrap->handle == NULL) {
+ lua_pop(L, 1); /* remove wrap from the stack */
+ lua_pushnil(L);
+ lbox_fio_pushsyserror(L);
+ return 2;
+ }
+
+ return 1;
+}
+
+static int
+lbox_fio_popen_read(struct lua_State *L)
+{
+ /* popen_read(self.fh, buf, size, seconds) */
+
+ void* fh = fio_popen_get_handle(L, 1);
+ uint32_t ctypeid;
+ char *buf = *(char **)luaL_checkcdata(L, 2, &ctypeid);
+ size_t len = lua_tonumber(L, 3);
+ int seconds = lua_tonumber(L, 4);
+
+ if (!len) {
+ lua_pushinteger(L, 0);
+ lua_pushinteger(L, STDOUT_FILENO);
+ return 2;
+ }
+
+ int output_number = 0;
+ int res = coio_popen_read(fh, buf, len, &output_number, seconds);
+
+ if (res < 0) {
+ lua_pushnil(L);
+ lua_pushnil(L);
+ lbox_fio_pushsyserror(L);
+ return 3;
+ }
+
+ lua_pushinteger(L, res);
+ lua_pushinteger(L, output_number);
+ return 2;
+}
+
+static int
+lbox_fio_popen_write(struct lua_State *L)
+{
+ void* fh = fio_popen_get_handle(L, 1);
+ const char *buf = lua_tostring(L, 2);
+ uint32_t ctypeid = 0;
+ if (buf == NULL)
+ buf = *(const char **)luaL_checkcdata(L, 2, &ctypeid);
+ size_t len = lua_tonumber(L, 3);
+
+ int res = coio_popen_write(fh, buf, len);
+ if (res < 0) {
+ lua_pushnil(L);
+ lbox_fio_pushsyserror(L);
+ return 2;
+ }
+ lua_pushinteger(L, res);
+ return 1;
+}
+
+static int
+lbox_fio_popen_get_status(struct lua_State *L)
+{
+ void* fh = fio_popen_get_handle(L, 1);
+ int signal_id = 0;
+ int exit_code = 0;
+ int res = coio_popen_get_status(fh, &signal_id, &exit_code);
+
+ switch (res) {
+ case POPEN_RUNNING:
+ lua_pushnil(L);
+ break;
+
+ case POPEN_KILLED:
+ lua_pushinteger(L, -signal_id);
+ break;
+
+ default:
+ lua_pushinteger(L, exit_code);
+ break;
+ }
+
+ return 1;
+}
+
+static int
+lbox_fio_popen_get_std_file_handle(struct lua_State *L)
+{
+ void* fh = fio_popen_get_handle(L, 1);
+ int file_no = lua_tonumber(L, 2);
+ int res = coio_popen_get_std_file_handle(fh, file_no);
+
+ if (res < 0)
+ lua_pushnil(L);
+ else
+ lua_pushinteger(L, res);
+ return 1;
+}
+
+static int
+lbox_fio_popen_kill(struct lua_State *L)
+{
+ void* fh = fio_popen_get_handle(L, 1);
+ int signal_id = lua_tonumber(L, 2);
+
+ int res = coio_popen_kill(fh, signal_id);
+ if (res < 0){
+ lua_pushnil(L);
+ lbox_fio_pushsyserror(L);
+ return 2;
+ } else {
+ lua_pushboolean(L, true);
+ return 1;
+ }
+}
+
+static int
+lbox_fio_popen_wait(struct lua_State *L)
+{
+ struct fio_popen_wrap *wrap = fio_popen_get_handle_wrap(L, 1);
+ void *fh = wrap->handle;
+ int timeout = lua_tonumber(L, 2);
+
+ /*
+ * On success function returns an
+ * exit code as a positive number
+ * or signal id as a negative number.
+ * If failed, rc is nul and err
+ * contains a error message.
+ */
+
+ int exit_code =0;
+ int res = coio_popen_wait(fh, timeout, &exit_code);
+ if (res < 0){
+ lua_pushnil(L);
+ lbox_fio_pushsyserror(L);
+ return 2;
+ } else {
+ /* Release the allocated resources */
+ coio_popen_destroy(fh);
+ wrap->handle = NULL;
+
+ lua_pushinteger(L, exit_code);
+ return 1;
+ }
+}
void
tarantool_lua_fio_init(struct lua_State *L)
{
+ static const struct luaL_Reg popen_data_meta[] = {
+ {"__gc", lbox_fio_popen_gc},
+ {NULL, NULL}
+ };
+ luaL_register_type(L, fio_popen_typename, popen_data_meta);
+
static const struct luaL_Reg fio_methods[] = {
{ "lstat", lbox_fio_lstat },
{ "stat", lbox_fio_stat },
@@ -747,6 +1029,13 @@ tarantool_lua_fio_init(struct lua_State *L)
{ "listdir", lbox_fio_listdir },
{ "fstat", lbox_fio_fstat },
{ "copyfile", lbox_fio_copyfile, },
+ { "popen", lbox_fio_popen },
+ { "popen_read", lbox_fio_popen_read },
+ { "popen_write", lbox_fio_popen_write },
+ { "popen_get_status", lbox_fio_popen_get_status },
+ { "popen_get_std_file_handle", lbox_fio_popen_get_std_file_handle },
+ { "popen_kill", lbox_fio_popen_kill },
+ { "popen_wait", lbox_fio_popen_wait },
{ NULL, NULL }
};
luaL_register(L, NULL, internal_methods);
diff --git a/src/lua/fio.lua b/src/lua/fio.lua
index 74a871134..e5b9bdf81 100644
--- a/src/lua/fio.lua
+++ b/src/lua/fio.lua
@@ -3,6 +3,7 @@
local fio = require('fio')
local ffi = require('ffi')
local buffer = require('buffer')
+local signal = require('signal')
ffi.cdef[[
int umask(int mask);
@@ -15,6 +16,12 @@ local const_char_ptr_t = ffi.typeof('const char *')
local internal = fio.internal
fio.internal = nil
+fio.STDIN = 0
+fio.STDOUT = 1
+fio.STDERR = 2
+fio.PIPE = -2
+fio.DEVNULL = -3
+
local function sprintf(fmt, ...)
if select('#', ...) == 0 then
return fmt
@@ -206,6 +213,261 @@ fio.open = function(path, flags, mode)
return fh
end
+local popen_methods = {}
+
+popen_methods.do_read = function(self, buf, size, seconds)
+ if size == nil or type(size) ~= 'number' then
+ error('fio.popen.read: invalid size argument')
+ end
+ if seconds == nil or type(seconds) ~= 'number' then
+ error('fio.popen.read: invalid seconds argument')
+ end
+
+ local tmpbuf
+ if not ffi.istype(const_char_ptr_t, buf) then
+ tmpbuf = buffer.ibuf()
+ buf = tmpbuf:reserve(size)
+ end
+
+ local res, output_no, err = internal.popen_read(self.fh, buf, size, seconds)
+ if res == nil then
+ if tmpbuf ~= nil then
+ tmpbuf:recycle()
+ end
+ return nil, nil, err
+ end
+
+ if tmpbuf ~= nil then
+ tmpbuf:alloc(res)
+ res = ffi.string(tmpbuf.rpos, tmpbuf:size())
+ tmpbuf:recycle()
+ end
+ return res, output_no
+end
+
+-- read stdout & stderr of the process started by fio.popen
+-- read() -> str, source
+-- read(buf) -> len, source
+-- read(size) -> str, source
+-- read(buf, size) -> len, source
+-- source contains id of the stream, fio.STDOUT or fio.STDERR
+popen_methods.read = function(self, buf, size)
+ if self.fh == nil then
+ return nil, nil, 'Invalid object'
+ end
+
+ if buf == nil and size == nil then
+ -- read()
+ size = 512
+ elseif ffi.istype(const_char_ptr_t, buf) and size == nil then
+ -- read(buf)
+ size = 512
+ elseif not ffi.istype(const_char_ptr_t, buf) and
+ buf ~= nil and size == nil then
+ -- read(size)
+ size = tonumber(buf)
+ buf = nil
+ elseif ffi.istype(const_char_ptr_t, buf) and size ~= nil then
+ -- read(buf, size)
+ size = tonumber(size)
+ else
+ error("fio.popen.read: invalid arguments")
+ end
+
+ return self:do_read(buf, size, -1)
+end
+
+-- read stdout & stderr of the process started by fio.popen
+-- read2(seconds) -> str, source
+-- read2(buf, seconds) -> len, source
+-- read2(size, seconds) -> str, source
+-- read2(buf, size, seconds) -> len, source
+-- source contains id of the stream, fio.STDOUT or fio.STDERR
+popen_methods.read2 = function(self, buf, size, seconds)
+ if self.fh == nil then
+ return nil, nil, 'Invalid object'
+ end
+
+ if not ffi.istype(const_char_ptr_t, buf) and buf ~= nil and
+ size == nil and seconds == nil then
+ -- read2(seconds)
+ seconds = tonumber(buf)
+ size = 512
+ buf = nil
+ elseif ffi.istype(const_char_ptr_t, buf) and size ~= nil and
+ seconds == nil then
+ -- read2(buf, seconds)
+ seconds = tonumber(size)
+ size = 512
+ elseif not ffi.istype(const_char_ptr_t, buf) and buf ~= nil and
+ size ~= nil and seconds == nil then
+ -- read2(size, seconds)
+ seconds = tonumber(size)
+ size = tonumber(buf)
+ buf = nil
+ elseif ffi.istype(const_char_ptr_t, buf) and size ~= nil and
+ seconds ~= nil then
+ -- read2(buf, size, seconds)
+ seconds = tonumber(seconds)
+ size = tonumber(size)
+ else
+ error("fio.popen.read2: invalid arguments")
+ end
+
+ return self:do_read(buf, size, seconds)
+end
+
+-- write(str)
+-- write(buf, len)
+popen_methods.write = function(self, data, len)
+ if type(data) == 'string' then
+ if len == nil then
+ len = string.len(data)
+ end
+ elseif not ffi.istype(const_char_ptr_t, data) then
+ data = tostring(data)
+ len = #data
+ end
+
+ local res, err = internal.popen_write(self.fh, data, tonumber(len))
+ if err ~= nil then
+ return false, err
+ end
+ return res >= 0
+end
+
+popen_methods.get_status = function(self)
+ if self.fh ~= nil then
+ return internal.popen_get_status(self.fh)
+ else
+ return self.exit_code
+ end
+end
+
+popen_methods.get_stdin = function (self)
+ if self.fh == nil then
+ return nil, 'Invalid object'
+ end
+
+ return internal.popen_get_std_file_handle(self.fh, fio.STDIN)
+end
+
+popen_methods.get_stdout = function (self)
+ if self.fh == nil then
+ return nil, 'Invalid object'
+ end
+
+ return internal.popen_get_std_file_handle(self.fh, fio.STDOUT)
+end
+
+popen_methods.get_stderr = function (self)
+ if self.fh == nil then
+ return nil, 'Invalid object'
+ end
+
+ return internal.popen_get_std_file_handle(self.fh, fio.STDERR)
+end
+
+popen_methods.kill = function(self, sig)
+ if self.fh == nil then
+ return nil, errno.strerror(errno.ESRCH)
+ end
+
+ if sig == nil then
+ sig = 'SIGTERM'
+ end
+ if type(sig) == 'string' then
+ sig = signal.c.signals[sig]
+ if sig == nil then
+ errno(errno.EINVAL)
+ return nil, sprintf("fio.popen.kill(): unknown signal: %s", sig)
+ end
+ else
+ sig = tonumber(sig)
+ end
+
+ return internal.popen_kill(self.fh, sig)
+end
+
+popen_methods.wait = function(self, timeout)
+ if self.fh == nil then
+ return nil, 'Invalid object'
+ end
+
+ if timeout == nil then
+ timeout = -1
+ else
+ timeout = tonumber(timeout)
+ end
+
+ local rc, err = internal.popen_wait(self.fh, timeout)
+ if rc ~= nil then
+ self.exit_code = tonumber(rc)
+ self.fh = nil
+ return rc
+ else
+ return nil,err
+ end
+end
+
+
+local popen_mt = { __index = popen_methods }
+
+fio.popen = function(params)
+ local argv = params.argv
+ local env = params.environment
+ local hstdin = params.stdin
+ local hstdout = params.stdout
+ local hstderr = params.stderr
+
+ if type(hstdin) == 'table' then
+ hstdin = hstdin.fh
+ end
+ if type(hstdout) == 'table' then
+ hstdout = hstdout.fh
+ end
+ if type(hstderr) == 'table' then
+ hstderr = hstderr.fh
+ end
+
+ if argv == nil or
+ type(argv) ~= 'table' or
+ table.getn(argv) < 1 then
+ local errmsg = [[Usage: fio.popen({parameters}),
+parameters - a table containing arguments to run a process.
+The following arguments are expected:
+
+argv - [mandatory] is a table of argument strings passed
+ to the new program.
+ By convention, the first of these strings should
+ contain the filename associated with the file
+ being executed.
+
+environment - [optional] is a table of strings,
+ conventionally of the form key=value,
+ which are passed as environment to the
+ new program.
+
+stdin - [optional] overrides the child process's
+ standard input.
+stdout - [optional] overrides the child process's
+ standard output.
+stderr - [optional] overrides the child process's
+ standard error output.
+]]
+ error(errmsg)
+ end
+
+ local fh,err = internal.popen(argv, env, hstdin, hstdout, hstderr)
+ if err ~= nil then
+ return nil, err
+ end
+
+ local pobj = {fh = fh}
+ setmetatable(pobj, popen_mt)
+ return pobj
+end
+
fio.pathjoin = function(...)
local i, path = 1, nil
diff --git a/src/lua/init.c b/src/lua/init.c
index 5ddc5a4d8..b1da6bf4b 100644
--- a/src/lua/init.c
+++ b/src/lua/init.c
@@ -56,6 +56,7 @@
#include "lua/msgpack.h"
#include "lua/pickle.h"
#include "lua/fio.h"
+#include "lua/lua_signal.h"
#include "lua/httpc.h"
#include "lua/utf8.h"
#include "digest.h"
@@ -447,6 +448,7 @@ tarantool_lua_init(const char *tarantool_bin, int argc, char **argv)
tarantool_lua_errno_init(L);
tarantool_lua_error_init(L);
tarantool_lua_fio_init(L);
+ tarantool_lua_signal_init(L);
tarantool_lua_socket_init(L);
tarantool_lua_pickle_init(L);
tarantool_lua_digest_init(L);
diff --git a/src/lua/lua_signal.c b/src/lua/lua_signal.c
new file mode 100644
index 000000000..924b0ab51
--- /dev/null
+++ b/src/lua/lua_signal.c
@@ -0,0 +1,99 @@
+/*
+ * Copyright 2010-2019, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the
+ * following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include "lua/lua_signal.h"
+#include <sys/types.h>
+#include <signal.h>
+
+#include <lua.h>
+#include <lauxlib.h>
+#include <lualib.h>
+
+#include "lua/utils.h"
+
+#ifndef PUSHTABLE
+#define PUSHTABLE(name, method, value) { \
+ lua_pushliteral(L, name); \
+ method(L, value); \
+ lua_settable(L, -3); \
+}
+#endif /*PUSHTABLE*/
+
+void
+tarantool_lua_signal_init(struct lua_State *L)
+{
+ static const struct luaL_Reg signal_methods[] = {
+ { NULL, NULL }
+ };
+
+ luaL_register_module(L, "signal", signal_methods);
+
+ lua_pushliteral(L, "c");
+ lua_newtable(L);
+
+ lua_pushliteral(L, "signals");
+ lua_newtable(L);
+
+ PUSHTABLE("SIGINT", lua_pushinteger, SIGINT);
+ PUSHTABLE("SIGILL", lua_pushinteger, SIGILL);
+ PUSHTABLE("SIGABRT", lua_pushinteger, SIGABRT);
+ PUSHTABLE("SIGFPE", lua_pushinteger, SIGFPE);
+ PUSHTABLE("SIGSEGV", lua_pushinteger, SIGSEGV);
+ PUSHTABLE("SIGTERM", lua_pushinteger, SIGTERM);
+
+ PUSHTABLE("SIGHUP", lua_pushinteger, SIGHUP);
+ PUSHTABLE("SIGQUIT", lua_pushinteger, SIGQUIT);
+ PUSHTABLE("SIGTRAP", lua_pushinteger, SIGTRAP);
+ PUSHTABLE("SIGKILL", lua_pushinteger, SIGKILL);
+ PUSHTABLE("SIGBUS", lua_pushinteger, SIGBUS);
+ PUSHTABLE("SIGSYS", lua_pushinteger, SIGSYS);
+ PUSHTABLE("SIGPIPE", lua_pushinteger, SIGPIPE);
+ PUSHTABLE("SIGALRM", lua_pushinteger, SIGALRM);
+
+ PUSHTABLE("SIGURG", lua_pushinteger, SIGURG);
+ PUSHTABLE("SIGSTOP", lua_pushinteger, SIGSTOP);
+ PUSHTABLE("SIGTSTP", lua_pushinteger, SIGTSTP);
+ PUSHTABLE("SIGCONT", lua_pushinteger, SIGCONT);
+ PUSHTABLE("SIGCHLD", lua_pushinteger, SIGCHLD);
+ PUSHTABLE("SIGTTIN", lua_pushinteger, SIGTTIN);
+ PUSHTABLE("SIGTTOU", lua_pushinteger, SIGTTOU);
+ PUSHTABLE("SIGPOLL", lua_pushinteger, SIGPOLL);
+ PUSHTABLE("SIGXCPU", lua_pushinteger, SIGXCPU);
+ PUSHTABLE("SIGXFSZ", lua_pushinteger, SIGXFSZ);
+ PUSHTABLE("SIGVTALRM", lua_pushinteger, SIGVTALRM);
+ PUSHTABLE("SIGPROF", lua_pushinteger, SIGPROF);
+ PUSHTABLE("SIGUSR1", lua_pushinteger, SIGUSR1);
+ PUSHTABLE("SIGUSR2", lua_pushinteger, SIGUSR2);
+ lua_settable(L, -3); /* "signals" */
+
+ lua_settable(L, -3); /* "c" */
+ lua_pop(L, 1);
+}
diff --git a/src/lua/lua_signal.h b/src/lua/lua_signal.h
new file mode 100644
index 000000000..814692077
--- /dev/null
+++ b/src/lua/lua_signal.h
@@ -0,0 +1,45 @@
+#ifndef INCLUDES_TARANTOOL_LUA_SIGNAL_H
+#define INCLUDES_TARANTOOL_LUA_SIGNAL_H
+
+/*
+ * Copyright 2010-2019, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the
+ * following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+#if defined(__cplusplus)
+extern "C" {
+#endif /* defined(__cplusplus) */
+
+struct lua_State;
+void tarantool_lua_signal_init(struct lua_State *L);
+
+#if defined(__cplusplus)
+} /* extern "C" */
+#endif /* defined(__cplusplus) */
+
+#endif /*INCLUDES_TARANTOOL_LUA_SIGNAL_H*/
diff --git a/src/main.cc b/src/main.cc
index 569ff4b5f..86258cc50 100644
--- a/src/main.cc
+++ b/src/main.cc
@@ -77,6 +77,7 @@
#include "box/session.h"
#include "systemd.h"
#include "crypto/crypto.h"
+#include "coio_popen.h"
static pid_t master_pid = getpid();
static struct pidfh *pid_file_handle;
@@ -328,7 +329,8 @@ signal_reset()
sigaction(SIGHUP, &sa, NULL) == -1 ||
sigaction(SIGWINCH, &sa, NULL) == -1 ||
sigaction(SIGSEGV, &sa, NULL) == -1 ||
- sigaction(SIGFPE, &sa, NULL) == -1)
+ sigaction(SIGFPE, &sa, NULL) == -1 ||
+ sigaction(SIGCHLD, &sa, NULL) == -1)
say_syserror("sigaction");
/* Unblock any signals blocked by libev. */
@@ -373,6 +375,23 @@ signal_init(void)
panic_syserror("sigaction");
}
+ struct sigaction sa_chld;
+ memset(&sa_chld, 0, sizeof(sa_chld));
+ sigemptyset(&sa_chld.sa_mask);
+
+ /*
+ * SA_NOCLDWAIT - do not transform children
+ * into zombies when they terminate.
+ * SA_NOCLDSTOP - do not receive notification
+ * when child processes stop
+ */
+ sa_chld.sa_flags = SA_SIGINFO | SA_NOCLDSTOP | SA_NOCLDWAIT;
+ sa_chld.sa_sigaction = coio_popen_child_is_dead;
+
+ if (sigaction(SIGCHLD, &sa_chld, NULL) == -1) {
+ panic_syserror("sigaction SIGCHLD");
+ }
+
ev_signal_init(&ev_sigs[0], sig_checkpoint, SIGUSR1);
ev_signal_init(&ev_sigs[1], signal_cb, SIGINT);
ev_signal_init(&ev_sigs[2], signal_cb, SIGTERM);
diff --git a/test/box-tap/fio_popen.sample.txt b/test/box-tap/fio_popen.sample.txt
new file mode 100644
index 000000000..43cba8c65
--- /dev/null
+++ b/test/box-tap/fio_popen.sample.txt
@@ -0,0 +1,5 @@
+AAA
+BBB
+CCC
+DDD
+
diff --git a/test/box-tap/fio_popen.test.lua b/test/box-tap/fio_popen.test.lua
new file mode 100755
index 000000000..0c2bd7e70
--- /dev/null
+++ b/test/box-tap/fio_popen.test.lua
@@ -0,0 +1,189 @@
+#!/usr/bin/env tarantool
+
+local tap = require('tap')
+local fio = require('fio')
+local errno = require('errno')
+local fiber = require('fiber')
+local test = tap.test()
+
+test:plan(5+9+11+9+8)
+
+-- Preliminaries
+local function read_stdout(app)
+ local ss = ""
+
+ local s,src,err = app:read(128)
+
+ while s ~= nil and s ~= "" do
+ ss = ss .. s
+
+ s,src,err = app:read(128)
+ end
+
+ return ss
+end
+
+-- Test 1. Run application, check its status, kill and wait
+local app1 = fio.popen({argv = {'/bin/sh', '-c', 'cat'},
+ stdout=fio.STDOUT,
+ stderr=fio.STDOUT})
+test:isnt(app1, nil, "#1. Starting a existing application")
+
+local rc = app1:get_status()
+test:is(rc, nil, "#1. Process is running")
+
+rc,err = app1:kill()
+test:is(rc, true, "#1. Sending kill(15)")
+
+rc,err = app1:wait(5)
+test:is(rc, -15, "#1. Process was killed")
+
+rc = app1:get_status()
+test:is(rc, -15, "#1. Process was killed 2")
+
+app1 = nil
+
+-- Test 2. Run application, write to stdin, read from stdout
+app1 = fio.popen({argv = {'/bin/sh', '-c', 'cat'},
+ stdout=fio.PIPE,
+ stdin=fio.PIPE,
+ stderr=fio.STDOUT})
+test:isnt(app1, nil, "#2. Starting a existing application")
+
+rc = app1:get_status()
+test:is(rc, nil, "#2. Process is running")
+
+local test2str = '123\n456\n789'
+
+app1:write(test2str)
+rc,src,err = app1:read(256)
+
+test:is(src, fio.STDOUT, "#2. Read from STDOUT")
+test:is(rc, test2str, "#2. Received exact string")
+
+test2str = 'abc\ndef\ngh'
+
+app1:write(test2str, 4)
+local rc2,src2,err = app1:read(6)
+
+test:is(err, nil, "#2. Reading ok")
+test:is(src2, fio.STDOUT, "#2. Read from STDOUT 2")
+test:is(rc2, 'abc\n', "#2. Received exact string 2")
+
+rc,err = app1:kill()
+test:is(rc, true, "#2. Sending kill(15)")
+
+rc,err = app1:wait()
+test:is(rc, -15, "#2. Process was killed")
+
+app1 = nil
+
+-- Test 3. read from stdout with timeout
+app1 = fio.popen({argv = {'/bin/sh', '-c', 'cat'},
+ stdout=fio.PIPE,
+ stdin=fio.PIPE,
+ stderr=fio.STDOUT})
+test:isnt(app1, nil, "#3. Starting a existing application")
+test:is(app1:get_stderr(), nil, "#3. STDERR is redirected")
+test:isnt(app1:get_stdout(), nil, "#3. STDOUT is available")
+test:isnt(app1:get_stdin(), nil, "#3. STDIN is available")
+
+
+rc = app1:get_status()
+test:is(rc, nil, "#3. Process is running")
+
+rc,src,err = app1:read2(256, 2)
+
+local e = errno()
+test:is(e, errno.ETIMEDOUT, "#3. Timeout")
+
+
+local test2str = '123\n456\n789'
+
+app1:write(test2str)
+rc,src,err = app1:read2(256, 2)
+
+test:is(err, nil, "#3. Reading ok")
+test:is(src, fio.STDOUT, "#3. Read from STDOUT")
+test:is(rc, test2str, "#3. Received exact string")
+
+rc,err = app1:kill('SIGHUP')
+test:is(rc, true, "#3. Sending kill(1)")
+
+rc,err = app1:wait()
+test:is(rc, -1, "#3. Process was killed")
+
+app1 = nil
+
+-- Test 4. Redirect from file
+local build_path = os.getenv("BUILDDIR")
+local txt_filename = fio.pathjoin(build_path, 'test/box-tap/fio_popen.sample.txt')
+
+local txt_file = fio.open(txt_filename, {'O_RDONLY'})
+test:isnt(txt_file, nil, "#4. Open existing file for reading")
+
+app1 = fio.popen({argv = {'/bin/sh', '-c', 'cat'},
+ stdout=fio.PIPE,
+ stdin=txt_file,
+ stderr=fio.STDOUT})
+test:isnt(app1, nil, "#4. Starting a existing application")
+test:is(app1:get_stderr(), nil, "#4. STDERR is redirected")
+test:isnt(app1:get_stdout(), nil, "#4. STDOUT is available")
+test:is(app1:get_stdin(), nil, "#4. STDIN is redirected")
+
+rc = app1:get_status()
+test:is(rc, nil, "#4. Process is running")
+
+rc,src,err = app1:read2(256, 2)
+
+test:is(src, fio.STDOUT, "#4. Read from STDOUT")
+
+local test2str = 'AAA\nBBB\nCCC\nDDD\n\n'
+
+test:is(rc, test2str, "#4. Received exact string")
+
+rc,err = app1:wait()
+test:is(rc, 0, "#4. Process's exited")
+
+app1 = nil
+txt_file:close()
+
+-- Test 5. Redirect output from one process to another
+local app_path = fio.pathjoin(build_path, 'test/box-tap/fio_popen_test1.sh')
+
+app1 = fio.popen({argv = {'/bin/sh', '-c', app_path},
+ stdout=fio.PIPE,
+ stderr=fio.STDOUT})
+
+test:isnt(app1, nil, "#5. Starting application 1")
+test:is(app1:get_stderr(), nil, "#5. STDERR is redirected")
+test:isnt(app1:get_stdout(), nil, "#5. STDOUT is available")
+test:isnt(app1:get_stdin(), nil, "#5. STDIN is available")
+
+fiber.sleep(1)
+
+local app2 = fio.popen({argv = {'/bin/sh', '-c', 'cat'},
+ stdout=fio.PIPE,
+ stdin=app1:get_stdout()})
+
+test:isnt(app2, nil, "#5. Starting application 2")
+
+local test2str = '1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n'
+
+rc = read_stdout(app2)
+
+test:is(rc, test2str, "#5. Received exact string")
+
+rc,err = app1:wait()
+test:is(rc, 0, "#5. Process's exited 1")
+
+rc,err = app2:wait()
+test:is(rc, 0, "#5. Process's exited 2")
+
+app1 = nil
+app2 = nil
+
+-- --------------------------------------------------------------
+test:check()
+os.exit(0)
+
diff --git a/test/box-tap/fio_popen_test1.sh b/test/box-tap/fio_popen_test1.sh
new file mode 100755
index 000000000..da54ee19a
--- /dev/null
+++ b/test/box-tap/fio_popen_test1.sh
@@ -0,0 +1,7 @@
+#!/bin/bash
+for i in {1..10}
+do
+ echo $i
+# sleep 1
+done
+
diff --git a/third_party/libeio/eio.c b/third_party/libeio/eio.c
index 7351d5dda..e433b1b3b 100644
--- a/third_party/libeio/eio.c
+++ b/third_party/libeio/eio.c
@@ -1741,7 +1741,24 @@ static int eio_threads;
static void ecb_cold
eio_prefork()
{
- eio_threads = etp_set_max_parallel(EIO_POOL, 0);
+ /*
+ * When fork() is called libeio shuts
+ * down all working threads.
+ * But it causes a deadlock if fork() was
+ * called from the libeio thread.
+ * To avoid this do not close the
+ * thread who called fork().
+ * This behaviour is acceptable for the
+ * case when fork() is immediately followed
+ * by exec().
+ * To clone a process call fork() from the
+ * main thread.
+ */
+
+ if (etp_is_in_pool_thread())
+ eio_threads = etp_get_max_parallel(EIO_POOL);
+ else
+ eio_threads = etp_set_max_parallel(EIO_POOL, 0);
}
static void ecb_cold
diff --git a/third_party/libeio/etp.c b/third_party/libeio/etp.c
index 42f7661eb..38c2d1f9b 100644
--- a/third_party/libeio/etp.c
+++ b/third_party/libeio/etp.c
@@ -164,6 +164,21 @@ struct etp_pool_user
xmutex_t lock;
};
+static __thread int is_eio_thread = 0;
+
+/**
+ * Check whether the current thread belongs to
+ * libeio thread pool or just a generic thread.
+ *
+ * @return 0 for generic thread
+ * @return 1 for libeio thread pool
+ **/
+ETP_API_DECL int ecb_cold
+etp_is_in_pool_thread()
+{
+ return is_eio_thread;
+}
+
/* worker threads management */
static void ecb_cold
@@ -322,6 +337,9 @@ X_THREAD_PROC (etp_proc)
self.pool = pool;
etp_pool_user user; /* per request */
+/* Distinguish libeio threads from the generic threads */
+ is_eio_thread = 1;
+
etp_proc_init ();
/* try to distribute timeouts somewhat evenly (nanosecond part) */
@@ -616,3 +634,13 @@ etp_set_max_parallel (etp_pool pool, unsigned int threads)
X_UNLOCK (pool->lock);
return retval;
}
+
+ETP_API_DECL int ecb_cold
+etp_get_max_parallel (etp_pool pool)
+{
+ int retval;
+ X_LOCK (pool->lock);
+ retval = pool->wanted;
+ X_UNLOCK (pool->lock);
+ return retval;
+}
diff --git a/third_party/libev/ev.c b/third_party/libev/ev.c
index 6a2648591..5fa8293a1 100644
--- a/third_party/libev/ev.c
+++ b/third_party/libev/ev.c
@@ -4214,6 +4214,8 @@ noinline
void
ev_signal_stop (EV_P_ ev_signal *w) EV_THROW
{
+ if (!loop)
+ return;
clear_pending (EV_A_ (W)w);
if (expect_false (!ev_is_active (w)))
return;
--
2.17.1
More information about the Tarantool-patches
mailing list