* [PATCH v4 1/2] core: Non-blocking io.popen
2019-07-02 7:36 [PATCH v4 0/2] core: Non-blocking io.popen Stanislav Zudin
@ 2019-07-02 7:36 ` Stanislav Zudin
2019-07-02 7:36 ` [PATCH v4 2/2] core: Non-blocking popen Stanislav Zudin
1 sibling, 0 replies; 3+ messages in thread
From: Stanislav Zudin @ 2019-07-02 7:36 UTC (permalink / raw)
To: tarantool-patches, vdavydov.dev, alexander.turenko; +Cc: Stanislav Zudin
Adds nonblocking implementation of popen.
The method is available in namespace fio.
fio.popen() returns an object providing facilities
for dealing with standard input/output, getting
status of the child process.
Closes #4031
@TarantoolBot document
Title: Nonblocking fio.popen
handle, err = fio.popen(parameters)
fio.popen starts a process and redirects its input/output.
parameters - a table containing arguments to run a process.
The following arguments are expected:
argv - [mandatory] is a table of argument strings passed to
the new program. By convention, the first of these strings should
contain the filename associated with the file being executed.
environment - [optional] is a table of strings, conventionally of the
form key=value, which are passed as environment to the new program.
If not specified a parent's environment is used.
By default stdin, stdout,stderr of the associated process are
available for writing/reading using object's methods
handle:write() and handle:read().
One can override default behavior to redirect streams to/from file
or to input/output of another process or to the default input/output
of the parent process.
stdin - [optional] overrides the child process's standard input.
stdout - [optional] overrides the child process's standard output.
stderr - [optional] overrides the the child process's standard
error output.
May accept one of the following values:
Handle of the file open with fio.open()
Handle of the standard input/output of another process,
open by fio.popen
A constant defining the parent's STDIN, STDOUT or STDERR.
A constants:
fio.PIPE - Opens a file descriptor available for reading/writing
or redirection to/from another process.
fio.DEVNULL - Makes fio.popen redirect the output to /dev/null.
On success returns an object providing methods for
communication with the running process.
Returns nil if underlying functions calls fail;
in this case the second return value, err, contains a error message.
The object created by fio.popen provides the following methods:
read()
write()
kill()
wait()
status()
stdin()
stdout()
stderr()
number handle:stdin()
Returns handle of the child process's standard input.
The handle is available only if it was created with
fio.PIPE option.
Use this handle to setup a redirection
from file or other process to the input of the associated process.
If handle is unavailable the method returns nil.
number handle:stdout()
number handle:stderr()
Return STDOUT and STDIN of the associated process accordingly.
See handle:stdin() for details.
rc,err = handle:wait(timeout)
The wait() waits for the associated process to terminate.
timeout - an integer specifies number of seconds to wait.
If the requested time has elapsed the method returns false,
the second return value, err, contains a error message.
To distinguish timeout from the the other errors use errno.
If timeout is nil, the method waits infinitely until
the associated process is terminated.
On success function returns true.
If failed, rc is false and err contains a error message.
If the associated process is terminated, one can use the following
methods get the exit status:
rc = handle:status()
returns nil if process is still running
== 0 if the process exited normally
error code > 0 if the process terminated with an error
-signal if the process was killed by a signal
rc, err = handle:kill(sig)
The kill() sends a specified signal to the associated process.
On success the method returns true,
if failed - false and error message.
If the sig is nil the default "SIGTERM" is being sent to the process.
If the signal is unknown then the method fails (with error EINVAL).
The method accepts string names of signal as well as their numeric
values.
rc,src,err = handle:read(buffer,size,timeout)
read stdout & stderr of the process started by fio.popen
Usage:
read(size) -> str, source, err
read(buf, size) -> length, source, err
read(size, timeout) -> str, source, err
read(buf, size, timeout) -> length, source, err
timeout - number of seconds to wait (optional)
source contains id of the stream, fio.STDOUT or fio.STDERR
err - error message if method has failed or nil on success
rc, err = handle:write(buf[, size])
rc, err = handle:write(opts), where opts are:
* buf
* size
* timeout
Writes specified number of bytes
On success returns number of written bytes.
If failed the rc is nil and err contains an error message.
---
src/CMakeLists.txt | 1 +
src/lib/core/CMakeLists.txt | 8 +
src/lib/core/coio_popen.c | 827 ++++++++++++++++++++++++++++++++
src/lib/core/coio_popen.h | 229 +++++++++
src/lib/core/coio_task.c | 2 +
src/lua/fio.c | 281 +++++++++++
src/lua/fio.lua | 219 +++++++++
src/lua/init.c | 2 +
src/lua/lua_signal.c | 155 ++++++
src/lua/lua_signal.h | 45 ++
src/main.cc | 8 +-
test/app-tap/fio_popen.test.lua | 309 ++++++++++++
test/app-tap/fio_popen_test1.sh | 6 +
test/app-tap/fio_popen_test2.sh | 7 +
test/app-tap/fio_popen_test3.sh | 5 +
15 files changed, 2103 insertions(+), 1 deletion(-)
create mode 100644 src/lib/core/coio_popen.c
create mode 100644 src/lib/core/coio_popen.h
create mode 100644 src/lua/lua_signal.c
create mode 100644 src/lua/lua_signal.h
create mode 100755 test/app-tap/fio_popen.test.lua
create mode 100755 test/app-tap/fio_popen_test1.sh
create mode 100755 test/app-tap/fio_popen_test2.sh
create mode 100755 test/app-tap/fio_popen_test3.sh
diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index 33b64f6a6..96f1d7513 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -120,6 +120,7 @@ set (server_sources
lua/string.c
lua/buffer.c
lua/swim.c
+ lua/lua_signal.c
${lua_sources}
${PROJECT_SOURCE_DIR}/third_party/lua-yaml/lyaml.cc
${PROJECT_SOURCE_DIR}/third_party/lua-yaml/b64.c
diff --git a/src/lib/core/CMakeLists.txt b/src/lib/core/CMakeLists.txt
index 66e430a25..64152f08a 100644
--- a/src/lib/core/CMakeLists.txt
+++ b/src/lib/core/CMakeLists.txt
@@ -27,8 +27,16 @@ set(core_sources
mpstream.c
port.c
decimal.c
+ coio_popen.c
)
+# Disable gcc compiler error
+if (CMAKE_COMPILER_IS_GNUCC)
+ set_source_files_properties(coio_popen.c PROPERTIES COMPILE_FLAGS
+ -Wno-clobbered)
+endif()
+
+
if (TARGET_OS_NETBSD)
# A workaround for "undefined reference to `__gcc_personality_v0'"
# on x86_64-rumprun-netbsd-gcc
diff --git a/src/lib/core/coio_popen.c b/src/lib/core/coio_popen.c
new file mode 100644
index 000000000..dcc50136f
--- /dev/null
+++ b/src/lib/core/coio_popen.c
@@ -0,0 +1,827 @@
+/*
+ * Copyright 2010-2019, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the
+ * following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <stddef.h>
+#include <signal.h>
+#include "coio_popen.h"
+#include "coio_task.h"
+#include "fiber.h"
+#include "fio.h"
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/wait.h>
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <pthread.h>
+#include <float.h>
+#include <sysexits.h>
+
+/*
+ * On OSX this global variable is not declared
+ * in <unistd.h>
+ */
+extern char **environ;
+
+
+struct popen_handle {
+ /* Process id */
+ pid_t pid;
+
+ /*
+ * Three descriptors:
+ * [0] write to stdin of the child process
+ * [1] read from stdout of the child process
+ * [2] read from stderr of the child process
+ * Valid only for pipe.
+ */
+ int fd[3];
+
+ /*
+ * Handle to /dev/null.
+ */
+ int devnull_fd;
+
+ /*
+ * Current process status.
+ * The SIGCHLD handler changes this status.
+ */
+ enum popen_status status;
+
+ /*
+ * Exit status of the associated process
+ * or number of the signal that caused the
+ * associated process to terminate.
+ */
+ int exit_code;
+};
+
+/*
+ * Map: (pid) => (popen_handle *)
+ */
+#define mh_name _popen_storage
+#define mh_key_t pid_t
+#define mh_node_t struct popen_handle*
+#define mh_arg_t void *
+#define mh_hash(a, arg) ((*a)->pid)
+#define mh_hash_key(a, arg) (a)
+#define mh_cmp(a, b, arg) (((*a)->pid) != ((*b)->pid))
+#define mh_cmp_key(a, b, arg) ((a) != ((*b)->pid))
+#define MH_SOURCE 1
+#include "salad/mhash.h"
+
+
+static pthread_mutex_t mutex;
+static struct mh_popen_storage_t* popen_hash_table = NULL;
+
+void
+popen_initialize()
+{
+ pthread_mutexattr_t errorcheck;
+ pthread_mutexattr_init(&errorcheck);
+ pthread_mutexattr_settype(&errorcheck,
+ PTHREAD_MUTEX_ERRORCHECK);
+ pthread_mutex_init(&mutex, &errorcheck);
+ pthread_mutexattr_destroy(&errorcheck);
+
+ popen_hash_table = mh_popen_storage_new();
+}
+
+static void
+popen_lock_data_list()
+{
+ pthread_mutex_lock(&mutex);
+}
+
+static void
+popen_unlock_data_list()
+{
+ pthread_mutex_unlock(&mutex);
+}
+
+static void
+popen_append_to_list(struct popen_handle *data)
+{
+ struct popen_handle **old = NULL;
+ mh_int_t id = mh_popen_storage_put(popen_hash_table,
+ (const struct popen_handle **)&data, &old, NULL);
+ (void)id;
+}
+
+static struct popen_handle *
+popen_lookup_data_by_pid(pid_t pid)
+{
+ mh_int_t pos = mh_popen_storage_find(popen_hash_table, pid, NULL);
+ if (pos == mh_end(popen_hash_table))
+ return NULL;
+ else {
+ struct popen_handle ** ptr =
+ mh_popen_storage_node(popen_hash_table, pos);
+ return *ptr;
+ }
+}
+
+static void
+popen_exclude_from_list(struct popen_handle *data)
+{
+ mh_popen_storage_remove(popen_hash_table,
+ (const struct popen_handle **)&data, NULL);
+}
+
+static struct popen_handle *
+popen_data_new()
+{
+ struct popen_handle *data =
+ (struct popen_handle *)calloc(1, sizeof(*data));
+ data->fd[0] = -1;
+ data->fd[1] = -1;
+ data->fd[2] = -1;
+ data->devnull_fd = -1;
+ data->status = POPEN_RUNNING;
+ return data;
+}
+
+enum pipe_end {
+ PIPE_READ = 0,
+ PIPE_WRITE = 1
+};
+
+static inline enum pipe_end
+ popen_opposite_pipe(enum pipe_end side)
+{
+ return (enum pipe_end)(side ^ 1);
+ /*
+ * The code is equal to:
+ * return (side == PIPE_READ) ? PIPE_WRITE
+ * : PIPE_READ;
+ */
+}
+
+static inline bool
+popen_create_pipe(int fd, int pipe_pair[2], enum pipe_end parent_side)
+{
+ if (fd == FIO_PIPE) {
+ if (pipe(pipe_pair) < 0 ||
+ fcntl(pipe_pair[parent_side], F_SETFL, O_NONBLOCK) < 0) {
+ return false;
+ }
+ }
+ return true;
+}
+
+static inline void
+popen_close_child_fd(int std_fd, int pipe_pair[2],
+ int *saved_fd, enum pipe_end child_side)
+{
+ if (std_fd == FIO_PIPE) {
+ /* Close child's side. */
+ close(pipe_pair[child_side]);
+
+ enum pipe_end parent_side = popen_opposite_pipe(child_side);
+ *saved_fd = pipe_pair[parent_side];
+ }
+}
+
+static inline void
+popen_close_pipe(int pipe_pair[2])
+{
+ if (pipe_pair[0] >= 0) {
+ close(pipe_pair[0]);
+ close(pipe_pair[1]);
+ }
+}
+
+
+/**
+ * Implementation of fio.popen.
+ * The function opens a process by creating a pipe
+ * forking.
+ *
+ * @param argv - is an array of character pointers
+ * to the arguments terminated by a null pointer.
+ *
+ * @param envp - is the pointer to an array
+ * of character pointers to the environment strings.
+ *
+ * @param stdin_fd - the file handle to be redirected to the
+ * child process's STDIN.
+ *
+ * @param stdout_fd - the file handle receiving the STDOUT
+ * output of the child process.
+ *
+ * @param stderr_fd - the file handle receiving the STDERR
+ * output of the child process.
+ *
+ * The stdin_fd, stdout_fd & stderr_fd accept file descriptors
+ * from open() or the following values:
+ *
+ * FIO_PIPE - opens a pipe, binds it with child's
+ * input/output. The pipe is available for reading/writing.
+ *
+ * FIO_DEVNULL - redirects output from process to /dev/null.
+ *
+ * @return handle of the pipe for reading or writing
+ * (depends on value of type).
+ * In a case of error returns NULL.
+ */
+static struct popen_handle *
+popen_new_impl(char **argv, char **envp,
+ int stdin_fd, int stdout_fd, int stderr_fd)
+{
+ bool popen_list_locked = false;
+ pid_t pid;
+ int pipe_rd[2] = {-1,-1};
+ int pipe_wr[2] = {-1,-1};
+ int pipe_er[2] = {-1,-1};
+ errno = 0;
+
+ struct popen_handle *data = popen_data_new();
+ if (data == NULL)
+ return NULL;
+
+ /*
+ * Setup a /dev/null if necessary.
+ */
+ bool read_devnull = (stdin_fd == FIO_DEVNULL);
+ bool write_devnull = (stdout_fd == FIO_DEVNULL) ||
+ (stderr_fd == FIO_DEVNULL);
+ int devnull_flags = O_RDWR;
+ if (!read_devnull)
+ devnull_flags = O_WRONLY;
+ else if (!write_devnull)
+ devnull_flags = O_RDONLY;
+
+ if (read_devnull || write_devnull) {
+ data->devnull_fd = open("/dev/null", devnull_flags);
+ if (data->devnull_fd < 0)
+ goto on_error;
+ else {
+ if (stdin_fd == FIO_DEVNULL)
+ stdin_fd = data->devnull_fd;
+ if (stdout_fd == FIO_DEVNULL)
+ stdout_fd = data->devnull_fd;
+ if (stderr_fd == FIO_DEVNULL)
+ stderr_fd = data->devnull_fd;
+ }
+ }
+
+ if (!popen_create_pipe(stdin_fd, pipe_rd, PIPE_WRITE))
+ goto on_error;
+ if (!popen_create_pipe(stdout_fd, pipe_wr, PIPE_READ))
+ goto on_error;
+ if (!popen_create_pipe(stderr_fd, pipe_er, PIPE_READ))
+ goto on_error;
+
+ /*
+ * Prepare data for the child process.
+ * There must be no branches, to avoid compiler
+ * error: "argument ‘xxx’ might be clobbered by
+ * ‘longjmp’ or ‘vfork’ [-Werror=clobbered]".
+ */
+ if (envp == NULL)
+ envp = environ;
+
+ /* Handles to be closed in child process */
+ int close_fd[3] = {-1, -1, -1};
+ /* Handles to be duplicated in child process */
+ int dup_fd[3] = {-1, -1, -1};
+ /* Handles to be closed after dup2 in child process */
+ int close_after_dup_fd[3] = {-1, -1, -1};
+
+ if (stdin_fd == FIO_PIPE) {
+ close_fd[STDIN_FILENO] = pipe_rd[PIPE_WRITE];
+ dup_fd[STDIN_FILENO] = pipe_rd[PIPE_READ];
+ } else if (stdin_fd != STDIN_FILENO) {
+ dup_fd[STDIN_FILENO] = stdin_fd;
+ close_after_dup_fd[STDIN_FILENO] = stdin_fd;
+ }
+
+ if (stdout_fd == FIO_PIPE) {
+ close_fd[STDOUT_FILENO] = pipe_wr[PIPE_READ];
+ dup_fd[STDOUT_FILENO] = pipe_wr[PIPE_WRITE];
+ } else if (stdout_fd != STDOUT_FILENO){
+ dup_fd[STDOUT_FILENO] = stdout_fd;
+ if (stdout_fd != STDERR_FILENO)
+ close_after_dup_fd[STDOUT_FILENO] = stdout_fd;
+ }
+
+ if (stderr_fd == FIO_PIPE) {
+ close_fd[STDERR_FILENO] = pipe_er[PIPE_READ];
+ dup_fd[STDERR_FILENO] = pipe_er[PIPE_WRITE];
+ } else if (stderr_fd != STDERR_FILENO) {
+ dup_fd[STDERR_FILENO] = stderr_fd;
+ if (stderr_fd != STDOUT_FILENO)
+ close_after_dup_fd[STDERR_FILENO] = stderr_fd;
+ }
+
+
+ popen_lock_data_list();
+ popen_list_locked = true;
+
+ pid = vfork();
+
+ if (pid < 0)
+ goto on_error;
+ else if (pid == 0) /* child */ {
+ /* Reset all signals to their defaults. */
+ struct sigaction sa;
+ memset(&sa, 0, sizeof(sa));
+ sigemptyset(&sa.sa_mask);
+ sa.sa_handler = SIG_DFL;
+
+ if (sigaction(SIGUSR1, &sa, NULL) == -1 ||
+ sigaction(SIGINT, &sa, NULL) == -1 ||
+ sigaction(SIGTERM, &sa, NULL) == -1 ||
+ sigaction(SIGHUP, &sa, NULL) == -1 ||
+ sigaction(SIGWINCH, &sa, NULL) == -1 ||
+ sigaction(SIGSEGV, &sa, NULL) == -1 ||
+ sigaction(SIGFPE, &sa, NULL) == -1 ||
+ sigaction(SIGCHLD, &sa, NULL) == -1)
+ exit(EX_OSERR);
+
+ /* Unblock any signals blocked by libev. */
+ sigset_t sigset;
+ sigfillset(&sigset);
+ if (sigprocmask(SIG_UNBLOCK, &sigset, NULL) == -1)
+ exit(EX_OSERR);
+
+ /* Setup stdin/stdout */
+ for(int i = 0; i < 3; ++i) {
+ if (close_fd[i] >= 0)
+ close(close_fd[i]);
+ if (dup_fd[i] >= 0)
+ dup2(dup_fd[i], i);
+ if (close_after_dup_fd[i] >= 0)
+ close(close_after_dup_fd[i]);
+ }
+
+ execve( argv[0], argv, envp);
+ exit(EX_OSERR);
+ unreachable();
+ }
+
+ /* Parent process */
+ popen_close_child_fd(stdin_fd, pipe_rd,
+ &data->fd[STDIN_FILENO], PIPE_READ);
+ popen_close_child_fd(stdout_fd, pipe_wr,
+ &data->fd[STDOUT_FILENO], PIPE_WRITE);
+ popen_close_child_fd(stderr_fd, pipe_er,
+ &data->fd[STDERR_FILENO], PIPE_WRITE);
+
+ data->pid = pid;
+
+on_cleanup:
+ if (data){
+ popen_append_to_list(data);
+ }
+
+ if (popen_list_locked)
+ popen_unlock_data_list();
+
+ if (argv){
+ for(int i = 0; argv[i] != NULL; ++i)
+ free(argv[i]);
+ free(argv);
+ }
+ if (envp && envp != environ) {
+ for(int i = 0; envp[i] != NULL; ++i)
+ free(envp[i]);
+ free(envp);
+ }
+
+ return data;
+
+on_error:
+ popen_close_pipe(pipe_rd);
+ popen_close_pipe(pipe_wr);
+ popen_close_pipe(pipe_er);
+
+ if (data) {
+ if (data->devnull_fd >= 0)
+ close(data->devnull_fd);
+ free(data);
+ }
+ data = NULL;
+
+ goto on_cleanup;
+ unreachable();
+}
+
+ssize_t
+popen_new(va_list ap)
+{
+ char **argv = va_arg(ap, char **);
+ char **envp = va_arg(ap, char **);
+ int stdin_fd = va_arg(ap, int);
+ int stdout_fd = va_arg(ap, int);
+ int stderr_fd = va_arg(ap, int);
+ struct popen_handle **handle = va_arg(ap, struct popen_handle **);
+
+ *handle = popen_new_impl(argv, envp, stdin_fd, stdout_fd, stderr_fd);
+ return (*handle) ? 0 : -1;
+}
+
+static void
+popen_close_handles(struct popen_handle *data)
+{
+ for(int i = 0; i < 3; ++i) {
+ if (data->fd[i] >= 0) {
+ close(data->fd[i]);
+ data->fd[i] = -1;
+ }
+ }
+ if (data->devnull_fd >= 0) {
+ close(data->devnull_fd);
+ data->devnull_fd = -1;
+ }
+}
+
+int
+popen_destroy(struct popen_handle *fh)
+{
+ assert(fh);
+
+ popen_lock_data_list();
+ popen_close_handles(fh);
+ popen_exclude_from_list(fh);
+ popen_unlock_data_list();
+
+ free(fh);
+ return 0;
+}
+
+/**
+ * Check if an errno, returned from a sio function, means a
+ * non-critical error: EAGAIN, EWOULDBLOCK, EINTR.
+ */
+static inline bool
+popen_wouldblock(int err)
+{
+ return err == EAGAIN || err == EWOULDBLOCK || err == EINTR;
+}
+
+static int
+popen_do_read(struct popen_handle *data, void *buf, size_t count, int *source_id)
+{
+ /*
+ * STDERR has higher priority, read it first.
+ */
+ int rc = 0;
+ errno = 0;
+ int fd_count = 0;
+ if (data->fd[STDERR_FILENO] >= 0) {
+ ++fd_count;
+ rc = read(data->fd[STDERR_FILENO], buf, count);
+
+ if (rc >= 0)
+ *source_id = STDERR_FILENO;
+ if (rc > 0)
+ return rc;
+
+ if (rc < 0 && !popen_wouldblock(errno))
+ return rc;
+
+ }
+
+ /*
+ * STDERR is not available or not ready, try STDOUT.
+ */
+ if (data->fd[STDOUT_FILENO] >= 0) {
+ ++fd_count;
+ rc = read(data->fd[STDOUT_FILENO], buf, count);
+
+ if (rc >= 0) {
+ *source_id = STDOUT_FILENO;
+ return rc;
+ }
+ }
+
+ if (!fd_count) {
+ /*
+ * There are no open handles for reading.
+ */
+ errno = EBADF;
+ rc = -1;
+ }
+ return rc;
+}
+
+static void
+popen_coio_create(struct ev_io *coio, int fd)
+{
+ coio->data = fiber();
+ ev_init(coio, (ev_io_cb) fiber_schedule_cb);
+ coio->fd = fd;
+}
+
+int
+popen_read(struct popen_handle *fh, void *buf, size_t count,
+ size_t *read_bytes, int *source_id,
+ ev_tstamp timeout)
+{
+ assert(fh);
+ if (timeout < 0.0)
+ timeout = DBL_MAX;
+
+ ev_tstamp start, delay;
+ evio_timeout_init(loop(), &start, &delay, timeout);
+
+ struct ev_io coio_rd;
+ struct ev_io coio_er;
+ popen_coio_create(&coio_er, fh->fd[STDERR_FILENO]);
+ popen_coio_create(&coio_rd, fh->fd[STDOUT_FILENO]);
+
+ int result = 0;
+
+ while (true) {
+ int rc = popen_do_read(fh, buf, count, source_id);
+ if (rc >= 0) {
+ *read_bytes = rc;
+ break;
+ }
+
+ if (!popen_wouldblock(errno)) {
+ result = -1;
+ break;
+ }
+
+ /*
+ * The handlers are not ready, yield.
+ */
+ if (!ev_is_active(&coio_rd) &&
+ fh->fd[STDOUT_FILENO] >= 0) {
+ ev_io_set(&coio_rd, fh->fd[STDOUT_FILENO], EV_READ);
+ ev_io_start(loop(), &coio_rd);
+ }
+ if (!ev_is_active(&coio_er) &&
+ fh->fd[STDERR_FILENO] >= 0) {
+ ev_io_set(&coio_er, fh->fd[STDERR_FILENO], EV_READ);
+ ev_io_start(loop(), &coio_er);
+ }
+ /*
+ * Yield control to other fibers until the
+ * timeout is reached.
+ */
+ bool is_timedout = fiber_yield_timeout(delay);
+ if (is_timedout) {
+ errno = ETIMEDOUT;
+ result = -1;
+ break;
+ }
+
+ if (fiber_is_cancelled()) {
+ errno = EINTR;
+ result = -1;
+ break;
+ }
+
+ evio_timeout_update(loop(), &start, &delay);
+ }
+
+ ev_io_stop(loop(), &coio_er);
+ ev_io_stop(loop(), &coio_rd);
+ return result;
+}
+
+int
+popen_write(struct popen_handle *fh, const void *buf, size_t count,
+ size_t *written, ev_tstamp timeout)
+{
+ assert(fh);
+ if (count == 0) {
+ *written = 0;
+ return 0;
+ }
+
+ if (timeout < 0.0)
+ timeout = DBL_MAX;
+
+ if (fh->fd[STDIN_FILENO] < 0) {
+ *written = 0;
+ errno = EBADF;
+ return -1;
+ }
+
+ ev_tstamp start, delay;
+ evio_timeout_init(loop(), &start, &delay, timeout);
+
+ struct ev_io coio;
+ popen_coio_create(&coio, fh->fd[STDIN_FILENO]);
+ int result = 0;
+
+ while(true) {
+ ssize_t rc = write(fh->fd[STDIN_FILENO], buf, count);
+ if (rc < 0 && !popen_wouldblock(errno)) {
+ result = -1;
+ break;
+ }
+
+ size_t urc = (size_t)rc;
+
+ if (urc == count) {
+ *written = count;
+ break;
+ }
+
+ if (rc > 0) {
+ buf += rc;
+ count -= urc;
+ }
+
+ /*
+ * The handlers are not ready, yield.
+ */
+ if (!ev_is_active(&coio)) {
+ ev_io_set(&coio, fh->fd[STDIN_FILENO], EV_WRITE);
+ ev_io_start(loop(), &coio);
+ }
+
+ /*
+ * Yield control to other fibers until the
+ * timeout is reached.
+ */
+ bool is_timedout = fiber_yield_timeout(delay);
+ if (is_timedout) {
+ errno = ETIMEDOUT;
+ result = -1;
+ break;
+ }
+
+ if (fiber_is_cancelled()) {
+ errno = EINTR;
+ result = -1;
+ break;
+ }
+
+ evio_timeout_update(loop(), &start, &delay);
+ }
+
+ ev_io_stop(loop(), &coio);
+ return result;
+}
+
+int
+popen_kill(struct popen_handle *fh, int signal_id)
+{
+ assert(fh);
+ return kill(fh->pid, signal_id);
+}
+
+int
+popen_wait(struct popen_handle *fh, ev_tstamp timeout, int *exit_code)
+{
+ assert(fh);
+ if (timeout < 0.0)
+ timeout = DBL_MAX;
+
+ ev_tstamp start, delay;
+ evio_timeout_init(loop(), &start, &delay, timeout);
+
+ int result = 0;
+
+ while (true) {
+ /* Wait for SIGCHLD */
+ int code = 0;
+
+ int rc = popen_get_status(fh, &code);
+ if (rc != POPEN_RUNNING) {
+ *exit_code = (rc == POPEN_EXITED) ? code
+ : -code;
+ break;
+ }
+
+ /*
+ * Yield control to other fibers until the
+ * timeout is reached.
+ * Let's sleep for 20 msec.
+ */
+ fiber_yield_timeout(0.02);
+
+ if (fiber_is_cancelled()) {
+ errno = EINTR;
+ result = -1;
+ break;
+ }
+
+ evio_timeout_update(loop(), &start, &delay);
+ bool is_timedout = (delay == 0.0);
+ if (is_timedout) {
+ errno = ETIMEDOUT;
+ result = -1;
+ break;
+ }
+ }
+
+ return result;
+}
+
+int
+popen_get_std_file_handle(struct popen_handle *fh, int file_no)
+{
+ assert(fh);
+ if (file_no < STDIN_FILENO || STDERR_FILENO < file_no){
+ errno = EINVAL;
+ return -1;
+ }
+
+ errno = 0;
+ return fh->fd[file_no];
+}
+
+int
+popen_get_status(struct popen_handle *fh, int *exit_code)
+{
+ assert(fh);
+ errno = 0;
+
+ if (exit_code)
+ *exit_code = fh->exit_code;
+
+ return fh->status;
+}
+
+/*
+ * evio data to control SIGCHLD
+ */
+static ev_child cw;
+
+static void
+popen_sigchld_cb(ev_loop *loop, ev_child *watcher, int revents)
+{
+ (void)loop;
+ (void)revents;
+
+ popen_lock_data_list();
+
+ struct popen_handle *data = popen_lookup_data_by_pid(watcher->rpid);
+ if (data) {
+ if (WIFEXITED(watcher->rstatus)) {
+ data->exit_code = WEXITSTATUS(watcher->rstatus);
+ data->status = POPEN_EXITED;
+ } else if (WIFSIGNALED(watcher->rstatus)) {
+ data->exit_code = WTERMSIG(watcher->rstatus);
+
+ if (WCOREDUMP(watcher->rstatus))
+ data->status = POPEN_DUMPED;
+ else
+ data->status = POPEN_KILLED;
+ } else {
+ /*
+ * The status is not determined, treat as EXITED
+ */
+ data->exit_code = EX_SOFTWARE;
+ data->status = POPEN_EXITED;
+ }
+
+ /*
+ * We shouldn't close file descriptors here.
+ * The child process may exit earlier than
+ * the parent process finishes reading data.
+ * In this case the reading fails.
+ */
+ }
+
+ popen_unlock_data_list();
+}
+
+void
+popen_setup_sigchld_handler()
+{
+ ev_child_init (&cw, popen_sigchld_cb, 0/*pid*/, 0);
+ ev_child_start(loop(), &cw);
+
+}
+
+void
+popen_reset_sigchld_handler()
+{
+ ev_child_stop(loop(), &cw);
+}
diff --git a/src/lib/core/coio_popen.h b/src/lib/core/coio_popen.h
new file mode 100644
index 000000000..57057e97b
--- /dev/null
+++ b/src/lib/core/coio_popen.h
@@ -0,0 +1,229 @@
+#ifndef TARANTOOL_LIB_CORE_COIO_POPEN_H_INCLUDED
+#define TARANTOOL_LIB_CORE_COIO_POPEN_H_INCLUDED
+/*
+ * Copyright 2010-2019, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the
+ * following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#if defined(__cplusplus)
+extern "C" {
+#endif /* defined(__cplusplus) */
+
+#include "evio.h"
+#include <stdarg.h>
+
+/**
+ * Special values of the file descriptors passed to fio.popen
+ * */
+enum {
+ /**
+ * Tells fio.popen to open a handle for
+ * direct reading/writing.
+ */
+ FIO_PIPE = -2,
+
+ /**
+ * Tells fio.popen to redirect the given standard
+ * stream into /dev/null.
+ */
+ FIO_DEVNULL = -3
+};
+
+struct popen_handle;
+
+/**
+ * Possible status of the process started via fio.popen
+ **/
+enum popen_status {
+
+ /**
+ * The process is alive and well.
+ */
+ POPEN_RUNNING = 1,
+
+ /**
+ * The process exited.
+ */
+ POPEN_EXITED = 2,
+
+ /**
+ * The process terminated by a signal.
+ */
+ POPEN_KILLED = 3,
+
+ /**
+ * The process terminated abnormally.
+ */
+ POPEN_DUMPED = 4
+};
+
+/**
+ * Initializes inner data of fio.popen
+ * */
+void
+popen_initialize();
+
+ssize_t
+popen_new(va_list ap);
+
+/**
+ * The function releases allocated resources.
+ * The function doesn't wait for the associated process
+ * to terminate.
+ *
+ * @param fh handle returned by fio.popen.
+ *
+ * @return 0 if the process is terminated
+ * @return -1 for an error
+ */
+int
+popen_destroy(struct popen_handle *fh);
+
+/**
+ * The function reads up to count bytes from the handle
+ * associated with the child process.
+ * Returns immediately
+ *
+ * @param fd handle returned by fio.popen.
+ * @param buf a buffer to be read into
+ * @param count size of buffer in bytes
+ * @param read_bytes A pointer to the
+ * variable that receives the number of bytes read.
+ * @param source_id A pointer to the variable that receives a
+ * source stream id, 1 - for STDOUT, 2 - for STDERR.
+ *
+ * @return 0 data were successfully read
+ * @return -1 an error occurred, see errno for error code
+ *
+ * If there is nothing to read yet function returns -1
+ * and errno set no EAGAIN.
+ */
+int
+popen_read(struct popen_handle *fh, void *buf, size_t count,
+ size_t *read_bytes, int *source_id,
+ ev_tstamp timeout);
+
+/**
+ * The function writes up to count bytes to the handle
+ * associated with the child process.
+ * Tries to write as much as possible without blocking
+ * and immediately returns.
+ *
+ * @param fd handle returned by fio.popen.
+ * @param buf a buffer to be written from
+ * @param count size of buffer in bytes
+ * @param written A pointer to the
+ * variable that receives the number of bytes actually written.
+ * If function fails the number of written bytes is undefined.
+ *
+ * @return 0 data were successfully written
+ * Compare values of <count> and <written> to check
+ * whether all data were written or not.
+ * @return -1 an error occurred, see errno for error code
+ *
+ * If the writing can block, function returns -1
+ * and errno set no EAGAIN.
+ */
+int
+popen_write(struct popen_handle *fh, const void *buf, size_t count,
+ size_t *written, ev_tstamp timeout);
+
+
+/**
+ * The function send the specified signal
+ * to the associated process.
+ *
+ * @param fd - handle returned by fio.popen.
+ *
+ * @return 0 on success
+ * @return -1 an error occurred, see errno for error code
+ */
+int
+popen_kill(struct popen_handle *fh, int signal_id);
+
+/**
+ * Wait for the associated process to terminate.
+ * The function doesn't release the allocated resources.
+ *
+ * @param fd handle returned by fio.popen.
+ *
+ * @param timeout number of second to wait before function exit with error.
+ * If function exited due to timeout the errno equals to ETIMEDOUT.
+ *
+ * @exit_code On success contains the exit code as a positive number
+ * or signal id as a negative number.
+
+ * @return On success function returns 0, and -1 on error.
+ */
+int
+popen_wait(struct popen_handle *fh, ev_tstamp timeout, int *exit_code);
+
+/**
+ * Returns descriptor of the specified file.
+ *
+ * @param fd - handle returned by fio.popen.
+ * @param file_no accepts one of the
+ * following values:
+ * STDIN_FILENO,
+ * STDOUT_FILENO,
+ * STDERR_FILENO
+ *
+ * @return file descriptor or -1 if not available
+ */
+int
+popen_get_std_file_handle(struct popen_handle *fh, int file_no);
+
+
+/**
+ * Returns status of the associated process.
+ *
+ * @param fd - handle returned by fio.popen.
+ *
+ * @param exit_code - if not NULL accepts the exit code
+ * if the process terminated normally or signal id
+ * if process was termianted by signal.
+ *
+ * @return one of the following values:
+ * POPEN_RUNNING if the process is alive
+ * POPEN_EXITED if the process was terminated normally
+ * POPEN_KILLED if the process was terminated by a signal
+ */
+int
+popen_get_status(struct popen_handle *fh, int *exit_code);
+
+void
+popen_setup_sigchld_handler();
+void
+popen_reset_sigchld_handler();
+
+#if defined(__cplusplus)
+} /* extern "C" */
+#endif /* defined(__cplusplus) */
+
+#endif /* TARANTOOL_LIB_CORE_COIO_POPEN_H_INCLUDED */
diff --git a/src/lib/core/coio_task.c b/src/lib/core/coio_task.c
index 908b336ed..e6a1b327f 100644
--- a/src/lib/core/coio_task.c
+++ b/src/lib/core/coio_task.c
@@ -40,6 +40,7 @@
#include "fiber.h"
#include "third_party/tarantool_ev.h"
+#include "coio_popen.h"
/*
* Asynchronous IO Tasks (libeio wrapper).
@@ -129,6 +130,7 @@ coio_on_stop(void *data)
void
coio_init(void)
{
+ popen_initialize();
eio_set_thread_on_start(coio_on_start, NULL);
eio_set_thread_on_stop(coio_on_stop, NULL);
}
diff --git a/src/lua/fio.c b/src/lua/fio.c
index 806f4256b..873ee1651 100644
--- a/src/lua/fio.c
+++ b/src/lua/fio.c
@@ -46,6 +46,9 @@
#include "lua/utils.h"
#include "coio_file.h"
+#include "coio_popen.h"
+
+static uint32_t CTID_STRUCT_POPEN_HANDLE_REF = 0;
static inline void
lbox_fio_pushsyserror(struct lua_State *L)
@@ -703,6 +706,269 @@ lbox_fio_copyfile(struct lua_State *L)
return lbox_fio_pushbool(L, coio_copyfile(source, dest) == 0);
}
+static bool
+popen_verify_argv(struct lua_State *L)
+{
+ if (!lua_istable(L, 1))
+ return false;
+ int num = (int)lua_objlen(L, 1); /*luaL_getn(L,1);*/
+ return num >= 1;
+}
+
+static char**
+popen_extract_strarray(struct lua_State *L, int index, int* array_size)
+{
+ if (lua_type(L, index) != LUA_TTABLE) {
+ if (array_size)
+ *array_size = 0;
+ return NULL;
+ }
+
+ size_t num = lua_objlen(L, index); /*luaL_getn(L,index);*/
+
+ char** array = calloc(num+1, sizeof(char*));
+ /*
+ * The last item in the array must be NULL
+ */
+
+ if (array == NULL)
+ return NULL;
+
+ for(size_t i = 0; i < num; ++i) {
+ lua_rawgeti(L, index, i+1);
+ size_t slen = 0;
+ const char* str = lua_tolstring(L, -1, &slen);
+ if (!str)
+ str = "";
+ array[i] = strdup(str);
+ lua_pop(L, 1);
+ }
+
+ if (array_size)
+ *array_size = num;
+ /*
+ * The number of elements doesn't include
+ * the trailing NULL pointer
+ */
+ return array;
+}
+
+static struct popen_handle *
+fio_popen_get_handle(lua_State *L, int idx)
+{
+ uint32_t cdata_type;
+ struct popen_handle **handle_ptr = luaL_checkcdata(L, idx, &cdata_type);
+ if (handle_ptr == NULL || cdata_type != CTID_STRUCT_POPEN_HANDLE_REF)
+ return NULL;
+ return *handle_ptr;
+}
+
+static void
+fio_popen_invalidate_handle(lua_State *L, int idx)
+{
+ uint32_t cdata_type;
+ struct popen_handle **handle_ptr = luaL_checkcdata(L, idx, &cdata_type);
+ if (handle_ptr != NULL && cdata_type == CTID_STRUCT_POPEN_HANDLE_REF) {
+ *handle_ptr = NULL;
+ }
+}
+
+static int
+lbox_fio_popen_gc(lua_State *L)
+{
+ struct popen_handle *handle = fio_popen_get_handle(L,1);
+
+ if (handle)
+ popen_destroy(handle);
+ return 0;
+}
+
+static int
+lbox_fio_popen(struct lua_State *L)
+{
+ if (lua_gettop(L) < 1) {
+ usage:
+ luaL_error(L, "fio.popen: Invalid arguments");
+ }
+
+ if (!popen_verify_argv(L))
+ goto usage;
+
+ int stdin_fd = FIO_PIPE;
+ int stdout_fd = FIO_PIPE;
+ int stderr_fd = FIO_PIPE;
+
+ char** argv = popen_extract_strarray(L, 1, NULL);
+ char** env = popen_extract_strarray(L, 2, NULL);
+
+ if (lua_isnumber(L, 3))
+ stdin_fd = lua_tonumber(L, 3);
+ if (lua_isnumber(L, 4))
+ stdout_fd = lua_tonumber(L, 4);
+ if (lua_isnumber(L, 5))
+ stderr_fd = lua_tonumber(L, 5);
+
+ struct popen_handle* handle = NULL;
+ if (coio_call(popen_new, argv, env, stdin_fd, stdout_fd,
+ stderr_fd, &handle) < 0) {
+ lua_pushnil(L);
+ lbox_fio_pushsyserror(L);
+ return 2;
+ } else {
+ luaL_pushcdata(L, CTID_STRUCT_POPEN_HANDLE_REF);
+ *(struct popen_handle **)
+ luaL_pushcdata(L, CTID_STRUCT_POPEN_HANDLE_REF) = handle;
+ lua_pushcfunction(L, lbox_fio_popen_gc);
+ luaL_setcdatagc(L, -2);
+
+ return 1;
+ }
+}
+
+static int
+lbox_fio_popen_read(struct lua_State *L)
+{
+ /* popen_read(self.fh, buf, size, seconds) */
+
+ void* fh = fio_popen_get_handle(L, 1);
+ uint32_t ctypeid;
+ char *buf = *(char **)luaL_checkcdata(L, 2, &ctypeid);
+ size_t len = lua_tonumber(L, 3);
+ ev_tstamp seconds = lua_tonumber(L, 4);
+
+ if (!len) {
+ lua_pushinteger(L, 0);
+ lua_pushinteger(L, STDOUT_FILENO);
+ return 2;
+ }
+
+ int output_number = 0;
+ size_t received = 0;
+ int rc = popen_read(fh, buf, len,
+ &received, &output_number,
+ seconds);
+ if (rc == 0) { /* The reading's succeeded */
+ lua_pushinteger(L, received);
+ lua_pushinteger(L, output_number);
+ return 2;
+ } else {
+ lua_pushnil(L);
+ lua_pushnil(L);
+ lbox_fio_pushsyserror(L);
+ return 3;
+ }
+}
+
+static int
+lbox_fio_popen_write(struct lua_State *L)
+{
+ struct popen_handle* fh = fio_popen_get_handle(L, 1);
+ const char *buf = lua_tostring(L, 2);
+ uint32_t ctypeid = 0;
+ if (buf == NULL)
+ buf = *(const char **)luaL_checkcdata(L, 2, &ctypeid);
+ size_t len = lua_tointeger(L, 3);
+ double timeout = lua_tonumber(L, 4);
+
+ size_t written = 0;
+ int rc = popen_write(fh, buf, len,
+ &written, timeout);
+ if (rc == 0 && written == len) {
+ /* The writing's succeeded */
+ lua_pushinteger(L, (ssize_t) written);
+ return 1;
+ } else {
+ lua_pushnil(L);
+ lbox_fio_pushsyserror(L);
+ return 2;
+ }
+}
+
+static int
+lbox_fio_popen_get_status(struct lua_State *L)
+{
+ struct popen_handle* fh = fio_popen_get_handle(L, 1);
+ int exit_code = 0;
+ int res = popen_get_status(fh, &exit_code);
+
+ switch (res) {
+ case POPEN_RUNNING:
+ lua_pushnil(L);
+ break;
+
+ case POPEN_KILLED:
+ lua_pushinteger(L, -exit_code);
+ break;
+
+ default:
+ lua_pushinteger(L, exit_code);
+ break;
+ }
+
+ return 1;
+}
+
+static int
+lbox_fio_popen_get_std_file_handle(struct lua_State *L)
+{
+ struct popen_handle* fh = fio_popen_get_handle(L, 1);
+ int file_no = lua_tonumber(L, 2);
+ int res = popen_get_std_file_handle(fh, file_no);
+
+ if (res < 0)
+ lua_pushnil(L);
+ else
+ lua_pushinteger(L, res);
+ return 1;
+}
+
+static int
+lbox_fio_popen_kill(struct lua_State *L)
+{
+ struct popen_handle* fh = fio_popen_get_handle(L, 1);
+ int signal_id = lua_tonumber(L, 2);
+
+ int res = popen_kill(fh, signal_id);
+ if (res < 0){
+ lua_pushboolean(L, false);
+ lbox_fio_pushsyserror(L);
+ return 2;
+ } else {
+ lua_pushboolean(L, true);
+ return 1;
+ }
+}
+
+static int
+lbox_fio_popen_wait(struct lua_State *L)
+{
+ struct popen_handle *fh = fio_popen_get_handle(L, 1);
+ assert(fh);
+ ev_tstamp timeout = lua_tonumber(L, 2);
+
+ /*
+ * On success function returns an
+ * exit code as a positive number
+ * or signal id as a negative number.
+ * If failed, rc is nul and err
+ * contains a error message.
+ */
+
+ int exit_code =0;
+ int res = popen_wait(fh, timeout, &exit_code);
+ if (res < 0){
+ lua_pushnil(L);
+ lbox_fio_pushsyserror(L);
+ return 2;
+ } else {
+ /* Release the allocated resources */
+ popen_destroy(fh);
+ fio_popen_invalidate_handle(L, 1);
+
+ lua_pushinteger(L, exit_code);
+ return 1;
+ }
+}
void
@@ -747,6 +1013,13 @@ tarantool_lua_fio_init(struct lua_State *L)
{ "listdir", lbox_fio_listdir },
{ "fstat", lbox_fio_fstat },
{ "copyfile", lbox_fio_copyfile, },
+ { "popen", lbox_fio_popen },
+ { "popen_read", lbox_fio_popen_read },
+ { "popen_write", lbox_fio_popen_write },
+ { "popen_get_status", lbox_fio_popen_get_status },
+ { "popen_get_std_file_handle", lbox_fio_popen_get_std_file_handle },
+ { "popen_kill", lbox_fio_popen_kill },
+ { "popen_wait", lbox_fio_popen_wait },
{ NULL, NULL }
};
luaL_register(L, NULL, internal_methods);
@@ -849,4 +1122,12 @@ tarantool_lua_fio_init(struct lua_State *L)
lua_settable(L, -3);
lua_pop(L, 1);
+
+ /* Get CTypeID for `struct tuple' */
+ int rc = luaL_cdef(L, "struct popen_handle;");
+ assert(rc == 0);
+ (void) rc;
+ CTID_STRUCT_POPEN_HANDLE_REF = luaL_ctypeid(L, "struct popen_handle &");
+ assert(CTID_STRUCT_POPEN_HANDLE_REF != 0);
+
}
diff --git a/src/lua/fio.lua b/src/lua/fio.lua
index ba8c47ec0..d07cac552 100644
--- a/src/lua/fio.lua
+++ b/src/lua/fio.lua
@@ -3,6 +3,8 @@
local fio = require('fio')
local ffi = require('ffi')
local buffer = require('buffer')
+local signal = require('signal')
+local errno = require('errno')
ffi.cdef[[
int umask(int mask);
@@ -15,6 +17,13 @@ local const_char_ptr_t = ffi.typeof('const char *')
local internal = fio.internal
fio.internal = nil
+fio.STDIN = 0
+fio.STDOUT = 1
+fio.STDERR = 2
+fio.PIPE = -2
+fio.DEVNULL = -3
+
+
local function sprintf(fmt, ...)
if select('#', ...) == 0 then
return fmt
@@ -206,6 +215,216 @@ fio.open = function(path, flags, mode)
return fh
end
+local popen_methods = {}
+
+-- read stdout & stderr of the process started by fio.popen
+-- Usage:
+-- read(size) -> str, source, err
+-- read(buf, size) -> length, source, err
+-- read(size, timeout) -> str, source, err
+-- read(buf, size, timeout) -> length, source, err
+--
+-- timeout - number of seconds to wait (optional)
+-- source contains id of the stream, fio.STDOUT or fio.STDERR
+-- err - error message if method has failed or nil on success
+popen_methods.read = function(self, buf, size, timeout)
+ if self.fh == nil then
+ return nil, nil, 'Invalid object'
+ end
+
+ local tmpbuf
+
+ if ffi.istype(const_char_ptr_t, buf) then
+ -- ext. buffer is specified
+ if type(size) ~= 'number' then
+ error('fio.popen.read: invalid size argument')
+ end
+ timeout = timeout or -1
+ elseif type(buf) == 'number' then
+ -- use temp. buffer
+ timeout = size or -1
+ size = buf
+
+ tmpbuf = buffer.ibuf()
+ buf = tmpbuf:reserve(size)
+ else
+ error("fio.popen.read: invalid arguments")
+ end
+
+ local res, output_no, err = internal.popen_read(self.fh, buf, size, timeout)
+ if res == nil then
+ if tmpbuf ~= nil then
+ tmpbuf:recycle()
+ end
+ return nil, nil, err
+ end
+
+ if tmpbuf ~= nil then
+ tmpbuf:alloc(res)
+ res = ffi.string(tmpbuf.rpos, tmpbuf:size())
+ tmpbuf:recycle()
+ end
+ return res, output_no
+end
+
+-- write(str)
+-- write(buf, len)
+popen_methods.write = function(self, data, size)
+ local timeout = -1.0
+ if type(data) == 'table' then
+ timeout = data.timeout or timeout
+ size = data.size or size
+ data = data.buf
+ end
+
+ if type(data) == 'string' then
+ if size == nil then
+ size = string.len(data)
+ end
+ elseif not ffi.istype(const_char_ptr_t, data) then
+ data = tostring(data)
+ size = #data
+ end
+
+ local res, err = internal.popen_write(self.fh, data, tonumber(size), tonumber(timeout))
+ if err ~= nil then
+ return false, err
+ end
+ return res >= 0
+end
+
+popen_methods.status = function(self)
+ if self.fh ~= nil then
+ return internal.popen_get_status(self.fh)
+ else
+ return self.exit_code
+ end
+end
+
+popen_methods.stdin = function (self)
+ if self.fh == nil then
+ return nil, 'Invalid object'
+ end
+
+ return internal.popen_get_std_file_handle(self.fh, fio.STDIN)
+end
+
+popen_methods.stdout = function (self)
+ if self.fh == nil then
+ return nil, 'Invalid object'
+ end
+
+ return internal.popen_get_std_file_handle(self.fh, fio.STDOUT)
+end
+
+popen_methods.stderr = function (self)
+ if self.fh == nil then
+ return nil, 'Invalid object'
+ end
+
+ return internal.popen_get_std_file_handle(self.fh, fio.STDERR)
+end
+
+popen_methods.kill = function(self, sig)
+ if self.fh == nil then
+ return false, errno.strerror(errno.ESRCH)
+ end
+
+ if sig == nil then
+ sig = 'SIGTERM'
+ end
+ if type(sig) == 'string' then
+ sig = signal.c.signals[sig]
+ if sig == nil then
+ errno(errno.EINVAL)
+ return false, sprintf("fio.popen.kill(): unknown signal: %s", sig)
+ end
+ else
+ sig = tonumber(sig)
+ end
+
+ return internal.popen_kill(self.fh, sig)
+end
+
+popen_methods.wait = function(self, timeout)
+ if self.fh == nil then
+ return false, 'Invalid object'
+ end
+
+ if timeout == nil then
+ timeout = -1
+ else
+ timeout = tonumber(timeout)
+ end
+
+ local rc, err = internal.popen_wait(self.fh, timeout)
+ if rc ~= nil then
+ self.exit_code = tonumber(rc)
+ self.fh = nil
+ return true
+ else
+ return false,err
+ end
+end
+
+
+local popen_mt = { __index = popen_methods }
+
+fio.popen = function(params)
+ local argv = params.argv
+ local env = params.environment
+ local hstdin = params.stdin
+ local hstdout = params.stdout
+ local hstderr = params.stderr
+
+ if type(hstdin) == 'table' then
+ hstdin = hstdin.fh
+ end
+ if type(hstdout) == 'table' then
+ hstdout = hstdout.fh
+ end
+ if type(hstderr) == 'table' then
+ hstderr = hstderr.fh
+ end
+
+ if argv == nil or
+ type(argv) ~= 'table' or
+ table.getn(argv) < 1 then
+ local errmsg = [[Usage: fio.popen({parameters}),
+parameters - a table containing arguments to run a process.
+The following arguments are expected:
+
+argv - [mandatory] is a table of argument strings passed
+ to the new program.
+ By convention, the first of these strings should
+ contain the filename associated with the file
+ being executed.
+
+environment - [optional] is a table of strings,
+ conventionally of the form key=value,
+ which are passed as environment to the
+ new program.
+
+stdin - [optional] overrides the child process's
+ standard input.
+stdout - [optional] overrides the child process's
+ standard output.
+stderr - [optional] overrides the child process's
+ standard error output.
+]]
+ error(errmsg)
+ end
+
+ local fh,err = internal.popen(argv, env, hstdin, hstdout, hstderr)
+ if err ~= nil then
+ return nil, err
+ end
+
+ local pobj = {fh = fh}
+ setmetatable(pobj, popen_mt)
+ return pobj
+end
+
fio.pathjoin = function(...)
local i, path = 1, nil
diff --git a/src/lua/init.c b/src/lua/init.c
index 9982828d9..0f0e9f0fc 100644
--- a/src/lua/init.c
+++ b/src/lua/init.c
@@ -56,6 +56,7 @@
#include "lua/msgpack.h"
#include "lua/pickle.h"
#include "lua/fio.h"
+#include "lua/lua_signal.h"
#include "lua/httpc.h"
#include "lua/utf8.h"
#include "lua/swim.h"
@@ -449,6 +450,7 @@ tarantool_lua_init(const char *tarantool_bin, int argc, char **argv)
tarantool_lua_fiber_channel_init(L);
tarantool_lua_errno_init(L);
tarantool_lua_error_init(L);
+ tarantool_lua_signal_init(L);
tarantool_lua_fio_init(L);
tarantool_lua_socket_init(L);
tarantool_lua_pickle_init(L);
diff --git a/src/lua/lua_signal.c b/src/lua/lua_signal.c
new file mode 100644
index 000000000..67c751073
--- /dev/null
+++ b/src/lua/lua_signal.c
@@ -0,0 +1,155 @@
+/*
+ * Copyright 2010-2019, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the
+ * following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include "lua/lua_signal.h"
+#include <sys/types.h>
+#include <signal.h>
+
+#include <lua.h>
+#include <lauxlib.h>
+#include <lualib.h>
+
+#include "lua/utils.h"
+
+#ifndef PUSHTABLE
+#define PUSHTABLE(name, method, value) { \
+ lua_pushliteral(L, name); \
+ method(L, value); \
+ lua_settable(L, -3); \
+}
+#endif /*PUSHTABLE*/
+
+void
+tarantool_lua_signal_init(struct lua_State *L)
+{
+ static const struct luaL_Reg signal_methods[] = {
+ { NULL, NULL }
+ };
+
+ luaL_register_module(L, "signal", signal_methods);
+
+ lua_pushliteral(L, "c");
+ lua_newtable(L);
+
+ lua_pushliteral(L, "signals");
+ lua_newtable(L);
+
+#ifdef SIGINT
+ PUSHTABLE("SIGINT", lua_pushinteger, SIGINT);
+#endif
+#ifdef SIGILL
+ PUSHTABLE("SIGILL", lua_pushinteger, SIGILL);
+#endif
+#ifdef SIGABRT
+ PUSHTABLE("SIGABRT", lua_pushinteger, SIGABRT);
+#endif
+#ifdef SIGFPE
+ PUSHTABLE("SIGFPE", lua_pushinteger, SIGFPE);
+#endif
+#ifdef SIGSEGV
+ PUSHTABLE("SIGSEGV", lua_pushinteger, SIGSEGV);
+#endif
+#ifdef SIGTERM
+ PUSHTABLE("SIGTERM", lua_pushinteger, SIGTERM);
+#endif
+
+#ifdef SIGHUP
+ PUSHTABLE("SIGHUP", lua_pushinteger, SIGHUP);
+#endif
+#ifdef SIGQUIT
+ PUSHTABLE("SIGQUIT", lua_pushinteger, SIGQUIT);
+#endif
+#ifdef SIGTRAP
+ PUSHTABLE("SIGTRAP", lua_pushinteger, SIGTRAP);
+#endif
+#ifdef SIGKILL
+ PUSHTABLE("SIGKILL", lua_pushinteger, SIGKILL);
+#endif
+#ifdef SIGBUS
+ PUSHTABLE("SIGBUS", lua_pushinteger, SIGBUS);
+#endif
+#ifdef SIGSYS
+ PUSHTABLE("SIGSYS", lua_pushinteger, SIGSYS);
+#endif
+#ifdef SIGPIPE
+ PUSHTABLE("SIGPIPE", lua_pushinteger, SIGPIPE);
+#endif
+#ifdef SIGALRM
+ PUSHTABLE("SIGALRM", lua_pushinteger, SIGALRM);
+#endif
+
+#ifdef SIGURG
+ PUSHTABLE("SIGURG", lua_pushinteger, SIGURG);
+#endif
+#ifdef SIGSTOP
+ PUSHTABLE("SIGSTOP", lua_pushinteger, SIGSTOP);
+#endif
+#ifdef SIGTSTP
+ PUSHTABLE("SIGTSTP", lua_pushinteger, SIGTSTP);
+#endif
+#ifdef SIGCONT
+ PUSHTABLE("SIGCONT", lua_pushinteger, SIGCONT);
+#endif
+#ifdef SIGCHLD
+ PUSHTABLE("SIGCHLD", lua_pushinteger, SIGCHLD);
+#endif
+#ifdef SIGTTIN
+ PUSHTABLE("SIGTTIN", lua_pushinteger, SIGTTIN);
+#endif
+#ifdef SIGTTOU
+ PUSHTABLE("SIGTTOU", lua_pushinteger, SIGTTOU);
+#endif
+#ifdef SIGPOLL
+ PUSHTABLE("SIGPOLL", lua_pushinteger, SIGPOLL);
+#endif
+#ifdef SIGXCPU
+ PUSHTABLE("SIGXCPU", lua_pushinteger, SIGXCPU);
+#endif
+#ifdef SIGXFSZ
+ PUSHTABLE("SIGXFSZ", lua_pushinteger, SIGXFSZ);
+#endif
+#ifdef SIGVTALRM
+ PUSHTABLE("SIGVTALRM", lua_pushinteger, SIGVTALRM);
+#endif
+#ifdef SIGPROF
+ PUSHTABLE("SIGPROF", lua_pushinteger, SIGPROF);
+#endif
+#ifdef SIGUSR1
+ PUSHTABLE("SIGUSR1", lua_pushinteger, SIGUSR1);
+#endif
+#ifdef SIGUSR2
+ PUSHTABLE("SIGUSR2", lua_pushinteger, SIGUSR2);
+#endif
+ lua_settable(L, -3); /* "signals" */
+
+ lua_settable(L, -3); /* "c" */
+ lua_pop(L, 1);
+}
diff --git a/src/lua/lua_signal.h b/src/lua/lua_signal.h
new file mode 100644
index 000000000..814692077
--- /dev/null
+++ b/src/lua/lua_signal.h
@@ -0,0 +1,45 @@
+#ifndef INCLUDES_TARANTOOL_LUA_SIGNAL_H
+#define INCLUDES_TARANTOOL_LUA_SIGNAL_H
+
+/*
+ * Copyright 2010-2019, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the
+ * following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+#if defined(__cplusplus)
+extern "C" {
+#endif /* defined(__cplusplus) */
+
+struct lua_State;
+void tarantool_lua_signal_init(struct lua_State *L);
+
+#if defined(__cplusplus)
+} /* extern "C" */
+#endif /* defined(__cplusplus) */
+
+#endif /*INCLUDES_TARANTOOL_LUA_SIGNAL_H*/
diff --git a/src/main.cc b/src/main.cc
index 569ff4b5f..1e4356241 100644
--- a/src/main.cc
+++ b/src/main.cc
@@ -77,6 +77,7 @@
#include "box/session.h"
#include "systemd.h"
#include "crypto/crypto.h"
+#include "coio_popen.h"
static pid_t master_pid = getpid();
static struct pidfh *pid_file_handle;
@@ -306,6 +307,8 @@ signal_free(void)
int i;
for (i = 0; i < ev_sig_count; i++)
ev_signal_stop(loop(), &ev_sigs[i]);
+
+ popen_reset_sigchld_handler();
}
/** Make sure the child has a default signal disposition. */
@@ -328,7 +331,8 @@ signal_reset()
sigaction(SIGHUP, &sa, NULL) == -1 ||
sigaction(SIGWINCH, &sa, NULL) == -1 ||
sigaction(SIGSEGV, &sa, NULL) == -1 ||
- sigaction(SIGFPE, &sa, NULL) == -1)
+ sigaction(SIGFPE, &sa, NULL) == -1 ||
+ sigaction(SIGCHLD, &sa, NULL) == -1)
say_syserror("sigaction");
/* Unblock any signals blocked by libev. */
@@ -373,6 +377,8 @@ signal_init(void)
panic_syserror("sigaction");
}
+ popen_setup_sigchld_handler();
+
ev_signal_init(&ev_sigs[0], sig_checkpoint, SIGUSR1);
ev_signal_init(&ev_sigs[1], signal_cb, SIGINT);
ev_signal_init(&ev_sigs[2], signal_cb, SIGTERM);
diff --git a/test/app-tap/fio_popen.test.lua b/test/app-tap/fio_popen.test.lua
new file mode 100755
index 000000000..c91b7f5d2
--- /dev/null
+++ b/test/app-tap/fio_popen.test.lua
@@ -0,0 +1,309 @@
+#!/usr/bin/env tarantool
+
+local tap = require('tap')
+local fio = require('fio')
+local errno = require('errno')
+local signal = require('signal')
+local fiber = require('fiber')
+local test = tap.test()
+
+test:plan(6+10+12+10+8+7+4+3)
+
+-- Preliminaries
+local function read_stdout(app)
+ local ss = ""
+
+ local s,src,err = app:read(128, 0.1)
+
+ while s ~= nil and s ~= "" do
+ ss = ss .. s
+
+ s,src,err = app:read(128, 0.1)
+ end
+
+ return ss
+end
+
+-- Test 1. Run application, check its status, kill and wait
+local app1 = fio.popen({argv = {'/bin/sh', '-c', 'cat'},
+ stdout=fio.STDOUT,
+ stderr=fio.STDOUT})
+test:isnt(app1, nil, "#1. Starting a existing application")
+
+local rc = app1:status()
+test:is(rc, nil, "#1. Process is running")
+
+rc,err = app1:kill()
+test:is(rc, true, "#1. Sending kill(15)")
+
+rc,err = app1:wait(5)
+test:is(rc, true, "#1. Process was killed")
+
+rc = app1:status()
+test:is(rc, -signal.c.signals['SIGTERM'], "#1. Process was killed 2")
+
+rc,src,err = app1:read(128,0.1)
+test:is(rc, nil, "#1. Cant read from the dead process")
+
+app1 = nil
+
+-- Test 2. Run application, write to stdin, read from stdout
+app1 = fio.popen({argv = {'/bin/sh', '-c', 'cat'},
+ stdout=fio.PIPE,
+ stdin=fio.PIPE,
+ stderr=fio.STDOUT})
+test:isnt(app1, nil, "#2. Starting a existing application")
+
+rc = app1:status()
+test:is(rc, nil, "#2. Process is running")
+
+local test2str = '123\n456\n789'
+
+app1:write(test2str)
+rc,src,err = app1:read(256)
+
+test:is(src, fio.STDOUT, "#2. Read from STDOUT")
+test:is(rc, test2str, "#2. Received exact string")
+
+test2str = 'abc\ndef\ngh'
+
+app1:write(test2str, 4)
+local rc2,src2,err = app1:read(6)
+
+test:is(err, nil, "#2. Reading ok")
+test:is(src2, fio.STDOUT, "#2. Read from STDOUT 2")
+test:is(rc2, 'abc\n', "#2. Received exact string 2")
+
+rc,err = app1:kill()
+test:is(rc, true, "#2. Sending kill(15)")
+
+rc,err = app1:wait()
+test:is(rc, true, "#2. Process is terminated")
+
+rc = app1:status()
+test:is(rc, -signal.c.signals['SIGTERM'], "#2. Process was killed")
+
+app1 = nil
+
+-- Test 3. read from stdout with timeout
+app1 = fio.popen({argv = {'/bin/sh', '-c', 'cat'},
+ stdout=fio.PIPE,
+ stdin=fio.PIPE,
+ stderr=fio.STDOUT})
+test:isnt(app1, nil, "#3. Starting a existing application")
+test:is(app1:stderr(), nil, "#3. STDERR is redirected")
+test:isnt(app1:stdout(), nil, "#3. STDOUT is available")
+test:isnt(app1:stdin(), nil, "#3. STDIN is available")
+
+
+rc = app1:status()
+test:is(rc, nil, "#3. Process is running")
+
+rc,src,err = app1:read(256, 0.5)
+
+local e = errno()
+test:is(e, errno.ETIMEDOUT, "#3. Timeout")
+
+
+local test2str = '123\n456\n789'
+
+app1:write(test2str)
+rc,src,err = app1:read(256, 0.5)
+
+test:is(err, nil, "#3. Reading ok")
+test:is(src, fio.STDOUT, "#3. Read from STDOUT")
+test:is(rc, test2str, "#3. Received exact string")
+
+rc,err = app1:kill('SIGHUP')
+test:is(rc, true, "#3. Sending kill(1)")
+
+rc,err = app1:wait()
+test:is(rc, true, "#3. Process was killed")
+rc = app1:status()
+test:is(rc, -signal.c.signals['SIGHUP'], "#3. Process was killed")
+
+app1 = nil
+
+-- Test 4. Redirect from file
+local tmpdir = fio.tempdir()
+local txt_filename = fio.pathjoin(tmpdir, 'fio_popen.sample.txt')
+fh1 = fio.open(txt_filename, { 'O_RDWR', 'O_TRUNC', 'O_CREAT' }, tonumber('0777', 8))
+
+local test2str = 'AAA\nBBB\nCCC\nDDD\n\n'
+
+fh1:write(test2str)
+fh1:close()
+
+local txt_file = fio.open(txt_filename, {'O_RDONLY'})
+test:isnt(txt_file, nil, "#4. Open existing file for reading")
+
+app1 = fio.popen({argv = {'/bin/sh', '-c', 'cat'},
+ stdout=fio.PIPE,
+ stdin=txt_file,
+ stderr=fio.STDOUT})
+test:isnt(app1, nil, "#4. Starting a existing application")
+test:is(app1:stderr(), nil, "#4. STDERR is redirected")
+test:isnt(app1:stdout(), nil, "#4. STDOUT is available")
+test:is(app1:stdin(), nil, "#4. STDIN is redirected")
+
+rc = app1:status()
+test:is(rc, nil, "#4. Process is running")
+
+rc,src,err = app1:read(256, 0.5)
+
+test:is(src, fio.STDOUT, "#4. Read from STDOUT")
+test:is(rc, test2str, "#4. Received exact string")
+
+rc,err = app1:wait()
+test:is(rc, true, "#4. Process is terminated")
+rc = app1:status()
+test:is(rc, 0, "#4. Process's exited")
+
+app1 = nil
+txt_file:close()
+fio.unlink(txt_filename)
+
+-- Test 5. Redirect output from one process to another
+local build_path = os.getenv("BUILDDIR")
+local app_path = fio.pathjoin(build_path, 'test/app-tap/fio_popen_test1.sh')
+
+app1 = fio.popen({argv = {'/bin/sh', '-c', app_path},
+ stdout=fio.PIPE,
+ stderr=fio.STDOUT})
+
+test:isnt(app1, nil, "#5. Starting application 1")
+test:is(app1:stderr(), nil, "#5. STDERR is redirected")
+test:isnt(app1:stdout(), nil, "#5. STDOUT is available")
+test:isnt(app1:stdin(), nil, "#5. STDIN is available")
+
+fiber.sleep(1)
+
+local app2 = fio.popen({argv = {'/bin/sh', '-c', 'cat'},
+ stdout=fio.PIPE,
+ stdin=app1:stdout()})
+
+test:isnt(app2, nil, "#5. Starting application 2")
+
+local test2str = '1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n'
+
+rc = read_stdout(app2)
+
+test:is(rc, test2str, "#5. Received exact string")
+
+rc,err = app1:wait()
+test:is(rc, true, "#5. Process's exited 1")
+
+rc,err = app2:wait()
+test:is(rc, true, "#5. Process's exited 2")
+
+app1 = nil
+app2 = nil
+
+-- Test 6. Write a lot
+app1 = fio.popen({argv = {'/bin/sh', '-c', 'cat'},
+ stdout=fio.PIPE,
+ stdin=fio.PIPE,
+ stderr=fio.STDOUT})
+test:isnt(app1, nil, "#6. Starting a existing application")
+
+rc = app1:status()
+test:is(rc, nil, "#6. Process is running")
+
+local expected_length = 0
+local received_length = 0
+local str_to_send = ''
+local str_to_receive = ''
+
+for i=0,1000 do
+
+ local ss = ''
+ for j = 0,100 do
+ local s = tostring(i*100+j) .. '\n'
+ ss = ss .. s
+ end
+
+ str_to_send = str_to_send .. ss
+ app1:write({buf=ss, timeout=1})
+ rc,src,err = app1:read(1024, 1)
+ if rc ~= nil then
+ received_length = received_length + string.len(rc)
+ str_to_receive = str_to_receive .. rc
+ end
+end
+
+expected_length = string.len(str_to_send)
+
+-- Read the rest data
+str_to_receive = str_to_receive .. read_stdout(app1)
+received_length = string.len(str_to_receive)
+
+test:is(received_length, expected_length, "#6. Received number of bytes")
+test:is(str_to_receive, str_to_send, "#6. Received exact string")
+
+rc,err = app1:kill()
+test:is(rc, true, "#6. Sending kill(15)")
+
+rc,err = app1:wait()
+test:is(rc, true, "#6. Process is terminated")
+
+rc = app1:status()
+test:is(rc, -signal.c.signals['SIGTERM'], "#6. Process was killed")
+
+app1 = nil
+
+-- Test 7. Read both STDOUT & STDERR
+local app_path = fio.pathjoin(build_path, 'test/app-tap/fio_popen_test2.sh')
+
+app1 = fio.popen({argv = {'/bin/sh', '-c', app_path},
+ stdin=fio.STDIN,
+ stdout=fio.PIPE,
+ stderr=fio.PIPE})
+
+test:isnt(app1, nil, "#7. Starting application")
+
+local result = {}
+result[fio.STDOUT] = ''
+result[fio.STDERR] = ''
+
+while rc ~= nil and rc ~= '' do
+ rc,src,err = app1:read(64, 0.5)
+ if rc ~= nil then
+ result[src] = result[src] .. rc
+ end
+end
+
+local test2str = '1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n'
+
+test:is(result[fio.STDOUT], test2str, "#7. Received STDOUT")
+test:is(result[fio.STDERR], test2str, "#7. Received STDERR")
+
+rc,err = app1:wait()
+test:is(rc, true, "#7. Process's exited")
+
+app1 = nil
+
+
+-- Test 8. Use custom environment variables
+local app_path = fio.pathjoin(build_path, 'test/app-tap/fio_popen_test3.sh')
+
+app1 = fio.popen({argv = {'/bin/sh', '-c', app_path},
+ environment = {'VAR1=Variable1', 'VAR2=Variable2','VAR3=Variable3'},
+ stdout=fio.PIPE,
+ stderr=fio.STDOUT})
+
+test:isnt(app1, nil, "#8. Starting a existing application")
+
+rc = read_stdout(app1)
+
+local test2str = 'Variable1\nVariable2\nVariable3\n'
+test:is(rc, test2str, "#8. Received values of env. variables")
+
+rc,err = app1:wait()
+test:is(rc, true, "#8. Process was killed")
+
+app1 = nil
+
+-- --------------------------------------------------------------
+test:check()
+os.exit(0)
+
diff --git a/test/app-tap/fio_popen_test1.sh b/test/app-tap/fio_popen_test1.sh
new file mode 100755
index 000000000..d04cb522b
--- /dev/null
+++ b/test/app-tap/fio_popen_test1.sh
@@ -0,0 +1,6 @@
+#!/bin/bash
+for i in {1..10}
+do
+ echo $i
+done
+
diff --git a/test/app-tap/fio_popen_test2.sh b/test/app-tap/fio_popen_test2.sh
new file mode 100755
index 000000000..b6fae9e71
--- /dev/null
+++ b/test/app-tap/fio_popen_test2.sh
@@ -0,0 +1,7 @@
+#!/bin/bash
+for i in {1..10}
+do
+ 1>&2 echo $i
+ echo $i
+done
+
diff --git a/test/app-tap/fio_popen_test3.sh b/test/app-tap/fio_popen_test3.sh
new file mode 100755
index 000000000..1ac3ede44
--- /dev/null
+++ b/test/app-tap/fio_popen_test3.sh
@@ -0,0 +1,5 @@
+#!/bin/bash
+
+echo $VAR1
+echo $VAR2
+echo $VAR3
--
2.17.1
^ permalink raw reply [flat|nested] 3+ messages in thread* [PATCH v4 2/2] core: Non-blocking popen
2019-07-02 7:36 [PATCH v4 0/2] core: Non-blocking io.popen Stanislav Zudin
2019-07-02 7:36 ` [PATCH v4 1/2] " Stanislav Zudin
@ 2019-07-02 7:36 ` Stanislav Zudin
1 sibling, 0 replies; 3+ messages in thread
From: Stanislav Zudin @ 2019-07-02 7:36 UTC (permalink / raw)
To: tarantool-patches, vdavydov.dev, alexander.turenko; +Cc: Stanislav Zudin
Changes fio.popen() interface and tests.
popen_wait() uses fiber_cond variable to wait for SIGCHLD.
Changes signature of popen:write() and popen:read()
The mutex is not used any more, everything works in a single thread.
Executes vfork in the caller thread.
The caller is responsible for releasing memory allocated for popen
arguments. Uses region_alloc instead of calloc.
Installs SIGCHLD handler in popen_init().
Disables signals before fork() and enables them back after fork()
fio.popen creates only one descriptor to /dev/null and shares it
between all child processes.
Calls popen_free() to release global resources.
Adds test for /dev/null.
popen:close() closes the specified file descriptor.
popen:shutdown() closes all descriptors and releases the resources.
The namespace 'signal' contains numeric constants instead of
symbol names. Thus popen.kill() accepts only numeric signal id.
Changes popen:wait() behavior and tests. It's allowed to read after
wait() call. To release resources explicitly call popen:shutdown().
Closes #4031
@TarantoolBot document
Title: Nonblocking fio.popen
handle, err = fio.popen(args, parameters)
fio.popen starts a process and redirects its input/output.
args - [mandatory] is a table of argument strings passed to
the new program. By convention, the first of these strings should
contain the filename associated with the file being executed.
parameters - a table containing arguments to run a process.
The following arguments are expected:
environment - [optional] is a table of strings, conventionally of the
form key=value, which are passed as environment to the new program.
If not specified a parent's environment is used.
By default stdin, stdout,stderr of the associated process are
available for writing/reading using object's methods
handle:write() and handle:read().
One can override default behavior to redirect streams to/from file
or to input/output of another process or to the default input/output
of the parent process.
stdin - [optional] overrides the child process's standard input.
stdout - [optional] overrides the child process's standard output.
stderr - [optional] overrides the the child process's standard
error output.
May accept one of the following values:
Handle of the file open with fio.open()
Handle of the standard input/output of another process,
open by fio.popen
A constant defining the parent's STDIN, STDOUT or STDERR.
A constants:
fio.PIPE - Opens a file descriptor available for reading/writing
or redirection to/from another process.
fio.DEVNULL - Makes fio.popen redirect the output to /dev/null.
On success returns an object providing methods for
communication with the running process.
Returns nil if underlying functions calls fail;
in this case the second return value, err, contains a error message.
The object created by fio.popen provides the following methods:
read()
write()
kill()
wait()
status()
stdin()
stdout()
stderr()
close()
shutdown()
number handle:stdin()
Returns handle of the child process's standard input.
The handle is available only if it was created with
fio.PIPE option.
Use this handle to setup a redirection
from file or other process to the input of the associated process.
If handle is unavailable the method returns nil.
number handle:stdout()
number handle:stderr()
Return STDOUT and STDIN of the associated process accordingly.
See handle:stdin() for details.
rc,err = handle:wait(timeout)
The wait() waits for the associated process to terminate.
timeout - an integer specifies number of seconds to wait.
If the requested time has elapsed the method returns false,
the second return value, err, contains a error message.
To distinguish timeout from the the other errors use errno.
If timeout is nil, the method waits infinitely until
the associated process is terminated.
On success function returns true.
If failed, rc is false and err contains a error message.
If the associated process is terminated, one can use the following
methods get the exit status:
rc = handle:status()
returns nil if process is still running
== 0 if the process exited normally
error code > 0 if the process terminated with an error
-signal if the process was killed by a signal
rc, err = handle:kill(sig)
The kill() sends a specified signal to the associated process.
On success the method returns true,
if failed - false and error message.
If the sig is nil the default SIGTERM is being sent to the process.
If the signal is unknown then the method fails (with error EINVAL).
The supported signals are defined in the namespace 'signal'.
rc,src,err = handle:read(buffer,size,timeout)
read stdout & stderr of the process started by fio.popen
Usage:
read(size) -> str, source, err
read(buf, size) -> length, source, err
read(size, timeout) -> str, source, err
read(buf, size, timeout) -> length, source, err
or read(opts), where opts are:
size
buf
timeout
timeout - number of seconds to wait (optional)
source contains id of the stream, fio.STDOUT or fio.STDERR
err - error message if method has failed or nil on success
The method is available if process was started with stdout
and/or stderr=fio.PIPE.
rc, written, err = handle:write(data[, size])
rc, written, err = handle:write(opts), where opts are:
data
size
timeout
Writes specified number of bytes
On success returns true and number of written bytes.
If failed the rc is false, written is 0 and err contains an error message.
In a case of timeout rc is false and written contains number of written bytes.
The method is available if process was started with stdin=fio.PIPE.
rc,err = handle:close(fd)
Closes specified file descriptor of the associated process.
fd is one of the following constants:
fio.STDIN, fio.STDOUT, fio.STDERR or nil.
If fd is not specified (or nil) all the specified descriptors
will be closed.
The method is applicable only to the descriptors specified as fio.PIPE.
handle:shutdown()
Releases all resources allocated by popen().
Returns true on success or false with an error message on fail.
If method succeeded all descriptors are closed, no methods are
allowed except status(). The latest one returns status of the
associated process in the moment of shutdown call.
---
src/lib/core/coio_popen.c | 647 +++++++++++++++-----------------
src/lib/core/coio_popen.h | 167 +++++----
src/lib/core/coio_task.c | 2 +-
src/lua/fio.c | 209 ++++++-----
src/lua/fio.lua | 104 +++--
src/lua/lua_signal.c | 80 ++--
src/main.cc | 6 +-
test/app-tap/fio_popen.test.lua | 173 ++++++---
8 files changed, 722 insertions(+), 666 deletions(-)
diff --git a/src/lib/core/coio_popen.c b/src/lib/core/coio_popen.c
index dcc50136f..c11295280 100644
--- a/src/lib/core/coio_popen.c
+++ b/src/lib/core/coio_popen.c
@@ -41,9 +41,9 @@
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
-#include <pthread.h>
#include <float.h>
#include <sysexits.h>
+#include "evio.h"
/*
* On OSX this global variable is not declared
@@ -62,26 +62,25 @@ struct popen_handle {
* [1] read from stdout of the child process
* [2] read from stderr of the child process
* Valid only for pipe.
+ * In any other cases they are set to -1.
*/
int fd[3];
- /*
- * Handle to /dev/null.
- */
- int devnull_fd;
+ /* Set to true if the process has terminated. */
+ bool terminated;
/*
- * Current process status.
- * The SIGCHLD handler changes this status.
+ * If the process has terminated, this variable stores
+ * the exit code if the process exited normally, by
+ * calling exit(), or -signo if the process was killed
+ * by a signal.
*/
- enum popen_status status;
+ int status;
/*
- * Exit status of the associated process
- * or number of the signal that caused the
- * associated process to terminate.
+ * popen_wait() waits for SIGCHLD within the variable.
*/
- int exit_code;
+ struct fiber_cond sigchld_cond;
};
/*
@@ -98,41 +97,48 @@ struct popen_handle {
#define MH_SOURCE 1
#include "salad/mhash.h"
+static void
+popen_setup_sigchld_handler();
+
+static void
+popen_reset_sigchld_handler();
-static pthread_mutex_t mutex;
-static struct mh_popen_storage_t* popen_hash_table = NULL;
+
+static struct mh_popen_storage_t *popen_hash_table = NULL;
+
+/*
+ * A file descriptor to /dev/null
+ * shared between all child processes
+ */
+static int popen_devnull_fd = -1;
void
-popen_initialize()
+popen_init()
{
- pthread_mutexattr_t errorcheck;
- pthread_mutexattr_init(&errorcheck);
- pthread_mutexattr_settype(&errorcheck,
- PTHREAD_MUTEX_ERRORCHECK);
- pthread_mutex_init(&mutex, &errorcheck);
- pthread_mutexattr_destroy(&errorcheck);
-
popen_hash_table = mh_popen_storage_new();
+ popen_setup_sigchld_handler();
}
-static void
-popen_lock_data_list()
+void
+popen_free()
{
- pthread_mutex_lock(&mutex);
-}
+ popen_reset_sigchld_handler();
-static void
-popen_unlock_data_list()
-{
- pthread_mutex_unlock(&mutex);
+ mh_popen_storage_delete(popen_hash_table);
+ popen_hash_table = NULL;
+
+ if (popen_devnull_fd >= 0) {
+ close(popen_devnull_fd);
+ popen_devnull_fd = -1;
+ }
}
static void
-popen_append_to_list(struct popen_handle *data)
+popen_append_to_list(struct popen_handle *handle)
{
struct popen_handle **old = NULL;
mh_int_t id = mh_popen_storage_put(popen_hash_table,
- (const struct popen_handle **)&data, &old, NULL);
+ (const struct popen_handle **)&handle, &old, NULL);
(void)id;
}
@@ -149,24 +155,41 @@ popen_lookup_data_by_pid(pid_t pid)
}
}
-static void
-popen_exclude_from_list(struct popen_handle *data)
+static struct popen_handle *
+popen_data_new()
{
- mh_popen_storage_remove(popen_hash_table,
- (const struct popen_handle **)&data, NULL);
+ struct popen_handle *handle =
+ (struct popen_handle *)calloc(1, sizeof(*handle));
+
+ if (handle == NULL) {
+ diag_set(OutOfMemory, sizeof(*handle),
+ "calloc", "struct popen_handle");
+ return NULL;
+ }
+
+ handle->fd[0] = -1;
+ handle->fd[1] = -1;
+ handle->fd[2] = -1;
+
+ fiber_cond_create(&handle->sigchld_cond);
+ return handle;
}
-static struct popen_handle *
-popen_data_new()
+void
+popen_data_free(struct popen_handle *handle)
{
- struct popen_handle *data =
- (struct popen_handle *)calloc(1, sizeof(*data));
- data->fd[0] = -1;
- data->fd[1] = -1;
- data->fd[2] = -1;
- data->devnull_fd = -1;
- data->status = POPEN_RUNNING;
- return data;
+ fiber_cond_destroy(&handle->sigchld_cond);
+ free(handle);
+}
+
+static int
+popen_get_devnull()
+{
+ if (popen_devnull_fd < 0) {
+ popen_devnull_fd = open("/dev/null", O_RDWR | O_CLOEXEC);
+ }
+
+ return popen_devnull_fd;
}
enum pipe_end {
@@ -174,23 +197,12 @@ enum pipe_end {
PIPE_WRITE = 1
};
-static inline enum pipe_end
- popen_opposite_pipe(enum pipe_end side)
-{
- return (enum pipe_end)(side ^ 1);
- /*
- * The code is equal to:
- * return (side == PIPE_READ) ? PIPE_WRITE
- * : PIPE_READ;
- */
-}
-
static inline bool
-popen_create_pipe(int fd, int pipe_pair[2], enum pipe_end parent_side)
+popen_prepare_fd(int fd, int *pipe_pair, enum pipe_end parent_end)
{
if (fd == FIO_PIPE) {
if (pipe(pipe_pair) < 0 ||
- fcntl(pipe_pair[parent_side], F_SETFL, O_NONBLOCK) < 0) {
+ fcntl(pipe_pair[parent_end], F_SETFL, O_NONBLOCK) < 0) {
return false;
}
}
@@ -198,20 +210,22 @@ popen_create_pipe(int fd, int pipe_pair[2], enum pipe_end parent_side)
}
static inline void
-popen_close_child_fd(int std_fd, int pipe_pair[2],
- int *saved_fd, enum pipe_end child_side)
+popen_setup_parent_fd(int std_fd, int *pipe_pair,
+ int *saved_fd, enum pipe_end child_end)
{
if (std_fd == FIO_PIPE) {
- /* Close child's side. */
- close(pipe_pair[child_side]);
+ /* Close child's end. */
+ close(pipe_pair[child_end]);
- enum pipe_end parent_side = popen_opposite_pipe(child_side);
- *saved_fd = pipe_pair[parent_side];
+ enum pipe_end parent_end = (child_end == PIPE_READ)
+ ? PIPE_WRITE
+ : PIPE_READ;
+ *saved_fd = pipe_pair[parent_end];
}
}
static inline void
-popen_close_pipe(int pipe_pair[2])
+popen_cleanup_fd(int *pipe_pair)
{
if (pipe_pair[0] >= 0) {
close(pipe_pair[0]);
@@ -219,85 +233,78 @@ popen_close_pipe(int pipe_pair[2])
}
}
+static void
+popen_disable_signals(sigset_t *prev)
+{
+ sigset_t sigset;
+ sigfillset(&sigset);
+ sigemptyset(prev);
+ if (pthread_sigmask(SIG_BLOCK, &sigset, prev) == -1)
+ say_syserror("sigprocmask off");
+}
-/**
- * Implementation of fio.popen.
- * The function opens a process by creating a pipe
- * forking.
- *
- * @param argv - is an array of character pointers
- * to the arguments terminated by a null pointer.
- *
- * @param envp - is the pointer to an array
- * of character pointers to the environment strings.
- *
- * @param stdin_fd - the file handle to be redirected to the
- * child process's STDIN.
- *
- * @param stdout_fd - the file handle receiving the STDOUT
- * output of the child process.
- *
- * @param stderr_fd - the file handle receiving the STDERR
- * output of the child process.
- *
- * The stdin_fd, stdout_fd & stderr_fd accept file descriptors
- * from open() or the following values:
- *
- * FIO_PIPE - opens a pipe, binds it with child's
- * input/output. The pipe is available for reading/writing.
- *
- * FIO_DEVNULL - redirects output from process to /dev/null.
- *
- * @return handle of the pipe for reading or writing
- * (depends on value of type).
- * In a case of error returns NULL.
- */
-static struct popen_handle *
-popen_new_impl(char **argv, char **envp,
+static void
+popen_restore_signals(sigset_t *prev)
+{
+ if (pthread_sigmask(SIG_SETMASK, prev, NULL) == -1)
+ say_syserror("sigprocmask on");
+}
+
+static void
+popen_reset_signals()
+{
+ /* Reset all signals to their defaults. */
+ struct sigaction sa;
+ memset(&sa, 0, sizeof(sa));
+ sigemptyset(&sa.sa_mask);
+ sa.sa_handler = SIG_DFL;
+
+ if (sigaction(SIGUSR1, &sa, NULL) == -1 ||
+ sigaction(SIGINT, &sa, NULL) == -1 ||
+ sigaction(SIGTERM, &sa, NULL) == -1 ||
+ sigaction(SIGHUP, &sa, NULL) == -1 ||
+ sigaction(SIGWINCH, &sa, NULL) == -1 ||
+ sigaction(SIGSEGV, &sa, NULL) == -1 ||
+ sigaction(SIGFPE, &sa, NULL) == -1 ||
+ sigaction(SIGCHLD, &sa, NULL) == -1)
+ exit(EX_OSERR);
+
+ /* Unblock any signals blocked by libev. */
+ sigset_t sigset;
+ sigfillset(&sigset);
+ if (sigprocmask(SIG_UNBLOCK, &sigset, NULL) == -1)
+ exit(EX_OSERR);
+}
+
+struct popen_handle *
+popen_new(char **argv, char **env,
int stdin_fd, int stdout_fd, int stderr_fd)
{
- bool popen_list_locked = false;
pid_t pid;
- int pipe_rd[2] = {-1,-1};
- int pipe_wr[2] = {-1,-1};
- int pipe_er[2] = {-1,-1};
+ int pipe_stdin[2] = {-1,-1};
+ int pipe_stdout[2] = {-1,-1};
+ int pipe_stderr[2] = {-1,-1};
errno = 0;
- struct popen_handle *data = popen_data_new();
- if (data == NULL)
+ struct popen_handle *handle = popen_data_new();
+ if (handle == NULL)
return NULL;
/*
* Setup a /dev/null if necessary.
*/
- bool read_devnull = (stdin_fd == FIO_DEVNULL);
- bool write_devnull = (stdout_fd == FIO_DEVNULL) ||
- (stderr_fd == FIO_DEVNULL);
- int devnull_flags = O_RDWR;
- if (!read_devnull)
- devnull_flags = O_WRONLY;
- else if (!write_devnull)
- devnull_flags = O_RDONLY;
-
- if (read_devnull || write_devnull) {
- data->devnull_fd = open("/dev/null", devnull_flags);
- if (data->devnull_fd < 0)
- goto on_error;
- else {
- if (stdin_fd == FIO_DEVNULL)
- stdin_fd = data->devnull_fd;
- if (stdout_fd == FIO_DEVNULL)
- stdout_fd = data->devnull_fd;
- if (stderr_fd == FIO_DEVNULL)
- stderr_fd = data->devnull_fd;
- }
- }
-
- if (!popen_create_pipe(stdin_fd, pipe_rd, PIPE_WRITE))
+ if (stdin_fd == FIO_DEVNULL)
+ stdin_fd = popen_get_devnull();
+ if (stdout_fd == FIO_DEVNULL)
+ stdout_fd = popen_get_devnull();
+ if (stderr_fd == FIO_DEVNULL)
+ stderr_fd = popen_get_devnull();
+
+ if (!popen_prepare_fd(stdin_fd, pipe_stdin, PIPE_WRITE))
goto on_error;
- if (!popen_create_pipe(stdout_fd, pipe_wr, PIPE_READ))
+ if (!popen_prepare_fd(stdout_fd, pipe_stdout, PIPE_READ))
goto on_error;
- if (!popen_create_pipe(stderr_fd, pipe_er, PIPE_READ))
+ if (!popen_prepare_fd(stderr_fd, pipe_stderr, PIPE_READ))
goto on_error;
/*
@@ -306,8 +313,8 @@ popen_new_impl(char **argv, char **envp,
* error: "argument ‘xxx’ might be clobbered by
* ‘longjmp’ or ‘vfork’ [-Werror=clobbered]".
*/
- if (envp == NULL)
- envp = environ;
+ if (env == NULL)
+ env = environ;
/* Handles to be closed in child process */
int close_fd[3] = {-1, -1, -1};
@@ -317,16 +324,16 @@ popen_new_impl(char **argv, char **envp,
int close_after_dup_fd[3] = {-1, -1, -1};
if (stdin_fd == FIO_PIPE) {
- close_fd[STDIN_FILENO] = pipe_rd[PIPE_WRITE];
- dup_fd[STDIN_FILENO] = pipe_rd[PIPE_READ];
+ close_fd[STDIN_FILENO] = pipe_stdin[PIPE_WRITE];
+ dup_fd[STDIN_FILENO] = pipe_stdin[PIPE_READ];
} else if (stdin_fd != STDIN_FILENO) {
dup_fd[STDIN_FILENO] = stdin_fd;
close_after_dup_fd[STDIN_FILENO] = stdin_fd;
}
if (stdout_fd == FIO_PIPE) {
- close_fd[STDOUT_FILENO] = pipe_wr[PIPE_READ];
- dup_fd[STDOUT_FILENO] = pipe_wr[PIPE_WRITE];
+ close_fd[STDOUT_FILENO] = pipe_stdout[PIPE_READ];
+ dup_fd[STDOUT_FILENO] = pipe_stdout[PIPE_WRITE];
} else if (stdout_fd != STDOUT_FILENO){
dup_fd[STDOUT_FILENO] = stdout_fd;
if (stdout_fd != STDERR_FILENO)
@@ -334,44 +341,23 @@ popen_new_impl(char **argv, char **envp,
}
if (stderr_fd == FIO_PIPE) {
- close_fd[STDERR_FILENO] = pipe_er[PIPE_READ];
- dup_fd[STDERR_FILENO] = pipe_er[PIPE_WRITE];
+ close_fd[STDERR_FILENO] = pipe_stderr[PIPE_READ];
+ dup_fd[STDERR_FILENO] = pipe_stderr[PIPE_WRITE];
} else if (stderr_fd != STDERR_FILENO) {
dup_fd[STDERR_FILENO] = stderr_fd;
if (stderr_fd != STDOUT_FILENO)
close_after_dup_fd[STDERR_FILENO] = stderr_fd;
}
-
- popen_lock_data_list();
- popen_list_locked = true;
+ sigset_t prev_sigset;
+ popen_disable_signals(&prev_sigset);
pid = vfork();
if (pid < 0)
goto on_error;
else if (pid == 0) /* child */ {
- /* Reset all signals to their defaults. */
- struct sigaction sa;
- memset(&sa, 0, sizeof(sa));
- sigemptyset(&sa.sa_mask);
- sa.sa_handler = SIG_DFL;
-
- if (sigaction(SIGUSR1, &sa, NULL) == -1 ||
- sigaction(SIGINT, &sa, NULL) == -1 ||
- sigaction(SIGTERM, &sa, NULL) == -1 ||
- sigaction(SIGHUP, &sa, NULL) == -1 ||
- sigaction(SIGWINCH, &sa, NULL) == -1 ||
- sigaction(SIGSEGV, &sa, NULL) == -1 ||
- sigaction(SIGFPE, &sa, NULL) == -1 ||
- sigaction(SIGCHLD, &sa, NULL) == -1)
- exit(EX_OSERR);
-
- /* Unblock any signals blocked by libev. */
- sigset_t sigset;
- sigfillset(&sigset);
- if (sigprocmask(SIG_UNBLOCK, &sigset, NULL) == -1)
- exit(EX_OSERR);
+ popen_reset_signals();
/* Setup stdin/stdout */
for(int i = 0; i < 3; ++i) {
@@ -383,103 +369,93 @@ popen_new_impl(char **argv, char **envp,
close(close_after_dup_fd[i]);
}
- execve( argv[0], argv, envp);
+ execve( argv[0], argv, env);
exit(EX_OSERR);
unreachable();
}
+ popen_restore_signals(&prev_sigset);
+
/* Parent process */
- popen_close_child_fd(stdin_fd, pipe_rd,
- &data->fd[STDIN_FILENO], PIPE_READ);
- popen_close_child_fd(stdout_fd, pipe_wr,
- &data->fd[STDOUT_FILENO], PIPE_WRITE);
- popen_close_child_fd(stderr_fd, pipe_er,
- &data->fd[STDERR_FILENO], PIPE_WRITE);
+ popen_setup_parent_fd(stdin_fd, pipe_stdin,
+ &handle->fd[STDIN_FILENO], PIPE_READ);
+ popen_setup_parent_fd(stdout_fd, pipe_stdout,
+ &handle->fd[STDOUT_FILENO], PIPE_WRITE);
+ popen_setup_parent_fd(stderr_fd, pipe_stderr,
+ &handle->fd[STDERR_FILENO], PIPE_WRITE);
- data->pid = pid;
+ handle->pid = pid;
on_cleanup:
- if (data){
- popen_append_to_list(data);
+ if (handle){
+ popen_append_to_list(handle);
}
- if (popen_list_locked)
- popen_unlock_data_list();
-
- if (argv){
- for(int i = 0; argv[i] != NULL; ++i)
- free(argv[i]);
- free(argv);
- }
- if (envp && envp != environ) {
- for(int i = 0; envp[i] != NULL; ++i)
- free(envp[i]);
- free(envp);
- }
-
- return data;
+ return handle;
on_error:
- popen_close_pipe(pipe_rd);
- popen_close_pipe(pipe_wr);
- popen_close_pipe(pipe_er);
-
- if (data) {
- if (data->devnull_fd >= 0)
- close(data->devnull_fd);
- free(data);
+ popen_cleanup_fd(pipe_stdin);
+ popen_cleanup_fd(pipe_stdout);
+ popen_cleanup_fd(pipe_stderr);
+
+ if (handle) {
+ popen_data_free(handle);
}
- data = NULL;
+ handle = NULL;
goto on_cleanup;
unreachable();
}
-ssize_t
-popen_new(va_list ap)
-{
- char **argv = va_arg(ap, char **);
- char **envp = va_arg(ap, char **);
- int stdin_fd = va_arg(ap, int);
- int stdout_fd = va_arg(ap, int);
- int stderr_fd = va_arg(ap, int);
- struct popen_handle **handle = va_arg(ap, struct popen_handle **);
-
- *handle = popen_new_impl(argv, envp, stdin_fd, stdout_fd, stderr_fd);
- return (*handle) ? 0 : -1;
-}
-
static void
-popen_close_handles(struct popen_handle *data)
+popen_close_fds(struct popen_handle *handle)
{
for(int i = 0; i < 3; ++i) {
- if (data->fd[i] >= 0) {
- close(data->fd[i]);
- data->fd[i] = -1;
+ if (handle->fd[i] >= 0) {
+ close(handle->fd[i]);
+ handle->fd[i] = -1;
}
}
- if (data->devnull_fd >= 0) {
- close(data->devnull_fd);
- data->devnull_fd = -1;
- }
}
int
-popen_destroy(struct popen_handle *fh)
+popen_destroy(struct popen_handle *handle)
{
- assert(fh);
+ assert(handle);
+
+ popen_close_fds(handle);
+ mh_popen_storage_remove(popen_hash_table,
+ (const struct popen_handle **)&handle, NULL);
+
+ popen_data_free(handle);
+ return 0;
+}
+
+int
+popen_close(struct popen_handle *handle, int fd)
+{
+ assert(handle);
+
+ errno = 0;
- popen_lock_data_list();
- popen_close_handles(fh);
- popen_exclude_from_list(fh);
- popen_unlock_data_list();
+ if (fd < 0)
+ popen_close_fds(handle);
+ else if (STDIN_FILENO <= fd && fd <= STDERR_FILENO){
+ if (handle->fd[fd] >= 0) {
+ close(handle->fd[fd]);
+ handle->fd[fd] = -1;
+ }
+
+ } else {
+ errno = EINVAL;
+ return -1;
+ }
- free(fh);
return 0;
}
/**
- * Check if an errno, returned from a sio function, means a
+ * Check if an errno, returned from a io functions, means a
* non-critical error: EAGAIN, EWOULDBLOCK, EINTR.
*/
static inline bool
@@ -488,18 +464,19 @@ popen_wouldblock(int err)
return err == EAGAIN || err == EWOULDBLOCK || err == EINTR;
}
-static int
-popen_do_read(struct popen_handle *data, void *buf, size_t count, int *source_id)
+static ssize_t
+popen_do_read(struct popen_handle *handle, void *buf, size_t count,
+ int *source_id)
{
/*
* STDERR has higher priority, read it first.
*/
- int rc = 0;
+ ssize_t rc = 0;
errno = 0;
int fd_count = 0;
- if (data->fd[STDERR_FILENO] >= 0) {
+ if (handle->fd[STDERR_FILENO] >= 0) {
++fd_count;
- rc = read(data->fd[STDERR_FILENO], buf, count);
+ rc = read(handle->fd[STDERR_FILENO], buf, count);
if (rc >= 0)
*source_id = STDERR_FILENO;
@@ -514,9 +491,9 @@ popen_do_read(struct popen_handle *data, void *buf, size_t count, int *source_id
/*
* STDERR is not available or not ready, try STDOUT.
*/
- if (data->fd[STDOUT_FILENO] >= 0) {
+ if (handle->fd[STDOUT_FILENO] >= 0) {
++fd_count;
- rc = read(data->fd[STDOUT_FILENO], buf, count);
+ rc = read(handle->fd[STDOUT_FILENO], buf, count);
if (rc >= 0) {
*source_id = STDOUT_FILENO;
@@ -534,37 +511,35 @@ popen_do_read(struct popen_handle *data, void *buf, size_t count, int *source_id
return rc;
}
-static void
-popen_coio_create(struct ev_io *coio, int fd)
+static inline void
+popen_coio_create(struct ev_io *coio)
{
coio->data = fiber();
ev_init(coio, (ev_io_cb) fiber_schedule_cb);
- coio->fd = fd;
}
-int
-popen_read(struct popen_handle *fh, void *buf, size_t count,
- size_t *read_bytes, int *source_id,
- ev_tstamp timeout)
+ssize_t
+popen_read(struct popen_handle *handle, void *buf, size_t count,
+ int *source_id,
+ double timeout)
{
- assert(fh);
- if (timeout < 0.0)
- timeout = DBL_MAX;
+ assert(handle);
+ assert (timeout >= 0.0);
ev_tstamp start, delay;
evio_timeout_init(loop(), &start, &delay, timeout);
- struct ev_io coio_rd;
- struct ev_io coio_er;
- popen_coio_create(&coio_er, fh->fd[STDERR_FILENO]);
- popen_coio_create(&coio_rd, fh->fd[STDOUT_FILENO]);
+ struct ev_io coio_stdout;
+ struct ev_io coio_stderr;
+ popen_coio_create(&coio_stderr);
+ popen_coio_create(&coio_stdout);
int result = 0;
while (true) {
- int rc = popen_do_read(fh, buf, count, source_id);
+ ssize_t rc = popen_do_read(handle, buf, count, source_id);
if (rc >= 0) {
- *read_bytes = rc;
+ result = rc;
break;
}
@@ -574,17 +549,19 @@ popen_read(struct popen_handle *fh, void *buf, size_t count,
}
/*
- * The handlers are not ready, yield.
+ * The descriptors are not ready, yield.
*/
- if (!ev_is_active(&coio_rd) &&
- fh->fd[STDOUT_FILENO] >= 0) {
- ev_io_set(&coio_rd, fh->fd[STDOUT_FILENO], EV_READ);
- ev_io_start(loop(), &coio_rd);
+ if (!ev_is_active(&coio_stdout) &&
+ handle->fd[STDOUT_FILENO] >= 0) {
+ ev_io_set(&coio_stdout, handle->fd[STDOUT_FILENO],
+ EV_READ);
+ ev_io_start(loop(), &coio_stdout);
}
- if (!ev_is_active(&coio_er) &&
- fh->fd[STDERR_FILENO] >= 0) {
- ev_io_set(&coio_er, fh->fd[STDERR_FILENO], EV_READ);
- ev_io_start(loop(), &coio_er);
+ if (!ev_is_active(&coio_stderr) &&
+ handle->fd[STDERR_FILENO] >= 0) {
+ ev_io_set(&coio_stderr, handle->fd[STDERR_FILENO],
+ EV_READ);
+ ev_io_start(loop(), &coio_stderr);
}
/*
* Yield control to other fibers until the
@@ -598,7 +575,7 @@ popen_read(struct popen_handle *fh, void *buf, size_t count,
}
if (fiber_is_cancelled()) {
- errno = EINTR;
+ diag_set(FiberIsCancelled);
result = -1;
break;
}
@@ -606,25 +583,24 @@ popen_read(struct popen_handle *fh, void *buf, size_t count,
evio_timeout_update(loop(), &start, &delay);
}
- ev_io_stop(loop(), &coio_er);
- ev_io_stop(loop(), &coio_rd);
+ ev_io_stop(loop(), &coio_stderr);
+ ev_io_stop(loop(), &coio_stdout);
return result;
}
int
-popen_write(struct popen_handle *fh, const void *buf, size_t count,
- size_t *written, ev_tstamp timeout)
+popen_write(struct popen_handle *handle, const void *buf, size_t count,
+ size_t *written, double timeout)
{
- assert(fh);
+ assert(handle);
+ assert(timeout >= 0.0);
+
if (count == 0) {
*written = 0;
return 0;
}
- if (timeout < 0.0)
- timeout = DBL_MAX;
-
- if (fh->fd[STDIN_FILENO] < 0) {
+ if (handle->fd[STDIN_FILENO] < 0) {
*written = 0;
errno = EBADF;
return -1;
@@ -634,33 +610,29 @@ popen_write(struct popen_handle *fh, const void *buf, size_t count,
evio_timeout_init(loop(), &start, &delay, timeout);
struct ev_io coio;
- popen_coio_create(&coio, fh->fd[STDIN_FILENO]);
+ popen_coio_create(&coio);
int result = 0;
while(true) {
- ssize_t rc = write(fh->fd[STDIN_FILENO], buf, count);
+ ssize_t rc = write(handle->fd[STDIN_FILENO], buf, count);
if (rc < 0 && !popen_wouldblock(errno)) {
result = -1;
break;
}
- size_t urc = (size_t)rc;
-
- if (urc == count) {
- *written = count;
- break;
- }
-
if (rc > 0) {
buf += rc;
- count -= urc;
+ *written += rc;
+ count -= rc;
+ if (count == 0)
+ break;
}
/*
- * The handlers are not ready, yield.
+ * The descriptors are not ready, yield.
*/
if (!ev_is_active(&coio)) {
- ev_io_set(&coio, fh->fd[STDIN_FILENO], EV_WRITE);
+ ev_io_set(&coio, handle->fd[STDIN_FILENO], EV_WRITE);
ev_io_start(loop(), &coio);
}
@@ -676,7 +648,7 @@ popen_write(struct popen_handle *fh, const void *buf, size_t count,
}
if (fiber_is_cancelled()) {
- errno = EINTR;
+ diag_set(FiberIsCancelled);
result = -1;
break;
}
@@ -689,83 +661,61 @@ popen_write(struct popen_handle *fh, const void *buf, size_t count,
}
int
-popen_kill(struct popen_handle *fh, int signal_id)
+popen_kill(struct popen_handle *handle, int signal_id)
{
- assert(fh);
- return kill(fh->pid, signal_id);
+ assert(handle);
+ if (handle->terminated){
+ errno = ESRCH;
+ return -1;
+ }
+
+ return kill(handle->pid, signal_id);
}
int
-popen_wait(struct popen_handle *fh, ev_tstamp timeout, int *exit_code)
+popen_wait(struct popen_handle *handle, double timeout)
{
- assert(fh);
- if (timeout < 0.0)
- timeout = DBL_MAX;
-
- ev_tstamp start, delay;
- evio_timeout_init(loop(), &start, &delay, timeout);
-
- int result = 0;
-
- while (true) {
- /* Wait for SIGCHLD */
- int code = 0;
+ assert(handle);
+ assert(timeout >= 0.0);
- int rc = popen_get_status(fh, &code);
- if (rc != POPEN_RUNNING) {
- *exit_code = (rc == POPEN_EXITED) ? code
- : -code;
- break;
- }
+ while(!fiber_is_cancelled())
+ {
+ if (popen_get_status(handle, NULL))
+ return 0;
- /*
- * Yield control to other fibers until the
- * timeout is reached.
- * Let's sleep for 20 msec.
- */
- fiber_yield_timeout(0.02);
-
- if (fiber_is_cancelled()) {
- errno = EINTR;
- result = -1;
- break;
- }
-
- evio_timeout_update(loop(), &start, &delay);
- bool is_timedout = (delay == 0.0);
- if (is_timedout) {
+ if (fiber_cond_wait_timeout(&handle->sigchld_cond,
+ timeout) != 0) {
errno = ETIMEDOUT;
- result = -1;
- break;
+ return -1;
}
}
-
- return result;
+ diag_set(FiberIsCancelled);
+ return -1;
}
int
-popen_get_std_file_handle(struct popen_handle *fh, int file_no)
+popen_get_std_file_handle(struct popen_handle *handle, int file_no)
{
- assert(fh);
+ assert(handle);
if (file_no < STDIN_FILENO || STDERR_FILENO < file_no){
errno = EINVAL;
return -1;
}
errno = 0;
- return fh->fd[file_no];
+ return handle->fd[file_no];
}
-int
-popen_get_status(struct popen_handle *fh, int *exit_code)
+bool
+popen_get_status(struct popen_handle *handle, int *status)
{
- assert(fh);
+ assert(handle);
errno = 0;
- if (exit_code)
- *exit_code = fh->exit_code;
+ if (status)
+ *status = handle->status;
- return fh->status;
+ return handle->terminated;
}
/*
@@ -779,27 +729,20 @@ popen_sigchld_cb(ev_loop *loop, ev_child *watcher, int revents)
(void)loop;
(void)revents;
- popen_lock_data_list();
-
- struct popen_handle *data = popen_lookup_data_by_pid(watcher->rpid);
- if (data) {
+ struct popen_handle *handle = popen_lookup_data_by_pid(watcher->rpid);
+ if (handle) {
if (WIFEXITED(watcher->rstatus)) {
- data->exit_code = WEXITSTATUS(watcher->rstatus);
- data->status = POPEN_EXITED;
+ handle->status = WEXITSTATUS(watcher->rstatus);
} else if (WIFSIGNALED(watcher->rstatus)) {
- data->exit_code = WTERMSIG(watcher->rstatus);
-
- if (WCOREDUMP(watcher->rstatus))
- data->status = POPEN_DUMPED;
- else
- data->status = POPEN_KILLED;
+ handle->status = -WTERMSIG(watcher->rstatus);
} else {
/*
- * The status is not determined, treat as EXITED
+ * The status is not determined, treat as exited
*/
- data->exit_code = EX_SOFTWARE;
- data->status = POPEN_EXITED;
+ handle->status = EX_SOFTWARE;
}
+ handle->terminated = true;
+ fiber_cond_broadcast(&handle->sigchld_cond);
/*
* We shouldn't close file descriptors here.
@@ -808,11 +751,9 @@ popen_sigchld_cb(ev_loop *loop, ev_child *watcher, int revents)
* In this case the reading fails.
*/
}
-
- popen_unlock_data_list();
}
-void
+static void
popen_setup_sigchld_handler()
{
ev_child_init (&cw, popen_sigchld_cb, 0/*pid*/, 0);
@@ -820,7 +761,7 @@ popen_setup_sigchld_handler()
}
-void
+static void
popen_reset_sigchld_handler()
{
ev_child_stop(loop(), &cw);
diff --git a/src/lib/core/coio_popen.h b/src/lib/core/coio_popen.h
index 57057e97b..b7eb358d9 100644
--- a/src/lib/core/coio_popen.h
+++ b/src/lib/core/coio_popen.h
@@ -35,12 +35,13 @@
extern "C" {
#endif /* defined(__cplusplus) */
-#include "evio.h"
#include <stdarg.h>
+#include <sys/types.h>
+#include <stdbool.h>
/**
* Special values of the file descriptors passed to fio.popen
- * */
+ */
enum {
/**
* Tells fio.popen to open a handle for
@@ -58,76 +59,103 @@ enum {
struct popen_handle;
/**
- * Possible status of the process started via fio.popen
- **/
-enum popen_status {
-
- /**
- * The process is alive and well.
- */
- POPEN_RUNNING = 1,
-
- /**
- * The process exited.
- */
- POPEN_EXITED = 2,
-
- /**
- * The process terminated by a signal.
- */
- POPEN_KILLED = 3,
-
- /**
- * The process terminated abnormally.
- */
- POPEN_DUMPED = 4
-};
+ * Initializes inner data of fio.popen
+ */
+void
+popen_init();
/**
- * Initializes inner data of fio.popen
- * */
+ * Release resources acquired by fio.popen
+ */
void
-popen_initialize();
+popen_free();
-ssize_t
-popen_new(va_list ap);
+/**
+ * The function opens a process by creating a pipe
+ * forking.
+ *
+ * @param argv - is an array of character pointers
+ * to the arguments terminated by a null pointer.
+ *
+ * @param env - is the pointer to an array
+ * of character pointers to the environment strings.
+ *
+ * @param stdin_fd - the file handle to be redirected to the
+ * child process's STDIN.
+ *
+ * @param stdout_fd - the file handle receiving the STDOUT
+ * output of the child process.
+ *
+ * @param stderr_fd - the file handle receiving the STDERR
+ * output of the child process.
+ *
+ * The stdin_fd, stdout_fd & stderr_fd accept file descriptors
+ * from open() or the following values:
+ *
+ * FIO_PIPE - opens a pipe, binds it with child's
+ * input/output. The pipe is available for reading/writing.
+ *
+ * FIO_DEVNULL - redirects output from process to /dev/null.
+ *
+ * @return handle of the pipe for reading or writing
+ * (depends on value of type).
+ * In a case of error returns NULL.
+ */
+struct popen_handle *
+popen_new(char **argv, char **env,
+ int stdin_fd, int stdout_fd, int stderr_fd);
/**
* The function releases allocated resources.
* The function doesn't wait for the associated process
* to terminate.
*
- * @param fh handle returned by fio.popen.
+ * @param handle - a handle returned by fio.popen.
*
* @return 0 if the process is terminated
* @return -1 for an error
*/
int
-popen_destroy(struct popen_handle *fh);
+popen_destroy(struct popen_handle *handle);
+
+/**
+ * The function closes the specified file descriptor.
+ *
+ * @param handle a handle returned by fio.popen.
+ * @param fd file descriptor to be closed.
+ * If fd = -1 then all open descriptors are
+ * being closed.
+ * The function doesn't change the state of
+ * the associated process.
+ *
+ * @return 0 on success
+ * @return -1 if input parameters are wrong.
+ */
+int
+popen_close(struct popen_handle *handle, int fd);
+
/**
* The function reads up to count bytes from the handle
* associated with the child process.
* Returns immediately
*
- * @param fd handle returned by fio.popen.
+ * @param handle a handle returned by fio.popen.
* @param buf a buffer to be read into
* @param count size of buffer in bytes
- * @param read_bytes A pointer to the
- * variable that receives the number of bytes read.
* @param source_id A pointer to the variable that receives a
* source stream id, 1 - for STDOUT, 2 - for STDERR.
*
- * @return 0 data were successfully read
- * @return -1 an error occurred, see errno for error code
+ * @return on success returns number of bytes read (>= 0)
+ * @return -1 on error, see errno for error code
*
- * If there is nothing to read yet function returns -1
- * and errno set no EAGAIN.
+ * On timeout function returns -1
+ * and errno set to ETIMEDOUT.
*/
-int
-popen_read(struct popen_handle *fh, void *buf, size_t count,
- size_t *read_bytes, int *source_id,
- ev_tstamp timeout);
+ssize_t
+popen_read(struct popen_handle *handle, void *buf, size_t count,
+ int *source_id,
+ double timeout);
/**
* The function writes up to count bytes to the handle
@@ -135,7 +163,7 @@ popen_read(struct popen_handle *fh, void *buf, size_t count,
* Tries to write as much as possible without blocking
* and immediately returns.
*
- * @param fd handle returned by fio.popen.
+ * @param handle a handle returned by fio.popen.
* @param buf a buffer to be written from
* @param count size of buffer in bytes
* @param written A pointer to the
@@ -147,47 +175,47 @@ popen_read(struct popen_handle *fh, void *buf, size_t count,
* whether all data were written or not.
* @return -1 an error occurred, see errno for error code
*
- * If the writing can block, function returns -1
- * and errno set no EAGAIN.
+ * On timeout, function returns -1
+ * and errno set no ETIMEDOUT. The 'written' contains number
+ * of written bytes before timeout.
*/
int
-popen_write(struct popen_handle *fh, const void *buf, size_t count,
- size_t *written, ev_tstamp timeout);
+popen_write(struct popen_handle *handle, const void *buf, size_t count,
+ size_t *written, double timeout);
/**
* The function send the specified signal
* to the associated process.
*
- * @param fd - handle returned by fio.popen.
+ * @param handle a handle returned by fio.popen.
*
* @return 0 on success
* @return -1 an error occurred, see errno for error code
*/
int
-popen_kill(struct popen_handle *fh, int signal_id);
+popen_kill(struct popen_handle *handle, int signal_id);
/**
* Wait for the associated process to terminate.
* The function doesn't release the allocated resources.
*
- * @param fd handle returned by fio.popen.
+ * If associated process is terminated the function returns immediately.
+ *
+ * @param handle a handle returned by fio.popen.
*
* @param timeout number of second to wait before function exit with error.
* If function exited due to timeout the errno equals to ETIMEDOUT.
*
- * @exit_code On success contains the exit code as a positive number
- * or signal id as a negative number.
-
* @return On success function returns 0, and -1 on error.
*/
int
-popen_wait(struct popen_handle *fh, ev_tstamp timeout, int *exit_code);
+popen_wait(struct popen_handle *handle, double timeout);
/**
* Returns descriptor of the specified file.
*
- * @param fd - handle returned by fio.popen.
+ * @param handle a handle returned by fio.popen.
* @param file_no accepts one of the
* following values:
* STDIN_FILENO,
@@ -197,30 +225,23 @@ popen_wait(struct popen_handle *fh, ev_tstamp timeout, int *exit_code);
* @return file descriptor or -1 if not available
*/
int
-popen_get_std_file_handle(struct popen_handle *fh, int file_no);
+popen_get_std_file_handle(struct popen_handle *handle, int file_no);
/**
* Returns status of the associated process.
*
- * @param fd - handle returned by fio.popen.
+ * @param handle a handle returned by fio.popen.
*
- * @param exit_code - if not NULL accepts the exit code
- * if the process terminated normally or signal id
- * if process was termianted by signal.
+ * @param status if not NULL accepts the exit code
+ * if the process terminated normally or -signal id
+ * if process was terminated by signal.
*
- * @return one of the following values:
- * POPEN_RUNNING if the process is alive
- * POPEN_EXITED if the process was terminated normally
- * POPEN_KILLED if the process was terminated by a signal
+ * @return false if process is alive and
+ * true if process was terminated
*/
-int
-popen_get_status(struct popen_handle *fh, int *exit_code);
-
-void
-popen_setup_sigchld_handler();
-void
-popen_reset_sigchld_handler();
+bool
+popen_get_status(struct popen_handle *handle, int *status);
#if defined(__cplusplus)
} /* extern "C" */
diff --git a/src/lib/core/coio_task.c b/src/lib/core/coio_task.c
index e6a1b327f..7e53204bf 100644
--- a/src/lib/core/coio_task.c
+++ b/src/lib/core/coio_task.c
@@ -130,7 +130,7 @@ coio_on_stop(void *data)
void
coio_init(void)
{
- popen_initialize();
+ popen_init();
eio_set_thread_on_start(coio_on_start, NULL);
eio_set_thread_on_stop(coio_on_stop, NULL);
}
diff --git a/src/lua/fio.c b/src/lua/fio.c
index 873ee1651..d80e5f499 100644
--- a/src/lua/fio.c
+++ b/src/lua/fio.c
@@ -47,6 +47,9 @@
#include "lua/utils.h"
#include "coio_file.h"
#include "coio_popen.h"
+#include "small/region.h"
+#include "core/fiber.h"
+
static uint32_t CTID_STRUCT_POPEN_HANDLE_REF = 0;
@@ -706,27 +709,27 @@ lbox_fio_copyfile(struct lua_State *L)
return lbox_fio_pushbool(L, coio_copyfile(source, dest) == 0);
}
-static bool
-popen_verify_argv(struct lua_State *L)
-{
- if (!lua_istable(L, 1))
- return false;
- int num = (int)lua_objlen(L, 1); /*luaL_getn(L,1);*/
- return num >= 1;
-}
-
-static char**
-popen_extract_strarray(struct lua_State *L, int index, int* array_size)
+/**
+ * Returns an array of strings passed in a form of lua table.
+ * The table must contain at least one string.
+ * @return an array satisfying the execve requirements:
+ * for N strings returns an array of N=1 elements,
+ * where the trailing element is NULL.
+ * To release memory call free() for each string and for the array itself.
+ */
+static char **
+lbox_fio_popen_get_args(struct lua_State *L, int index, struct region *region)
{
- if (lua_type(L, index) != LUA_TTABLE) {
- if (array_size)
- *array_size = 0;
+ if (!lua_istable(L, index)) {
return NULL;
}
- size_t num = lua_objlen(L, index); /*luaL_getn(L,index);*/
+ size_t num = lua_objlen(L, index);
+ if (num < 1)
+ return NULL;
- char** array = calloc(num+1, sizeof(char*));
+ char **array = region_alloc(region,
+ (num + 1) * sizeof(char*));
/*
* The last item in the array must be NULL
*/
@@ -738,23 +741,20 @@ popen_extract_strarray(struct lua_State *L, int index, int* array_size)
lua_rawgeti(L, index, i+1);
size_t slen = 0;
const char* str = lua_tolstring(L, -1, &slen);
- if (!str)
- str = "";
- array[i] = strdup(str);
+ char* str2 = region_alloc(region, (slen +1) * sizeof(char));
+ strcpy(str2, str);
+ str2[slen] = '\0';
+ array[i] = str2;
lua_pop(L, 1);
}
- if (array_size)
- *array_size = num;
- /*
- * The number of elements doesn't include
- * the trailing NULL pointer
- */
+ array[num] = NULL;
+
return array;
}
static struct popen_handle *
-fio_popen_get_handle(lua_State *L, int idx)
+lbox_fio_popen_get_handle(lua_State *L, int idx)
{
uint32_t cdata_type;
struct popen_handle **handle_ptr = luaL_checkcdata(L, idx, &cdata_type);
@@ -764,7 +764,7 @@ fio_popen_get_handle(lua_State *L, int idx)
}
static void
-fio_popen_invalidate_handle(lua_State *L, int idx)
+lbox_fio_popen_invalidate_handle(lua_State *L, int idx)
{
uint32_t cdata_type;
struct popen_handle **handle_ptr = luaL_checkcdata(L, idx, &cdata_type);
@@ -776,7 +776,7 @@ fio_popen_invalidate_handle(lua_State *L, int idx)
static int
lbox_fio_popen_gc(lua_State *L)
{
- struct popen_handle *handle = fio_popen_get_handle(L,1);
+ struct popen_handle *handle = lbox_fio_popen_get_handle(L,1);
if (handle)
popen_destroy(handle);
@@ -784,22 +784,23 @@ lbox_fio_popen_gc(lua_State *L)
}
static int
-lbox_fio_popen(struct lua_State *L)
-{
- if (lua_gettop(L) < 1) {
+lbox_fio_popen(struct lua_State *L) {
+ if (lua_gettop(L) < 5) {
usage:
luaL_error(L, "fio.popen: Invalid arguments");
}
- if (!popen_verify_argv(L))
- goto usage;
-
int stdin_fd = FIO_PIPE;
int stdout_fd = FIO_PIPE;
int stderr_fd = FIO_PIPE;
- char** argv = popen_extract_strarray(L, 1, NULL);
- char** env = popen_extract_strarray(L, 2, NULL);
+ size_t region_svp = region_used(&fiber()->gc);
+ char **argv = lbox_fio_popen_get_args(L, 1, &fiber()->gc);
+ if (argv == NULL) {
+ region_truncate(&fiber()->gc, region_svp);
+ goto usage;
+ }
+ char** env = lbox_fio_popen_get_args(L, 2, &fiber()->gc);
if (lua_isnumber(L, 3))
stdin_fd = lua_tonumber(L, 3);
@@ -808,9 +809,13 @@ lbox_fio_popen(struct lua_State *L)
if (lua_isnumber(L, 5))
stderr_fd = lua_tonumber(L, 5);
- struct popen_handle* handle = NULL;
- if (coio_call(popen_new, argv, env, stdin_fd, stdout_fd,
- stderr_fd, &handle) < 0) {
+ struct popen_handle* handle = popen_new(argv, env,
+ stdin_fd, stdout_fd, stderr_fd);
+
+ /* release argv & env */
+ region_truncate(&fiber()->gc, region_svp);
+
+ if (handle == NULL) {
lua_pushnil(L);
lbox_fio_pushsyserror(L);
return 2;
@@ -820,7 +825,6 @@ lbox_fio_popen(struct lua_State *L)
luaL_pushcdata(L, CTID_STRUCT_POPEN_HANDLE_REF) = handle;
lua_pushcfunction(L, lbox_fio_popen_gc);
luaL_setcdatagc(L, -2);
-
return 1;
}
}
@@ -828,13 +832,13 @@ lbox_fio_popen(struct lua_State *L)
static int
lbox_fio_popen_read(struct lua_State *L)
{
- /* popen_read(self.fh, buf, size, seconds) */
-
- void* fh = fio_popen_get_handle(L, 1);
+ void* handle = lbox_fio_popen_get_handle(L, 1);
uint32_t ctypeid;
char *buf = *(char **)luaL_checkcdata(L, 2, &ctypeid);
size_t len = lua_tonumber(L, 3);
ev_tstamp seconds = lua_tonumber(L, 4);
+ if (seconds < 0)
+ seconds = TIMEOUT_INFINITY;
if (!len) {
lua_pushinteger(L, 0);
@@ -843,12 +847,11 @@ lbox_fio_popen_read(struct lua_State *L)
}
int output_number = 0;
- size_t received = 0;
- int rc = popen_read(fh, buf, len,
- &received, &output_number,
- seconds);
- if (rc == 0) { /* The reading's succeeded */
- lua_pushinteger(L, received);
+ ssize_t rc = popen_read(handle, buf, len,
+ &output_number,
+ seconds);
+ if (rc >= 0) { /* The reading's succeeded */
+ lua_pushinteger(L, rc);
lua_pushinteger(L, output_number);
return 2;
} else {
@@ -862,48 +865,45 @@ lbox_fio_popen_read(struct lua_State *L)
static int
lbox_fio_popen_write(struct lua_State *L)
{
- struct popen_handle* fh = fio_popen_get_handle(L, 1);
+ struct popen_handle* handle = lbox_fio_popen_get_handle(L, 1);
const char *buf = lua_tostring(L, 2);
uint32_t ctypeid = 0;
if (buf == NULL)
buf = *(const char **)luaL_checkcdata(L, 2, &ctypeid);
size_t len = lua_tointeger(L, 3);
double timeout = lua_tonumber(L, 4);
+ if (timeout < 0)
+ timeout = TIMEOUT_INFINITY;
size_t written = 0;
- int rc = popen_write(fh, buf, len,
+ int rc = popen_write(handle, buf, len,
&written, timeout);
- if (rc == 0 && written == len) {
+ if (rc == 0) {
/* The writing's succeeded */
- lua_pushinteger(L, (ssize_t) written);
- return 1;
+ lua_pushboolean(L, true);
+ lua_pushinteger(L, written);
+ return 2;
} else {
- lua_pushnil(L);
+ if (errno != ETIMEDOUT)
+ written = 0;
+ lua_pushboolean(L, false);
+ lua_pushinteger(L, written);
lbox_fio_pushsyserror(L);
- return 2;
+ return 3;
}
}
static int
lbox_fio_popen_get_status(struct lua_State *L)
{
- struct popen_handle* fh = fio_popen_get_handle(L, 1);
- int exit_code = 0;
- int res = popen_get_status(fh, &exit_code);
+ struct popen_handle* handle = lbox_fio_popen_get_handle(L, 1);
+ int status = 0;
+ bool terminated = popen_get_status(handle, &status);
- switch (res) {
- case POPEN_RUNNING:
- lua_pushnil(L);
- break;
-
- case POPEN_KILLED:
- lua_pushinteger(L, -exit_code);
- break;
-
- default:
- lua_pushinteger(L, exit_code);
- break;
- }
+ if (terminated)
+ lua_pushinteger(L, status);
+ else
+ lua_pushnil(L);
return 1;
}
@@ -911,9 +911,9 @@ lbox_fio_popen_get_status(struct lua_State *L)
static int
lbox_fio_popen_get_std_file_handle(struct lua_State *L)
{
- struct popen_handle* fh = fio_popen_get_handle(L, 1);
+ struct popen_handle* handle = lbox_fio_popen_get_handle(L, 1);
int file_no = lua_tonumber(L, 2);
- int res = popen_get_std_file_handle(fh, file_no);
+ int res = popen_get_std_file_handle(handle, file_no);
if (res < 0)
lua_pushnil(L);
@@ -925,10 +925,10 @@ lbox_fio_popen_get_std_file_handle(struct lua_State *L)
static int
lbox_fio_popen_kill(struct lua_State *L)
{
- struct popen_handle* fh = fio_popen_get_handle(L, 1);
+ struct popen_handle* handle = lbox_fio_popen_get_handle(L, 1);
int signal_id = lua_tonumber(L, 2);
- int res = popen_kill(fh, signal_id);
+ int res = popen_kill(handle, signal_id);
if (res < 0){
lua_pushboolean(L, false);
lbox_fio_pushsyserror(L);
@@ -942,30 +942,53 @@ lbox_fio_popen_kill(struct lua_State *L)
static int
lbox_fio_popen_wait(struct lua_State *L)
{
- struct popen_handle *fh = fio_popen_get_handle(L, 1);
- assert(fh);
+ struct popen_handle *handle = lbox_fio_popen_get_handle(L, 1);
+ assert(handle);
ev_tstamp timeout = lua_tonumber(L, 2);
+ if (timeout < 0)
+ timeout = TIMEOUT_INFINITY;
- /*
- * On success function returns an
- * exit code as a positive number
- * or signal id as a negative number.
- * If failed, rc is nul and err
- * contains a error message.
- */
+ int res = popen_wait(handle, timeout);
+ if (res < 0){
+ lua_pushboolean(L, false);
+ lbox_fio_pushsyserror(L);
+ return 2;
+ } else {
+ lua_pushboolean(L, true);
+ return 1;
+ }
+}
- int exit_code =0;
- int res = popen_wait(fh, timeout, &exit_code);
+static int
+lbox_fio_popen_shutdown(struct lua_State *L)
+{
+ struct popen_handle *handle = lbox_fio_popen_get_handle(L, 1);
+ assert(handle);
+ int res = popen_destroy(handle);
if (res < 0){
- lua_pushnil(L);
+ lua_pushboolean(L, false);
lbox_fio_pushsyserror(L);
return 2;
} else {
- /* Release the allocated resources */
- popen_destroy(fh);
- fio_popen_invalidate_handle(L, 1);
+ lbox_fio_popen_invalidate_handle(L, 1);
+ lua_pushboolean(L, true);
+ return 1;
+ }
+}
- lua_pushinteger(L, exit_code);
+static int
+lbox_fio_popen_close(struct lua_State *L)
+{
+ struct popen_handle *handle = lbox_fio_popen_get_handle(L, 1);
+ assert(handle);
+ int fd = lua_tointeger(L, 2);
+ int res = popen_close(handle, fd);
+ if (res < 0){
+ lua_pushboolean(L, false);
+ lbox_fio_pushsyserror(L);
+ return 2;
+ } else {
+ lua_pushboolean(L, true);
return 1;
}
}
@@ -1020,6 +1043,8 @@ tarantool_lua_fio_init(struct lua_State *L)
{ "popen_get_std_file_handle", lbox_fio_popen_get_std_file_handle },
{ "popen_kill", lbox_fio_popen_kill },
{ "popen_wait", lbox_fio_popen_wait },
+ { "popen_shutdown", lbox_fio_popen_shutdown },
+ { "popen_close", lbox_fio_popen_close },
{ NULL, NULL }
};
luaL_register(L, NULL, internal_methods);
diff --git a/src/lua/fio.lua b/src/lua/fio.lua
index d07cac552..7b9c159a4 100644
--- a/src/lua/fio.lua
+++ b/src/lua/fio.lua
@@ -22,6 +22,7 @@ fio.STDOUT = 1
fio.STDERR = 2
fio.PIPE = -2
fio.DEVNULL = -3
+fio.TIMEOUT_INFINITY = 500 * 365 * 86400
local function sprintf(fmt, ...)
@@ -232,6 +233,12 @@ popen_methods.read = function(self, buf, size, timeout)
return nil, nil, 'Invalid object'
end
+ if type(buf) == 'table' then
+ timeout = buf.timeout or timeout
+ size = buf.size or size
+ buf = buf.buf
+ end
+
local tmpbuf
if ffi.istype(const_char_ptr_t, buf) then
@@ -239,12 +246,15 @@ popen_methods.read = function(self, buf, size, timeout)
if type(size) ~= 'number' then
error('fio.popen.read: invalid size argument')
end
- timeout = timeout or -1
+ timeout = timeout or fio.TIMEOUT_INFINITY
elseif type(buf) == 'number' then
-- use temp. buffer
- timeout = size or -1
+ timeout = size or fio.TIMEOUT_INFINITY
size = buf
+ tmpbuf = buffer.ibuf()
+ buf = tmpbuf:reserve(size)
+ elseif buf == nil and type(size) == 'number' then
tmpbuf = buffer.ibuf()
buf = tmpbuf:reserve(size)
else
@@ -270,7 +280,7 @@ end
-- write(str)
-- write(buf, len)
popen_methods.write = function(self, data, size)
- local timeout = -1.0
+ local timeout = fio.TIMEOUT_INFINITY
if type(data) == 'table' then
timeout = data.timeout or timeout
size = data.size or size
@@ -286,11 +296,8 @@ popen_methods.write = function(self, data, size)
size = #data
end
- local res, err = internal.popen_write(self.fh, data, tonumber(size), tonumber(timeout))
- if err ~= nil then
- return false, err
- end
- return res >= 0
+ return internal.popen_write(self.fh, data,
+ tonumber(size), tonumber(timeout))
end
popen_methods.status = function(self)
@@ -331,17 +338,9 @@ popen_methods.kill = function(self, sig)
end
if sig == nil then
- sig = 'SIGTERM'
- end
- if type(sig) == 'string' then
- sig = signal.c.signals[sig]
- if sig == nil then
- errno(errno.EINVAL)
- return false, sprintf("fio.popen.kill(): unknown signal: %s", sig)
- end
- else
- sig = tonumber(sig)
+ sig = signal.SIGTERM
end
+ sig = tonumber(sig)
return internal.popen_kill(self.fh, sig)
end
@@ -352,14 +351,23 @@ popen_methods.wait = function(self, timeout)
end
if timeout == nil then
- timeout = -1
+ timeout = fio.TIMEOUT_INFINITY
else
timeout = tonumber(timeout)
end
- local rc, err = internal.popen_wait(self.fh, timeout)
+ return internal.popen_wait(self.fh, timeout)
+end
+
+popen_methods.shutdown = function(self)
+ if self.fh == nil then
+ return false, 'Invalid object'
+ end
+
+ local c = internal.popen_get_status(self.fh)
+ local rc, err = internal.popen_shutdown(self.fh)
if rc ~= nil then
- self.exit_code = tonumber(rc)
+ self.exit_code = c
self.fh = nil
return true
else
@@ -367,15 +375,25 @@ popen_methods.wait = function(self, timeout)
end
end
+popen_methods.close = function(self, fd)
+ if self.fh == nil then
+ return false, 'Invalid object'
+ end
+
+ if fd == nil then
+ fd = -1
+ end
+
+ return internal.popen_close(self.fh, tonumber(fd))
+end
local popen_mt = { __index = popen_methods }
-fio.popen = function(params)
- local argv = params.argv
- local env = params.environment
- local hstdin = params.stdin
- local hstdout = params.stdout
- local hstderr = params.stderr
+fio.popen = function(args, opts)
+ local env = opts and opts.env
+ local hstdin = opts and opts.stdin
+ local hstdout = opts and opts.stdout
+ local hstderr = opts and opts.stderr
if type(hstdin) == 'table' then
hstdin = hstdin.fh
@@ -387,35 +405,13 @@ fio.popen = function(params)
hstderr = hstderr.fh
end
- if argv == nil or
- type(argv) ~= 'table' or
- table.getn(argv) < 1 then
- local errmsg = [[Usage: fio.popen({parameters}),
-parameters - a table containing arguments to run a process.
-The following arguments are expected:
-
-argv - [mandatory] is a table of argument strings passed
- to the new program.
- By convention, the first of these strings should
- contain the filename associated with the file
- being executed.
-
-environment - [optional] is a table of strings,
- conventionally of the form key=value,
- which are passed as environment to the
- new program.
-
-stdin - [optional] overrides the child process's
- standard input.
-stdout - [optional] overrides the child process's
- standard output.
-stderr - [optional] overrides the child process's
- standard error output.
-]]
- error(errmsg)
+ if args == nil or
+ type(args) ~= 'table' or
+ table.getn(args) < 1 then
+ error('Usage: fio.popen({args}[, {opts}])')
end
- local fh,err = internal.popen(argv, env, hstdin, hstdout, hstderr)
+ local fh,err = internal.popen(args, env, hstdin, hstdout, hstderr)
if err ~= nil then
return nil, err
end
diff --git a/src/lua/lua_signal.c b/src/lua/lua_signal.c
index 67c751073..5527a7eda 100644
--- a/src/lua/lua_signal.c
+++ b/src/lua/lua_signal.c
@@ -40,116 +40,108 @@
#include "lua/utils.h"
#ifndef PUSHTABLE
-#define PUSHTABLE(name, method, value) { \
- lua_pushliteral(L, name); \
- method(L, value); \
- lua_settable(L, -3); \
+#define PUSHTABLE(name, value) { \
+ lua_pushstring(L, name); \
+ lua_pushinteger(L, value); \
+ lua_rawset(L, -3); \
}
#endif /*PUSHTABLE*/
void
tarantool_lua_signal_init(struct lua_State *L)
{
- static const struct luaL_Reg signal_methods[] = {
- { NULL, NULL }
+ static const struct luaL_Reg internal_methods[] = {
+ { NULL, NULL }
};
- luaL_register_module(L, "signal", signal_methods);
-
- lua_pushliteral(L, "c");
- lua_newtable(L);
-
- lua_pushliteral(L, "signals");
- lua_newtable(L);
+ luaL_register_module(L, "signal", internal_methods);
#ifdef SIGINT
- PUSHTABLE("SIGINT", lua_pushinteger, SIGINT);
+ PUSHTABLE("SIGINT", SIGINT);
#endif
#ifdef SIGILL
- PUSHTABLE("SIGILL", lua_pushinteger, SIGILL);
+ PUSHTABLE("SIGILL", SIGILL);
#endif
#ifdef SIGABRT
- PUSHTABLE("SIGABRT", lua_pushinteger, SIGABRT);
+ PUSHTABLE("SIGABRT", SIGABRT);
#endif
#ifdef SIGFPE
- PUSHTABLE("SIGFPE", lua_pushinteger, SIGFPE);
+ PUSHTABLE("SIGFPE", SIGFPE);
#endif
#ifdef SIGSEGV
- PUSHTABLE("SIGSEGV", lua_pushinteger, SIGSEGV);
+ PUSHTABLE("SIGSEGV", SIGSEGV);
#endif
#ifdef SIGTERM
- PUSHTABLE("SIGTERM", lua_pushinteger, SIGTERM);
+ PUSHTABLE("SIGTERM", SIGTERM);
#endif
#ifdef SIGHUP
- PUSHTABLE("SIGHUP", lua_pushinteger, SIGHUP);
+ PUSHTABLE("SIGHUP", SIGHUP);
#endif
#ifdef SIGQUIT
- PUSHTABLE("SIGQUIT", lua_pushinteger, SIGQUIT);
+ PUSHTABLE("SIGQUIT", SIGQUIT);
#endif
#ifdef SIGTRAP
- PUSHTABLE("SIGTRAP", lua_pushinteger, SIGTRAP);
+ PUSHTABLE("SIGTRAP", SIGTRAP);
#endif
#ifdef SIGKILL
- PUSHTABLE("SIGKILL", lua_pushinteger, SIGKILL);
+ PUSHTABLE("SIGKILL", SIGKILL);
#endif
#ifdef SIGBUS
- PUSHTABLE("SIGBUS", lua_pushinteger, SIGBUS);
+ PUSHTABLE("SIGBUS", SIGBUS);
#endif
#ifdef SIGSYS
- PUSHTABLE("SIGSYS", lua_pushinteger, SIGSYS);
+ PUSHTABLE("SIGSYS", SIGSYS);
#endif
#ifdef SIGPIPE
- PUSHTABLE("SIGPIPE", lua_pushinteger, SIGPIPE);
+ PUSHTABLE("SIGPIPE", SIGPIPE);
#endif
#ifdef SIGALRM
- PUSHTABLE("SIGALRM", lua_pushinteger, SIGALRM);
+ PUSHTABLE("SIGALRM", SIGALRM);
#endif
#ifdef SIGURG
- PUSHTABLE("SIGURG", lua_pushinteger, SIGURG);
+ PUSHTABLE("SIGURG", SIGURG);
#endif
#ifdef SIGSTOP
- PUSHTABLE("SIGSTOP", lua_pushinteger, SIGSTOP);
+ PUSHTABLE("SIGSTOP", SIGSTOP);
#endif
#ifdef SIGTSTP
- PUSHTABLE("SIGTSTP", lua_pushinteger, SIGTSTP);
+ PUSHTABLE("SIGTSTP", SIGTSTP);
#endif
#ifdef SIGCONT
- PUSHTABLE("SIGCONT", lua_pushinteger, SIGCONT);
+ PUSHTABLE("SIGCONT", SIGCONT);
#endif
#ifdef SIGCHLD
- PUSHTABLE("SIGCHLD", lua_pushinteger, SIGCHLD);
+ PUSHTABLE("SIGCHLD", SIGCHLD);
#endif
#ifdef SIGTTIN
- PUSHTABLE("SIGTTIN", lua_pushinteger, SIGTTIN);
+ PUSHTABLE("SIGTTIN", SIGTTIN);
#endif
#ifdef SIGTTOU
- PUSHTABLE("SIGTTOU", lua_pushinteger, SIGTTOU);
+ PUSHTABLE("SIGTTOU", SIGTTOU);
#endif
#ifdef SIGPOLL
- PUSHTABLE("SIGPOLL", lua_pushinteger, SIGPOLL);
+ PUSHTABLE("SIGPOLL", SIGPOLL);
#endif
#ifdef SIGXCPU
- PUSHTABLE("SIGXCPU", lua_pushinteger, SIGXCPU);
+ PUSHTABLE("SIGXCPU", SIGXCPU);
#endif
#ifdef SIGXFSZ
- PUSHTABLE("SIGXFSZ", lua_pushinteger, SIGXFSZ);
+ PUSHTABLE("SIGXFSZ", SIGXFSZ);
#endif
#ifdef SIGVTALRM
- PUSHTABLE("SIGVTALRM", lua_pushinteger, SIGVTALRM);
+ PUSHTABLE("SIGVTALRM", SIGVTALRM);
#endif
#ifdef SIGPROF
- PUSHTABLE("SIGPROF", lua_pushinteger, SIGPROF);
+ PUSHTABLE("SIGPROF", SIGPROF);
#endif
#ifdef SIGUSR1
- PUSHTABLE("SIGUSR1", lua_pushinteger, SIGUSR1);
+ PUSHTABLE("SIGUSR1", SIGUSR1);
#endif
#ifdef SIGUSR2
- PUSHTABLE("SIGUSR2", lua_pushinteger, SIGUSR2);
+ PUSHTABLE("SIGUSR2", SIGUSR2);
#endif
- lua_settable(L, -3); /* "signals" */
- lua_settable(L, -3); /* "c" */
- lua_pop(L, 1);
+ lua_pop(L, 1); /* signal */
}
diff --git a/src/main.cc b/src/main.cc
index 1e4356241..5cb106adc 100644
--- a/src/main.cc
+++ b/src/main.cc
@@ -307,8 +307,6 @@ signal_free(void)
int i;
for (i = 0; i < ev_sig_count; i++)
ev_signal_stop(loop(), &ev_sigs[i]);
-
- popen_reset_sigchld_handler();
}
/** Make sure the child has a default signal disposition. */
@@ -377,8 +375,6 @@ signal_init(void)
panic_syserror("sigaction");
}
- popen_setup_sigchld_handler();
-
ev_signal_init(&ev_sigs[0], sig_checkpoint, SIGUSR1);
ev_signal_init(&ev_sigs[1], signal_cb, SIGINT);
ev_signal_init(&ev_sigs[2], signal_cb, SIGTERM);
@@ -605,6 +601,8 @@ tarantool_free(void)
/* Shutdown worker pool. Waits until threads terminate. */
coio_shutdown();
+ popen_free();
+
box_free();
title_free(main_argc, main_argv);
diff --git a/test/app-tap/fio_popen.test.lua b/test/app-tap/fio_popen.test.lua
index c91b7f5d2..fe488a30d 100755
--- a/test/app-tap/fio_popen.test.lua
+++ b/test/app-tap/fio_popen.test.lua
@@ -7,7 +7,7 @@ local signal = require('signal')
local fiber = require('fiber')
local test = tap.test()
-test:plan(6+10+12+10+8+7+4+3)
+test:plan(7+11+13+11+10+8+5+4+8+6)
-- Preliminaries
local function read_stdout(app)
@@ -25,9 +25,8 @@ local function read_stdout(app)
end
-- Test 1. Run application, check its status, kill and wait
-local app1 = fio.popen({argv = {'/bin/sh', '-c', 'cat'},
- stdout=fio.STDOUT,
- stderr=fio.STDOUT})
+local app1 = fio.popen({'/bin/sh', '-c', 'cat'},
+ {stdout=fio.STDOUT, stderr=fio.STDOUT})
test:isnt(app1, nil, "#1. Starting a existing application")
local rc = app1:status()
@@ -40,18 +39,20 @@ rc,err = app1:wait(5)
test:is(rc, true, "#1. Process was killed")
rc = app1:status()
-test:is(rc, -signal.c.signals['SIGTERM'], "#1. Process was killed 2")
+test:is(rc, -signal.SIGTERM, "#1. Process was killed 2")
rc,src,err = app1:read(128,0.1)
test:is(rc, nil, "#1. Cant read from the dead process")
+rc,err = app1:shutdown()
+test:is(rc, true, "#1. Release resources")
+
app1 = nil
-- Test 2. Run application, write to stdin, read from stdout
-app1 = fio.popen({argv = {'/bin/sh', '-c', 'cat'},
- stdout=fio.PIPE,
- stdin=fio.PIPE,
- stderr=fio.STDOUT})
+app1 = fio.popen({'/bin/sh', '-c', 'cat'},{stdout=fio.PIPE,
+ stdin=fio.PIPE,
+ stderr=fio.STDOUT})
test:isnt(app1, nil, "#2. Starting a existing application")
rc = app1:status()
@@ -68,11 +69,6 @@ test:is(rc, test2str, "#2. Received exact string")
test2str = 'abc\ndef\ngh'
app1:write(test2str, 4)
-local rc2,src2,err = app1:read(6)
-
-test:is(err, nil, "#2. Reading ok")
-test:is(src2, fio.STDOUT, "#2. Read from STDOUT 2")
-test:is(rc2, 'abc\n', "#2. Received exact string 2")
rc,err = app1:kill()
test:is(rc, true, "#2. Sending kill(15)")
@@ -80,16 +76,24 @@ test:is(rc, true, "#2. Sending kill(15)")
rc,err = app1:wait()
test:is(rc, true, "#2. Process is terminated")
+local rc2,src2,err = app1:read(6)
+
+test:is(err, nil, "#2. Reading ok after wait()")
+test:is(src2, fio.STDOUT, "#2. Read from STDOUT 2")
+test:is(rc2, 'abc\n', "#2. Received exact string 2")
+
+rc,err = app1:shutdown()
+test:is(rc, true, "#2. Release resources")
+
rc = app1:status()
-test:is(rc, -signal.c.signals['SIGTERM'], "#2. Process was killed")
+test:is(rc, -signal.SIGTERM, "#2. Process was killed")
app1 = nil
-- Test 3. read from stdout with timeout
-app1 = fio.popen({argv = {'/bin/sh', '-c', 'cat'},
- stdout=fio.PIPE,
- stdin=fio.PIPE,
- stderr=fio.STDOUT})
+app1 = fio.popen({'/bin/sh', '-c', 'cat'},{stdout=fio.PIPE,
+ stdin=fio.PIPE,
+ stderr=fio.STDOUT})
test:isnt(app1, nil, "#3. Starting a existing application")
test:is(app1:stderr(), nil, "#3. STDERR is redirected")
test:isnt(app1:stdout(), nil, "#3. STDOUT is available")
@@ -99,7 +103,7 @@ test:isnt(app1:stdin(), nil, "#3. STDIN is available")
rc = app1:status()
test:is(rc, nil, "#3. Process is running")
-rc,src,err = app1:read(256, 0.5)
+rc,src,err = app1:read({size=256,timeout=0.5})
local e = errno()
test:is(e, errno.ETIMEDOUT, "#3. Timeout")
@@ -114,13 +118,17 @@ test:is(err, nil, "#3. Reading ok")
test:is(src, fio.STDOUT, "#3. Read from STDOUT")
test:is(rc, test2str, "#3. Received exact string")
-rc,err = app1:kill('SIGHUP')
+rc,err = app1:kill(signal.SIGHUP)
test:is(rc, true, "#3. Sending kill(1)")
rc,err = app1:wait()
test:is(rc, true, "#3. Process was killed")
+
+rc,err = app1:shutdown()
+test:is(rc, true, "#3. Release resources")
+
rc = app1:status()
-test:is(rc, -signal.c.signals['SIGHUP'], "#3. Process was killed")
+test:is(rc, -signal.SIGHUP, "#3. Process was killed")
app1 = nil
@@ -137,10 +145,9 @@ fh1:close()
local txt_file = fio.open(txt_filename, {'O_RDONLY'})
test:isnt(txt_file, nil, "#4. Open existing file for reading")
-app1 = fio.popen({argv = {'/bin/sh', '-c', 'cat'},
- stdout=fio.PIPE,
- stdin=txt_file,
- stderr=fio.STDOUT})
+app1 = fio.popen({'/bin/sh', '-c', 'cat'},{stdout=fio.PIPE,
+ stdin=txt_file,
+ stderr=fio.STDOUT})
test:isnt(app1, nil, "#4. Starting a existing application")
test:is(app1:stderr(), nil, "#4. STDERR is redirected")
test:isnt(app1:stdout(), nil, "#4. STDOUT is available")
@@ -156,6 +163,10 @@ test:is(rc, test2str, "#4. Received exact string")
rc,err = app1:wait()
test:is(rc, true, "#4. Process is terminated")
+
+rc,err = app1:shutdown()
+test:is(rc, true, "#4. Release resources")
+
rc = app1:status()
test:is(rc, 0, "#4. Process's exited")
@@ -167,9 +178,8 @@ fio.unlink(txt_filename)
local build_path = os.getenv("BUILDDIR")
local app_path = fio.pathjoin(build_path, 'test/app-tap/fio_popen_test1.sh')
-app1 = fio.popen({argv = {'/bin/sh', '-c', app_path},
- stdout=fio.PIPE,
- stderr=fio.STDOUT})
+app1 = fio.popen({'/bin/sh', '-c', app_path},{stdout=fio.PIPE,
+ stderr=fio.STDOUT})
test:isnt(app1, nil, "#5. Starting application 1")
test:is(app1:stderr(), nil, "#5. STDERR is redirected")
@@ -178,9 +188,8 @@ test:isnt(app1:stdin(), nil, "#5. STDIN is available")
fiber.sleep(1)
-local app2 = fio.popen({argv = {'/bin/sh', '-c', 'cat'},
- stdout=fio.PIPE,
- stdin=app1:stdout()})
+local app2 = fio.popen({'/bin/sh', '-c', 'cat'},
+ {stdout=fio.PIPE, stdin=app1:stdout()})
test:isnt(app2, nil, "#5. Starting application 2")
@@ -196,14 +205,19 @@ test:is(rc, true, "#5. Process's exited 1")
rc,err = app2:wait()
test:is(rc, true, "#5. Process's exited 2")
+rc,err = app1:shutdown()
+test:is(rc, true, "#5. Release resources 1")
+rc,err = app2:shutdown()
+test:is(rc, true, "#5. Release resources 2")
+
app1 = nil
app2 = nil
-- Test 6. Write a lot
-app1 = fio.popen({argv = {'/bin/sh', '-c', 'cat'},
- stdout=fio.PIPE,
- stdin=fio.PIPE,
- stderr=fio.STDOUT})
+app1 = fio.popen({'/bin/sh', '-c', 'cat'},
+ { stdout=fio.PIPE,
+ stdin=fio.PIPE,
+ stderr=fio.STDOUT})
test:isnt(app1, nil, "#6. Starting a existing application")
rc = app1:status()
@@ -246,18 +260,20 @@ test:is(rc, true, "#6. Sending kill(15)")
rc,err = app1:wait()
test:is(rc, true, "#6. Process is terminated")
+rc,err = app1:shutdown()
+test:is(rc, true, "#6. Release resources")
+
rc = app1:status()
-test:is(rc, -signal.c.signals['SIGTERM'], "#6. Process was killed")
+test:is(rc, -signal.SIGTERM, "#6. Process was killed")
app1 = nil
-- Test 7. Read both STDOUT & STDERR
local app_path = fio.pathjoin(build_path, 'test/app-tap/fio_popen_test2.sh')
-app1 = fio.popen({argv = {'/bin/sh', '-c', app_path},
- stdin=fio.STDIN,
- stdout=fio.PIPE,
- stderr=fio.PIPE})
+app1 = fio.popen({'/bin/sh', '-c', app_path},{stdin=fio.STDIN,
+ stdout=fio.PIPE,
+ stderr=fio.PIPE})
test:isnt(app1, nil, "#7. Starting application")
@@ -279,6 +295,8 @@ test:is(result[fio.STDERR], test2str, "#7. Received STDERR")
rc,err = app1:wait()
test:is(rc, true, "#7. Process's exited")
+rc,err = app1:shutdown()
+test:is(rc, true, "#7. Release resources")
app1 = nil
@@ -286,10 +304,10 @@ app1 = nil
-- Test 8. Use custom environment variables
local app_path = fio.pathjoin(build_path, 'test/app-tap/fio_popen_test3.sh')
-app1 = fio.popen({argv = {'/bin/sh', '-c', app_path},
- environment = {'VAR1=Variable1', 'VAR2=Variable2','VAR3=Variable3'},
- stdout=fio.PIPE,
- stderr=fio.STDOUT})
+app1 = fio.popen({'/bin/sh', '-c', app_path},{
+ env = {'VAR1=Variable1', 'VAR2=Variable2','VAR3=Variable3'},
+ stdout=fio.PIPE,
+ stderr=fio.STDOUT})
test:isnt(app1, nil, "#8. Starting a existing application")
@@ -300,9 +318,74 @@ test:is(rc, test2str, "#8. Received values of env. variables")
rc,err = app1:wait()
test:is(rc, true, "#8. Process was killed")
+rc,err = app1:shutdown()
+test:is(rc, true, "#8. Release resources")
+
+app1 = nil
+
+-- Test 9. Run application, use close()
+app1 = fio.popen({'/bin/sh', '-c', 'cat'},{stdout=fio.PIPE,
+ stdin=fio.PIPE,
+ stderr=fio.STDOUT})
+test:isnt(app1, nil, "#9. Starting a existing application")
+
+rc = app1:status()
+test:is(rc, nil, "#9. Process is running")
+
+local test2str = '123\n456\n789'
+
+app1:write(test2str)
+rc,src,err = app1:read(256)
+
+test:is(src, fio.STDOUT, "#9. Read from STDOUT")
+test:is(rc, test2str, "#9. Received exact string")
+
+rc,err = app1:close(fio.STDIN)
+test:is(rc, true, "#9. Close STDIN")
+
+rc,err = app1:wait()
+test:is(rc, true, "#9. Process is terminated")
+
+rc,err = app1:shutdown()
+test:is(rc, true, "#9. Release resources")
+
+rc = app1:status()
+test:is(rc, 0, "#9. Process has exited")
app1 = nil
+-- Test 10. Run application, redirect stdout to dev/null
+app1 = fio.popen({'/bin/sh', '-c', 'cat'},{stdout=fio.DEVNULL,
+ stdin=fio.PIPE,
+ stderr=fio.STDOUT})
+test:isnt(app1, nil, "#10. Starting a existing application")
+
+rc = app1:status()
+test:is(rc, nil, "#10. Process is running")
+for i=0,1000 do
+
+ local ss = ''
+ for j = 0,100 do
+ local s = tostring(i*100+j) .. '\n'
+ ss = ss .. s
+ end
+
+ app1:write(ss)
+end
+
+rc,err = app1:close(fio.STDIN)
+test:is(rc, true, "#10. Close STDIN")
+
+rc,err = app1:wait()
+test:is(rc, true, "#10. Process is terminated")
+
+rc,err = app1:shutdown()
+test:is(rc, true, "#10. Release resources")
+
+rc = app1:status()
+test:is(rc, 0, "#10. Process has exited")
+
+app1 = nil
-- --------------------------------------------------------------
test:check()
os.exit(0)
--
2.17.1
^ permalink raw reply [flat|nested] 3+ messages in thread