From 8533513aff3cc5bdb03b82ef503489ee8683478e Mon Sep 17 00:00:00 2001 From: Daan De Meyer Date: Wed, 12 Nov 2025 17:53:47 +0100 Subject: [PATCH 1/8] Introduce support for running code in fibers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Traditionally, asynchronous programming in systemd has been achieved using sd-event along with the asynchronous interfaces of sd-bus and sd-varlink. This works well when the system is reacting to events and all code triggered by those events can run without blocking. In these scenarios, the global Manager object is passed as userdata to the callback, and the callback can use the stack as usual, declaring local state and ensuring proper cleanup via _cleanup_. Control flow structures, such as loops, work as expected, and everything runs smoothly. However, challenges arise when the code needs to perform long-running operations within these callbacks. Since the system cannot block execution within the callback, we can't directly invoke a long-running operation and wait for its result without introducing complexities. Instead, we need to initiate the long-running task, register for completion with sd-event, sd-bus, or sd-varlink, and provide a callback to be invoked when the operation completes. This callback, however, only receives a single userdata pointer, which forces us to bundle all local variables into a struct and pass it along as part of the callback. On top of that, after queuing the asynchronous operation, the caller continues executing. As the caller's stack unwinds when the function exits, the resources and state within the local scope may be prematurely cleaned up. Therefore, the struct must store copies of the local variables or ensure proper reference counting to prevent premature resource cleanup. When multiple long-running operations need to be initiated within a loop, the complexity grows further. We must introduce additional shared state to track the completion of all operations before we can run any code that depends on their results. Furthermore, since the daemon may be shut down at any time, we must track the lifecycle of each long-running operation in the global Manager struct, ensuring proper cleanup even when stack unwinding can no longer manage the resources for us. Fibers, or green threads, provide a more natural way of handling asynchronous operations. By enabling cooperative multitasking within a single thread, fibers allow us to write code that looks like it’s running synchronously, but with the ability to yield control at predefined points, such as when waiting for long-running tasks to complete. With fibers, we can simplify the control flow by running asynchronous operations within a fiber, allowing us to "pause" execution while waiting for the long-running operation to finish and then "resume" the operation once it's complete. This eliminates the need for multiple callback chains, extensive state tracking, and the potential pitfalls of stack unwinding. This commit introduces the ability to execute long-running operations in a non-blocking manner while maintaining the simplicity and readability of synchronous code. The fiber-based approach will significantly improve the handling of complex workflows, making the code easier to write and maintain. The implementation is based on ucontext.h's makecontext() (with a fallback to the venerable sigaltstack() approach on musl), sigsetjmp()/siglongjmp() and sd-event. ucontext.h provides us with alternate stacks that we can switch between. We use sigsetjmp()/siglongjmp() instead of swapcontext() because the latter forcibly saves/restores a per context signal mask every time it is called. Using sigsetjmp()/siglongjmp(), we can avoid the unnecessary syscall and maintain a per thread signal mask, which makes much more sense than having a per fiber signal mask. The default stack size is the same as a regular thread. Because we use mmap() to allocate the stack, the memory won't actually be used until it is paged in by the kernel, so we don't actually use 8MB per fiber. To integrate fibers with the event loop, each fiber is assigned a deferred event source which resumes the fiber when enabled. The deferred event source is oneshot by default so the fiber will run immediately until it yields or suspends. If it yields, the deferred event source is enabled again (oneshot) immediately. If it suspends, before it suspends, one or more event sources are registered with sd-event that will enable the deferred event source (oneshot) to resume the fiber once the operation it is waiting for completes. Yielding or suspending the fiber is done by calling sd_fiber_yield() or sd_fiber_suspend() respectively. Both of these return zero on success or any error value from the async operation that caused the fiber to resume. This is also how fiber cancellation is implemented. When a fiber is cancelled, sd_fiber_yield() and sd_fiber_suspend() will return ECANCELED when the fiber is resumed, allowing the fiber to unwind its stack (which allows cleanup to happen automatically) and finish. Instead of having applications work directly with fibers, we hide them behind a generic futures interface to represent long-running operations, regardless of whether those operations are running on a fiber or not. Aside from fibers, the futures library (sd-future) will for example allow waiting for sd-event sources and doing sd-bus calls in the background as well. Fibers can suspend until a future is ready with sd_fiber_await() or by having the future wake up the fiber explicitly in its callback. A future always defaults to waking up the current fiber. Each future kind plugs into the library by providing an sd_future_ops vtable (alloc, free, cancel, set_priority). The library treats the impl pointer returned by alloc() as a black box. Future Implementations retrieve it via sd_future_get_private(). A future starts in SD_FUTURE_PENDING and transitions exactly once to SD_FUTURE_RESOLVED, carrying an integer result. Consumers can react to that transition either by installing a one-shot callback with sd_future_set_callback() (callback-style code) or by waiting on it from a fiber via sd_fiber_await() (synchronous-looking fiber code). sd_fiber_await() is itself built on a "wait future" that resolves when its target resolves; sd_future_new_wait() exposes the same primitive directly so non-fiber callers can chain futures without involving a fiber. Cancellation is cooperative: sd_future_cancel() invokes the future impl's cancel callback, which is responsible for tearing down its work and ultimately resolving the promise with -ECANCELED. For fiber futures this is what surfaces as the ECANCELED return from sd_fiber_yield()/sd_fiber_suspend() mentioned above. Fire-and-forget fibers — created by passing a NULL ret to sd_fiber_new() — take a self-reference on their future so they outlive the caller's scope. The self-ref is dropped when the fiber resolves. This floating mechanism (sd_fiber_set_floating()) is restricted to fiber futures because they uniquely guarantee resolution; allowing it for arbitrary future kinds would risk silent leaks for kinds that may never resolve. Note that fiber cleanup depends on the runtime operating normally. Each fiber's _cleanup_-style cleanups live on the fiber's own stack and run only when the fiber is resumed and allowed to unwind, which requires a working event loop to drive it to completion. The exit event source registered for top-level fibers ensures unwind on a normal sd_event_exit(), but if the event loop itself terminates abnormally (e.g. an unrecoverable allocation failure mid-dispatch) before all fibers have resolved, their stacks never unwind and any resources they own leak. The code lives in libsystemd as sd-future (not exported) for the following reasons: - We may want to make this a public libsystemd API in the future - The code can't live in src/basic as it makes heavy use of sd-event - The code can't live in src/shared as sd-bus and sd-event make use of it The log and log-context headers are updated with functions to allow fibers to have their own log prefix and log context. --- meson.build | 12 +- src/basic/architecture.h | 6 + src/basic/basic-forward.h | 2 + src/basic/log-context.c | 8 + src/basic/log-context.h | 2 + src/basic/log.c | 6 + src/basic/log.h | 2 + src/include/override/sys/mman.h | 7 + src/libsystemd/meson.build | 15 +- src/libsystemd/sd-common/sd-forward.h | 6 + src/libsystemd/sd-event/event-future.c | 118 ++ src/libsystemd/sd-event/event-future.h | 7 + src/libsystemd/sd-future/fiber.c | 812 ++++++++++++ src/libsystemd/sd-future/sd-future.c | 263 ++++ src/libsystemd/sd-future/test-fiber.c | 1171 +++++++++++++++++ src/systemd/_sd-common.h | 14 + src/systemd/meson.build | 1 + src/systemd/sd-future.h | 101 ++ .../TEST-02-UNITTESTS/meson.build | 2 +- test/test-link-abi.py | 1 + 20 files changed, 2552 insertions(+), 4 deletions(-) create mode 100644 src/libsystemd/sd-event/event-future.c create mode 100644 src/libsystemd/sd-event/event-future.h create mode 100644 src/libsystemd/sd-future/fiber.c create mode 100644 src/libsystemd/sd-future/sd-future.c create mode 100644 src/libsystemd/sd-future/test-fiber.c create mode 100644 src/systemd/sd-future.h diff --git a/meson.build b/meson.build index 1d2d59e4fa..56f73ea30d 100644 --- a/meson.build +++ b/meson.build @@ -1017,6 +1017,14 @@ else endif conf.set10('HAVE_LIBCRYPT', have) +# musl declares the ucontext.h functions but does not implement them; the fiber bootstrap in +# src/libsystemd/sd-future/fiber.c relies on them, so on musl we have to link to libucontext. +if get_option('libc') == 'musl' + libucontext = dependency('libucontext') +else + libucontext = [] +endif + bpf_framework = get_option('bpf-framework') bpf_compiler = get_option('bpf-compiler') libbpf = dependency('libbpf', @@ -1720,6 +1728,7 @@ libsystemd_includes = [basic_includes, include_directories( 'src/libsystemd/sd-common', 'src/libsystemd/sd-device', 'src/libsystemd/sd-event', + 'src/libsystemd/sd-future', 'src/libsystemd/sd-hwdb', 'src/libsystemd/sd-id128', 'src/libsystemd/sd-journal', @@ -1760,7 +1769,7 @@ libsystemd = shared_library( link_with : [libc_wrapper_static, libbasic_static], link_whole : [libsystemd_static], - dependencies : [userspace], + dependencies : [libucontext, userspace], link_depends : libsystemd_sym, install : true, install_tag: 'libsystemd', @@ -1782,6 +1791,7 @@ if static_libsystemd != 'false' dependencies : [libgcrypt_cflags, liblz4_cflags, libm, + libucontext, libxz_cflags, libzstd_cflags, userspace], diff --git a/src/basic/architecture.h b/src/basic/architecture.h index ee91ea0d34..350a72d087 100644 --- a/src/basic/architecture.h +++ b/src/basic/architecture.h @@ -242,4 +242,10 @@ Architecture uname_architecture(void); # error "Please register your architecture here!" #endif +#if defined(__hppa__) || defined(__hppa64__) +# define STACK_GROWS_UP 1 +#else +# define STACK_GROWS_UP 0 +#endif + DECLARE_STRING_TABLE_LOOKUP(architecture, Architecture); diff --git a/src/basic/basic-forward.h b/src/basic/basic-forward.h index 5edd494f78..5f9109cb17 100644 --- a/src/basic/basic-forward.h +++ b/src/basic/basic-forward.h @@ -111,10 +111,12 @@ typedef enum UnitNameMangle UnitNameMangle; typedef enum UnitType UnitType; typedef enum WaitFlags WaitFlags; +typedef struct Fiber Fiber; typedef struct Hashmap Hashmap; typedef struct HashmapBase HashmapBase; typedef struct IteratedCache IteratedCache; typedef struct Iterator Iterator; +typedef struct LogContext LogContext; typedef struct OrderedHashmap OrderedHashmap; typedef struct OrderedSet OrderedSet; typedef struct Set Set; diff --git a/src/basic/log-context.c b/src/basic/log-context.c index a05b4b1980..44eb7cd36f 100644 --- a/src/basic/log-context.c +++ b/src/basic/log-context.c @@ -177,6 +177,14 @@ size_t log_context_num_fields(void) { return _log_context_num_fields; } +void log_context_swap(LogContext **log_context, size_t *num_fields) { + assert(log_context); + assert(num_fields); + + SWAP_TWO(_log_context, *log_context); + SWAP_TWO(_log_context_num_fields, *num_fields); +} + void _reset_log_level(int *saved_log_level) { assert(saved_log_level); diff --git a/src/basic/log-context.h b/src/basic/log-context.h index ca112fa862..638d2ed913 100644 --- a/src/basic/log-context.h +++ b/src/basic/log-context.h @@ -66,6 +66,8 @@ size_t log_context_num_contexts(void); /* Returns the number of fields in all attached log contexts. */ size_t log_context_num_fields(void); +void log_context_swap(LogContext **log_context, size_t *num_fields); + void _reset_log_level(int *saved_log_level); #define _LOG_CONTEXT_SET_LOG_LEVEL(level, l) \ diff --git a/src/basic/log.c b/src/basic/log.c index d8b441bfad..29702abd44 100644 --- a/src/basic/log.c +++ b/src/basic/log.c @@ -87,6 +87,12 @@ bool _log_message_dummy = false; /* Always false */ } \ } while (false) +void log_prefix_swap(const char **prefix) { + assert(prefix); + + SWAP_TWO(log_prefix, *prefix); +} + static void log_close_console(void) { /* See comment in log_close_journal() */ (void) safe_close_above_stdio(TAKE_FD(console_fd)); diff --git a/src/basic/log.h b/src/basic/log.h index 46a4339de5..cb4e8b5d0f 100644 --- a/src/basic/log.h +++ b/src/basic/log.h @@ -380,6 +380,8 @@ int log_syntax_parse_error_internal( void log_setup(void); const char* _log_set_prefix(const char *prefix, bool force); + +void log_prefix_swap(const char **prefix); static inline const char* _log_unset_prefixp(const char **p) { assert(p); _log_set_prefix(*p, true); diff --git a/src/include/override/sys/mman.h b/src/include/override/sys/mman.h index 30ef92b835..9961ea4b21 100644 --- a/src/include/override/sys/mman.h +++ b/src/include/override/sys/mman.h @@ -18,3 +18,10 @@ static_assert(MFD_NOEXEC_SEAL == 0x0008U, ""); #else static_assert(MFD_EXEC == 0x0010U, ""); #endif + +/* since Linux 6.13 / glibc-2.42 */ +#ifndef MADV_GUARD_INSTALL +# define MADV_GUARD_INSTALL 102 +#else +static_assert(MADV_GUARD_INSTALL == 102, ""); +#endif diff --git a/src/libsystemd/meson.build b/src/libsystemd/meson.build index c6cf0fad34..061ff6213f 100644 --- a/src/libsystemd/meson.build +++ b/src/libsystemd/meson.build @@ -33,6 +33,7 @@ sd_daemon_sources = files('sd-daemon/sd-daemon.c') ############################################################ sd_event_sources = files( + 'sd-event/event-future.c', 'sd-event/event-util.c', 'sd-event/sd-event.c', ) @@ -75,6 +76,13 @@ sd_device_sources = files( ############################################################ +sd_future_sources = files( + 'sd-future/fiber.c', + 'sd-future/sd-future.c', +) + +############################################################ + sd_login_sources = files('sd-login/sd-login.c') ############################################################ @@ -135,8 +143,9 @@ libsystemd_sources = files( 'sd-resolve/sd-resolve.c', ) + sd_journal_sources + sd_id128_sources + sd_daemon_sources \ + sd_event_sources + sd_bus_sources + sd_device_sources \ - + sd_login_sources + sd_json_sources + sd_varlink_sources \ - + sd_path_sources + sd_netlink_sources + sd_network_sources + + sd_future_sources + sd_login_sources + sd_json_sources \ + + sd_varlink_sources + sd_path_sources + sd_netlink_sources \ + + sd_network_sources sources += libsystemd_sources @@ -151,6 +160,7 @@ libsystemd_static = static_library( link_with : [libc_wrapper_static, libbasic_static], dependencies : [libm, + libucontext, userspace], build_by_default : false) @@ -179,6 +189,7 @@ simple_tests += files( 'sd-bus/test-bus-vtable.c', 'sd-device/test-device-util.c', 'sd-device/test-sd-device-monitor.c', + 'sd-future/test-fiber.c', 'sd-hwdb/test-sd-hwdb.c', 'sd-id128/test-id128.c', 'sd-journal/test-audit-type.c', diff --git a/src/libsystemd/sd-common/sd-forward.h b/src/libsystemd/sd-common/sd-forward.h index 8abe655209..96ab84e982 100644 --- a/src/libsystemd/sd-common/sd-forward.h +++ b/src/libsystemd/sd-common/sd-forward.h @@ -127,3 +127,9 @@ typedef struct sd_resolve sd_resolve; typedef struct sd_resolve_query sd_resolve_query; typedef struct sd_hwdb sd_hwdb; + +typedef struct sd_future sd_future; + +typedef int (*sd_future_func_t)(sd_future *f); +typedef int (*sd_fiber_func_t)(void *userdata); +typedef _sd_destroy_t sd_fiber_destroy_t; diff --git a/src/libsystemd/sd-event/event-future.c b/src/libsystemd/sd-event/event-future.c new file mode 100644 index 0000000000..4595a7a6fb --- /dev/null +++ b/src/libsystemd/sd-event/event-future.c @@ -0,0 +1,118 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ + +#include "sd-event.h" +#include "sd-future.h" + +#include "alloc-util.h" +#include "errno-util.h" +#include "event-future.h" + +typedef struct TimeFuture { + sd_event_source *source; + + /* Result the future resolves with on natural expiry (vs. cancellation). 0 for normal sleep, + * non-zero (e.g. -ETIMEDOUT) lets a fiber waiting on this future resume with that error. */ + int result; +} TimeFuture; + +static void* time_future_alloc(void) { + return new0(TimeFuture, 1); +} + +static void time_future_free(sd_future *f) { + TimeFuture *tf = ASSERT_PTR(sd_future_get_private(ASSERT_PTR(f))); + sd_event_source_unref(tf->source); + free(tf); +} + +static int time_future_cancel(sd_future *f) { + TimeFuture *tf = ASSERT_PTR(sd_future_get_private(ASSERT_PTR(f))); + int r; + + r = sd_event_source_set_enabled(tf->source, SD_EVENT_OFF); + RET_GATHER(r, sd_future_resolve(f, -ECANCELED)); + return r; +} + +static int time_future_set_priority(sd_future *f, int64_t priority) { + TimeFuture *tf = ASSERT_PTR(sd_future_get_private(ASSERT_PTR(f))); + return sd_event_source_set_priority(tf->source, priority); +} + +static const sd_future_ops time_future_ops = { + .size = sizeof(sd_future_ops), + .alloc = time_future_alloc, + .free = time_future_free, + .cancel = time_future_cancel, + .set_priority = time_future_set_priority, +}; + +static int time_handler(sd_event_source *s, usec_t usec, void *userdata) { + sd_future *f = ASSERT_PTR(userdata); + TimeFuture *tf = ASSERT_PTR(sd_future_get_private(f)); + + return sd_future_resolve(f, tf->result); +} + +typedef int (*event_add_time_func)( + sd_event *e, + sd_event_source **ret, + clockid_t clock, + uint64_t usec, + uint64_t accuracy, + sd_event_time_handler_t callback, + void *userdata); + +static int future_new_time_internal( + event_add_time_func add_time, + sd_event *e, + clockid_t clock, + uint64_t usec, + uint64_t accuracy, + int result, + sd_future **ret) { + + int r; + + assert(add_time); + assert(e); + assert(ret); + + if (IN_SET(sd_event_get_state(e), SD_EVENT_EXITING, SD_EVENT_FINISHED)) + return -ECANCELED; + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + r = sd_future_new(&time_future_ops, &f); + if (r < 0) + return r; + + TimeFuture *tf = sd_future_get_private(f); + tf->result = result; + + r = add_time(e, &tf->source, clock, usec, accuracy, time_handler, f); + if (r < 0) + return r; + + if (sd_fiber_is_running()) { + int64_t priority; + + r = sd_fiber_get_priority(&priority); + if (r < 0) + return r; + + r = sd_event_source_set_priority(tf->source, priority); + if (r < 0) + return r; + } + + *ret = TAKE_PTR(f); + return 0; +} + +int future_new_time(sd_event *e, clockid_t clock, uint64_t usec, uint64_t accuracy, int result, sd_future **ret) { + return future_new_time_internal(sd_event_add_time, e, clock, usec, accuracy, result, ret); +} + +int future_new_time_relative(sd_event *e, clockid_t clock, uint64_t usec, uint64_t accuracy, int result, sd_future **ret) { + return future_new_time_internal(sd_event_add_time_relative, e, clock, usec, accuracy, result, ret); +} diff --git a/src/libsystemd/sd-event/event-future.h b/src/libsystemd/sd-event/event-future.h new file mode 100644 index 0000000000..7e956906eb --- /dev/null +++ b/src/libsystemd/sd-event/event-future.h @@ -0,0 +1,7 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +#pragma once + +#include "sd-forward.h" + +int future_new_time(sd_event *e, clockid_t clock, uint64_t usec, uint64_t accuracy, int result, sd_future **ret); +int future_new_time_relative(sd_event *e, clockid_t clock, uint64_t usec, uint64_t accuracy, int result, sd_future **ret); diff --git a/src/libsystemd/sd-future/fiber.c b/src/libsystemd/sd-future/fiber.c new file mode 100644 index 0000000000..48d26a27dd --- /dev/null +++ b/src/libsystemd/sd-future/fiber.c @@ -0,0 +1,812 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#if HAVE_VALGRIND_VALGRIND_H +#include +#endif + +#include "sd-event.h" +#include "sd-future.h" + +#include "alloc-util.h" +#include "architecture.h" +#include "errno-util.h" +#include "event-future.h" +#include "log-context.h" +#include "log.h" +#include "memory-util.h" +#include "pthread-util.h" +#include "time-util.h" + +#if HAS_FEATURE_ADDRESS_SANITIZER +#include +#endif + +/* glibc's _FORTIFY_SOURCE wraps siglongjmp() with __longjmp_chk, which asserts that the target SP is below + * the current SP. That assumption is incompatible with fiber switching, where the target SP lives on a + * separately-mmap'd stack and can be at any address relative to the caller. The fortify redirect happens + * in 's declaration of siglongjmp; we sidestep it by declaring our own alias that links + * directly to the unchecked "siglongjmp" symbol. musl doesn't fortify setjmp.h, so the alias is a plain + * synonym there. */ +_noreturn_ extern void siglongjmp_unchecked(sigjmp_buf env, int val) __asm__("siglongjmp"); + +static thread_local Fiber *current_fiber = NULL; + +typedef enum FiberState { + FIBER_STATE_INITIAL, + FIBER_STATE_READY, + FIBER_STATE_SUSPENDED, + FIBER_STATE_CANCELLED, + FIBER_STATE_COMPLETED, + _FIBER_STATE_MAX, + _FIBER_STATE_INVALID = -EINVAL, +} FiberState; + +typedef struct Fiber { + struct iovec stack; + sigjmp_buf context; /* Where to jump to when entering or resuming the fiber. */ + sigjmp_buf resume_context; /* Where to jump back to when the fiber yields or completes. */ + + /* Caller's stack range, recorded by fiber_run() on each entry so the fiber's siglongjmp back + * out (in fiber_swap() or the trampoline's terminate path) can hand AddressSanitizer the + * destination stack info. With ucontext this comes for free via uc_link/uc_stack; sigjmp_buf + * is opaque and doesn't carry it. */ + struct iovec resume_stack; + + FiberState state; + int result; /* Either resume error code or final return value */ + + sd_future *floating; /* Self-ref held while the fiber is floating; dropped on resolve. */ + + sd_event *event; + sd_event_source *defer_event_source; + sd_event_source *exit_event_source; + + char *name; + int64_t priority; + sd_fiber_func_t func; + void *userdata; + sd_fiber_destroy_t destroy; + + /* Storage for the swap performed in fiber_run(): while the fiber is suspended these hold the + * fiber's own log state; while it is running they hold the caller's log state. The active state + * always lives in the thread-locals in log.c / log-context.c. */ + LIST_HEAD(LogContext, log_context); + size_t log_context_num_fields; + const char *log_prefix; + +#if HAVE_VALGRIND_VALGRIND_H + unsigned stack_id; +#endif +} Fiber; + +static Fiber* fiber_get_current(void) { + return current_fiber; +} + +static void fiber_set_current(Fiber *f) { + current_fiber = f; +} + +static int fiber_allocate_stack(size_t size, void **ret) { + void *stack = NULL; + int r; + + /* The effective stack size is one page less than the given size, because we have to use + * one page as the guard page for the stack. */ + + assert(size > 0 && size % page_size() == 0); + assert(ret); + + stack = mmap(/* addr= */ NULL, size, + PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS | MAP_STACK, + /* fd= */ -EBADF, /* offset= */ 0); + if (stack == MAP_FAILED) + return -errno; + + /* Place the guard page where stack overflow will hit it: the high end on architectures + * where the stack grows up (PA-RISC), the low end everywhere else. fiber_stack_usable() + * mirrors this with the inverse offset. */ + void *guard = STACK_GROWS_UP ? (uint8_t*) stack + size - page_size() : stack; + + /* Prefer MADV_GUARD_INSTALL (Linux 6.13+): unlike mprotect(PROT_NONE) it doesn't split + * the VMA, so guard installation skips the mmap-lock contention and per-guard VMA cost. + * Fall back to mprotect on older kernels, which return EINVAL for unknown advice. + * FIXME: delete when baseline above 6.13. */ + r = RET_NERRNO(madvise(guard, page_size(), MADV_GUARD_INSTALL)); + if (r == -EINVAL) + r = RET_NERRNO(mprotect(guard, page_size(), PROT_NONE)); + if (r < 0) { + (void) munmap(stack, size); + return r; + } + + *ret = TAKE_PTR(stack); + return 0; +} + +/* Usable stack range of a fiber: the full mmap region minus the guard page. Single source of + * truth for the layout assumed by fiber_allocate_stack(); every consumer (ucontext ss_sp, + * ASAN handoff iovecs, Valgrind stack registration) goes through here. + * + * iov_base is the lowest usable byte regardless of growth direction — that matches POSIX's + * definition of stack_t.ss_sp, so libc's makecontext() handles the direction for us. Only the + * guard page placement (and hence iov_base's offset within the mapping) varies. */ +static struct iovec fiber_stack_usable(const struct iovec *stack) { + assert(stack); + assert(stack->iov_len > page_size()); + return (struct iovec) { + .iov_base = STACK_GROWS_UP ? stack->iov_base : (uint8_t*) stack->iov_base + page_size(), + .iov_len = stack->iov_len - page_size(), + }; +} + +static inline void start_switch_stack(void **fake_stack_save, const struct iovec *dest) { +#if HAS_FEATURE_ADDRESS_SANITIZER + __sanitizer_start_switch_fiber(fake_stack_save, + dest ? dest->iov_base : NULL, + dest ? dest->iov_len : 0); +#endif +} + +static inline void finish_switch_stack(void *fake_stack_save) { +#if HAS_FEATURE_ADDRESS_SANITIZER + __sanitizer_finish_switch_fiber(fake_stack_save, NULL, NULL); +#endif +} + +/* Refresh f->resume_stack from whoever is currently the running fiber, so the next siglongjmp() out + * of f (in the trampoline or fiber_swap()) can hand the right destination stack to ASAN. Must be + * called before fiber_set_current(f) — relies on fiber_get_current() returning the caller. */ +static void fiber_set_resume_stack(Fiber *f, Fiber *resume) { + assert(f); + + if (resume) + f->resume_stack = fiber_stack_usable(&resume->stack); + else + f->resume_stack = (struct iovec) {}; +} + +_noreturn_ static void fiber_entry_point(void) { + Fiber *f = ASSERT_PTR(fiber_get_current()); + void *fake_stack_save = NULL; + + assert(f->func); + assert(IN_SET(f->state, FIBER_STATE_INITIAL, FIBER_STATE_READY, FIBER_STATE_CANCELLED)); + + finish_switch_stack(NULL); + + /* Capture our resumable point on the fiber's stack, then bounce back to whoever last set + * f->resume_context. On bootstrap that's fiber_bootstrap(); on every subsequent yield it's + * the most recent fiber_run(). sigsetjmp(buf, 0) skips the signal-mask save: switching is + * thread-shared with respect to signal masks. */ + if (sigsetjmp(f->context, /* savemask= */ 0) == 0) { + start_switch_stack(&fake_stack_save, &f->resume_stack); + siglongjmp_unchecked(f->resume_context, /* val= */ 1); + } + + /* Re-entered for real via fiber_run()'s siglongjmp(f->context). */ + finish_switch_stack(fake_stack_save); + + /* Block scope so the cleanups attached to LOG_SET_PREFIX / LOG_CONTEXT_PUSH_KEY_VALUE fire + * before the siglongjmp below — siglongjmp skips _cleanup_ attributes, so we have to make + * sure the scope ends via a normal control-flow path first. */ + { + LOG_SET_PREFIX(f->name); + LOG_CONTEXT_PUSH_KEY_VALUE("FIBER=", f->name); + + f->result = f->state == FIBER_STATE_CANCELLED ? -ECANCELED : f->func(f->userdata); + f->state = FIBER_STATE_COMPLETED; + } + + /* Pass NULL fake_stack_save to discard the fiber's fake stack since the fiber is done. */ + start_switch_stack(NULL, &f->resume_stack); + + /* Bounce back to whichever fiber_run() call most recently entered us. resume_context is + * per-fiber so nested fiber_run() — e.g. a bus method dispatched as a fiber handler while + * sd_event_loop() itself runs in a fiber — is safe. */ + siglongjmp_unchecked(f->resume_context, 1); + assert_not_reached(); +} + +static int fiber_init(Fiber *f) { + ucontext_t old_uc, uc; + void *fake_stack_save = NULL; + + assert(f); + + if (getcontext(&uc) < 0) + return -errno; + + struct iovec fiber_stack = fiber_stack_usable(&f->stack); + + uc.uc_link = NULL; /* Unused: trampoline siglongjmps out instead of returning. */ + uc.uc_stack.ss_sp = fiber_stack.iov_base; + uc.uc_stack.ss_size = fiber_stack.iov_len; + uc.uc_stack.ss_flags = 0; + + Fiber *prev = fiber_get_current(); + fiber_set_current(f); + + makecontext(&uc, fiber_entry_point, /* argc= */ 0); + + fiber_set_resume_stack(f, prev); + if (sigsetjmp(f->resume_context, /* savemask= */ 0) == 0) { + start_switch_stack(&fake_stack_save, &fiber_stack); + if (swapcontext(&old_uc, &uc) < 0) { + finish_switch_stack(fake_stack_save); + fiber_set_current(prev); + return -errno; + } + assert_not_reached(); /* Trampoline siglongjmps back; swapcontext doesn't return. */ + } + + finish_switch_stack(fake_stack_save); + + fiber_set_current(prev); + return 0; +} + +/* Swap the thread-local log prefix and log context with the values stashed in f. While the fiber is + * suspended, f holds the fiber's own log state; while it's running, f holds the caller's log state. The + * swap is its own inverse, so the same call drives both directions. */ +static void fiber_swap_log_state(Fiber *f) { + assert(f); + log_prefix_swap(&f->log_prefix); + log_context_swap(&f->log_context, &f->log_context_num_fields); +} + +static void reset_current_fiber(void) { + /* Restore the caller's log state stashed in the running fiber (if any) before clearing + * current_fiber. Without this, the child of a fork() that happened mid-fiber would inherit the + * fiber's log prefix / context list in its thread-locals even though no fiber is running. */ + Fiber *f = fiber_get_current(); + if (f) + fiber_swap_log_state(f); + fiber_set_current(NULL); +} + +static sd_event_source* fiber_current_event_source(Fiber *f) { + assert(f); + assert(f->state != FIBER_STATE_COMPLETED); + assert(f->event); + + return sd_event_get_state(f->event) == SD_EVENT_EXITING ? f->exit_event_source : f->defer_event_source; +} + +static int atfork_ret; + +static void install_atfork(void) { + /* __register_atfork() either returns 0 or -ENOMEM, in its glibc implementation. Since it's + * only half-documented (glibc doesn't document it but LSB does — though only superficially) + * we'll check for errors only in the most generic fashion possible. */ + atfork_ret = pthread_atfork(/* prepare= */ NULL, /* parent= */ NULL, reset_current_fiber); + if (atfork_ret != 0) + log_debug_errno(atfork_ret, "pthread_atfork() failed: %m"); +} + +static void fiber_resolve(sd_future *f) { + Fiber *fiber = ASSERT_PTR(sd_future_get_private(ASSERT_PTR(f))); + + fiber->defer_event_source = sd_event_source_disable_unref(fiber->defer_event_source); + fiber->exit_event_source = sd_event_source_disable_unref(fiber->exit_event_source); + /* The floating self-ref (if any) is potentially the last ref keeping the fiber alive — moving it + * into a local _cleanup_ slot ensures sd_future_resolve() runs callbacks and waiters while f is + * still valid; the local's cleanup drops the ref afterwards, at which point no further f->... + * access can happen. */ + _unused_ _cleanup_(sd_future_unrefp) sd_future *floating = TAKE_PTR(fiber->floating); + sd_future_resolve(f, fiber->result); +} + +static void fiber_enter(Fiber *fiber, Fiber *prev, void **fake_stack_save) { + fiber_set_current(fiber); + fiber_swap_log_state(fiber); + + struct iovec fiber_stack = fiber_stack_usable(&fiber->stack); + start_switch_stack(fake_stack_save, &fiber_stack); + fiber_set_resume_stack(fiber, prev); +} + +static void fiber_leave(Fiber *fiber, Fiber *prev, void *fake_stack_save) { + finish_switch_stack(fake_stack_save); + fiber_swap_log_state(fiber); + fiber_set_current(prev); +} + +static int fiber_run(sd_future *f) { + Fiber *fiber = ASSERT_PTR(sd_future_get_private(ASSERT_PTR(f))); + int r; + + if (fiber->state == FIBER_STATE_COMPLETED) + return -ESTALE; + + assert(IN_SET(fiber->state, FIBER_STATE_INITIAL, FIBER_STATE_READY, FIBER_STATE_CANCELLED)); + + static pthread_once_t atfork_once = PTHREAD_ONCE_INIT; + r = pthread_once(&atfork_once, install_atfork); + if (r != 0) + return -r; + if (atfork_ret != 0) + return -atfork_ret; + + LOG_SET_PREFIX(fiber->name); + LOG_CONTEXT_PUSH_KEY_VALUE("FIBER=", fiber->name); + + log_debug("Scheduling fiber"); + + /* Save the previously-current fiber (if any) so we can restore it when this fiber yields or + * completes. This matters when fiber_run() is invoked from within another fiber (e.g. an + * sd-event dispatch that happens to be running inside a fiber context itself): the + * LOG_SET_PREFIX/LOG_CONTEXT_PUSH above attached to whichever fiber was current at that moment, + * and their scope-level cleanup must see the same fiber_get_current() when it runs to detach + * them from the correct list. */ + Fiber *prev = fiber_get_current(); + void *fake_stack_save = NULL; + fiber_enter(fiber, prev, &fake_stack_save); + + /* This is where we start executing the fiber. Once it yields, we continue here as if nothing + * happened. resume_context captures this point; the fiber siglongjmps back to it. */ + if (sigsetjmp(fiber->resume_context, 0) == 0) + siglongjmp_unchecked(fiber->context, 1); + + fiber_leave(fiber, prev, fake_stack_save); + + switch (fiber->state) { + + case FIBER_STATE_COMPLETED: + if (fiber->result < 0 && fiber->result != -ECANCELED) + log_debug_errno(fiber->result, "Fiber failed with error: %m"); + else + log_debug("Fiber finished executing"); + + fiber_resolve(f); + break; + + case FIBER_STATE_CANCELLED: + case FIBER_STATE_READY: + log_debug("Fiber yielded execution"); + + r = sd_event_source_set_enabled(fiber_current_event_source(fiber), SD_EVENT_ONESHOT); + if (r < 0) + return r; + break; + + case FIBER_STATE_SUSPENDED: + log_debug("Fiber suspended execution"); + /* Fiber is waiting for something - don't re-queue it */ + break; + + default: + assert_not_reached(); + } + + return 0; +} + +static int fiber_cancel(sd_future *f) { + Fiber *fiber = ASSERT_PTR(sd_future_get_private(ASSERT_PTR(f))); + int r; + + assert(fiber != fiber_get_current()); + + if (IN_SET(fiber->state, FIBER_STATE_COMPLETED, FIBER_STATE_CANCELLED)) + return 0; + + if (fiber->state == FIBER_STATE_INITIAL) { + /* The fiber's stack was allocated but never entered, so there are no scope-level cleanups + * waiting to run. Skip the dispatch round-trip that would just have fiber_entry_point() + * fall straight through with -ECANCELED, and settle the future right here — mirroring the + * FIBER_STATE_COMPLETED branch of fiber_run(). */ + fiber->result = -ECANCELED; + fiber->state = FIBER_STATE_COMPLETED; + fiber_resolve(f); + return 1; + } + + /* Once we cancel a fiber, we want to immediately resume it with -ECANCELED. */ + r = sd_event_source_set_enabled(fiber_current_event_source(fiber), SD_EVENT_ONESHOT); + if (r < 0) + return r; + + fiber->state = FIBER_STATE_CANCELLED; + + return 1; +} + +static int fiber_on_defer(sd_event_source *s, void *userdata) { + sd_future *f = ASSERT_PTR(userdata); + return fiber_run(f); +} + +static int fiber_on_exit(sd_event_source *s, void *userdata) { + sd_future *f = ASSERT_PTR(userdata); + Fiber *fiber = ASSERT_PTR(sd_future_get_private(f)); + int r; + + /* The fiber may already have completed via the regular defer path before sd_event_exit() + * fires the exit source; in that case there's nothing left to drive and we'd otherwise + * trip fiber_run()'s -ESTALE return, which sd_event would log spuriously and disable the + * source for. */ + if (fiber->state == FIBER_STATE_COMPLETED) + return 0; + + /* If fiber_cancel() returned 1 the fiber was just marked cancelled and its deferred/exit event + * source was re-armed; we let the event loop dispatch that source on the next iteration so it goes + * through the normal fiber_on_defer/fiber_on_exit path rather than running it recursively here. */ + r = fiber_cancel(f); + if (r != 0) + return r; + + return fiber_run(f); +} + +static void* fiber_alloc(void) { + return new0(Fiber, 1); +} + +static void fiber_free(sd_future *f) { + Fiber *fiber = ASSERT_PTR(sd_future_get_private(f)); + + /* To make sure all memory is deallocated, the fiber has to have completed by the time we free it to + * make sure its stack has finished unwinding (which will invoke the registered cleanup functions). + * As this function may get called when not running on a fiber ourselves, we can't guarantee here + * that we can run the fiber to completion ourselves, so we insist that this happens before we get + * here. To ensure fibers are cleaned up before exiting the event loop, exit handlers are added for + * fibers created outside of existing fibers. For fibers created within running fibers, unwinding the + * outer fiber should take care of cleaning up any created child fibers (for example using + * sd_future_cancel_wait_unref()). + * + * FIBER_STATE_INITIAL is also accepted: the stack was allocated but never entered, so there are no + * registered cleanups to run. This covers the partial-construction failure path in sd_fiber_new() + * as well as fibers that are unrefed before the event loop ever dispatches them. */ + assert(IN_SET(fiber->state, FIBER_STATE_INITIAL, FIBER_STATE_COMPLETED)); + + if (fiber->destroy) + fiber->destroy(fiber->userdata); + +#if HAVE_VALGRIND_VALGRIND_H + if (fiber->stack.iov_base) + VALGRIND_STACK_DEREGISTER(fiber->stack_id); +#endif + + if (fiber->stack.iov_base) + (void) munmap(fiber->stack.iov_base, fiber->stack.iov_len); + + sd_event_source_disable_unref(fiber->defer_event_source); + sd_event_source_disable_unref(fiber->exit_event_source); + sd_event_unref(fiber->event); + + free(fiber->name); + free(fiber); +} + +sd_future* sd_fiber_get_current(void) { + Fiber *f = fiber_get_current(); + if (!f) + return NULL; + + return sd_event_source_get_userdata(fiber_current_event_source(f)); +} + +int sd_fiber_is_running(void) { + return !!fiber_get_current(); +} + +sd_event* sd_fiber_get_event(void) { + Fiber *f = fiber_get_current(); + assert_return(f, NULL); + return f->event; +} + +int sd_fiber_get_priority(int64_t *ret) { + Fiber *f = fiber_get_current(); + + assert_return(ret, -EINVAL); + assert_return(f, -ESRCH); + + *ret = f->priority; + return 0; +} + +static int fiber_swap(FiberState state) { + Fiber *f = ASSERT_PTR(fiber_get_current()); + + f->state = state; + + void *fake_stack_save = NULL; + + if (sigsetjmp(f->context, 0) == 0) { + start_switch_stack(&fake_stack_save, &f->resume_stack); + siglongjmp_unchecked(f->resume_context, 1); + } + + finish_switch_stack(fake_stack_save); + + /* When we get here, we've been resumed. */ + + if (f->state == FIBER_STATE_CANCELLED) + return -ECANCELED; + + /* sd_fiber_resume() stashes the resumer's value (an async wakeup error from a deadline + * timer, an io_uring CQE result, etc.) into f->result for us to surface here. Consume it + * unconditionally so it doesn't pollute subsequent suspends or the fiber's eventual return + * value — both negative errors and positive payloads (byte counts, accepted fds, revents + * masks) are valid resume values. */ + return TAKE_GENERIC(f->result, int, 0); +} + +int sd_fiber_yield(void) { + assert_return(fiber_get_current(), -ESRCH); + return fiber_swap(FIBER_STATE_READY); +} + +int sd_fiber_suspend(void) { + assert_return(fiber_get_current(), -ESRCH); + return fiber_swap(FIBER_STATE_SUSPENDED); +} + +static int fiber_set_priority(sd_future *f, int64_t priority) { + Fiber *fiber = ASSERT_PTR(sd_future_get_private(ASSERT_PTR(f))); + int r = 0; + + if (fiber->defer_event_source) + RET_GATHER(r, sd_event_source_set_priority(fiber->defer_event_source, priority)); + + if (fiber->exit_event_source) + RET_GATHER(r, sd_event_source_set_priority(fiber->exit_event_source, priority)); + + if (r >= 0) + fiber->priority = priority; + + return r; +} + +static const sd_future_ops fiber_future_ops; + +int sd_fiber_resume(sd_future *f, int result) { + assert_return(f, -EINVAL); + assert_return(sd_future_get_ops(f) == &fiber_future_ops, -EINVAL); + + Fiber *fiber = ASSERT_PTR(sd_future_get_private(f)); + + if (fiber->state != FIBER_STATE_SUSPENDED) + return 0; + + /* Stash the result so fiber_swap() returns it from sd_fiber_suspend(). */ + fiber->result = result; + fiber->state = FIBER_STATE_READY; + return sd_event_source_set_enabled(fiber_current_event_source(fiber), SD_EVENT_ONESHOT); +} + +/* The fiber_future ops pass the Fiber pointer through as the future's private state. The fiber resolves + * its own future once it finishes running, so fiber_cancel() intentionally does not resolve. */ +static const sd_future_ops fiber_future_ops = { + .size = sizeof(sd_future_ops), + .alloc = fiber_alloc, + .free = fiber_free, + .cancel = fiber_cancel, + .set_priority = fiber_set_priority, +}; + +int sd_fiber_new(sd_event *e, const char *name, sd_fiber_func_t func, void *userdata, sd_fiber_destroy_t destroy, sd_future **ret) { + int r; + + assert_return(e, -EINVAL); + assert_return(name, -EINVAL); + assert_return(func, -EINVAL); + + if (IN_SET(sd_event_get_state(e), SD_EVENT_EXITING, SD_EVENT_FINISHED)) + return -ECANCELED; + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + r = sd_future_new(&fiber_future_ops, &f); + if (r < 0) + return r; + + Fiber *fiber = ASSERT_PTR(sd_future_get_private(f)); + + struct rlimit rl = { RLIM_INFINITY, RLIM_INFINITY }; + if (getrlimit(RLIMIT_STACK, &rl) < 0) + log_debug_errno(errno, "Reading RLIMIT_STACK failed, ignoring: %m"); + if (rl.rlim_cur == RLIM_INFINITY) + rl.rlim_cur = 8U * U64_MB; /* Same as the default thread stack size */ + + /* Reserve room for the guard page so the usable region stays above PTHREAD_STACK_MIN, which + * is what libc/pthread routines (e.g. TLS setup on musl) assume. */ + size_t stack_len = ROUND_UP(rl.rlim_cur, page_size()); + if (stack_len < (size_t) PTHREAD_STACK_MIN + page_size()) + stack_len = ROUND_UP((size_t) PTHREAD_STACK_MIN + page_size(), page_size()); + + *fiber = (Fiber) { + .stack.iov_len = stack_len, + .state = FIBER_STATE_INITIAL, + .name = strdup(name), + .func = func, + .userdata = userdata, + .event = sd_event_ref(e), + }; + if (!fiber->name) + return -ENOMEM; + + r = fiber_allocate_stack(fiber->stack.iov_len, &fiber->stack.iov_base); + if (r < 0) + return r; + +#if HAVE_VALGRIND_VALGRIND_H + /* Register the usable stack range (above the guard page) before fiber_bootstrap() so the + * trampoline's first sigsetjmp doesn't trip Valgrind's stack-tracking heuristics. */ + struct iovec usable = fiber_stack_usable(&fiber->stack); + fiber->stack_id = VALGRIND_STACK_REGISTER( + usable.iov_base, + (uint8_t*) usable.iov_base + usable.iov_len); +#endif + + r = fiber_init(fiber); + if (r < 0) + return r; + + /* Execution of the fiber is driven by two event sources, one deferred, one exit. The exit event + * source kicks in when sd_event_exit() is called, as from that point onwards only exit event + * sources will be dispatched. */ + + r = sd_event_add_defer(e, &fiber->defer_event_source, fiber_on_defer, f); + if (r < 0) + return r; + + r = sd_event_source_set_description(fiber->defer_event_source, fiber->name); + if (r < 0) + return r; + + r = sd_event_add_exit(e, &fiber->exit_event_source, fiber_on_exit, f); + if (r < 0) + return r; + + r = sd_event_source_set_description(fiber->exit_event_source, fiber->name); + if (r < 0) + return r; + + /* If we're on a fiber, we'll rely on the parent fiber to cancel this fiber if the event loop is + * exiting. Otherwise, we'll trigger cancellation of this fiber via the exit event source. Why cancel + * via the exit event source? We can only run the fiber while the event loop is active, so we need to + * make sure all fibers finish running before the event loop is finished, which an exit event source + * allows us to do. */ + r = sd_event_source_set_enabled(fiber->exit_event_source, sd_fiber_is_running() ? SD_EVENT_OFF : SD_EVENT_ONESHOT); + if (r < 0) + return r; + + /* Stays in FIBER_STATE_INITIAL until the event loop first dispatches it via fiber_run(). */ + + if (ret) + *ret = TAKE_PTR(f); + else { + /* Fire-and-forget: the fiber is guaranteed to resolve (via completion, cancellation, or + * the event loop exit handler), so making the future floating cleans it up. */ + r = sd_fiber_set_floating(f, true); + if (r < 0) + return r; + } + + /* We only take ownership of the given userdata pointer on success so assign the destroy callback + * at the very end so we don't clean up the userdata pointer on failure. */ + fiber->destroy = destroy; + + return 0; +} + +int sd_fiber_set_floating(sd_future *f, int b) { + assert_return(f, -EINVAL); + assert_return(sd_future_get_ops(f) == &fiber_future_ops, -EINVAL); + + Fiber *fiber = ASSERT_PTR(sd_future_get_private(f)); + + if (!!fiber->floating == !!b) + return 0; + + /* The floating self-ref keeps the future alive until the fiber resolves; fiber_run() drops it + * in the COMPLETED branch. Only valid for fiber futures because fibers uniquely guarantee + * resolution (via completion, cancellation, or the event loop exit handler). */ + if (b) + fiber->floating = sd_future_ref(f); + else + fiber->floating = sd_future_unref(fiber->floating); + + return 0; +} + +int sd_fiber_get_floating(sd_future *f) { + assert_return(f, -EINVAL); + assert_return(sd_future_get_ops(f) == &fiber_future_ops, -EINVAL); + + Fiber *fiber = ASSERT_PTR(sd_future_get_private(f)); + return !!fiber->floating; +} + +int sd_fiber_sleep(uint64_t usec) { + Fiber *f = fiber_get_current(); + int r; + + if (!f) + return usleep_safe(usec); + + if (usec == 0) + return sd_fiber_yield(); + + /* Match usleep_safe(USEC_INFINITY): suspend indefinitely. Passing USEC_INFINITY to + * sd_event_add_time_relative() would overflow into -EOVERFLOW. */ + if (usec == USEC_INFINITY) + return sd_fiber_suspend(); + + assert(f->event); + + _cleanup_(sd_future_cancel_wait_unrefp) sd_future *timer = NULL; + r = future_new_time_relative( + f->event, + CLOCK_MONOTONIC, + usec, + /* accuracy= */ 1, + /* result= */ 0, + &timer); + if (r < 0) + return r; + + return sd_fiber_suspend(); +} + +int sd_fiber_await(sd_future *target) { + sd_future *f = sd_fiber_get_current(); + int r; + + assert_return(f, -ESRCH); + assert_return(target, -EINVAL); + assert_return(target != f, -EDEADLK); + + Fiber *fiber = ASSERT_PTR(sd_future_get_private(f)); + + if (sd_future_state(target) == SD_FUTURE_RESOLVED) + return sd_future_result(target); + + /* Note that we do allow waiting for other fibers when the event loop is exiting, since waiting for + * other fibers does not require adding new event sources to the event loop. */ + if (sd_event_get_state(fiber->event) == SD_EVENT_FINISHED) + return -ECANCELED; + + _cleanup_(sd_future_cancel_wait_unrefp) sd_future *wait = NULL; + r = sd_future_new_wait(target, &wait); + if (r < 0) + return r; + + return sd_fiber_suspend(); +} + +sd_future* sd_fiber_timeout(uint64_t timeout) { + Fiber *fiber = fiber_get_current(); + int r; + + assert_return(fiber, NULL); + + if (timeout == USEC_INFINITY) + return NULL; + + sd_future *timer; + r = future_new_time_relative( + fiber->event, + CLOCK_MONOTONIC, + timeout, + /* accuracy= */ 1, + /* result= */ -ETIME, + &timer); + if (r < 0) + return NULL; /* On allocation failure no timer is armed and the scope becomes a no-op. + * Errors here are rare; if the caller cares they can compare to NULL. */ + + return timer; +} diff --git a/src/libsystemd/sd-future/sd-future.c b/src/libsystemd/sd-future/sd-future.c new file mode 100644 index 0000000000..ba3a1c52d2 --- /dev/null +++ b/src/libsystemd/sd-future/sd-future.c @@ -0,0 +1,263 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ + +#include "sd-future.h" + +#include "alloc-util.h" +#include "errno-util.h" +#include "log.h" +#include "macro.h" +#include "set.h" + +struct sd_future { + unsigned n_ref; + + int state; + int result; + + Set *waiters; + + sd_future_func_t callback; + void *userdata; + + const sd_future_ops *ops; + + /* Opaque per-future state owned by the future implementation (the code that called + * sd_future_new()). The ops vtable above receives this pointer in its callbacks, and + * external code can fetch it via sd_future_get_private(). */ + void *private; +}; + +static int fiber_resume_trampoline(sd_future *f) { + /* The future's result is what the fiber should resume with. Impls choose the value at + * resolution time — e.g. a deadline timer resolves with -ETIME, a wait future resolves + * with the target's result, a normal IO/sleep future resolves with 0 on success. */ + return sd_fiber_resume(sd_future_get_userdata(f), sd_future_result(f)); +} + +int sd_future_resolve(sd_future *f, int result) { + int r = 0; + + assert_return(f, -EINVAL); + + if (f->state != SD_FUTURE_PENDING) + return 0; + + /* Hold a self-ref across callback/waiter dispatch: callbacks (e.g. bus_fiber_resolved() + * dropping the tracking-set's ref) may legitimately release what would otherwise be the + * last reference, and we still access f->waiters below. The cleanup unrefs at scope exit, + * which is when freeing is safe again. */ + _unused_ _cleanup_(sd_future_unrefp) sd_future *self = sd_future_ref(f); + + f->state = SD_FUTURE_RESOLVED; + f->result = result; + + if (f->callback) + RET_GATHER(r, f->callback(f)); + + /* We'd like the set to not be modified while iterating over it, hence take ownership over it in + * a local variable. Otherwise code invoked via sd_future_resolve() could try to modify the set while + * we're iterating over it (for example wait_future_free()). */ + Set *waiters = TAKE_PTR(f->waiters); + sd_future *w; + SET_FOREACH(w, waiters) + RET_GATHER(r, sd_future_resolve(w, result)); + + set_free(waiters); + + return r; +} + +static sd_future* sd_future_free(sd_future *f) { + if (!f) + return NULL; + + if (f->state == SD_FUTURE_PENDING) + sd_future_resolve(f, -ECANCELED); + + set_free(f->waiters); + + if (f->ops->free) + f->ops->free(f); + + return mfree(f); +} + +DEFINE_TRIVIAL_REF_UNREF_FUNC(sd_future, sd_future, sd_future_free); +DEFINE_POINTER_ARRAY_CLEAR_FUNC(sd_future*, sd_future_unref); +DEFINE_POINTER_ARRAY_FREE_FUNC(sd_future*, sd_future_unref); + +sd_future* sd_future_cancel_wait_unref(sd_future *f) { + int r; + + if (!f) + return NULL; + + /* We have to be able to suspend until the fiber we're waiting for finishes, and that's only + * possible if we're running on a fiber ourselves. */ + if (!sd_fiber_is_running()) + return sd_future_unref(f); + + r = sd_future_cancel(f); + if (r < 0) + log_debug_errno(r, "Failed to cancel future, ignoring: %m"); + + if (f->state == SD_FUTURE_PENDING) { + /* Fast path: when f's resolve callback already targets the current fiber (the default for + * futures created on this fiber), we can suspend directly and let the existing trampoline + * wake us up — no need to allocate a wait future just to learn about the resolution. + * Otherwise fall back to sd_fiber_await() which sets up an explicit waiter. */ + if (f->callback == fiber_resume_trampoline && f->userdata == sd_fiber_get_current()) + r = sd_fiber_suspend(); + else + r = sd_fiber_await(f); + if (r < 0 && r != -ECANCELED) + log_debug_errno(r, "Failed to wait for future to finish, ignoring: %m"); + } + + return sd_future_unref(f); +} + +DEFINE_POINTER_ARRAY_CLEAR_FUNC(sd_future*, sd_future_cancel_wait_unref); +DEFINE_POINTER_ARRAY_FREE_FUNC(sd_future*, sd_future_cancel_wait_unref); + +int sd_future_new(const sd_future_ops *ops, sd_future **ret) { + assert_return(ops, -EINVAL); + assert_return(ops->size >= endoffsetof_field(sd_future_ops, set_priority), -EINVAL); + assert_return(ops->alloc, -EINVAL); + assert_return(ops->free, -EINVAL); + assert_return(ret, -EINVAL); + + sd_future *f = new(sd_future, 1); + if (!f) + return -ENOMEM; + + *f = (sd_future) { + .n_ref = 1, + .state = SD_FUTURE_PENDING, + .ops = ops, + }; + + f->private = ops->alloc(); + if (!f->private) { + free(f); + return -ENOMEM; + } + + /* If we're being created on a fiber, default the callback to resuming that fiber on resolve — + * this is almost always what you want, and it saves the usual set_callback boilerplate before + * sd_fiber_suspend(). Callers that want different behavior can override with + * sd_future_set_callback(). */ + sd_future *fiber = sd_fiber_get_current(); + if (fiber) + (void) sd_future_set_callback(f, fiber_resume_trampoline, fiber); + + *ret = f; + return 0; +} + +int sd_future_state(sd_future *f) { + assert_return(f, -EINVAL); + return f->state; +} + +int sd_future_result(sd_future *f) { + assert_return(f, -EINVAL); + assert_return(f->state == SD_FUTURE_RESOLVED, -EBUSY); + return f->result; +} + +void* sd_future_get_userdata(sd_future *f) { + assert_return(f, NULL); + return f->userdata; +} + +void* sd_future_get_private(sd_future *f) { + assert_return(f, NULL); + return f->private; +} + +const sd_future_ops* sd_future_get_ops(sd_future *f) { + assert_return(f, NULL); + return f->ops; +} + +int sd_future_set_callback(sd_future *f, sd_future_func_t callback, void *userdata) { + assert_return(f, -EINVAL); + + f->callback = callback; + f->userdata = userdata; + return 0; +} + +int sd_future_set_priority(sd_future *f, int64_t priority) { + assert_return(f, -EINVAL); + assert_return(f->state == SD_FUTURE_PENDING, -ESTALE); + assert_return(f->ops->set_priority, -EOPNOTSUPP); + + return f->ops->set_priority(f, priority); +} + +int sd_future_cancel(sd_future *f) { + assert_return(f, -EINVAL); + assert_return(f->ops->cancel, -EOPNOTSUPP); + + if (f->state == SD_FUTURE_RESOLVED) + return 0; + + return f->ops->cancel(f); +} + +typedef struct WaitFuture { + sd_future *target; +} WaitFuture; + +static void* wait_future_alloc(void) { + return new0(WaitFuture, 1); +} + +static void wait_future_free(sd_future *f) { + WaitFuture *wf = ASSERT_PTR(sd_future_get_private(ASSERT_PTR(f))); + + set_remove(wf->target->waiters, f); + sd_future_unref(wf->target); + free(wf); +} + +static int wait_future_cancel(sd_future *f) { + WaitFuture *wf = ASSERT_PTR(sd_future_get_private(ASSERT_PTR(f))); + + set_remove(wf->target->waiters, f); + return sd_future_resolve(f, -ECANCELED); +} + +static const sd_future_ops wait_future_ops = { + .size = sizeof(sd_future_ops), + .alloc = wait_future_alloc, + .free = wait_future_free, + .cancel = wait_future_cancel, +}; + +int sd_future_new_wait(sd_future *target, sd_future **ret) { + int r; + + assert_return(target, -EINVAL); + assert_return(ret, -EINVAL); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + r = sd_future_new(&wait_future_ops, &f); + if (r < 0) + return r; + + WaitFuture *wf = sd_future_get_private(f); + wf->target = sd_future_ref(target); + + if (target->state == SD_FUTURE_RESOLVED) + r = sd_future_resolve(f, target->result); + else + r = set_ensure_put(&target->waiters, &trivial_hash_ops, f); + if (r < 0) + return r; + + *ret = TAKE_PTR(f); + return 0; +} diff --git a/src/libsystemd/sd-future/test-fiber.c b/src/libsystemd/sd-future/test-fiber.c new file mode 100644 index 0000000000..2760b919a0 --- /dev/null +++ b/src/libsystemd/sd-future/test-fiber.c @@ -0,0 +1,1171 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ + +#include + +#if HAVE_VALGRIND_VALGRIND_H +# include +#endif + +#include "sd-event.h" +#include "sd-future.h" + +#include "architecture.h" +#include "log-context.h" +#include "memory-util.h" +#include "pidref.h" +#include "process-util.h" +#include "tests.h" +#include "time-util.h" + +static int simple_fiber(void *userdata) { + int *value = ASSERT_PTR(userdata); + return *value; +} + +TEST(fiber_simple) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + int value = 5; + ASSERT_OK(sd_fiber_new(e, "simple", simple_fiber, &value, NULL, &f)); + ASSERT_OK(sd_event_loop(e)); + ASSERT_EQ(sd_future_result(f), 5); +} + +/* Fiber that yields once */ +static int yielding_fiber(void *userdata) { + int *counter = userdata; + (*counter)++; + + sd_fiber_yield(); + + (*counter)++; + return 0; +} + +/* Test: Single fiber that yields */ +TEST(fiber_single_yield) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + int counter = 0; + ASSERT_OK(sd_fiber_new(e, "yielding", yielding_fiber, &counter, NULL, &f)); + + /* First iteration: fiber runs until first yield */ + ASSERT_EQ(counter, 0); + ASSERT_OK_POSITIVE(sd_event_run(e, 0)); + ASSERT_EQ(counter, 1); + + /* Second iteration: fiber runs from yield to completion */ + ASSERT_OK_POSITIVE(sd_event_run(e, 0)); + ASSERT_EQ(counter, 2); + + /* No more fibers to run */ + ASSERT_OK_ZERO(sd_event_loop(e)); +} + +static int counting_fiber(void *userdata) { + int counter = 0; + + for (int i = 0; i < 5; i++) { + counter++; + sd_fiber_yield(); + } + + return counter; +} + +/* Test: Multiple fibers yielding cooperatively */ +TEST(fiber_multiple_yield) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + sd_future *fibers[5] = {}; + CLEANUP_ELEMENTS(fibers, sd_future_unref_array_clear); + + for (size_t i = 0; i < ELEMENTSOF(fibers); i++) { + _cleanup_free_ char *name = NULL; + ASSERT_OK(asprintf(&name, "counting-%zu", i)); + ASSERT_OK(sd_fiber_new(e, name, counting_fiber, NULL, NULL, &fibers[i])); + } + + ASSERT_OK(sd_event_loop(e)); + + for (size_t i = 0; i < ELEMENTSOF(fibers); i++) + ASSERT_OK_EQ(sd_future_result(fibers[i]), 5); +} + +static int priority_fiber(void *userdata) { + int *counter = ASSERT_PTR(userdata); + + (*counter)++; + sd_fiber_yield(); + + return *counter; +} + +/* Test: Priority-based scheduling */ +TEST(fiber_priority_ascending) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + sd_future *fibers[5] = {}; + CLEANUP_ELEMENTS(fibers, sd_future_unref_array_clear); + int counter = 0; + + for (size_t i = 0; i < ELEMENTSOF(fibers); i++) { + _cleanup_free_ char *name = NULL; + ASSERT_OK(asprintf(&name, "priority-%zu", i)); + ASSERT_OK(sd_fiber_new(e, name, priority_fiber, &counter, NULL, &fibers[i])); + ASSERT_OK(sd_future_set_priority(fibers[i], i)); + } + + ASSERT_OK(sd_event_loop(e)); + + /* The fibers have ascending priorities, so we the first one to run to completion, + * followed by the second one, etc. */ + + for (size_t i = 0; i < ELEMENTSOF(fibers); i++) + ASSERT_EQ(sd_future_result(fibers[i]), (int) i + 1); +} + +TEST(fiber_priority_identical) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + sd_future *fibers[5] = {}; + CLEANUP_ELEMENTS(fibers, sd_future_unref_array_clear); + int counter = 0; + + for (size_t i = 0; i < ELEMENTSOF(fibers); i++) { + _cleanup_free_ char *name = NULL; + ASSERT_OK(asprintf(&name, "priority-%zu", i)); + ASSERT_OK(sd_fiber_new(e, name, priority_fiber, &counter, NULL, &fibers[i])); + } + + ASSERT_OK(sd_event_loop(e)); + + /* The fibers have the same priorities, so we expect all of them to run once first, and then they'll + * all run again another time, so they should all return the same value. */ + + for (size_t i = 0; i < ELEMENTSOF(fibers); i++) + ASSERT_EQ(sd_future_result(fibers[i]), (int) 5); +} + +static int error_fiber(void *userdata) { + return -ENOENT; +} + +TEST(fiber_error_return) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "error", error_fiber, NULL, NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_EQ(sd_future_result(f), -ENOENT); +} + +static int cancel_fiber(void *userdata) { + return sd_fiber_yield(); +} + +TEST(fiber_cancel_basic) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + int value = 42; + ASSERT_OK(sd_fiber_new(e, "cancel", cancel_fiber, &value, NULL, &f)); + + ASSERT_OK(sd_future_cancel(f)); + ASSERT_OK(sd_event_loop(e)); + ASSERT_ERROR(sd_future_result(f), ECANCELED); +} + +static int fiber_that_yields(void *userdata) { + int *yield_count = userdata; + int r; + + for (int i = 0; i < 5; i++) { + (*yield_count)++; + r = sd_fiber_yield(); + if (r < 0) + return r; /* Propagate cancellation error */ + } + + return 0; +} + +/* Test: fiber_yield() returns error when fiber is cancelled externally */ +TEST(fiber_cancel_propagation_via_yield) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + int yield_count = 0; + ASSERT_OK(sd_fiber_new(e, "yielding", fiber_that_yields, &yield_count, NULL, &f)); + + ASSERT_OK_POSITIVE(sd_event_run(e, 0)); + ASSERT_EQ(yield_count, 1); + ASSERT_OK_POSITIVE(sd_event_run(e, 0)); + ASSERT_EQ(yield_count, 2); + + ASSERT_OK(sd_future_cancel(f)); + + ASSERT_OK(sd_event_loop(e)); + + /* sd_fiber should have been cancelled */ + ASSERT_ERROR(sd_future_result(f), ECANCELED); + ASSERT_EQ(yield_count, 2); +} + +/* Test: Cancel a fiber that has already completed */ +TEST(fiber_cancel_completed) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + int value = 42; + ASSERT_OK(sd_fiber_new(e, "simple", simple_fiber, &value, NULL, &f)); + + /* Run the fiber to completion */ + ASSERT_OK(sd_event_loop(e)); + + /* Canceling a completed fiber should be a no-op */ + ASSERT_OK(sd_future_cancel(f)); + ASSERT_EQ(sd_future_result(f), 42); +} + +static int multiple_yield_fiber(void *userdata) { + int *counter = userdata; + int r; + + for (int i = 0; i < 3; i++) { + (*counter)++; + r = sd_fiber_yield(); + if (r < 0) + return r; + } + + return 0; +} + +/* Test: Cancel one fiber among multiple */ +TEST(fiber_cancel_one_of_many) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + sd_future *fibers[3] = {}; + CLEANUP_ELEMENTS(fibers, sd_future_unref_array_clear); + int counters[3] = {0, 0, 0}; + for (size_t i = 0; i < ELEMENTSOF(fibers); i++) + ASSERT_OK(sd_fiber_new(e, "multiple-yield", multiple_yield_fiber, &counters[i], NULL, &fibers[i])); + + /* Run one iteration - all fibers yield after incrementing once */ + ASSERT_OK_POSITIVE(sd_event_run(e, 0)); + ASSERT_OK_POSITIVE(sd_event_run(e, 0)); + ASSERT_OK_POSITIVE(sd_event_run(e, 0)); + ASSERT_EQ(counters[0], 1); + ASSERT_EQ(counters[1], 1); + ASSERT_EQ(counters[2], 1); + + /* Cancel the second fiber */ + ASSERT_OK(sd_future_cancel(fibers[1])); + + /* Run to completion */ + ASSERT_OK(sd_event_loop(e)); + + /* First and third fibers should complete normally */ + ASSERT_EQ(counters[0], 3); + ASSERT_EQ(counters[2], 3); + ASSERT_EQ(sd_future_result(fibers[0]), 0); + ASSERT_EQ(sd_future_result(fibers[2]), 0); + + /* Second fiber should be canceled with counter at 1 */ + ASSERT_EQ(counters[1], 1); + ASSERT_EQ(sd_future_result(fibers[1]), -ECANCELED); +} + +/* Test: sd_fiber_await() - wait for a fiber to complete */ +static int slow_fiber(void *userdata) { + int *counter = userdata; + + for (int i = 0; i < 3; i++) { + (*counter)++; + sd_fiber_yield(); + } + + return 42; +} + +static int waiting_fiber(void *userdata) { + sd_future *target = userdata; + int r; + + r = sd_fiber_await(target); + if (r < 0) + return r; + + r = sd_future_result(target); + return r == 42 ? 0 : -EIO; +} + +TEST(fiber_wait_for_basic) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + /* Create target fiber with lower priority (runs second) */ + _cleanup_(sd_future_unrefp) sd_future *target = NULL, *waiter = NULL; + int counter = 0; + ASSERT_OK(sd_fiber_new(e, "slow", slow_fiber, &counter, NULL, &target)); + ASSERT_OK(sd_future_set_priority(target, 1)); + + /* Create waiter fiber with higher priority (runs first) */ + ASSERT_OK(sd_fiber_new(e, "waiting", waiting_fiber, target, NULL, &waiter)); + ASSERT_OK(sd_future_set_priority(waiter, 0)); + + ASSERT_OK(sd_event_loop(e)); + + ASSERT_OK(sd_future_result(waiter)); + ASSERT_OK_EQ(sd_future_result(target), 42); + ASSERT_EQ(counter, 3); +} + +/* Test: wait for already completed fiber */ +static int wait_for_completed_fiber(void *userdata) { + sd_future *target = userdata; + int r; + + r = sd_fiber_await(target); + if (r < 0) + return r; + + return sd_future_result(target); +} + +TEST(fiber_wait_for_completed) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *target = NULL, *waiter = NULL; + int value = 100; + + /* Create target fiber with higher priority (runs first) */ + ASSERT_OK(sd_fiber_new(e, "simple", simple_fiber, &value, NULL, &target)); + ASSERT_OK(sd_future_set_priority(target, 0)); + /* Create waiter fiber with lower priority (runs second, after target completes) */ + ASSERT_OK(sd_fiber_new(e, "wait-for-completed", wait_for_completed_fiber, target, NULL, &waiter)); + ASSERT_OK(sd_future_set_priority(waiter, 1)); + + ASSERT_OK(sd_event_loop(e)); + + ASSERT_OK_EQ(sd_future_result(waiter), 100); + ASSERT_OK_EQ(sd_future_result(target), 100); +} + +/* Test: awaiting an already-resolved future returns the future's result directly */ +static int await_resolved_fiber(void *userdata) { + sd_future *target = userdata; + + ASSERT_EQ((int) sd_future_state(target), (int) SD_FUTURE_RESOLVED); + ASSERT_OK_EQ(sd_fiber_await(target), 77); + return 0; +} + +TEST(fiber_await_resolved_returns_result) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *target = NULL, *waiter = NULL; + int value = 77; + + /* Higher-priority target runs to completion before the waiter starts. */ + ASSERT_OK(sd_fiber_new(e, "target", simple_fiber, &value, NULL, &target)); + ASSERT_OK(sd_future_set_priority(target, 0)); + ASSERT_OK(sd_fiber_new(e, "await-resolved", await_resolved_fiber, target, NULL, &waiter)); + ASSERT_OK(sd_future_set_priority(waiter, 1)); + + ASSERT_OK(sd_event_loop(e)); + + ASSERT_OK(sd_future_result(waiter)); + ASSERT_OK_EQ(sd_future_result(target), 77); +} + +/* Test: wait for cancelled fiber */ +static int wait_for_cancelled_fiber(void *userdata) { + sd_future *target = userdata; + int r; + + r = sd_fiber_await(target); + if (r < 0) + return r; + + return sd_future_result(target); +} + +TEST(fiber_wait_for_cancelled) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *target = NULL, *waiter = NULL; + int counter = 0; + ASSERT_OK(sd_fiber_new(e, "yielding", fiber_that_yields, &counter, NULL, &target)); + ASSERT_OK(sd_fiber_new(e, "wait-for-cancelled", wait_for_cancelled_fiber, target, NULL, &waiter)); + + ASSERT_OK_POSITIVE(sd_event_run(e, 0)); + ASSERT_OK_POSITIVE(sd_event_run(e, 0)); + + ASSERT_OK(sd_future_cancel(target)); + + ASSERT_OK(sd_event_loop(e)); + + ASSERT_ERROR(sd_future_result(waiter), ECANCELED); + ASSERT_ERROR(sd_future_result(target), ECANCELED); +} + +/* Test: multiple fibers waiting for the same target */ +static int multi_waiter_fiber(void *userdata) { + sd_future *target = userdata; + int r; + + r = sd_fiber_await(target); + if (r < 0) + return r; + + return sd_future_result(target); +} + +TEST(fiber_wait_for_multiple_waiters) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *target = NULL; + int counter = 0; + ASSERT_OK(sd_fiber_new(e, "slow", slow_fiber, &counter, NULL, &target)); + + sd_future *waiters[3] = {}; + CLEANUP_ELEMENTS(waiters, sd_future_unref_array_clear); + for (size_t i = 0; i < ELEMENTSOF(waiters); i++) + ASSERT_OK(sd_fiber_new(e, "multi-waiter", multi_waiter_fiber, target, NULL, &waiters[i])); + + ASSERT_OK(sd_event_loop(e)); + + for (size_t i = 0; i < ELEMENTSOF(waiters); i++) + ASSERT_OK_EQ(sd_future_result(waiters[i]), 42); + + ASSERT_OK_EQ(sd_future_result(target), 42); + ASSERT_EQ(counter, 3); +} + +/* Test: chain of waiting fibers */ +static int chain_waiter_fiber(void *userdata) { + sd_future *target = userdata; + int r; + + r = sd_fiber_await(target); + if (r < 0) + return r; + + r = sd_future_result(target); + return r + 1; +} + +TEST(fiber_wait_for_chain) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + sd_future *fibers[5] = {}; + CLEANUP_ELEMENTS(fibers, sd_future_unref_array_clear); + int value = 10; + + ASSERT_OK(sd_fiber_new(e, "simple", simple_fiber, &value, NULL, &fibers[0])); + + /* Each subsequent fiber waits for the previous and adds 1 */ + for (size_t i = 1; i < ELEMENTSOF(fibers); i++) + ASSERT_OK(sd_fiber_new(e, "chain-waiter", chain_waiter_fiber, fibers[i - 1], NULL, &fibers[i])); + + ASSERT_OK(sd_event_loop(e)); + + /* Check results: 10, 11, 12, 13, 14 */ + for (size_t i = 0; i < ELEMENTSOF(fibers); i++) + ASSERT_OK_EQ(sd_future_result(fibers[i]), 10 + (int) i); +} + +static int nested_run_inner_fiber(void *userdata) { + int *counter = ASSERT_PTR(userdata); + + (*counter)++; + int r = sd_fiber_yield(); + if (r < 0) + return r; + (*counter)++; + + return 0; +} + +static int nested_run_outer_fiber(void *userdata) { + int *counter = ASSERT_PTR(userdata); + _cleanup_(sd_event_unrefp) sd_event *inner = NULL; + _cleanup_(sd_future_unrefp) sd_future *nested = NULL; + int r; + + /* Yield once before the nested loop: this forces the outer fiber to later resume through its own + * siglongjmp back to its resume_context after the inner fiber_run() has executed, which is + * exactly the path that breaks when the resume context is stored thread-globally instead of + * per-fiber. */ + r = sd_fiber_yield(); + if (r < 0) + return r; + + r = sd_event_new(&inner); + if (r < 0) + return r; + + r = sd_event_set_exit_on_idle(inner, true); + if (r < 0) + return r; + + /* Spawn a fiber on the inner event loop. Driving it via sd_event_loop(inner) causes fiber_run() to + * be invoked while we are already executing inside fiber_run() for the outer fiber. */ + r = sd_fiber_new(inner, "inner", nested_run_inner_fiber, counter, NULL, &nested); + if (r < 0) + return r; + + r = sd_event_loop(inner); + if (r < 0) + return r; + + r = sd_future_result(nested); + if (r < 0) + return r; + + /* Yield again after the inner loop has returned. If the outer fiber's resume context was clobbered + * by the nested fiber_run(), the siglongjmp underneath this yield would jump into an already + * unwound stack frame. */ + return sd_fiber_yield(); +} + +TEST(fiber_nested_run) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *outer = NULL; + int counter = 0; + ASSERT_OK(sd_fiber_new(e, "outer", nested_run_outer_fiber, &counter, NULL, &outer)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(outer)); + + /* The inner fiber incremented the counter once before yielding and once after resuming. */ + ASSERT_EQ(counter, 2); +} + +static int nested_current_check_inner_fiber(void *userdata) { + sd_future **slots = ASSERT_PTR(userdata); + + slots[1] = sd_fiber_get_current(); + int r = sd_fiber_yield(); + if (r < 0) + return r; + /* After resuming, the current fiber must still be us, not the outer fiber that was current when + * fiber_run() re-entered. */ + if (sd_fiber_get_current() != slots[1]) + return -EBADF; + + return 0; +} + +static int nested_current_check_outer_fiber(void *userdata) { + sd_future **slots = ASSERT_PTR(userdata); + _cleanup_(sd_event_unrefp) sd_event *inner = NULL; + _cleanup_(sd_future_unrefp) sd_future *nested = NULL; + int r; + + slots[0] = sd_fiber_get_current(); + + r = sd_event_new(&inner); + if (r < 0) + return r; + + r = sd_event_set_exit_on_idle(inner, true); + if (r < 0) + return r; + + r = sd_fiber_new(inner, "inner", nested_current_check_inner_fiber, slots, NULL, &nested); + if (r < 0) + return r; + + r = sd_event_loop(inner); + if (r < 0) + return r; + + r = sd_future_result(nested); + if (r < 0) + return r; + + /* After the nested fiber_run() has returned, the current fiber must have been restored to the + * outer fiber rather than left as NULL or pointing at the (now freed) inner fiber. */ + if (sd_fiber_get_current() != slots[0]) + return -EBADF; + + return 0; +} + +TEST(fiber_nested_run_current_restored) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + sd_future *slots[2] = {}; + _cleanup_(sd_future_unrefp) sd_future *outer = NULL; + ASSERT_OK(sd_fiber_new(e, "outer", nested_current_check_outer_fiber, slots, NULL, &outer)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(outer)); + + ASSERT_NOT_NULL(slots[0]); + ASSERT_NOT_NULL(slots[1]); + ASSERT_TRUE(slots[0] != slots[1]); +} + +static int nested_cancellation_fiber(void *userdata) { + int *counter = ASSERT_PTR(userdata); + _cleanup_(sd_future_cancel_wait_unrefp) sd_future *nested = NULL; + int r; + + if (*counter >= 5) + return sd_fiber_sleep(10 * USEC_PER_SEC); + + (*counter)++; + + _cleanup_free_ char *name = NULL; + if (asprintf(&name, "nested-cancellation-%i", *counter) < 0) + return -ENOMEM; + + /* Create a nested fiber within this fiber */ + r = sd_fiber_new(sd_fiber_get_event(), name, nested_cancellation_fiber, counter, NULL, &nested); + if (r < 0) + return r; + + /* Wait for the nested fiber to complete */ + r = sd_fiber_await(nested); + if (r < 0) + return r; + + /* If we got here without cancellation, verify the nested fiber completed */ + return sd_future_result(nested); +} + +static int exit_loop_fiber(void *userdata) { + /* Just exit the event loop, causing the outer fiber to be cancelled */ + return sd_event_exit(sd_fiber_get_event(), 0); +} + +TEST(fiber_nested_cancellation) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + + int counter = 0; + + /* Create outer fiber with higher priority (runs first) */ + _cleanup_(sd_future_unrefp) sd_future *outer = NULL; + ASSERT_OK(sd_fiber_new(e, "outer", nested_cancellation_fiber, &counter, NULL, &outer)); + + /* Create exit fiber with lower priority (runs after all nested fibers have suspended) */ + _cleanup_(sd_future_unrefp) sd_future *exit_fiber = NULL; + ASSERT_OK(sd_fiber_new(e, "exit-loop", exit_loop_fiber, NULL, NULL, &exit_fiber)); + ASSERT_OK(sd_future_set_priority(exit_fiber, 1)); + + /* Run the event loop - the exit fiber should cause it to exit, + * which should cancel the outer fiber, which should cancel the nested fiber, and so forth. */ + ASSERT_OK(sd_event_loop(e)); + + /* The exit fiber should have completed successfully */ + ASSERT_OK(sd_future_result(exit_fiber)); + + /* The outer fiber should have been cancelled */ + ASSERT_ERROR(sd_future_result(outer), ECANCELED); + + /* The nested fiber was created and incremented counter once before being cancelled */ + ASSERT_GT(counter, 0); +} + +static int nested_fiber_cleanup_nested_fiber(void *userdata) { + int *counter = ASSERT_PTR(userdata); + int r; + + r = sd_fiber_sleep(10 * USEC_PER_SEC); + if (r == -ECANCELED) + (*counter)++; + else if (r < 0) + return r; + + return 0; +} + +static int nested_fiber_cleanup_fiber(void *userdata) { + _cleanup_(sd_future_cancel_wait_unrefp) sd_future *nested = NULL; + int r; + + /* Create a nested fiber within this fiber. */ + r = sd_fiber_new(sd_fiber_get_event(), "nested", nested_fiber_cleanup_nested_fiber, userdata, NULL, &nested); + if (r < 0) + return r; + + /* Yield and then exit, the nested fiber should be cancelled. */ + return sd_fiber_yield(); +} + +TEST(nested_fiber_cleanup) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *outer = NULL; + int counter = 0; + ASSERT_OK(sd_fiber_new(e, "outer", nested_fiber_cleanup_fiber, &counter, NULL, &outer)); + + ASSERT_OK(sd_event_loop(e)); + + /* The outer fiber should have finished normally */ + ASSERT_OK(sd_future_result(outer)); + + /* The nested fiber was created and incremented its counter once when it was cancelled. */ + ASSERT_GT(counter, 0); +} + +static int priority_check_fiber(void *userdata) { + int64_t *ret = ASSERT_PTR(userdata); + + /* Verify that sd_fiber_get_priority() returns the value set via sd_future_set_priority() */ + ASSERT_OK(sd_fiber_get_priority(ret)); + + /* Exercise sd_fiber_sleep() which internally creates a time future. This verifies that the priority + * is correctly propagated to the time event source (via f->time.source, not f->io.source). */ + return sd_fiber_sleep(1); +} + +TEST(fiber_priority_get) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + int64_t got_priority = 0; + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "priority-check", priority_check_fiber, &got_priority, NULL, &f)); + ASSERT_OK(sd_future_set_priority(f, 10)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(f)); + + /* Verify priority was stored and retrievable */ + ASSERT_EQ(got_priority, 10); +} + +static int floating_fiber(void *userdata) { + int *counter = ASSERT_PTR(userdata); + + (*counter)++; + int r = sd_fiber_yield(); + if (r < 0) + return r; + (*counter)++; + + return 0; +} + +TEST(fiber_floating) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + int counter = 0; + ASSERT_OK(sd_fiber_new(e, "floating", floating_fiber, &counter, NULL, &f)); + + ASSERT_OK_ZERO(sd_fiber_get_floating(f)); + ASSERT_OK(sd_fiber_set_floating(f, true)); + ASSERT_OK_POSITIVE(sd_fiber_get_floating(f)); + + /* Drop our handle: the floating ref keeps the future alive until the fiber resolves, after + * which the self-unref frees it. If this didn't work we'd either leak (visible under ASan) or + * trip fiber_free()'s "state == COMPLETED" assertion. */ + f = sd_future_unref(f); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_EQ(counter, 2); +} + +static int drop_extra_ref(sd_future *f) { + /* Drop an extra ref the test installed before the callback fires. After this returns, the + * floating self-ref is the only thing keeping the future alive — exercising the path where + * the floating unref in fiber_run() is the last unref. */ + sd_future_unref(f); + return 0; +} + +TEST(fiber_floating_callback_drops_ref) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + sd_future *f = NULL; + int counter = 0; + ASSERT_OK(sd_fiber_new(e, "floating-cb", floating_fiber, &counter, NULL, &f)); + + ASSERT_OK(sd_fiber_set_floating(f, true)); + + /* Bump the ref for the callback to drop, then install the callback. */ + sd_future_ref(f); + ASSERT_OK(sd_future_set_callback(f, drop_extra_ref, NULL)); + + /* Drop our handle. Refs remaining: floating self-ref + the extra ref the callback will drop. */ + f = sd_future_unref(f); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_EQ(counter, 2); +} + +TEST(fiber_floating_toggle) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + int counter = 0; + ASSERT_OK(sd_fiber_new(e, "floating-toggle", floating_fiber, &counter, NULL, &f)); + + /* Toggling floating on and off again should leave the refcount unchanged: set_floating(true) + * takes a ref and set_floating(false) drops it. If the accounting were off, the subsequent + * event loop would either free the future while the fiber still runs (fiber_free assertion) + * or leak it. */ + ASSERT_OK(sd_fiber_set_floating(f, true)); + ASSERT_OK(sd_fiber_set_floating(f, false)); + ASSERT_OK_ZERO(sd_fiber_get_floating(f)); + + /* Setting floating to the same value twice should be a no-op. */ + ASSERT_OK(sd_fiber_set_floating(f, false)); + ASSERT_OK(sd_fiber_set_floating(f, true)); + ASSERT_OK(sd_fiber_set_floating(f, true)); + + /* Drop our handle; the still-floating ref drives cleanup. */ + f = sd_future_unref(f); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_EQ(counter, 2); +} + +/* Test: SD_FIBER_TIMEOUT scope expires while the fiber is suspended with no other wakeup source. */ +static int timeout_suspend_fiber(void *userdata) { + SD_FIBER_TIMEOUT(50 * USEC_PER_MSEC); + + /* Plain suspend with no other future to wake us — only the deadline timer can resume. */ + return sd_fiber_suspend(); +} + +TEST(fiber_timeout_suspend_expires) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "timeout-suspend", timeout_suspend_fiber, NULL, NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_ERROR(sd_future_result(f), ETIME); +} + +/* Test: SD_FIBER_TIMEOUT scope around a sleep that finishes before the deadline expires; the + * cleanup must cancel the timer cleanly without leaving a stale wakeup. */ +static int timeout_in_time_fiber(void *userdata) { + SD_FIBER_TIMEOUT(1 * USEC_PER_SEC); + return sd_fiber_sleep(10 * USEC_PER_MSEC); +} + +TEST(fiber_timeout_sleep_in_time) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "in-time", timeout_in_time_fiber, NULL, NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK_ZERO(sd_future_result(f)); +} + +/* Test: SD_FIBER_TIMEOUT(USEC_INFINITY) is a no-op — no timer is created and the fiber completes + * normally. */ +static int timeout_infinite_fiber(void *userdata) { + SD_FIBER_TIMEOUT(USEC_INFINITY); + return sd_fiber_sleep(10 * USEC_PER_MSEC); +} + +TEST(fiber_timeout_infinite_no_op) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "infinite", timeout_infinite_fiber, NULL, NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK_ZERO(sd_future_result(f)); +} + +/* Test: SD_FIBER_WITH_TIMEOUT block form returns -ETIME from the suspend inside it. */ +static int with_timeout_block_fiber(void *userdata) { + int r = 0; + SD_FIBER_WITH_TIMEOUT(50 * USEC_PER_MSEC) + r = sd_fiber_suspend(); + return r; +} + +TEST(fiber_with_timeout_block) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "with-timeout", with_timeout_block_fiber, NULL, NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_ERROR(sd_future_result(f), ETIME); +} + +/* Test: nested SD_FIBER_TIMEOUT — inner scope's timer fires first; once we're back in just the + * outer scope, suspending again must time out via the still-armed outer timer. */ +static int nested_timeout_fiber(void *userdata) { + int *fired = ASSERT_PTR(userdata); + + SD_FIBER_TIMEOUT(50 * USEC_PER_MSEC); /* outer */ + + SD_FIBER_WITH_TIMEOUT(20 * USEC_PER_MSEC) { /* inner — expires first */ + int r = sd_fiber_suspend(); + if (r != -ETIME) + return -ENOTRECOVERABLE; + (*fired)++; + } + + /* Inner scope is gone, but the outer timer is still armed (it only used ~20ms of its + * 100ms budget). Suspending again must eventually wake us with -ETIME. */ + int r = sd_fiber_suspend(); + if (r != -ETIME) + return -ENOTRECOVERABLE; + (*fired)++; + + return 0; +} + +TEST(fiber_timeout_nested) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + int fired = 0; + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "nested-timeout", nested_timeout_fiber, &fired, NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK_ZERO(sd_future_result(f)); + ASSERT_EQ(fired, 2); +} + +/* Test: signal mask is per-thread, not per-fiber. Changes one fiber makes via pthread_sigmask + * must be visible to other fibers on the same thread, both while the modifying fiber is + * suspended and after it resumes. The fiber switch (sigsetjmp/siglongjmp with savesigs=0) + * deliberately doesn't save or restore the mask. */ +static int sigmask_peer_fiber(void *userdata) { + sigset_t set, current; + + /* The waiter blocked SIGUSR1 before await'ing us; the per-thread mask should still + * have it blocked here. */ + ASSERT_OK_ZERO(-pthread_sigmask(SIG_SETMASK, NULL, ¤t)); + ASSERT_TRUE(sigismember(¤t, SIGUSR1)); + + ASSERT_OK(sigemptyset(&set)); + ASSERT_OK(sigaddset(&set, SIGUSR1)); + ASSERT_OK_ZERO(-pthread_sigmask(SIG_UNBLOCK, &set, NULL)); + + return 0; +} + +static int sigmask_waiter_fiber(void *userdata) { + sd_future *peer = ASSERT_PTR(userdata); + sigset_t set, current; + + ASSERT_OK(sigemptyset(&set)); + ASSERT_OK(sigaddset(&set, SIGUSR1)); + ASSERT_OK_ZERO(-pthread_sigmask(SIG_BLOCK, &set, NULL)); + + ASSERT_OK_ZERO(-pthread_sigmask(SIG_SETMASK, NULL, ¤t)); + ASSERT_TRUE(sigismember(¤t, SIGUSR1)); + + int r = sd_fiber_await(peer); + if (r < 0) + return r; + + /* The peer unblocked SIGUSR1 while we were suspended. The change is per-thread, so + * we must observe it here. */ + ASSERT_OK_ZERO(-pthread_sigmask(SIG_SETMASK, NULL, ¤t)); + ASSERT_FALSE(sigismember(¤t, SIGUSR1)); + + return 0; +} + +TEST(fiber_signal_mask_is_per_thread) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + sigset_t saved; + ASSERT_OK_ZERO(-pthread_sigmask(SIG_SETMASK, NULL, &saved)); + + _cleanup_(sd_future_unrefp) sd_future *waiter = NULL, *peer = NULL; + ASSERT_OK(sd_fiber_new(e, "sigmask-peer", sigmask_peer_fiber, NULL, NULL, &peer)); + ASSERT_OK(sd_future_set_priority(peer, 1)); + ASSERT_OK(sd_fiber_new(e, "sigmask-waiter", sigmask_waiter_fiber, peer, NULL, &waiter)); + ASSERT_OK(sd_future_set_priority(waiter, 0)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(waiter)); + ASSERT_OK(sd_future_result(peer)); + + ASSERT_OK_ZERO(-pthread_sigmask(SIG_SETMASK, &saved, NULL)); +} + +/* Test: log context is per-fiber. fiber_run() swaps the thread-local log context (and prefix) with + * a per-fiber stash on entry and exit, so fields pushed by one fiber must not leak into another + * fiber that runs while the first is suspended, and must be restored when the first resumes. */ +static int log_context_peer_fiber(void *userdata) { + size_t *peer_observed = ASSERT_PTR(userdata); + + /* The waiter pushed a field before await'ing us. If log context were shared across fibers, + * we would observe it here. Record what we see and let the caller verify. */ + *peer_observed = log_context_num_fields(); + + return 0; +} + +static int log_context_waiter_fiber(void *userdata) { + sd_future *peer = ASSERT_PTR(userdata); + + size_t before_push = log_context_num_fields(); + + LOG_CONTEXT_PUSH("WAITER=here"); + size_t after_push = log_context_num_fields(); + if (after_push != before_push + 1) + return -EBADF; + + int r = sd_fiber_await(peer); + if (r < 0) + return r; + + /* Our pushed field must be visible again after the peer ran and resumed us. */ + if (log_context_num_fields() != after_push) + return -EBADF; + + return 0; +} + +TEST(fiber_log_context_per_fiber) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + size_t baseline = log_context_num_fields(); + + size_t peer_observed = 0; + _cleanup_(sd_future_unrefp) sd_future *waiter = NULL, *peer = NULL; + ASSERT_OK(sd_fiber_new(e, "log-peer", log_context_peer_fiber, &peer_observed, NULL, &peer)); + ASSERT_OK(sd_future_set_priority(peer, 1)); + ASSERT_OK(sd_fiber_new(e, "log-waiter", log_context_waiter_fiber, peer, NULL, &waiter)); + ASSERT_OK(sd_future_set_priority(waiter, 0)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(waiter)); + ASSERT_OK(sd_future_result(peer)); + + /* Inside the peer, only the peer's own FIBER= field (pushed by fiber_run) should have been + * active — the waiter's WAITER= push must have been swapped out. */ + ASSERT_EQ(peer_observed, baseline + 1); + + /* The thread-local log context should be exactly as it was before the test ran. */ + ASSERT_EQ(log_context_num_fields(), baseline); +} + +static int stack_overflow_fiber(void *userdata) { + volatile char anchor; + size_t pagesz = page_size(); + + /* Walk one page at a time away from the fiber's current SP toward the guard page, + * writing one byte per page until the kernel raises a fatal signal. On downward + * stacks we walk to lower addresses (guard at the base); on upward stacks like + * hppa we walk to higher addresses (guard at the top of the mapping). The 64 MiB + * ceiling is purely a safety net so the test fails loudly instead of looping if + * the guard isn't there. */ + for (size_t i = 1; i < (64U * U64_MB) / pagesz; i++) { + uintptr_t off = i * pagesz; + volatile char *p = (volatile char *) (STACK_GROWS_UP + ? (uintptr_t) &anchor + off + : (uintptr_t) &anchor - off); + *p = 0; + } + return 0; +} + +TEST(fiber_stack_guard) { +#if HAS_FEATURE_ADDRESS_SANITIZER + (void) log_tests_skipped("ASan intercepts deliberate stack OOB writes"); + return; +#endif +#if HAVE_VALGRIND_VALGRIND_H + if (RUNNING_ON_VALGRIND) { + (void) log_tests_skipped("Valgrind intercepts deliberate stack OOB writes"); + return; + } +#endif + + _cleanup_(pidref_done) PidRef pidref = PIDREF_NULL; + int r = pidref_safe_fork("(stack-overflow)", FORK_RESET_SIGNALS|FORK_LOG, &pidref); + ASSERT_OK(r); + + if (r == 0) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "overflow", stack_overflow_fiber, NULL, NULL, &f)); + (void) sd_event_loop(e); + _exit(EXIT_SUCCESS); /* unreachable if the guard fires */ + } + + siginfo_t si; + ASSERT_OK(pidref_wait_for_terminate(&pidref, &si)); + ASSERT_TRUE(IN_SET(si.si_code, CLD_KILLED, CLD_DUMPED)); + ASSERT_TRUE(IN_SET(si.si_status, SIGSEGV, SIGBUS)); +} + +DEFINE_TEST_MAIN(LOG_DEBUG); diff --git a/src/systemd/_sd-common.h b/src/systemd/_sd-common.h index f9c9a2627d..8da080bf18 100644 --- a/src/systemd/_sd-common.h +++ b/src/systemd/_sd-common.h @@ -69,6 +69,20 @@ typedef void (*_sd_destroy_t)(void *userdata); # define _SD_STRINGIFY(x) _SD_XSTRINGIFY(x) #endif +/* Mirror of CONCATENATE / UNIQ from macro-fundamental.h, available to public sd-* headers. */ +#ifndef _SD_CONCATENATE +# define _SD_XCONCATENATE(x, y) x ## y +# define _SD_CONCATENATE(x, y) _SD_XCONCATENATE(x, y) +#endif + +#ifndef _SD_UNIQ +# ifdef __COUNTER__ +# define _SD_UNIQ __COUNTER__ +# else +# define _SD_UNIQ __LINE__ +# endif +#endif + #ifndef _SD_BEGIN_DECLARATIONS # ifdef __cplusplus # define _SD_BEGIN_DECLARATIONS \ diff --git a/src/systemd/meson.build b/src/systemd/meson.build index ad455d73b2..c35f811245 100644 --- a/src/systemd/meson.build +++ b/src/systemd/meson.build @@ -36,6 +36,7 @@ _not_installed_headers = [ 'sd-dhcp6-option.h', 'sd-dhcp6-protocol.h', 'sd-dns-resolver.h', + 'sd-future.h', 'sd-ipv4acd.h', 'sd-ipv4ll.h', 'sd-lldp-rx.h', diff --git a/src/systemd/sd-future.h b/src/systemd/sd-future.h new file mode 100644 index 0000000000..9d0d03acf4 --- /dev/null +++ b/src/systemd/sd-future.h @@ -0,0 +1,101 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +#ifndef foosdfuturefoo +#define foosdfuturefoo + +/*** + systemd is free software; you can redistribute it and/or modify it + under the terms of the GNU Lesser General Public License as published by + the Free Software Foundation; either version 2.1 of the License, or + (at your option) any later version. + + systemd is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public License + along with systemd; If not, see . +***/ + +#include "_sd-common.h" + +_SD_BEGIN_DECLARATIONS; + +typedef struct sd_event sd_event; +typedef struct sd_future sd_future; +typedef struct sd_future_ops sd_future_ops; +typedef int (*sd_future_func_t)(sd_future *f); +typedef int (*sd_fiber_func_t)(void *userdata); +typedef _sd_destroy_t sd_fiber_destroy_t; + +struct sd_future_ops { + size_t size; + void* (*alloc)(void); + void (*free)(sd_future *f); + int (*cancel)(sd_future *f); + int (*set_priority)(sd_future *f, int64_t priority); +}; + +__extension__ typedef enum _SD_ENUM_TYPE_S64(sd_future_state_t) { + SD_FUTURE_PENDING, + SD_FUTURE_RESOLVED, + _SD_ENUM_FORCE_S64(SD_FUTURE_STATE) +} sd_future_state_t; + +int sd_future_new(const sd_future_ops *ops, sd_future **ret); +int sd_future_cancel(sd_future *f); +int sd_future_resolve(sd_future *f, int result); + +_SD_DECLARE_TRIVIAL_REF_UNREF_FUNC(sd_future); +_SD_DEFINE_POINTER_CLEANUP_FUNC(sd_future, sd_future_unref); +void sd_future_unref_array_clear(sd_future *array[], size_t n); +void sd_future_unref_array(sd_future *array[], size_t n); + +sd_future* sd_future_cancel_wait_unref(sd_future *f); +_SD_DEFINE_POINTER_CLEANUP_FUNC(sd_future, sd_future_cancel_wait_unref); +void sd_future_cancel_wait_unref_array_clear(sd_future *array[], size_t n); +void sd_future_cancel_wait_unref_array(sd_future *array[], size_t n); + +int sd_future_state(sd_future *f); +int sd_future_result(sd_future *f); +void* sd_future_get_userdata(sd_future *f); +void* sd_future_get_private(sd_future *f); +const sd_future_ops* sd_future_get_ops(sd_future *f); + +int sd_future_set_callback(sd_future *f, sd_future_func_t callback, void *userdata); +int sd_future_set_priority(sd_future *f, int64_t priority); + +int sd_future_new_wait(sd_future *target, sd_future **ret); + +int sd_fiber_new(sd_event *e, const char *name, sd_fiber_func_t func, void *userdata, sd_fiber_destroy_t destroy, sd_future **ret); + +int sd_fiber_set_floating(sd_future *f, int b); +int sd_fiber_get_floating(sd_future *f); + +int sd_fiber_is_running(void); +sd_future* sd_fiber_get_current(void); +int sd_fiber_get_priority(int64_t *ret); +sd_event* sd_fiber_get_event(void); + +int sd_fiber_yield(void); +int sd_fiber_sleep(uint64_t usec); +int sd_fiber_await(sd_future *target); +int sd_fiber_suspend(void); +int sd_fiber_resume(sd_future *f, int result); + +sd_future* sd_fiber_timeout(uint64_t timeout); + +#define SD_FIBER_TIMEOUT(timeout) _SD_FIBER_TIMEOUT(_SD_UNIQ, (timeout)) +#define _SD_FIBER_TIMEOUT(uniq, timeout) \ + sd_future *_SD_CONCATENATE(_sd_fto_, uniq) __attribute__((cleanup(sd_future_cancel_wait_unrefp), unused)) = sd_fiber_timeout(timeout) + +#define SD_FIBER_WITH_TIMEOUT(timeout) _SD_FIBER_WITH_TIMEOUT(_SD_UNIQ, (timeout)) +#define _SD_FIBER_WITH_TIMEOUT(uniq, timeout) \ + for (sd_future *_SD_CONCATENATE(_sd_fto_, uniq) __attribute__((cleanup(sd_future_cancel_wait_unrefp), unused)) = sd_fiber_timeout(timeout), \ + *_SD_CONCATENATE(_sd_fto_b_, uniq) = (sd_future*) (uintptr_t) 1; \ + _SD_CONCATENATE(_sd_fto_b_, uniq); \ + _SD_CONCATENATE(_sd_fto_b_, uniq) = NULL) + +_SD_END_DECLARATIONS; + +#endif diff --git a/test/integration-tests/TEST-02-UNITTESTS/meson.build b/test/integration-tests/TEST-02-UNITTESTS/meson.build index 9be6f2a6d9..b074903c00 100644 --- a/test/integration-tests/TEST-02-UNITTESTS/meson.build +++ b/test/integration-tests/TEST-02-UNITTESTS/meson.build @@ -3,7 +3,7 @@ integration_tests += [ integration_test_template + { 'name' : fs.name(meson.current_source_dir()), - 'coredump-exclude-regex' : '/(bash|python3.[0-9]+|systemd-executor)$', + 'coredump-exclude-regex' : '/(bash|python3.[0-9]+|systemd-executor|test-fiber)$', 'cmdline' : integration_test_template['cmdline'] + [ ''' diff --git a/test/test-link-abi.py b/test/test-link-abi.py index a36e6c32d7..8a935a513f 100755 --- a/test/test-link-abi.py +++ b/test/test-link-abi.py @@ -28,6 +28,7 @@ LIBC_LIB_PREFIXES: tuple[str, ...] = ( 'libm.so.', 'libresolv.so.', 'libc.musl-', + 'libucontext.', ) # GCC runtime support libraries (stack unwinding, soft-float helpers, C++ standard library). The From cf4c65afa86021a750de38bbed192eeb1c9fd425 Mon Sep 17 00:00:00 2001 From: Daan De Meyer Date: Sat, 25 Apr 2026 22:31:58 +0200 Subject: [PATCH 2/8] sd-future: add fiber-aware non-blocking I/O wrappers Add a family of sd_fiber_*() I/O wrappers that, when called from a fiber, behave like blocking I/O from the caller's perspective but yield to the event loop instead of blocking the thread: sd_fiber_read / sd_fiber_write sd_fiber_readv / sd_fiber_writev sd_fiber_recv / sd_fiber_send sd_fiber_connect sd_fiber_recvmsg / sd_fiber_sendmsg sd_fiber_recvfrom / sd_fiber_sendto sd_fiber_accept sd_fiber_ppoll Most of them share a single helper, fiber_io_operation(), which when invoked outside a fiber falls through to the underlying syscall directly, preserving the regular blocking behaviour. Inside a fiber the helper flips the fd to non-blocking (restoring its original mode on return), tries the syscall once on the fast path, and on EAGAIN/ EWOULDBLOCK creates an sd-event-backed IO future via future_new_io(), suspends the fiber, and retries the syscall once the event source fires. future_new_io() itself is added to sd-event/event-future.{c,h} as a new IoFuture kind. It wraps sd_event_add_io() into an sd_future: oneshot enable, EPOLLERR translated via SO_ERROR (suppressed for non-sockets), and the fd duplicated with F_DUPFD_CLOEXEC to avoid EEXIST when multiple sources watch the same descriptor. Together these let fiber-using code write straight-line socket and pipe I/O without bundling state into callbacks. --- src/basic/io-util.c | 9 + src/basic/io-util.h | 6 + src/libsystemd/meson.build | 2 + src/libsystemd/sd-event/event-future.c | 118 ++ src/libsystemd/sd-event/event-future.h | 1 + src/libsystemd/sd-future/fiber-io.c | 471 ++++++++ src/libsystemd/sd-future/test-fiber-io.c | 1388 ++++++++++++++++++++++ src/systemd/sd-future.h | 25 + 8 files changed, 2020 insertions(+) create mode 100644 src/libsystemd/sd-future/fiber-io.c create mode 100644 src/libsystemd/sd-future/test-fiber-io.c diff --git a/src/basic/io-util.c b/src/basic/io-util.c index 103aa2a7cd..b4f643b572 100644 --- a/src/basic/io-util.c +++ b/src/basic/io-util.c @@ -3,6 +3,7 @@ #include #include #include +#include /* IWYU pragma: keep */ #include #include @@ -10,6 +11,14 @@ #include "io-util.h" #include "time-util.h" +/* EPOLL_POLL_COMMON_MASK in io-util.h treats POLL* and EPOLL* as interchangeable; verify it. */ +assert_cc((uint32_t) POLLIN == EPOLLIN); +assert_cc((uint32_t) POLLOUT == EPOLLOUT); +assert_cc((uint32_t) POLLERR == EPOLLERR); +assert_cc((uint32_t) POLLHUP == EPOLLHUP); +assert_cc((uint32_t) POLLPRI == EPOLLPRI); +assert_cc((uint32_t) POLLRDHUP == EPOLLRDHUP); + int flush_fd(int fd) { int count = 0; diff --git a/src/basic/io-util.h b/src/basic/io-util.h index 918108c023..ab52dc8db6 100644 --- a/src/basic/io-util.h +++ b/src/basic/io-util.h @@ -3,6 +3,12 @@ #include "basic-forward.h" +/* The intersection of poll() and epoll_wait() event masks. Linux defines POLL* and EPOLL* with the + * same numeric values for these — see the assert_cc()s in io-util.c — so this mask can be used + * interchangeably as a `revents` (poll) or `events` (epoll) bitset. */ +#define EPOLL_POLL_COMMON_MASK \ + (EPOLLIN | EPOLLOUT | EPOLLERR | EPOLLHUP | EPOLLPRI | EPOLLRDHUP) + int flush_fd(int fd); ssize_t loop_read(int fd, void *buf, size_t nbytes, bool do_poll); diff --git a/src/libsystemd/meson.build b/src/libsystemd/meson.build index 061ff6213f..3c624051f0 100644 --- a/src/libsystemd/meson.build +++ b/src/libsystemd/meson.build @@ -77,6 +77,7 @@ sd_device_sources = files( ############################################################ sd_future_sources = files( + 'sd-future/fiber-io.c', 'sd-future/fiber.c', 'sd-future/sd-future.c', ) @@ -190,6 +191,7 @@ simple_tests += files( 'sd-device/test-device-util.c', 'sd-device/test-sd-device-monitor.c', 'sd-future/test-fiber.c', + 'sd-future/test-fiber-io.c', 'sd-hwdb/test-sd-hwdb.c', 'sd-id128/test-id128.c', 'sd-journal/test-audit-type.c', diff --git a/src/libsystemd/sd-event/event-future.c b/src/libsystemd/sd-event/event-future.c index 4595a7a6fb..8c9960fcdd 100644 --- a/src/libsystemd/sd-event/event-future.c +++ b/src/libsystemd/sd-event/event-future.c @@ -6,6 +6,124 @@ #include "alloc-util.h" #include "errno-util.h" #include "event-future.h" +#include "fd-util.h" + +typedef struct IoFuture { + sd_event_source *source; +} IoFuture; + +static void* io_future_alloc(void) { + return new0(IoFuture, 1); +} + +static void io_future_free(sd_future *f) { + IoFuture *iof = ASSERT_PTR(sd_future_get_private(ASSERT_PTR(f))); + sd_event_source_unref(iof->source); + free(iof); +} + +static int io_future_cancel(sd_future *f) { + IoFuture *iof = ASSERT_PTR(sd_future_get_private(ASSERT_PTR(f))); + int r = 0; + + RET_GATHER(r, sd_event_source_set_enabled(iof->source, SD_EVENT_OFF)); + RET_GATHER(r, sd_future_resolve(f, -ECANCELED)); + return r; +} + +static int io_future_set_priority(sd_future *f, int64_t priority) { + IoFuture *iof = ASSERT_PTR(sd_future_get_private(ASSERT_PTR(f))); + return sd_event_source_set_priority(iof->source, priority); +} + +static const sd_future_ops io_future_ops = { + .size = sizeof(sd_future_ops), + .alloc = io_future_alloc, + .free = io_future_free, + .cancel = io_future_cancel, + .set_priority = io_future_set_priority, +}; + +static int io_handler(sd_event_source *s, int fd, uint32_t revents, void *userdata) { + sd_future *f = ASSERT_PTR(userdata); + + /* Resolve with the revents mask on success (matching io_uring poll_add's CQE convention) so + * callers can read it directly off the future result. EPOLLERR is the one exception: surface + * the actual socket error via SO_ERROR so callers like sd_fiber_connect() can return -errno + * directly without re-querying. */ + if (FLAGS_SET(revents, EPOLLERR)) { + int error = 0; + socklen_t len = sizeof(error); + + int r = RET_NERRNO(getsockopt(fd, SOL_SOCKET, SO_ERROR, &error, &len)); + if (r == -ENOTSOCK) + return sd_future_resolve(f, (int) revents); + if (r >= 0 && error != 0) + return sd_future_resolve(f, -error); + if (r >= 0) + /* EPOLLERR was reported but SO_ERROR returned no pending error (e.g. + * already consumed elsewhere). Surface the revents mask so the caller + * still sees the error condition rather than mistaking it for success. */ + return sd_future_resolve(f, (int) revents); + /* On any other getsockopt() error fall through and resolve the future with that + * error so the waiting fiber wakes up rather than hanging forever. */ + return sd_future_resolve(f, r); + } + + return sd_future_resolve(f, (int) revents); +} + +int future_new_io(sd_event *e, int fd, uint32_t events, sd_future **ret) { + int r; + + assert(e); + assert(fd >= 0); + assert(ret); + + if (IN_SET(sd_event_get_state(e), SD_EVENT_EXITING, SD_EVENT_FINISHED)) + return -ECANCELED; + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + r = sd_future_new(&io_future_ops, &f); + if (r < 0) + return r; + + IoFuture *iof = sd_future_get_private(f); + + /* Duplicate fd to avoid EEXIST from epoll when adding the same fd multiple times */ + _cleanup_close_ int fd_copy = fcntl(fd, F_DUPFD_CLOEXEC, 3); + if (fd_copy < 0) + return -errno; + + r = sd_event_add_io(e, &iof->source, fd_copy, events, io_handler, f); + if (r < 0) + return r; + + r = sd_event_source_set_io_fd_own(iof->source, true); + if (r < 0) + return r; + + TAKE_FD(fd_copy); + + r = sd_event_source_set_enabled(iof->source, SD_EVENT_ONESHOT); + if (r < 0) + return r; + + if (sd_fiber_is_running()) { + int64_t priority; + + r = sd_fiber_get_priority(&priority); + if (r < 0) + return r; + + r = sd_event_source_set_priority(iof->source, priority); + if (r < 0) + return r; + } + + *ret = TAKE_PTR(f); + return 0; +} typedef struct TimeFuture { sd_event_source *source; diff --git a/src/libsystemd/sd-event/event-future.h b/src/libsystemd/sd-event/event-future.h index 7e956906eb..3bc275e7b7 100644 --- a/src/libsystemd/sd-event/event-future.h +++ b/src/libsystemd/sd-event/event-future.h @@ -3,5 +3,6 @@ #include "sd-forward.h" +int future_new_io(sd_event *e, int fd, uint32_t events, sd_future **ret); int future_new_time(sd_event *e, clockid_t clock, uint64_t usec, uint64_t accuracy, int result, sd_future **ret); int future_new_time_relative(sd_event *e, clockid_t clock, uint64_t usec, uint64_t accuracy, int result, sd_future **ret); diff --git a/src/libsystemd/sd-future/fiber-io.c b/src/libsystemd/sd-future/fiber-io.c new file mode 100644 index 0000000000..823e890718 --- /dev/null +++ b/src/libsystemd/sd-future/fiber-io.c @@ -0,0 +1,471 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ + +#include +#include /* IWYU pragma: keep */ +#include +#include +#include +#include + +#include "sd-event.h" +#include "sd-future.h" + +#include "alloc-util.h" +#include "errno-util.h" +#include "event-future.h" +#include "fd-util.h" +#include "io-util.h" +#include "time-util.h" + +typedef ssize_t (*FiberIOFunc)(int fd, void *args); + +static ssize_t fiber_io_operation( + int fd, + uint32_t events, + FiberIOFunc func, + void *args) { + _cleanup_(nonblock_resetp) int reset_fd = -EBADF; + int r; + + assert(fd >= 0); + assert(func); + + if (!sd_fiber_is_running()) + return func(fd, args); + + sd_event *e = sd_fiber_get_event(); + assert(e); + + r = fd_nonblock(fd, true); + if (r < 0) + return r; + if (r > 0) + reset_fd = fd; + + ssize_t n = func(fd, args); + if (n >= 0 || !ERRNO_IS_NEG_TRANSIENT(n)) + return n; + + _cleanup_(sd_future_cancel_wait_unrefp) sd_future *io = NULL; + r = future_new_io(e, fd, events, &io); + if (r < 0) + return r; + + r = sd_fiber_suspend(); + if (r < 0) + return r; + + return func(fd, args); +} + +typedef struct ReadArgs { + void *buf; + size_t count; +} ReadArgs; + +static ssize_t read_callback(int fd, void *args) { + ReadArgs *a = ASSERT_PTR(args); + ssize_t n; + + n = read(fd, a->buf, a->count); + return n >= 0 ? n : -errno; +} + +ssize_t sd_fiber_read(int fd, void *buf, size_t count) { + assert_return(fd >= 0, -EBADF); + assert_return(buf || count == 0, -EINVAL); + + return fiber_io_operation(fd, EPOLLIN, read_callback, &(ReadArgs) { + .buf = buf, + .count = count, + }); +} + +typedef struct WriteArgs { + const void *buf; + size_t count; +} WriteArgs; + +static ssize_t write_callback(int fd, void *args) { + WriteArgs *a = ASSERT_PTR(args); + ssize_t n; + + n = write(fd, a->buf, a->count); + return n >= 0 ? n : -errno; +} + +ssize_t sd_fiber_write(int fd, const void *buf, size_t count) { + assert_return(fd >= 0, -EBADF); + assert_return(buf || count == 0, -EINVAL); + + return fiber_io_operation(fd, EPOLLOUT, write_callback, &(WriteArgs) { + .buf = buf, + .count = count, + }); +} + +typedef struct ReadvArgs { + const struct iovec *iov; + int iovcnt; +} ReadvArgs; + +static ssize_t readv_callback(int fd, void *args) { + ReadvArgs *a = ASSERT_PTR(args); + ssize_t n; + + n = readv(fd, a->iov, a->iovcnt); + return n >= 0 ? n : -errno; +} + +ssize_t sd_fiber_readv(int fd, const struct iovec *iov, int iovcnt) { + assert_return(fd >= 0, -EBADF); + assert_return(iov || iovcnt == 0, -EINVAL); + + return fiber_io_operation(fd, EPOLLIN, readv_callback, &(ReadvArgs) { + .iov = iov, + .iovcnt = iovcnt, + }); +} + +typedef struct WritevArgs { + const struct iovec *iov; + int iovcnt; +} WritevArgs; + +static ssize_t writev_callback(int fd, void *args) { + WritevArgs *a = ASSERT_PTR(args); + ssize_t n; + + n = writev(fd, a->iov, a->iovcnt); + return n >= 0 ? n : -errno; +} + +ssize_t sd_fiber_writev(int fd, const struct iovec *iov, int iovcnt) { + assert_return(fd >= 0, -EBADF); + assert_return(iov || iovcnt == 0, -EINVAL); + + return fiber_io_operation(fd, EPOLLOUT, writev_callback, &(WritevArgs) { + .iov = iov, + .iovcnt = iovcnt, + }); +} + +typedef struct RecvArgs { + void *buf; + size_t len; + int flags; +} RecvArgs; + +static ssize_t recv_callback(int fd, void *args) { + RecvArgs *a = ASSERT_PTR(args); + ssize_t n; + + n = recv(fd, a->buf, a->len, a->flags); + return n >= 0 ? n : -errno; +} + +ssize_t sd_fiber_recv(int sockfd, void *buf, size_t len, int flags) { + assert_return(sockfd >= 0, -EBADF); + assert_return(buf || len == 0, -EINVAL); + + return fiber_io_operation(sockfd, EPOLLIN, recv_callback, &(RecvArgs) { + .buf = buf, + .len = len, + .flags = flags, + }); +} + +typedef struct SendArgs { + const void *buf; + size_t len; + int flags; +} SendArgs; + +static ssize_t send_callback(int fd, void *args) { + SendArgs *a = ASSERT_PTR(args); + ssize_t n; + + n = send(fd, a->buf, a->len, a->flags); + return n >= 0 ? n : -errno; +} + +ssize_t sd_fiber_send(int sockfd, const void *buf, size_t len, int flags) { + assert_return(sockfd >= 0, -EBADF); + assert_return(buf || len == 0, -EINVAL); + + return fiber_io_operation(sockfd, EPOLLOUT, send_callback, &(SendArgs) { + .buf = buf, + .len = len, + .flags = flags, + }); +} + +int sd_fiber_connect(int sockfd, const struct sockaddr *addr, socklen_t addrlen) { + _cleanup_(nonblock_resetp) int reset_fd = -EBADF; + int r; + + assert_return(sockfd >= 0, -EBADF); + assert_return(addr, -EINVAL); + + if (!sd_fiber_is_running()) + return RET_NERRNO(connect(sockfd, addr, addrlen)); + + sd_event *e = sd_fiber_get_event(); + assert(e); + + r = fd_nonblock(sockfd, true); + if (r < 0) + return r; + if (r > 0) + reset_fd = sockfd; + + r = RET_NERRNO(connect(sockfd, addr, addrlen)); + if (r != -EINPROGRESS) + return r; + + _cleanup_(sd_future_cancel_wait_unrefp) sd_future *io = NULL; + r = future_new_io(e, sockfd, EPOLLOUT, &io); + if (r < 0) + return r; + + /* future_new_io resolves with the revents mask on success; translate any positive value + * (e.g. POLLOUT) back to the connect(2) success status. */ + r = sd_fiber_suspend(); + return r > 0 ? 0 : r; +} + +typedef struct RecvmsgArgs { + struct msghdr *msg; + int flags; +} RecvmsgArgs; + +static ssize_t recvmsg_callback(int fd, void *args) { + RecvmsgArgs *a = ASSERT_PTR(args); + ssize_t n; + + n = recvmsg(fd, a->msg, a->flags); + return n >= 0 ? n : -errno; +} + +ssize_t sd_fiber_recvmsg(int sockfd, struct msghdr *msg, int flags) { + assert_return(sockfd >= 0, -EBADF); + assert_return(msg, -EINVAL); + + return fiber_io_operation(sockfd, EPOLLIN, recvmsg_callback, &(RecvmsgArgs) { + .msg = msg, + .flags = flags, + }); +} + +typedef struct SendmsgArgs { + const struct msghdr *msg; + int flags; +} SendmsgArgs; + +static ssize_t sendmsg_callback(int fd, void *args) { + SendmsgArgs *a = ASSERT_PTR(args); + ssize_t n; + + n = sendmsg(fd, a->msg, a->flags); + return n >= 0 ? n : -errno; +} + +ssize_t sd_fiber_sendmsg(int sockfd, const struct msghdr *msg, int flags) { + assert_return(sockfd >= 0, -EBADF); + assert_return(msg, -EINVAL); + + return fiber_io_operation(sockfd, EPOLLOUT, sendmsg_callback, &(SendmsgArgs) { + .msg = msg, + .flags = flags, + }); +} + +static ssize_t recvfrom_callback(int fd, void *args) { + RecvmsgArgs *a = ASSERT_PTR(args); + ssize_t n; + + n = recvmsg(fd, a->msg, a->flags); + return n >= 0 ? n : -errno; +} + +ssize_t sd_fiber_recvfrom(int sockfd, void *buf, size_t len, int flags, struct sockaddr *src_addr, socklen_t *addrlen) { + ssize_t n; + + assert_return(sockfd >= 0, -EBADF); + assert_return(buf || len == 0, -EINVAL); + assert_return(!src_addr || addrlen, -EINVAL); + + /* io_uring has no direct recvfrom prep helper, so emulate via recvmsg with a single-iovec + * msghdr. The kernel updates msg_namelen in place; we copy it back to *addrlen below. */ + struct iovec iov = { .iov_base = buf, .iov_len = len }; + struct msghdr msg = { + .msg_name = src_addr, + .msg_namelen = src_addr ? *addrlen : 0, + .msg_iov = &iov, + .msg_iovlen = 1, + }; + + n = fiber_io_operation(sockfd, EPOLLIN, recvfrom_callback, &(RecvmsgArgs) { + .msg = &msg, + .flags = flags, + }); + if (n < 0) + return n; + + if (addrlen) + *addrlen = msg.msg_namelen; + + return n; +} + +static ssize_t sendto_callback(int fd, void *args) { + SendmsgArgs *a = ASSERT_PTR(args); + ssize_t n; + + n = sendmsg(fd, a->msg, a->flags); + return n >= 0 ? n : -errno; +} + +ssize_t sd_fiber_sendto(int sockfd, const void *buf, size_t len, int flags, const struct sockaddr *dest_addr, socklen_t addrlen) { + assert_return(sockfd >= 0, -EBADF); + assert_return(buf || len == 0, -EINVAL); + + struct iovec iov = { .iov_base = (void *) buf, .iov_len = len }; + struct msghdr msg = { + .msg_name = (void *) dest_addr, + .msg_namelen = dest_addr ? addrlen : 0, + .msg_iov = &iov, + .msg_iovlen = 1, + }; + + return fiber_io_operation(sockfd, EPOLLOUT, sendto_callback, &(SendmsgArgs) { + .msg = &msg, + .flags = flags, + }); +} + +typedef struct AcceptArgs { + struct sockaddr *addr; + socklen_t *addrlen; + int flags; +} AcceptArgs; + +static ssize_t accept_callback(int fd, void *args) { + AcceptArgs *a = ASSERT_PTR(args); + + return RET_NERRNO(accept4(fd, a->addr, a->addrlen, a->flags)); +} + +int sd_fiber_accept(int sockfd, struct sockaddr *addr, socklen_t *addrlen, int flags) { + assert_return(sockfd >= 0, -EBADF); + + return fiber_io_operation(sockfd, EPOLLIN, accept_callback, &(AcceptArgs) { + .addr = addr, + .addrlen = addrlen, + .flags = flags, + }); +} + +int sd_fiber_ppoll(struct pollfd *fds, size_t n_fds, const struct timespec *timeout, const sigset_t *sigmask) { + int r; + + assert_return(fds || n_fds == 0, -EINVAL); + + if (!sd_fiber_is_running()) + return RET_NERRNO(ppoll(fds, n_fds, timeout, sigmask)); + + /* When on a fiber signals are handled via sd-event hence we should never mess around with the + * signal mask when running on a fiber. */ + assert_return(!sigmask, -EOPNOTSUPP); + + sd_event *e = sd_fiber_get_event(); + assert(e); + + /* No fds to wait on and no timeout means there's nothing that could ever wake the fiber up, + * since unlike raw ppoll() we cannot use signal delivery as a wakeup. Signals received while + * the fiber is suspended are handled by sd-event via signalfd, in which case the signal handler + * is expected to cancel the fiber via sd_future_cancel() if a wakeup is desired. */ + if (n_fds == 0 && !timeout) + return -EINVAL; + + bool zero_timeout = timeout && timeout->tv_sec == 0 && timeout->tv_nsec == 0; + + /* Try polling with zero timeout first to see if any are immediately ready. */ + r = RET_NERRNO(ppoll(fds, n_fds, &(const struct timespec) {}, /* sigmask= */ NULL)); + if (zero_timeout || r != 0) /* Either error or some fds are ready */ + return r; + + sd_future **futures = NULL; + CLEANUP_ARRAY(futures, n_fds, sd_future_cancel_wait_unref_array); + + futures = new0(sd_future*, n_fds); + if (!futures) + return -ENOMEM; + + /* Set up I/O event sources for all valid fds. POLL* and EPOLL* share their bit values (see + * EPOLL_POLL_COMMON_MASK in io-util.h), so we can pass the user-supplied event mask through + * to either backend without translation. */ + size_t n_io_futures = 0; + for (size_t i = 0; i < n_fds; i++) { + if (fds[i].fd < 0) + continue; + + uint32_t events = fds[i].events & EPOLL_POLL_COMMON_MASK; + if (events == 0) + continue; + + r = future_new_io(e, fds[i].fd, events, &futures[i]); + if (r < 0) + return r; + + n_io_futures++; + } + + /* A timeout that overflows usec_t saturates to USEC_INFINITY in timespec_load(); treat that + * like "no timeout" (matches sd_fiber_sleep(USEC_INFINITY)) rather than letting + * sd_event_add_time_relative() reject it with -EOVERFLOW — standard ppoll() would just + * wait a very long time. */ + usec_t usec = timeout ? timespec_load(timeout) : USEC_INFINITY; + + /* If every fd was skipped (negative or empty event mask) and we'd have no timer, there's + * nothing that could ever wake the fiber up — same situation as n_fds == 0 && !timeout, + * just not detectable upfront. Refuse rather than suspend forever. */ + if (n_io_futures == 0 && usec == USEC_INFINITY) + return -EINVAL; + + _cleanup_(sd_future_cancel_wait_unrefp) sd_future *timer = NULL; + if (usec != USEC_INFINITY) { + r = future_new_time_relative( + e, + CLOCK_MONOTONIC, + usec, + /* accuracy= */ 1, + /* result= */ 0, + &timer); + if (r < 0) + return r; + } + + r = sd_fiber_suspend(); + if (r < 0 && r != -ETIME) + return r; + + /* Always sweep fds with a non-blocking ppoll(): the timer and an fd readiness can resolve in + * the same event-loop tick (or the fd can become ready between the timer firing and us being + * scheduled), and ppoll() semantics give events precedence over the timeout in that case. */ + int n = RET_NERRNO(ppoll(fds, n_fds, &(const struct timespec) {}, /* sigmask= */ NULL)); + if (n != 0) + return n; + + /* No fds ready: distinguish our own timer from an external -ETIME. */ + if (timer && sd_future_state(timer) == SD_FUTURE_RESOLVED) + return 0; + + /* An IO future resolved with a revents mask (r > 0) but the readiness was already consumed + * by the time we swept — report 0 rather than leaking the bitmask as a (bogus) ppoll fd + * count to the caller. */ + if (r > 0) + return 0; + + return r; +} diff --git a/src/libsystemd/sd-future/test-fiber-io.c b/src/libsystemd/sd-future/test-fiber-io.c new file mode 100644 index 0000000000..aef38950b6 --- /dev/null +++ b/src/libsystemd/sd-future/test-fiber-io.c @@ -0,0 +1,1388 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ + +#include +#include +#include +#include +#include + +#include "sd-event.h" +#include "sd-future.h" + +#include "fd-util.h" +#include "tests.h" +#include "time-util.h" + +/* Test: Basic pipe I/O with sd-event */ + +typedef struct PipeIOContext { + int *pipefd; + int order; +} PipeIOContext; + +static int pipe_read_fiber(void *userdata) { + PipeIOContext *ctx = ASSERT_PTR(userdata); + char buf[64]; + ssize_t n; + + n = sd_fiber_read(ctx->pipefd[0], buf, sizeof(buf)); + if (n < 0) + return (int) n; + + /* Verify we read "hello" */ + if (n != 5 || memcmp(buf, "hello", 5) != 0) + return -EIO; + + return (int) n; +} + +TEST(fiber_io_basic) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC | O_NONBLOCK)); + + PipeIOContext ctx = { .pipefd = pipefd }; + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "pipe-read", pipe_read_fiber, &ctx, NULL, &f)); + + /* Write data to the pipe */ + ASSERT_OK_EQ_ERRNO(write(pipefd[1], "hello", 5), 5); + + /* Run the scheduler - should process the I/O */ + ASSERT_OK(sd_event_loop(e)); + + /* Verify fiber read the data */ + ASSERT_OK_EQ(sd_future_result(f), 5); +} + +static int pipe_read_order_fiber(void *userdata) { + PipeIOContext *ctx = ASSERT_PTR(userdata); + char buf[64]; + ssize_t n; + + /* Record that the read fiber started before attempting the blocking read */ + ASSERT_EQ(ctx->order, 0); + ctx->order = 1; + + n = sd_fiber_read(ctx->pipefd[0], buf, sizeof(buf)); + if (n < 0) + return (int) n; + + /* After resuming, verify the write fiber ran while we were suspended */ + ASSERT_EQ(ctx->order, 2); + + /* Verify we read "hello" */ + if (n != 5 || memcmp(buf, "hello", 5) != 0) + return -EIO; + + return (int) n; +} + +static int pipe_write_order_fiber(void *userdata) { + PipeIOContext *ctx = ASSERT_PTR(userdata); + + /* Verify the read fiber already ran and suspended before we started */ + ASSERT_EQ(ctx->order, 1); + ctx->order = 2; + + return sd_fiber_write(ctx->pipefd[1], "hello", STRLEN("hello")); +} + +TEST(fiber_io_read_write) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC | O_NONBLOCK)); + + PipeIOContext ctx = { .pipefd = pipefd }; + + /* Higher priority for the read fiber, which will run first and then suspend because no data is + * available. The write fiber will run second, write data to the pipe, causing the read fiber to get + * resumed. */ + _cleanup_(sd_future_unrefp) sd_future *fr = NULL, *fw = NULL; + ASSERT_OK(sd_fiber_new(e, "pipe-read", pipe_read_order_fiber, &ctx, NULL, &fr)); + ASSERT_OK(sd_future_set_priority(fr, 0)); + ASSERT_OK(sd_fiber_new(e, "pipe-write", pipe_write_order_fiber, &ctx, NULL, &fw)); + ASSERT_OK(sd_future_set_priority(fw, 1)); + + /* Run the scheduler - should process the I/O */ + ASSERT_OK(sd_event_loop(e)); + + /* Verify both fibers completed and the full read->suspend->write->resume sequence occurred */ + ASSERT_OK_EQ(sd_future_result(fr), 5); + ASSERT_OK_EQ(sd_future_result(fw), 5); +} + +/* Test: Multiple concurrent reads */ +static int concurrent_read_fiber(void *userdata) { + int *args = userdata; + int fd = args[0]; + int expected = args[1]; + char buf[64]; + ssize_t n; + + n = sd_fiber_read(fd, buf, sizeof buf); + if (n < 0) + return (int) n; + + if (n != 1 || buf[0] != (char) expected) + return -EIO; + + return 0; +} + +TEST(fiber_io_concurrent) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + sd_future *fibers[3] = {}; + CLEANUP_ELEMENTS(fibers, sd_future_unref_array_clear); + + /* Create 3 pipes and 3 fibers */ + int pipes[3][2]; + int args[3][2]; + for (size_t i = 0; i < ELEMENTSOF(fibers); i++) { + ASSERT_OK_ERRNO(pipe2(pipes[i], O_CLOEXEC | O_NONBLOCK)); + args[i][0] = pipes[i][0]; + args[i][1] = 'A' + i; + ASSERT_OK(sd_fiber_new(e, "concurrent-read", concurrent_read_fiber, args[i], NULL, &fibers[i])); + } + + /* Write data in reverse order */ + ASSERT_EQ(write(pipes[2][1], "C", 1), 1); + ASSERT_EQ(write(pipes[1][1], "B", 1), 1); + ASSERT_EQ(write(pipes[0][1], "A", 1), 1); + + /* Run until all complete */ + ASSERT_OK(sd_event_loop(e)); + + /* All should complete successfully */ + for (size_t i = 0; i < ELEMENTSOF(fibers); i++) { + ASSERT_OK(sd_future_result(fibers[i])); + safe_close_pair(pipes[i]); + } +} + +/* Test: Cancel fiber during I/O */ +static int blocking_read_fiber(void *userdata) { + int fd = PTR_TO_INT(userdata); + char buf[64]; + ssize_t n; + + n = sd_fiber_read(fd, buf, sizeof(buf)); + return (int) n; +} + +TEST(fiber_io_cancel) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC | O_NONBLOCK)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "blocking-read", blocking_read_fiber, INT_TO_PTR(pipefd[0]), NULL, &f)); + + /* Run once - fiber will suspend on read */ + ASSERT_OK_POSITIVE(sd_event_run(e, 0)); + + /* Fiber should be suspended now - add explicit check via state tracking */ + + /* Cancel the fiber */ + ASSERT_OK(sd_future_cancel(f)); + + /* Run to completion */ + ASSERT_OK(sd_event_loop(e)); + + /* Should be cancelled */ + ASSERT_ERROR(sd_future_result(f), ECANCELED); +} + +TEST(fiber_io_fallback) { + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC)); /* Note: blocking pipe */ + + char buf[STRLEN("fallback")] = {}; + ASSERT_OK_EQ(sd_fiber_write(pipefd[1], "fallback", sizeof(buf)), (ssize_t) sizeof(buf)); + ASSERT_OK_EQ(sd_fiber_read(pipefd[0], buf, sizeof(buf)), (ssize_t) sizeof(buf)); +} + +static int pipe_readv_order_fiber(void *userdata) { + PipeIOContext *ctx = ASSERT_PTR(userdata); + char buf1[5], buf2[5]; + struct iovec iov[] = { + { .iov_base = buf1, .iov_len = sizeof(buf1) }, + { .iov_base = buf2, .iov_len = sizeof(buf2) }, + }; + ssize_t n; + + /* Record that the read fiber started before attempting the blocking read */ + ASSERT_EQ(ctx->order, 0); + ctx->order = 1; + + /* This will initially block since no data is available */ + n = sd_fiber_readv(ctx->pipefd[0], iov, ELEMENTSOF(iov)); + if (n < 0) + return (int) n; + + /* After resuming, verify the write fiber ran while we were suspended */ + ASSERT_EQ(ctx->order, 2); + + if (n != 10 || memcmp(buf1, "fiber", 5) != 0 || memcmp(buf2, "readv", 5) != 0) + return -EIO; + + return (int) n; +} + +static int pipe_writev_order_fiber(void *userdata) { + PipeIOContext *ctx = ASSERT_PTR(userdata); + const char *part1 = "fiber"; + const char *part2 = "readv"; + struct iovec iov[] = { + { .iov_base = (void*) part1, .iov_len = 5 }, + { .iov_base = (void*) part2, .iov_len = 5 }, + }; + + /* Verify the read fiber already ran and suspended before we started */ + ASSERT_EQ(ctx->order, 1); + ctx->order = 2; + + return sd_fiber_writev(ctx->pipefd[1], iov, ELEMENTSOF(iov)); +} + +TEST(fiber_io_readv_writev) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC | O_NONBLOCK)); + + PipeIOContext ctx = { .pipefd = pipefd }; + + /* Higher priority for the read fiber, which will run first and then suspend because no data is + * available. The write fiber will run second, write data to the pipe, causing the read fiber to get + * resumed. */ + _cleanup_(sd_future_unrefp) sd_future *fr = NULL, *fw = NULL; + ASSERT_OK(sd_fiber_new(e, "pipe-readv", pipe_readv_order_fiber, &ctx, NULL, &fr)); + ASSERT_OK(sd_future_set_priority(fr, 0)); + ASSERT_OK(sd_fiber_new(e, "pipe-writev", pipe_writev_order_fiber, &ctx, NULL, &fw)); + ASSERT_OK(sd_future_set_priority(fw, 1)); + + /* Run the scheduler - should process the I/O */ + ASSERT_OK(sd_event_loop(e)); + + /* Verify both fibers completed and the full read->suspend->write->resume sequence occurred */ + ASSERT_OK_EQ(sd_future_result(fr), 10); + ASSERT_OK_EQ(sd_future_result(fw), 10); +} + +static int concurrent_readv_fiber(void *userdata) { + int *args = userdata; + int fd = args[0]; + int expected1 = args[1]; + int expected2 = args[2]; + char buf1[1], buf2[1]; + struct iovec iov[] = { + { .iov_base = buf1, .iov_len = sizeof(buf1) }, + { .iov_base = buf2, .iov_len = sizeof(buf2) }, + }; + ssize_t n; + + n = sd_fiber_readv(fd, iov, ELEMENTSOF(iov)); + if (n < 0) + return (int) n; + + if (n != 2 || buf1[0] != (char) expected1 || buf2[0] != (char) expected2) + return -EIO; + + return 0; +} + +TEST(fiber_io_readv_concurrent) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + sd_future *fibers[3] = {}; + CLEANUP_ELEMENTS(fibers, sd_future_unref_array_clear); + + /* Create 3 pipes and 3 fibers */ + int pipes[3][2]; + int args[3][3]; + for (size_t i = 0; i < ELEMENTSOF(fibers); i++) { + ASSERT_OK_ERRNO(pipe2(pipes[i], O_CLOEXEC | O_NONBLOCK)); + args[i][0] = pipes[i][0]; + args[i][1] = 'A' + i; + args[i][2] = 'a' + i; + ASSERT_OK(sd_fiber_new(e, "concurrent-readv", concurrent_readv_fiber, args[i], NULL, &fibers[i])); + } + + /* Write data in reverse order */ + ASSERT_EQ(write(pipes[2][1], "Cc", 2), 2); + ASSERT_EQ(write(pipes[1][1], "Bb", 2), 2); + ASSERT_EQ(write(pipes[0][1], "Aa", 2), 2); + + /* Run until all complete */ + ASSERT_OK(sd_event_loop(e)); + + /* All should complete successfully */ + for (size_t i = 0; i < ELEMENTSOF(fibers); i++) { + ASSERT_OK(sd_future_result(fibers[i])); + safe_close_pair(pipes[i]); + } +} + +typedef struct SocketIOContext { + int *sockfd; + int order; +} SocketIOContext; + +static int socket_send_order_fiber(void *userdata) { + SocketIOContext *ctx = ASSERT_PTR(userdata); + + /* Verify the recv fiber already ran and suspended before we started */ + ASSERT_EQ(ctx->order, 1); + ctx->order = 2; + + return sd_fiber_send(ctx->sockfd[0], "socket", STRLEN("socket"), 0); +} + +static int socket_recv_order_fiber(void *userdata) { + SocketIOContext *ctx = ASSERT_PTR(userdata); + char buf[64]; + ssize_t n; + + /* Record that the recv fiber started before attempting the blocking recv */ + ASSERT_EQ(ctx->order, 0); + ctx->order = 1; + + n = sd_fiber_recv(ctx->sockfd[1], buf, sizeof(buf), 0); + if (n < 0) + return (int) n; + + /* After resuming, verify the send fiber ran while we were suspended */ + ASSERT_EQ(ctx->order, 2); + + /* Verify we received "socket" */ + if (n != 6 || memcmp(buf, "socket", 6) != 0) + return -EIO; + + return (int) n; +} + +TEST(fiber_io_recv_send) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int sockfd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0, sockfd)); + + SocketIOContext ctx = { .sockfd = sockfd }; + + /* Higher priority for the recv fiber, which will run first and suspend */ + _cleanup_(sd_future_unrefp) sd_future *fs = NULL, *fr = NULL; + ASSERT_OK(sd_fiber_new(e, "socket-recv", socket_recv_order_fiber, &ctx, NULL, &fr)); + ASSERT_OK(sd_future_set_priority(fr, 0)); + ASSERT_OK(sd_fiber_new(e, "socket-send", socket_send_order_fiber, &ctx, NULL, &fs)); + ASSERT_OK(sd_future_set_priority(fs, 1)); + + ASSERT_OK(sd_event_loop(e)); + + /* Verify both fibers completed and the full recv->suspend->send->resume sequence occurred */ + ASSERT_OK_EQ(sd_future_result(fr), 6); + ASSERT_OK_EQ(sd_future_result(fs), 6); +} + +static int socket_recv_peek_fiber(void *userdata) { + int sockfd = PTR_TO_INT(userdata); + char buf1[64], buf2[64]; + ssize_t n1, n2; + + /* First peek at the data */ + n1 = sd_fiber_recv(sockfd, buf1, sizeof(buf1), MSG_PEEK); + if (n1 < 0) + return (int) n1; + + /* Then actually read it */ + n2 = sd_fiber_recv(sockfd, buf2, sizeof(buf2), 0); + if (n2 < 0) + return (int) n2; + + /* Both should have read the same data */ + if (n1 != n2 || memcmp(buf1, buf2, n1) != 0) + return -EIO; + + if (n1 != 4 || memcmp(buf1, "peek", 4) != 0) + return -EIO; + + return 0; +} + +TEST(fiber_io_recv_peek) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int sockfd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0, sockfd)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "socket-recv-peek", socket_recv_peek_fiber, INT_TO_PTR(sockfd[1]), NULL, &f)); + + /* Write data to the socket */ + ASSERT_OK_EQ_ERRNO(write(sockfd[0], "peek", 4), 4); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(f)); +} + +static int socket_connect_fiber(void *userdata) { + struct sockaddr_un *addr = userdata; + _cleanup_close_ int sockfd = -EBADF; + + sockfd = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0); + if (sockfd < 0) + return -errno; + + return sd_fiber_connect(sockfd, (struct sockaddr*) addr, sizeof(*addr)); +} + +TEST(fiber_io_connect) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + /* Create listening socket with abstract namespace */ + _cleanup_close_ int listen_fd = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0); + ASSERT_OK(listen_fd); + + /* Use abstract socket (starts with null byte) */ + struct sockaddr_un addr = { + .sun_family = AF_UNIX, + }; + addr.sun_path[0] = '\0'; + snprintf(addr.sun_path + 1, sizeof(addr.sun_path) - 1, "test-fiber-connect-%d", getpid()); + + ASSERT_OK(bind(listen_fd, (struct sockaddr*) &addr, sizeof(addr))); + ASSERT_OK(listen(listen_fd, 1)); + + /* Create fiber to connect */ + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "socket-connect", socket_connect_fiber, &addr, NULL, &f)); + + /* Run the event loop - connection should complete */ + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(f)); +} + +static int socket_sendmsg_fiber(void *userdata) { + int sockfd = PTR_TO_INT(userdata); + struct iovec iov = { + .iov_base = (void*) "message", + .iov_len = STRLEN("message"), + }; + struct msghdr msg = { + .msg_iov = &iov, + .msg_iovlen = 1, + }; + + return sd_fiber_sendmsg(sockfd, &msg, 0); +} + +static int socket_recvmsg_fiber(void *userdata) { + int sockfd = PTR_TO_INT(userdata); + char buf[64]; + struct iovec iov = { + .iov_base = buf, + .iov_len = sizeof(buf), + }; + struct msghdr msg = { + .msg_iov = &iov, + .msg_iovlen = 1, + }; + ssize_t n; + + n = sd_fiber_recvmsg(sockfd, &msg, 0); + if (n < 0) + return (int) n; + + if (n != 7 || memcmp(buf, "message", 7) != 0) + return -EIO; + + return (int) n; +} + +TEST(fiber_io_recvmsg_sendmsg) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int sockfd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0, sockfd)); + + _cleanup_(sd_future_unrefp) sd_future *fs = NULL, *fr = NULL; + ASSERT_OK(sd_fiber_new(e, "socket-recvmsg", socket_recvmsg_fiber, INT_TO_PTR(sockfd[1]), NULL, &fr)); + ASSERT_OK(sd_future_set_priority(fr, 1)); + ASSERT_OK(sd_fiber_new(e, "socket-sendmsg", socket_sendmsg_fiber, INT_TO_PTR(sockfd[0]), NULL, &fs)); + ASSERT_OK(sd_future_set_priority(fs, 0)); + + ASSERT_OK(sd_event_loop(e)); + + ASSERT_OK_EQ(sd_future_result(fr), 7); + ASSERT_OK_EQ(sd_future_result(fs), 7); +} + +static int socket_sendto_fiber(void *userdata) { + int sockfd = PTR_TO_INT(userdata); + + /* For socketpair dgram sockets, we can use NULL address since they're connected */ + return sd_fiber_sendto(sockfd, "datagram", STRLEN("datagram"), 0, NULL, 0); +} + +static int socket_recvfrom_fiber(void *userdata) { + int sockfd = PTR_TO_INT(userdata); + char buf[64]; + struct sockaddr_un addr; + socklen_t addr_len = sizeof(addr); + ssize_t n; + + n = sd_fiber_recvfrom(sockfd, buf, sizeof(buf), 0, + (struct sockaddr*) &addr, &addr_len); + if (n < 0) + return (int) n; + + if (n != 8 || memcmp(buf, "datagram", 8) != 0) + return -EIO; + + return (int) n; +} + +TEST(fiber_io_recvfrom_sendto) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int sockfd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_DGRAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0, sockfd)); + + _cleanup_(sd_future_unrefp) sd_future *fs = NULL, *fr = NULL; + ASSERT_OK(sd_fiber_new(e, "socket-recvfrom", socket_recvfrom_fiber, INT_TO_PTR(sockfd[1]), NULL, &fr)); + ASSERT_OK(sd_future_set_priority(fr, 1)); + ASSERT_OK(sd_fiber_new(e, "socket-sendto", socket_sendto_fiber, INT_TO_PTR(sockfd[0]), NULL, &fs)); + ASSERT_OK(sd_future_set_priority(fs, 0)); + + ASSERT_OK(sd_event_loop(e)); + + ASSERT_OK_EQ(sd_future_result(fr), 8); + ASSERT_OK_EQ(sd_future_result(fs), 8); +} + +static int socket_sendmsg_fd_fiber(void *userdata) { + int *args = userdata; + int sockfd = args[0]; + int fd_to_send = args[1]; + struct iovec iov = { + .iov_base = (void*) "X", + .iov_len = 1, + }; + union { + struct cmsghdr cmsghdr; + uint8_t buf[CMSG_SPACE(sizeof(int))]; + } control = {}; + struct msghdr msg = { + .msg_iov = &iov, + .msg_iovlen = 1, + .msg_control = &control, + .msg_controllen = sizeof(control), + }; + struct cmsghdr *cmsg; + + cmsg = CMSG_FIRSTHDR(&msg); + cmsg->cmsg_level = SOL_SOCKET; + cmsg->cmsg_type = SCM_RIGHTS; + cmsg->cmsg_len = CMSG_LEN(sizeof(int)); + memcpy(CMSG_DATA(cmsg), &fd_to_send, sizeof(int)); + + return sd_fiber_sendmsg(sockfd, &msg, 0); +} + +static int socket_recvmsg_fd_fiber(void *userdata) { + int sockfd = PTR_TO_INT(userdata); + char buf[1]; + struct iovec iov = { + .iov_base = buf, + .iov_len = sizeof(buf), + }; + union { + struct cmsghdr cmsghdr; + uint8_t buf[CMSG_SPACE(sizeof(int))]; + } control = {}; + struct msghdr msg = { + .msg_iov = &iov, + .msg_iovlen = 1, + .msg_control = &control, + .msg_controllen = sizeof(control), + }; + struct cmsghdr *cmsg; + int received_fd; + ssize_t n; + + n = sd_fiber_recvmsg(sockfd, &msg, 0); + if (n < 0) + return (int) n; + + if (n != 1 || buf[0] != 'X') + return -EIO; + + /* Extract the file descriptor */ + cmsg = CMSG_FIRSTHDR(&msg); + if (!cmsg || cmsg->cmsg_level != SOL_SOCKET || cmsg->cmsg_type != SCM_RIGHTS) + return -EIO; + + memcpy(&received_fd, CMSG_DATA(cmsg), sizeof(int)); + + /* Verify we can use the fd */ + if (fcntl(received_fd, F_GETFD) < 0) + return -errno; + + close(received_fd); + return 0; +} + +TEST(fiber_io_sendmsg_recvmsg_fd) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int sockfd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0, sockfd)); + + /* Create a test file descriptor to send */ + _cleanup_close_ int test_fd = open("/dev/null", O_RDONLY | O_CLOEXEC); + ASSERT_OK_ERRNO(test_fd); + + _cleanup_(sd_future_unrefp) sd_future *fs = NULL, *fr = NULL; + int args[2] = { sockfd[0], test_fd }; + ASSERT_OK(sd_fiber_new(e, "socket-recvmsg-fd", socket_recvmsg_fd_fiber, INT_TO_PTR(sockfd[1]), NULL, &fr)); + ASSERT_OK(sd_future_set_priority(fr, 1)); + ASSERT_OK(sd_fiber_new(e, "socket-sendmsg-fd", socket_sendmsg_fd_fiber, args, NULL, &fs)); + ASSERT_OK(sd_future_set_priority(fs, 0)); + + ASSERT_OK(sd_event_loop(e)); + + ASSERT_OK(sd_future_result(fr)); + ASSERT_OK_EQ(sd_future_result(fs), 1); +} + +TEST(fiber_io_socket_fallback) { + _cleanup_close_pair_ int sockfd[2] = EBADF_PAIR; + char buf[STRLEN("fallback")] = {}; + + ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC, 0, sockfd)); + + /* Test send/recv without fiber context */ + ASSERT_OK_EQ(sd_fiber_send(sockfd[0], "fallback", sizeof(buf), 0), (ssize_t) sizeof(buf)); + ASSERT_OK_EQ(sd_fiber_recv(sockfd[1], buf, sizeof(buf), 0), (ssize_t) sizeof(buf)); + + /* Test sendto/recvfrom without fiber context */ + ASSERT_OK_EQ(sd_fiber_sendto(sockfd[0], "fallback", sizeof(buf), 0, NULL, 0), (ssize_t) sizeof(buf)); + ASSERT_OK_EQ(sd_fiber_recvfrom(sockfd[1], buf, sizeof(buf), 0, NULL, NULL), (ssize_t) sizeof(buf)); +} + +static int blocking_recv_fiber(void *userdata) { + int sockfd = PTR_TO_INT(userdata); + char buf[64]; + + return sd_fiber_recv(sockfd, buf, sizeof(buf), 0); +} + +TEST(fiber_io_socket_cancel) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int sockfd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0, sockfd)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "blocking-recv", blocking_recv_fiber, INT_TO_PTR(sockfd[0]), NULL, &f)); + + /* Run once - fiber will suspend on recv */ + ASSERT_OK_POSITIVE(sd_event_run(e, 0)); + + /* Cancel the fiber */ + ASSERT_OK(sd_future_cancel(f)); + + /* Run to completion */ + ASSERT_OK(sd_event_loop(e)); + + /* Should be cancelled */ + ASSERT_ERROR(sd_future_result(f), ECANCELED); +} + +/* Test: Basic accept operation */ +static int accept_fiber(void *userdata) { + int listen_fd = PTR_TO_INT(userdata); + struct sockaddr_un addr; + socklen_t addr_len = sizeof(addr); + int client_fd; + + client_fd = sd_fiber_accept(listen_fd, (struct sockaddr*) &addr, &addr_len, SOCK_CLOEXEC); + if (client_fd < 0) + return client_fd; + + close(client_fd); + return 0; +} + +TEST(fiber_io_accept_basic) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + /* Create listening socket with abstract namespace */ + _cleanup_close_ int listen_fd = -EBADF; + ASSERT_OK_ERRNO(listen_fd = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0)); + + struct sockaddr_un addr = { + .sun_family = AF_UNIX, + }; + addr.sun_path[0] = '\0'; + snprintf(addr.sun_path + 1, sizeof(addr.sun_path) - 1, "test-fiber-accept-%d", getpid()); + + ASSERT_OK_ERRNO(bind(listen_fd, (struct sockaddr*) &addr, sizeof(addr))); + ASSERT_OK_ERRNO(listen(listen_fd, 1)); + + /* Create fiber to accept connection */ + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "accept", accept_fiber, INT_TO_PTR(listen_fd), NULL, &f)); + + /* Connect from outside fiber context */ + _cleanup_close_ int connect_fd = -EBADF; + ASSERT_OK(connect_fd = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0)); + ASSERT_OK(connect(connect_fd, (struct sockaddr*) &addr, sizeof(addr))); + + /* Run the event loop - accept should complete */ + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(f)); +} + +/* Test: Multiple sequential accepts */ +static int accept_multiple_fiber(void *userdata) { + int listen_fd = PTR_TO_INT(userdata); + struct sockaddr_un addr; + socklen_t addr_len; + int count = 0; + + for (int i = 0; i < 3; i++) { + _cleanup_close_ int client_fd = -EBADF; + + addr_len = sizeof(addr); + client_fd = sd_fiber_accept(listen_fd, (struct sockaddr*) &addr, &addr_len, SOCK_CLOEXEC); + if (client_fd < 0) + return client_fd; + + count++; + } + + return count; +} + +TEST(fiber_io_accept_multiple) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + /* Create listening socket */ + _cleanup_close_ int listen_fd = -EBADF; + ASSERT_OK(listen_fd = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0)); + + struct sockaddr_un addr = { + .sun_family = AF_UNIX, + }; + addr.sun_path[0] = '\0'; + snprintf(addr.sun_path + 1, sizeof(addr.sun_path) - 1, "test-fiber-accept-multi-%d", getpid()); + + ASSERT_OK(bind(listen_fd, (struct sockaddr*) &addr, sizeof(addr))); + ASSERT_OK(listen(listen_fd, 5)); + + /* Create fiber to accept multiple connections */ + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "accept-multiple", accept_multiple_fiber, INT_TO_PTR(listen_fd), NULL, &f)); + + /* Connect multiple times */ + int connect_fds[3] = { -EBADF, -EBADF, -EBADF }; + for (size_t i = 0; i < 3; i++) { + connect_fds[i] = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0); + ASSERT_OK(connect_fds[i]); + ASSERT_OK(connect(connect_fds[i], (struct sockaddr*) &addr, sizeof(addr))); + } + + /* Run the event loop */ + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK_EQ(sd_future_result(f), 3); + + /* Clean up connection fds */ + for (size_t i = 0; i < 3; i++) + safe_close(connect_fds[i]); +} + +/* Test: Accept and exchange data */ +static int accept_and_read_fiber(void *userdata) { + int listen_fd = PTR_TO_INT(userdata); + _cleanup_close_ int client_fd = -EBADF; + char buf[64]; + ssize_t n; + + client_fd = sd_fiber_accept(listen_fd, NULL, NULL, SOCK_CLOEXEC); + if (client_fd < 0) + return client_fd; + + n = sd_fiber_read(client_fd, buf, sizeof(buf)); + if (n < 0) + return (int) n; + + if (n != 5 || memcmp(buf, "hello", 5) != 0) + return -EIO; + + return 0; +} + +TEST(fiber_io_accept_and_read) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + /* Create listening socket */ + _cleanup_close_ int listen_fd = -EBADF; + ASSERT_OK(listen_fd = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0)); + + struct sockaddr_un addr = { + .sun_family = AF_UNIX, + }; + addr.sun_path[0] = '\0'; + snprintf(addr.sun_path + 1, sizeof(addr.sun_path) - 1, "test-fiber-accept-read-%d", getpid()); + + ASSERT_OK(bind(listen_fd, (struct sockaddr*) &addr, sizeof(addr))); + ASSERT_OK(listen(listen_fd, 1)); + + /* Create fiber to accept and read */ + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "accept-and-read", accept_and_read_fiber, INT_TO_PTR(listen_fd), NULL, &f)); + + /* Connect and send data */ + _cleanup_close_ int connect_fd = -EBADF; + ASSERT_OK(connect_fd = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0)); + ASSERT_OK(connect(connect_fd, (struct sockaddr*) &addr, sizeof(addr))); + ASSERT_OK_EQ_ERRNO(write(connect_fd, "hello", 5), 5); + + /* Run the event loop */ + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(f)); +} + +/* Test: poll with single fd ready immediately */ +static int poll_immediate_fiber(void *userdata) { + int *pipefd = userdata; + struct pollfd fds[] = { + { .fd = pipefd[0], .events = POLLIN }, + }; + int r; + + r = sd_fiber_ppoll(fds, ELEMENTSOF(fds), NULL, NULL); + if (r < 0) + return r; + + /* Should have one fd ready */ + if (r != 1) + return -EIO; + + if (!(fds[0].revents & POLLIN)) + return -EIO; + + return 0; +} + +TEST(fiber_poll_immediate) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC | O_NONBLOCK)); + + /* Write data before creating fiber */ + ASSERT_OK_EQ_ERRNO(write(pipefd[1], "X", 1), 1); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "poll-immediate", poll_immediate_fiber, pipefd, NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(f)); +} + +/* Test: poll with fd that becomes ready after suspension */ +static int poll_fiber(void *userdata) { + int *pipefd = userdata; + struct pollfd fds[] = { + { .fd = pipefd[0], .events = POLLIN }, + }; + int r; + + r = sd_fiber_ppoll(fds, ELEMENTSOF(fds), NULL, NULL); + if (r < 0) + return r; + + if (r != 1 || !(fds[0].revents & POLLIN)) + return -EIO; + + /* Read the data */ + char buf[1]; + if (read(pipefd[0], buf, 1) != 1 || buf[0] != 'Y') + return -EIO; + + return 0; +} + +TEST(fiber_poll) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC | O_NONBLOCK)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "poll-suspend", poll_fiber, pipefd, NULL, &f)); + + /* Run once - fiber will suspend on poll */ + ASSERT_OK_POSITIVE(sd_event_run(e, 0)); + + /* Write data to wake it up */ + ASSERT_OK_EQ_ERRNO(write(pipefd[1], "Y", 1), 1); + + /* Complete execution */ + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(f)); +} + +/* Test: poll with multiple fds */ +static int poll_multiple_fiber(void *userdata) { + int (*pipes)[2] = userdata; + struct pollfd fds[] = { + { .fd = pipes[0][0], .events = POLLIN }, + { .fd = pipes[1][0], .events = POLLIN }, + { .fd = pipes[2][0], .events = POLLIN }, + }; + int r; + + r = sd_fiber_ppoll(fds, ELEMENTSOF(fds), NULL, NULL); + if (r < 0) + return r; + + /* Should have all three ready */ + if (r != 3) + return -EIO; + + for (size_t i = 0; i < 3; i++) { + if (!(fds[i].revents & POLLIN)) + return -EIO; + + char buf[1]; + if (read(fds[i].fd, buf, 1) != 1 || buf[0] != (char) ('A' + i)) + return -EIO; + } + + return 0; +} + +TEST(fiber_poll_multiple) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + /* Create three pipes */ + int pipes[3][2]; + for (size_t i = 0; i < 3; i++) + ASSERT_OK_ERRNO(pipe2(pipes[i], O_CLOEXEC | O_NONBLOCK)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "poll-multiple", poll_multiple_fiber, pipes, NULL, &f)); + + /* Run once - fiber will suspend waiting for data */ + ASSERT_OK_POSITIVE(sd_event_run(e, 0)); + + /* Write to all three pipes in different order */ + ASSERT_OK_EQ_ERRNO(write(pipes[2][1], "C", 1), 1); + ASSERT_OK_EQ_ERRNO(write(pipes[0][1], "A", 1), 1); + ASSERT_OK_EQ_ERRNO(write(pipes[1][1], "B", 1), 1); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(f)); + + for (size_t i = 0; i < 3; i++) + safe_close_pair(pipes[i]); +} + +/* Test: poll with POLLOUT (write readiness) */ +static int poll_pollout_fiber(void *userdata) { + int *pipefd = userdata; + struct pollfd fds[] = { + { .fd = pipefd[1], .events = POLLOUT }, + }; + int r; + + r = sd_fiber_ppoll(fds, ELEMENTSOF(fds), NULL, NULL); + if (r < 0) + return r; + + if (r != 1 || !(fds[0].revents & POLLOUT)) + return -EIO; + + /* Pipe should be writable */ + if (write(pipefd[1], "Z", 1) != 1) + return -errno; + + return 0; +} + +TEST(fiber_poll_pollout) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC | O_NONBLOCK)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "poll-pollout", poll_pollout_fiber, pipefd, NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(f)); + + /* Verify data was written */ + char buf[1]; + ASSERT_OK_EQ_ERRNO(read(pipefd[0], buf, 1), 1); + ASSERT_EQ(buf[0], 'Z'); +} + +/* Test: poll with timeout that expires */ +static int poll_timeout_fiber(void *userdata) { + int *pipefd = userdata; + struct pollfd fds[] = { + { .fd = pipefd[0], .events = POLLIN }, + }; + int r; + + /* Poll with 100ms timeout - no data will arrive */ + r = sd_fiber_ppoll(fds, ELEMENTSOF(fds), &(struct timespec) { .tv_nsec = 100 * NSEC_PER_MSEC }, NULL); + if (r < 0) + return r; + + /* Should timeout with no fds ready */ + if (r != 0) + return -EIO; + + return 0; +} + +TEST(fiber_poll_timeout) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC | O_NONBLOCK)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "poll-timeout", poll_timeout_fiber, pipefd, NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(f)); +} + +/* Test: poll with zero timeout (should not block) */ +static int poll_zero_timeout_fiber(void *userdata) { + int *pipefd = userdata; + struct pollfd fds[] = { + { .fd = pipefd[0], .events = POLLIN }, + }; + int r; + + /* Poll with zero timeout - should return immediately */ + r = sd_fiber_ppoll(fds, ELEMENTSOF(fds), &(struct timespec) {}, NULL); + if (r < 0) + return r; + + /* No data available, so should return 0 */ + if (r != 0) + return -EIO; + + /* Now write data */ + if (write(pipefd[1], "Q", 1) != 1) + return -errno; + + /* Poll again with zero timeout - should see data */ + r = sd_fiber_ppoll(fds, ELEMENTSOF(fds), NULL, NULL); + if (r < 0) + return r; + + if (r != 1 || !(fds[0].revents & POLLIN)) + return -EIO; + + return 0; +} + +TEST(fiber_poll_zero_timeout) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC | O_NONBLOCK)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "poll-zero-timeout", poll_zero_timeout_fiber, pipefd, NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(f)); +} + +/* Test: poll with zero fds and zero timeout (should return immediately) */ +static int poll_zero_fds_fiber(void *userdata) { + return sd_fiber_ppoll(NULL, 0, &(struct timespec) {}, NULL); +} + +TEST(fiber_poll_zero_fds) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "poll-zero-fds", poll_zero_fds_fiber, NULL, NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK_EQ(sd_future_result(f), 0); +} + +/* Test: poll with zero fds and no timeout has no possible wakeup, must reject with -EINVAL */ +static int poll_zero_fds_no_timeout_fiber(void *userdata) { + return sd_fiber_ppoll(NULL, 0, NULL, NULL); +} + +TEST(fiber_poll_zero_fds_no_timeout) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "poll-zero-fds-no-timeout", poll_zero_fds_no_timeout_fiber, NULL, NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_ERROR(sd_future_result(f), EINVAL); +} + +/* Test: poll with negative fd (should be ignored) */ +static int poll_negative_fd_fiber(void *userdata) { + int *pipefd = userdata; + struct pollfd fds[] = { + { .fd = -1, .events = POLLIN }, + { .fd = pipefd[0], .events = POLLIN }, + }; + int r; + + r = sd_fiber_ppoll(fds, ELEMENTSOF(fds), NULL, NULL); + if (r < 0) + return r; + + /* Only the second fd should be ready */ + if (r != 1 || !(fds[1].revents & POLLIN)) + return -EIO; + + /* First fd should have no events */ + if (fds[0].revents != 0) + return -EIO; + + return 0; +} + +TEST(fiber_poll_negative_fd) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC | O_NONBLOCK)); + + /* Write data before creating fiber */ + ASSERT_OK_EQ_ERRNO(write(pipefd[1], "N", 1), 1); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "poll-negative-fd", poll_negative_fd_fiber, pipefd, NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(f)); +} + +/* Test: Multiple fibers waiting on the same fd */ +typedef struct SharedFdArgs { + int pipefd; + int *counter; +} SharedFdArgs; + +static int shared_fd_read_fiber(void *userdata) { + SharedFdArgs *args = ASSERT_PTR(userdata); + char buf[1]; + ssize_t n; + + n = sd_fiber_read(args->pipefd, buf, sizeof(buf)); + if (n < 0) + return (int) n; + + if (n != 1) + return -EIO; + + /* Increment counter to track successful reads */ + (*args->counter)++; + + return 0; +} + +TEST(fiber_io_same_fd_multiple_fibers) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC | O_NONBLOCK)); + + /* Create 3 fibers all waiting on the same pipe read end */ + sd_future *fibers[3] = {}; + CLEANUP_ELEMENTS(fibers, sd_future_unref_array_clear); + SharedFdArgs args[3]; + int counter = 0; + + for (size_t i = 0; i < ELEMENTSOF(fibers); i++) { + args[i].pipefd = pipefd[0]; + args[i].counter = &counter; + ASSERT_OK(sd_fiber_new(e, "shared-fd-read", shared_fd_read_fiber, &args[i], NULL, &fibers[i])); + } + + /* All fibers should suspend waiting for data */ + for (size_t i = 0; i < ELEMENTSOF(fibers); i++) + ASSERT_OK_POSITIVE(sd_event_run(e, 0)); + + /* Write 3 bytes - each byte will wake one fiber */ + ASSERT_OK_EQ_ERRNO(write(pipefd[1], "ABC", 3), 3); + + /* Run until all fibers complete */ + ASSERT_OK(sd_event_loop(e)); + + /* All should complete successfully and each should have read one byte */ + for (size_t i = 0; i < ELEMENTSOF(fibers); i++) + ASSERT_OK(sd_future_result(fibers[i])); + + ASSERT_EQ(counter, 3); +} + +static int blocking_fd_preserve_fiber(void *userdata) { + int *pipefd = ASSERT_PTR(userdata); + char buf[8] = {}; + ssize_t n; + + /* The pipe has data pre-filled, so this should succeed immediately on the fast path. + * This exercises the fd blocking state restore: fiber_io_operation() temporarily sets the fd + * to nonblocking, and must restore it to blocking on the success path. */ + n = sd_fiber_read(pipefd[0], buf, sizeof(buf)); + if (n < 0) + return (int) n; + + if ((size_t) n != sizeof(buf) || memcmp(buf, "blocking", sizeof(buf)) != 0) + return -EIO; + + return 0; +} + +TEST(fiber_io_blocking_fd_preserved) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + /* Create a blocking pipe (no O_NONBLOCK) */ + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC)); + + /* Pre-fill the pipe so the read will succeed immediately */ + ASSERT_OK_EQ_ERRNO(write(pipefd[1], "blocking", 8), 8); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "blocking-fd-preserve", blocking_fd_preserve_fiber, pipefd, NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(f)); + + /* Verify the read end is still in blocking mode after the fiber completed */ + ASSERT_OK_ZERO(fd_nonblock(pipefd[0], false)); +} + +static int socket_connect_blocking_fiber(void *userdata) { + struct sockaddr_un *addr = userdata; + _cleanup_close_ int sockfd = -EBADF; + + /* Use a blocking socket (no SOCK_NONBLOCK). sd_fiber_connect() should temporarily set it + * to nonblocking, handle the EINPROGRESS path with getsockopt(SO_ERROR), and restore + * the blocking state. */ + sockfd = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC, 0); + if (sockfd < 0) + return -errno; + + int r = sd_fiber_connect(sockfd, (struct sockaddr*) addr, sizeof(*addr)); + if (r < 0) + return r; + + /* Verify the socket is back in blocking mode */ + r = fd_nonblock(sockfd, false); + if (r < 0) + return r; + if (r > 0) + return -EBUSY; /* fd was nonblocking, but should have been restored to blocking */ + + return 0; +} + +TEST(fiber_io_connect_blocking) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + /* Create listening socket */ + _cleanup_close_ int listen_fd = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0); + ASSERT_OK(listen_fd); + + struct sockaddr_un addr = { + .sun_family = AF_UNIX, + }; + addr.sun_path[0] = '\0'; + snprintf(addr.sun_path + 1, sizeof(addr.sun_path) - 1, "test-fiber-connect-blocking-%d", getpid()); + + ASSERT_OK(bind(listen_fd, (struct sockaddr*) &addr, sizeof(addr))); + ASSERT_OK(listen(listen_fd, 1)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "connect-blocking", socket_connect_blocking_fiber, &addr, NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(f)); +} + +DEFINE_TEST_MAIN(LOG_DEBUG); diff --git a/src/systemd/sd-future.h b/src/systemd/sd-future.h index 9d0d03acf4..5e0fa22525 100644 --- a/src/systemd/sd-future.h +++ b/src/systemd/sd-future.h @@ -17,10 +17,18 @@ along with systemd; If not, see . ***/ +#include + #include "_sd-common.h" _SD_BEGIN_DECLARATIONS; +struct iovec; +struct pollfd; +struct sockaddr; +struct msghdr; +struct timespec; + typedef struct sd_event sd_event; typedef struct sd_future sd_future; typedef struct sd_future_ops sd_future_ops; @@ -96,6 +104,23 @@ sd_future* sd_fiber_timeout(uint64_t timeout); _SD_CONCATENATE(_sd_fto_b_, uniq); \ _SD_CONCATENATE(_sd_fto_b_, uniq) = NULL) +/* Fiber I/O operations - use sd-event for non-blocking I/O when in fiber context */ +ssize_t sd_fiber_read(int fd, void *buf, size_t count); +ssize_t sd_fiber_write(int fd, const void *buf, size_t count); +ssize_t sd_fiber_readv(int fd, const struct iovec *iov, int iovcnt); +ssize_t sd_fiber_writev(int fd, const struct iovec *iov, int iovcnt); +ssize_t sd_fiber_recv(int sockfd, void *buf, size_t len, int flags); +ssize_t sd_fiber_send(int sockfd, const void *buf, size_t len, int flags); +int sd_fiber_connect(int sockfd, const struct sockaddr *addr, socklen_t addrlen); +ssize_t sd_fiber_recvmsg(int sockfd, struct msghdr *msg, int flags); +ssize_t sd_fiber_sendmsg(int sockfd, const struct msghdr *msg, int flags); +ssize_t sd_fiber_recvfrom(int sockfd, void *buf, size_t len, int flags, struct sockaddr *src_addr, socklen_t *addrlen); +ssize_t sd_fiber_sendto(int sockfd, const void *buf, size_t len, int flags, const struct sockaddr *dest_addr, socklen_t addrlen); +int sd_fiber_accept(int sockfd, struct sockaddr *addr, socklen_t *addrlen, int flags); +#ifndef __STRICT_ANSI__ +int sd_fiber_ppoll(struct pollfd *fds, size_t n_fds, const struct timespec *timeout, const sigset_t *sigmask); +#endif + _SD_END_DECLARATIONS; #endif From 7bc793e21f2d4bf67bd311545270bc515fe63ad9 Mon Sep 17 00:00:00 2001 From: Daan De Meyer Date: Sat, 25 Apr 2026 22:06:54 +0200 Subject: [PATCH 3/8] sd-future: make src/basic blocking helpers fiber-aware MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Some helpers in src/basic — ppoll_usec_full() (used by fd_wait_for_event()), loop_read(), loop_read_exact(), loop_write_full() and pidref_wait_for_terminate_full() — block the calling thread. That's the right behaviour outside a fiber but not inside one, where blocking the thread also stalls every other fiber running on the same event loop. Rewriting every caller to pick a fiber or non-fiber variant explicitly would be a lot of churn and would split otherwise-shared code paths in two. Instead, the helpers detect at runtime whether they're running on a fiber and dispatch to a suspending variant when they are. FiberOps in fiber-ops.h holds five function pointers (ppoll, read, write, timeout, cancel_wait_unref); a fiber_ops global constant is populated whenever we enter a fiber with functions that delegate to suspending variants of common syscalls. With this approach, the variants themselves stay in libsystemd which is required because they make use of sd-event. - loop_read()/loop_read_exact() take the fiber read hook on a fiber unless the caller asked for a non-blocking attempt (do_poll=false) and the fd is already non-blocking — in that case we fall through to read() to preserve the existing return-EAGAIN-immediately semantic. The hook itself suspends on EAGAIN until data is available, so neither the do_poll knob nor the explicit fd_wait_for_event() retry loop are needed on the fiber path. - loop_write_full() likewise takes the fiber write hook on a fiber, except when timeout=0 with an already-non-blocking fd (preserving the fast-return-EAGAIN semantic). The fiber path runs inside a FIBER_OPS_TIMEOUT() scope so the caller's timeout is honoured via a deadline future, mirroring SD_FIBER_TIMEOUT() but reachable from src/basic without pulling in sd-future.h. - pidref_wait_for_terminate_full() polls the pidfd via fd_wait_for_event() before each waitid() when either a finite timeout is set or we're on a fiber, and requires pidref->fd >= 0 in those cases (returning -ENOMEDIUM otherwise — extending the rule that already applied to finite timeouts). The poll suspends the fiber via the ppoll hook above; the subsequent waitid() doesn't block because the pidfd is already signalled. --- src/basic/basic-forward.h | 1 + src/basic/fiber-ops.c | 51 ++ src/basic/fiber-ops.h | 34 ++ src/basic/io-util.c | 84 +++- src/basic/meson.build | 1 + src/basic/pidref.c | 23 +- src/libsystemd/meson.build | 1 + src/libsystemd/sd-future/fiber.c | 17 +- src/libsystemd/sd-future/test-fiber-ops.c | 574 ++++++++++++++++++++++ 9 files changed, 763 insertions(+), 23 deletions(-) create mode 100644 src/basic/fiber-ops.c create mode 100644 src/basic/fiber-ops.h create mode 100644 src/libsystemd/sd-future/test-fiber-ops.c diff --git a/src/basic/basic-forward.h b/src/basic/basic-forward.h index 5f9109cb17..6fa9f3bf86 100644 --- a/src/basic/basic-forward.h +++ b/src/basic/basic-forward.h @@ -112,6 +112,7 @@ typedef enum UnitType UnitType; typedef enum WaitFlags WaitFlags; typedef struct Fiber Fiber; +typedef struct FiberOps FiberOps; typedef struct Hashmap Hashmap; typedef struct HashmapBase HashmapBase; typedef struct IteratedCache IteratedCache; diff --git a/src/basic/fiber-ops.c b/src/basic/fiber-ops.c new file mode 100644 index 0000000000..330630f9dc --- /dev/null +++ b/src/basic/fiber-ops.c @@ -0,0 +1,51 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +#include +#include +#include + +#include "errno-util.h" +#include "fiber-ops.h" + +static thread_local const FiberOps *fiber_ops = NULL; + +bool fiber_ops_is_set(void) { + return fiber_ops != NULL; +} + +void fiber_ops_set(const FiberOps *ops) { + fiber_ops = ops; +} + +int fiber_ops_ppoll(struct pollfd *fds, size_t n_fds, const struct timespec *timeout, const sigset_t *sigmask) { + if (fiber_ops) + return fiber_ops->ppoll(fds, n_fds, timeout, sigmask); + + return RET_NERRNO(ppoll(fds, n_fds, timeout, sigmask)); +} + +ssize_t fiber_ops_read(int fd, void *buf, size_t count) { + if (fiber_ops) + return fiber_ops->read(fd, buf, count); + + ssize_t n = read(fd, buf, count); + return n < 0 ? -errno : n; +} + +ssize_t fiber_ops_write(int fd, const void *buf, size_t count) { + if (fiber_ops) + return fiber_ops->write(fd, buf, count); + + return RET_NERRNO(write(fd, buf, count)); +} + +sd_future* fiber_ops_timeout(uint64_t timeout) { + assert(fiber_ops); + + return fiber_ops->timeout(timeout); +} + +sd_future* fiber_ops_cancel_wait_unref(sd_future *f) { + assert(fiber_ops); + + return fiber_ops->cancel_wait_unref(f); +} diff --git a/src/basic/fiber-ops.h b/src/basic/fiber-ops.h new file mode 100644 index 0000000000..64bd82353b --- /dev/null +++ b/src/basic/fiber-ops.h @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +#pragma once + +#include "basic-forward.h" + +typedef struct sd_future sd_future; + +/* Hooks installed on a fiber so that functions in src/basic can transparently defer to the suspending + * variants in sd-future when invoked from a running fiber. Populated by sd_fiber_new() with pointers to the + * implementations in fiber-ops.c. */ +typedef struct FiberOps { + int (*ppoll)(struct pollfd *fds, size_t n_fds, const struct timespec *timeout, const sigset_t *sigmask); + ssize_t (*read)(int fd, void *buf, size_t count); + ssize_t (*write)(int fd, const void *buf, size_t count); + sd_future* (*timeout)(uint64_t timeout); + sd_future* (*cancel_wait_unref)(sd_future *f); +} FiberOps; + +bool fiber_ops_is_set(void); +void fiber_ops_set(const FiberOps *fiber_ops); + +int fiber_ops_ppoll(struct pollfd *fds, size_t n_fds, const struct timespec *timeout, const sigset_t *sigmask); +ssize_t fiber_ops_read(int fd, void *buf, size_t count); +ssize_t fiber_ops_write(int fd, const void *buf, size_t count); + +/* Mirror of SD_FIBER_TIMEOUT() for code under src/basic that doesn't include sd-future.h: dispatches + * through FiberOps so the actual sd_fiber_timeout() implementation lives in libsystemd. */ +sd_future* fiber_ops_timeout(uint64_t timeout); +sd_future* fiber_ops_cancel_wait_unref(sd_future *f); +DEFINE_TRIVIAL_CLEANUP_FUNC(sd_future*, fiber_ops_cancel_wait_unref); + +#define FIBER_OPS_TIMEOUT(timeout) _FIBER_OPS_TIMEOUT(UNIQ, (timeout)) +#define _FIBER_OPS_TIMEOUT(uniq, timeout) \ + _unused_ _cleanup_(fiber_ops_cancel_wait_unrefp) sd_future *UNIQ_T(_fot_, uniq) = fiber_ops_timeout(timeout) diff --git a/src/basic/io-util.c b/src/basic/io-util.c index b4f643b572..3dbc3670a4 100644 --- a/src/basic/io-util.c +++ b/src/basic/io-util.c @@ -1,5 +1,6 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ +#include #include #include #include @@ -8,6 +9,7 @@ #include #include "errno-util.h" +#include "fiber-ops.h" #include "io-util.h" #include "time-util.h" @@ -69,25 +71,44 @@ ssize_t loop_read(int fd, void *buf, size_t nbytes, bool do_poll) { if (nbytes > (size_t) SSIZE_MAX) return -EINVAL; + /* do_poll == false means "don't wait, return what we have if EAGAIN". If the fd is already + * non-blocking, read() can't block the thread, so the non-fiber path satisfies that semantic + * correctly even from a fiber. Only use the fiber path when the fd is blocking (where read() + * would otherwise block the entire event loop). */ + int flags = 0; + if (fiber_ops_is_set() && !do_poll) { + flags = fcntl(fd, F_GETFL); + if (flags < 0) + return -errno; + } + do { ssize_t k; - k = read(fd, p, nbytes); - if (k < 0) { - if (errno == EINTR) - continue; + if (fiber_ops_is_set() && (do_poll || !FLAGS_SET(flags, O_NONBLOCK))) { + /* On a fiber the read op suspends on EAGAIN until data is available, so we don't + * need a separate poll step or the do_poll knob. */ + k = fiber_ops_read(fd, p, nbytes); + if (k < 0) + return n > 0 ? n : k; + } else { + k = read(fd, p, nbytes); + if (k < 0) { + if (errno == EINTR) + continue; - if (errno == EAGAIN && do_poll) { + if (errno == EAGAIN && do_poll) { - /* We knowingly ignore any return value here, - * and expect that any error/EOF is reported - * via read() */ + /* We knowingly ignore any return value here, + * and expect that any error/EOF is reported + * via read() */ - (void) fd_wait_for_event(fd, POLLIN, USEC_INFINITY); - continue; + (void) fd_wait_for_event(fd, POLLIN, USEC_INFINITY); + continue; + } + + return n > 0 ? n : -errno; } - - return n > 0 ? n : -errno; } if (k == 0) @@ -137,6 +158,37 @@ int loop_write_full(int fd, const void *buf, size_t nbytes, usec_t timeout) { p = buf; } + /* timeout == 0 means "don't wait, return -EAGAIN if not ready". If the fd is already + * non-blocking, write() can't block the thread, so the non-fiber path satisfies that + * semantic correctly even from a fiber. Only use the fiber path when the fd is blocking + * (where write() would otherwise block the entire event loop). */ + int flags = 0; + if (fiber_ops_is_set() && timeout == 0) { + flags = fcntl(fd, F_GETFL); + if (flags < 0) + return -errno; + } + + if (fiber_ops_is_set() && !FLAGS_SET(flags, O_NONBLOCK)) { + /* On a fiber the write op suspends on EAGAIN until the fd is writable; honor the + * caller's timeout via a deadline scope. */ + FIBER_OPS_TIMEOUT(timestamp_is_set(timeout) ? timeout : USEC_INFINITY); + + while (nbytes > 0) { + ssize_t k = fiber_ops_write(fd, p, nbytes); + if (k < 0) + return (int) k; + if (_unlikely_(nbytes > 0 && k == 0)) /* Can't really happen */ + return -EIO; + + assert((size_t) k <= nbytes); + p += k; + nbytes -= k; + } + + return 0; + } + /* When timeout is 0 or USEC_INFINITY this is not used. But we initialize it to a sensible value. */ end = timestamp_is_set(timeout) ? usec_add(now(CLOCK_MONOTONIC), timeout) : USEC_INFINITY; @@ -220,11 +272,9 @@ int ppoll_usec_full(struct pollfd *fds, size_t n_fds, usec_t timeout, const sigs if (n_fds == 0 && timeout == 0) return 0; - r = ppoll(fds, n_fds, timeout == USEC_INFINITY ? NULL : TIMESPEC_STORE(timeout), ss); - if (r < 0) - return -errno; - if (r == 0) - return 0; + r = fiber_ops_ppoll(fds, n_fds, timeout == USEC_INFINITY ? NULL : TIMESPEC_STORE(timeout), ss); + if (r <= 0) + return r; for (size_t i = 0, n = r; i < n_fds && n > 0; i++) { if (fds[i].revents == 0) diff --git a/src/basic/meson.build b/src/basic/meson.build index 7cc62c7b8b..bae72fbcb2 100644 --- a/src/basic/meson.build +++ b/src/basic/meson.build @@ -36,6 +36,7 @@ basic_sources = files( 'ether-addr-util.c', 'extract-word.c', 'fd-util.c', + 'fiber-ops.c', 'fileio.c', 'filesystems.c', 'format-ifname.c', diff --git a/src/basic/pidref.c b/src/basic/pidref.c index 10ff9a63b1..33131c0586 100644 --- a/src/basic/pidref.c +++ b/src/basic/pidref.c @@ -7,6 +7,7 @@ #include "alloc-util.h" #include "errno-util.h" #include "fd-util.h" +#include "fiber-ops.h" #include "format-util.h" #include "hash-funcs.h" #include "io-util.h" @@ -466,16 +467,28 @@ int pidref_wait_for_terminate_full(PidRef *pidref, usec_t timeout, siginfo_t *re if (pidref->pid == 1 || pidref_is_self(pidref)) return -ECHILD; - if (timeout != USEC_INFINITY && pidref->fd < 0) + if (pidref->fd < 0 && (timeout != USEC_INFINITY || fiber_ops_is_set())) return -ENOMEDIUM; usec_t ts = timeout == USEC_INFINITY ? USEC_INFINITY : usec_add(now(CLOCK_MONOTONIC), timeout); + /* Poll the pidfd before waitid() if either there's a finite timeout (so we can honor it) or + * we're on a fiber (so fd_wait_for_event() can suspend us instead of blocking the event loop + * inside waitid()). Otherwise let waitid() block directly. The precondition above guarantees + * pidref->fd >= 0 in both cases. */ + bool poll_first = ts != USEC_INFINITY || fiber_ops_is_set(); + for (;;) { - if (ts != USEC_INFINITY) { - usec_t left = usec_sub_unsigned(ts, now(CLOCK_MONOTONIC)); - if (left == 0) - return -ETIMEDOUT; + if (poll_first) { + usec_t left; + + if (ts == USEC_INFINITY) + left = USEC_INFINITY; + else { + left = usec_sub_unsigned(ts, now(CLOCK_MONOTONIC)); + if (left == 0) + return -ETIMEDOUT; + } r = fd_wait_for_event(pidref->fd, POLLIN, left); if (r == 0) diff --git a/src/libsystemd/meson.build b/src/libsystemd/meson.build index 3c624051f0..91a78155d2 100644 --- a/src/libsystemd/meson.build +++ b/src/libsystemd/meson.build @@ -192,6 +192,7 @@ simple_tests += files( 'sd-device/test-sd-device-monitor.c', 'sd-future/test-fiber.c', 'sd-future/test-fiber-io.c', + 'sd-future/test-fiber-ops.c', 'sd-hwdb/test-sd-hwdb.c', 'sd-id128/test-id128.c', 'sd-journal/test-audit-type.c', diff --git a/src/libsystemd/sd-future/fiber.c b/src/libsystemd/sd-future/fiber.c index 48d26a27dd..64dee8411d 100644 --- a/src/libsystemd/sd-future/fiber.c +++ b/src/libsystemd/sd-future/fiber.c @@ -20,6 +20,7 @@ #include "architecture.h" #include "errno-util.h" #include "event-future.h" +#include "fiber-ops.h" #include "log-context.h" #include "log.h" #include "memory-util.h" @@ -270,8 +271,10 @@ static void reset_current_fiber(void) { * current_fiber. Without this, the child of a fork() that happened mid-fiber would inherit the * fiber's log prefix / context list in its thread-locals even though no fiber is running. */ Fiber *f = fiber_get_current(); - if (f) + if (f) { fiber_swap_log_state(f); + fiber_ops_set(NULL); + } fiber_set_current(NULL); } @@ -307,9 +310,19 @@ static void fiber_resolve(sd_future *f) { sd_future_resolve(f, fiber->result); } +static const FiberOps fiber_ops = { + .ppoll = sd_fiber_ppoll, + .read = sd_fiber_read, + .write = sd_fiber_write, + .timeout = sd_fiber_timeout, + .cancel_wait_unref = sd_future_cancel_wait_unref, +}; + static void fiber_enter(Fiber *fiber, Fiber *prev, void **fake_stack_save) { fiber_set_current(fiber); fiber_swap_log_state(fiber); + if (!prev) + fiber_ops_set(&fiber_ops); struct iovec fiber_stack = fiber_stack_usable(&fiber->stack); start_switch_stack(fake_stack_save, &fiber_stack); @@ -318,6 +331,8 @@ static void fiber_enter(Fiber *fiber, Fiber *prev, void **fake_stack_save) { static void fiber_leave(Fiber *fiber, Fiber *prev, void *fake_stack_save) { finish_switch_stack(fake_stack_save); + if (!prev) + fiber_ops_set(NULL); fiber_swap_log_state(fiber); fiber_set_current(prev); } diff --git a/src/libsystemd/sd-future/test-fiber-ops.c b/src/libsystemd/sd-future/test-fiber-ops.c new file mode 100644 index 0000000000..f2c603fda0 --- /dev/null +++ b/src/libsystemd/sd-future/test-fiber-ops.c @@ -0,0 +1,574 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ + +#include +#include +#include + +#include "sd-event.h" +#include "sd-future.h" + +#include "alloc-util.h" +#include "cleanup-util.h" +#include "fd-util.h" +#include "io-util.h" +#include "pidref.h" +#include "process-util.h" +#include "tests.h" +#include "time-util.h" + +/* Test: wait_for_terminate basic functionality */ +static int wait_simple_fiber(void *userdata) { + _cleanup_(pidref_done_sigkill_wait) PidRef pidref = PIDREF_NULL; + siginfo_t si; + int r; + + /* Fork a child that exits immediately */ + r = pidref_safe_fork("(test-child)", FORK_RESET_SIGNALS|FORK_DEATHSIG_SIGTERM|FORK_LOG, &pidref); + if (r < 0) + return r; + + if (r == 0) + _exit(42); + + /* Parent - wait for child */ + r = pidref_wait_for_terminate(&pidref, &si); + if (r < 0) + return r; + + pidref_done(&pidref); + + /* Verify child exited with status 42 */ + if (si.si_code != CLD_EXITED || si.si_status != 42) + return -EIO; + + return 0; +} + +TEST(wait_for_terminate_fiber_basic) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "wait-simple", wait_simple_fiber, NULL, /* destroy= */ NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(f)); +} + +/* Test: wait_for_terminate with multiple children */ +static int wait_multiple_fiber(void *userdata) { + PidRef pidrefs[3] = { PIDREF_NULL, PIDREF_NULL, PIDREF_NULL }; + siginfo_t si; + int r; + + /* Fork three children with different exit codes */ + for (size_t i = 0; i < 3; i++) { + r = pidref_safe_fork("(test-child)", FORK_RESET_SIGNALS|FORK_DEATHSIG_SIGTERM|FORK_LOG, &pidrefs[i]); + if (r < 0) + goto cleanup; + + if (r == 0) + /* Child process */ + _exit(10 + i); + } + + /* Wait for all three in order */ + for (size_t i = 0; i < 3; i++) { + r = pidref_wait_for_terminate(&pidrefs[i], &si); + if (r < 0) + goto cleanup; + + pidref_done(&pidrefs[i]); + + if (si.si_code != CLD_EXITED || si.si_status != (int) (10 + i)) { + r = -EIO; + goto cleanup; + } + } + + return 0; + +cleanup: + for (size_t i = 0; i < 3; i++) + pidref_done(&pidrefs[i]); + + return r; +} + +TEST(wait_for_terminate_fiber_multiple) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "wait-multiple", wait_multiple_fiber, NULL, /* destroy= */ NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(f)); +} + +static int concurrent_wait_fiber(void *userdata) { + _cleanup_(pidref_done_sigkill_wait) PidRef pidref = PIDREF_NULL; + siginfo_t si; + int r; + + r = pidref_safe_fork("(test-child)", FORK_RESET_SIGNALS|FORK_DEATHSIG_SIGTERM|FORK_LOG, &pidref); + if (r < 0) + return r; + + if (r == 0) + /* Child exits with specified status */ + _exit(PTR_TO_INT(userdata)); + + r = pidref_wait_for_terminate(&pidref, &si); + if (r < 0) + return r; + + pidref_done(&pidref); + + if (si.si_code != CLD_EXITED || si.si_status != PTR_TO_INT(userdata)) + return -EIO; + + return 0; +} + +TEST(wait_for_terminate_fiber_concurrent) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + sd_future *fibers[3] = {}; + CLEANUP_ELEMENTS(fibers, sd_future_unref_array_clear); + + /* Create 3 fibers, each waiting for a different child */ + for (size_t i = 0; i < ELEMENTSOF(fibers); i++) + ASSERT_OK(sd_fiber_new(e, "concurrent-wait", concurrent_wait_fiber, INT_TO_PTR(20 + i), /* destroy= */ NULL, &fibers[i])); + + ASSERT_OK(sd_event_loop(e)); + + /* All fibers should complete successfully */ + for (size_t i = 0; i < ELEMENTSOF(fibers); i++) + ASSERT_OK(sd_future_result(fibers[i])); +} + +typedef struct LoopIOContext { + int *pipefd; + const char *data; + size_t len; + int order; +} LoopIOContext; + +static int loop_read_suspend_fiber(void *userdata) { + LoopIOContext *ctx = ASSERT_PTR(userdata); + char buf[64]; + + ASSERT_EQ(ctx->order, 0); + ctx->order = 1; + + ssize_t n = loop_read(ctx->pipefd[0], buf, sizeof(buf), /* do_poll= */ true); + + /* While we were suspended, the writer fiber should have run. */ + ASSERT_EQ(ctx->order, 2); + + if (n < 0) + return (int) n; + if ((size_t) n != ctx->len || memcmp(buf, ctx->data, ctx->len) != 0) + return -EIO; + + return (int) n; +} + +static int loop_write_suspend_fiber(void *userdata) { + LoopIOContext *ctx = ASSERT_PTR(userdata); + + ASSERT_EQ(ctx->order, 1); + ctx->order = 2; + + int r = loop_write(ctx->pipefd[1], ctx->data, ctx->len); + if (r < 0) + return r; + + /* Close the write end so the reader sees EOF after reading the data. */ + ctx->pipefd[1] = safe_close(ctx->pipefd[1]); + return 0; +} + +/* Test: two fibers cooperatively pass a small payload through a blocking pipe using the suspending + * loop helpers. Exercises the non-blocking flip, event-loop yielding, and the blocking-mode restore. */ +TEST(loop_read_write_suspend) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC)); + + static const char payload[] = "loop-suspend"; + LoopIOContext ctx = { + .pipefd = pipefd, + .data = payload, + .len = sizeof(payload) - 1, + }; + + _cleanup_(sd_future_unrefp) sd_future *fr = NULL, *fw = NULL; + ASSERT_OK(sd_fiber_new(e, "loop-read", loop_read_suspend_fiber, &ctx, /* destroy= */ NULL, &fr)); + ASSERT_OK(sd_future_set_priority(fr, 0)); + ASSERT_OK(sd_fiber_new(e, "loop-write", loop_write_suspend_fiber, &ctx, /* destroy= */ NULL, &fw)); + ASSERT_OK(sd_future_set_priority(fw, 1)); + + ASSERT_OK(sd_event_loop(e)); + + ASSERT_OK_EQ(sd_future_result(fr), (int) ctx.len); + ASSERT_OK_ZERO(sd_future_result(fw)); + + /* The read fd started out blocking and loop_read() must have restored it before returning. */ + ASSERT_OK_ZERO(fcntl(pipefd[0], F_GETFL) & O_NONBLOCK); +} + +static int loop_read_exact_short_fiber(void *userdata) { + int fd = PTR_TO_INT(userdata); + char buf[16]; + + /* Requesting more bytes than the peer writes should return -EIO once EOF is hit. */ + return loop_read_exact(fd, buf, sizeof(buf), /* do_poll= */ true); +} + +/* Test: loop_read_exact() returns -EIO when the peer closes early. */ +TEST(loop_read_exact_short) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "loop-read-exact", loop_read_exact_short_fiber, + INT_TO_PTR(pipefd[0]), /* destroy= */ NULL, &f)); + + /* Write a few bytes and close the write end — less than the fiber asked for. */ + ASSERT_OK_EQ_ERRNO(write(pipefd[1], "abc", 3), (ssize_t) 3); + pipefd[1] = safe_close(pipefd[1]); + + ASSERT_OK(sd_event_loop(e)); + + ASSERT_ERROR(sd_future_result(f), EIO); +} + +typedef struct LoopWriteTimeoutContext { + int fd; + int result; +} LoopWriteTimeoutContext; + +static int loop_write_timeout_fiber(void *userdata) { + LoopWriteTimeoutContext *ctx = ASSERT_PTR(userdata); + + /* Try to write much more than the pipe buffer can hold with a short timeout. The write will + * succeed partially and then hit -ETIME after exhausting the timeout while blocked. */ + static const char big_buf[128 * 1024] = { 0 }; + ctx->result = loop_write_full(ctx->fd, big_buf, sizeof(big_buf), 100 * USEC_PER_MSEC); + return 0; +} + +/* Test: loop_write_full() returns -ETIME when the peer never drains. */ +TEST(loop_write_full_timeout) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC)); + + /* Shrink the pipe buffer to its minimum (one page) so the 128K write below is guaranteed to block + * regardless of the architecture's page size. The default pipe buffer is 16 pages, which on + * 64K-page architectures (e.g. ppc64le) is 1 MiB — enough to absorb the entire write without ever + * blocking, defeating the purpose of the timeout. */ + ASSERT_OK_ERRNO(fcntl(pipefd[1], F_SETPIPE_SZ, 1)); + + LoopWriteTimeoutContext ctx = { .fd = pipefd[1], .result = 0 }; + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "loop-write-timeout", loop_write_timeout_fiber, &ctx, /* destroy= */ NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + + ASSERT_OK_ZERO(sd_future_result(f)); + ASSERT_ERROR(ctx.result, ETIME); +} + +typedef struct PpollDispatchContext { + int *pipefd; + int order; +} PpollDispatchContext; + +static int ppoll_dispatch_read_fiber(void *userdata) { + PpollDispatchContext *ctx = ASSERT_PTR(userdata); + struct pollfd pfd = { + .fd = ctx->pipefd[0], + .events = POLLIN, + }; + + ASSERT_EQ(ctx->order, 0); + ctx->order = 1; + + /* Direct ppoll_usec() call from a fiber must dispatch through sd_fiber_poll(), suspending the + * fiber instead of blocking the entire thread. If dispatch fails, the writer fiber never gets a + * chance to run and the test deadlocks. */ + int r = ppoll_usec(&pfd, 1, USEC_INFINITY); + if (r < 0) + return r; + + ASSERT_EQ(ctx->order, 2); + + if (r != 1 || !FLAGS_SET(pfd.revents, POLLIN)) + return -EIO; + + return 0; +} + +static int ppoll_dispatch_write_fiber(void *userdata) { + PpollDispatchContext *ctx = ASSERT_PTR(userdata); + + ASSERT_EQ(ctx->order, 1); + ctx->order = 2; + + if (write(ctx->pipefd[1], "x", 1) < 0) + return -errno; + + return 0; +} + +/* Test: ppoll_usec() called from a fiber dispatches through the FiberOps hook to sd_fiber_poll(), + * yielding to the event loop instead of blocking. */ +TEST(ppoll_usec_dispatch) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC | O_NONBLOCK)); + + PpollDispatchContext ctx = { .pipefd = pipefd }; + + _cleanup_(sd_future_unrefp) sd_future *fr = NULL, *fw = NULL; + ASSERT_OK(sd_fiber_new(e, "ppoll-read", ppoll_dispatch_read_fiber, &ctx, /* destroy= */ NULL, &fr)); + ASSERT_OK(sd_future_set_priority(fr, 0)); + ASSERT_OK(sd_fiber_new(e, "ppoll-write", ppoll_dispatch_write_fiber, &ctx, /* destroy= */ NULL, &fw)); + ASSERT_OK(sd_future_set_priority(fw, 1)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(fr)); + ASSERT_OK(sd_future_result(fw)); +} + +static int loop_write_zero_timeout_nonblock_fiber(void *userdata) { + int fd = PTR_TO_INT(userdata); + + /* Fill the pipe so the next write would block. The fd is non-blocking, so on a fiber + * loop_write_full(timeout=0) must take the non-fiber path and return -EAGAIN immediately + * rather than suspending. */ + static const char big_buf[128 * 1024] = { 0 }; + return loop_write_full(fd, big_buf, sizeof(big_buf), /* timeout= */ 0); +} + +/* Test: timeout == 0 on a non-blocking fd from a fiber preserves the "don't wait" semantic and + * returns -EAGAIN when the pipe buffer is full, instead of suspending the fiber. */ +TEST(loop_write_zero_timeout_nonblock) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC | O_NONBLOCK)); + ASSERT_OK_ERRNO(fcntl(pipefd[1], F_SETPIPE_SZ, 1)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "loop-write-zt-nb", loop_write_zero_timeout_nonblock_fiber, + INT_TO_PTR(pipefd[1]), /* destroy= */ NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_ERROR(sd_future_result(f), EAGAIN); +} + +typedef struct LoopWriteZeroBlockingContext { + int *pipefd; + size_t total; + int order; +} LoopWriteZeroBlockingContext; + +static int loop_write_zero_blocking_writer_fiber(void *userdata) { + LoopWriteZeroBlockingContext *ctx = ASSERT_PTR(userdata); + + ASSERT_EQ(ctx->order, 0); + ctx->order = 1; + + /* timeout == 0 on a *blocking* fd from a fiber: the fast EAGAIN return isn't possible, so + * loop_write_full() takes the fiber path. The reader fiber drains the pipe, letting our + * write complete via fiber suspension/resume. */ + _cleanup_free_ char *big_buf = malloc0(ctx->total); + ASSERT_NOT_NULL(big_buf); + int r = loop_write_full(ctx->pipefd[1], big_buf, ctx->total, /* timeout= */ 0); + + ASSERT_EQ(ctx->order, 2); + return r; +} + +static int loop_write_zero_blocking_reader_fiber(void *userdata) { + LoopWriteZeroBlockingContext *ctx = ASSERT_PTR(userdata); + + ASSERT_EQ(ctx->order, 1); + ctx->order = 2; + + _cleanup_free_ char *buf = malloc(ctx->total); + ASSERT_NOT_NULL(buf); + ssize_t n = loop_read(ctx->pipefd[0], buf, ctx->total, /* do_poll= */ true); + if (n < 0) + return (int) n; + return (int) n; +} + +/* Test: timeout == 0 on a blocking fd from a fiber takes the fiber path (suspends until the peer + * drains) instead of blocking the entire thread. */ +TEST(loop_write_zero_timeout_blocking) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC)); + ASSERT_OK_ERRNO(fcntl(pipefd[1], F_SETPIPE_SZ, 1)); + + /* F_SETPIPE_SZ rounds up to the kernel's pipe minimum (typically a page); query the actual + * size and write more than that, so the write must wait on the reader regardless of page size. */ + int pipe_sz = fcntl(pipefd[1], F_GETPIPE_SZ); + ASSERT_OK_ERRNO(pipe_sz); + + LoopWriteZeroBlockingContext ctx = { .pipefd = pipefd, .total = (size_t) pipe_sz * 2 }; + + _cleanup_(sd_future_unrefp) sd_future *fw = NULL, *fr = NULL; + ASSERT_OK(sd_fiber_new(e, "loop-write-zt-blk", loop_write_zero_blocking_writer_fiber, + &ctx, /* destroy= */ NULL, &fw)); + ASSERT_OK(sd_future_set_priority(fw, 0)); + ASSERT_OK(sd_fiber_new(e, "loop-read-zt-blk", loop_write_zero_blocking_reader_fiber, + &ctx, /* destroy= */ NULL, &fr)); + ASSERT_OK(sd_future_set_priority(fr, 1)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK(sd_future_result(fw)); + ASSERT_OK_EQ(sd_future_result(fr), (int) ctx.total); +} + +static int loop_read_no_poll_nonblock_fiber(void *userdata) { + int fd = PTR_TO_INT(userdata); + char buf[64]; + + /* Empty non-blocking pipe + do_poll=false: on a fiber loop_read() must take the non-fiber + * path and return -EAGAIN immediately rather than suspending. */ + return (int) loop_read(fd, buf, sizeof(buf), /* do_poll= */ false); +} + +/* Test: do_poll == false on a non-blocking fd from a fiber preserves the "don't wait" semantic + * and returns -EAGAIN when no data is available, instead of suspending the fiber. */ +TEST(loop_read_no_poll_nonblock) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC | O_NONBLOCK)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(e, "loop-read-np-nb", loop_read_no_poll_nonblock_fiber, + INT_TO_PTR(pipefd[0]), /* destroy= */ NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_ERROR(sd_future_result(f), EAGAIN); +} + +typedef struct LoopReadNoPollBlockingContext { + int *pipefd; + const char *data; + size_t len; + int order; +} LoopReadNoPollBlockingContext; + +static int loop_read_no_poll_blocking_reader_fiber(void *userdata) { + LoopReadNoPollBlockingContext *ctx = ASSERT_PTR(userdata); + char buf[64]; + + ASSERT_EQ(ctx->order, 0); + ctx->order = 1; + + /* do_poll == false on a *blocking* fd from a fiber: the fast EAGAIN return isn't possible, + * so loop_read() takes the fiber path and suspends until the writer fiber feeds data. */ + ssize_t n = loop_read(ctx->pipefd[0], buf, sizeof(buf), /* do_poll= */ false); + + ASSERT_EQ(ctx->order, 2); + + if (n < 0) + return (int) n; + if ((size_t) n != ctx->len || memcmp(buf, ctx->data, ctx->len) != 0) + return -EIO; + + return (int) n; +} + +static int loop_read_no_poll_blocking_writer_fiber(void *userdata) { + LoopReadNoPollBlockingContext *ctx = ASSERT_PTR(userdata); + + ASSERT_EQ(ctx->order, 1); + ctx->order = 2; + + int r = loop_write(ctx->pipefd[1], ctx->data, ctx->len); + if (r < 0) + return r; + + ctx->pipefd[1] = safe_close(ctx->pipefd[1]); + return 0; +} + +/* Test: do_poll == false on a blocking fd from a fiber takes the fiber path (suspends until the + * peer feeds data) instead of blocking the entire thread. */ +TEST(loop_read_no_poll_blocking) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC)); + + static const char payload[] = "no-poll"; + LoopReadNoPollBlockingContext ctx = { + .pipefd = pipefd, + .data = payload, + .len = sizeof(payload) - 1, + }; + + _cleanup_(sd_future_unrefp) sd_future *fr = NULL, *fw = NULL; + ASSERT_OK(sd_fiber_new(e, "loop-read-np-blk", loop_read_no_poll_blocking_reader_fiber, + &ctx, /* destroy= */ NULL, &fr)); + ASSERT_OK(sd_future_set_priority(fr, 0)); + ASSERT_OK(sd_fiber_new(e, "loop-write-np-blk", loop_read_no_poll_blocking_writer_fiber, + &ctx, /* destroy= */ NULL, &fw)); + ASSERT_OK(sd_future_set_priority(fw, 1)); + + ASSERT_OK(sd_event_loop(e)); + ASSERT_OK_EQ(sd_future_result(fr), (int) ctx.len); + ASSERT_OK_ZERO(sd_future_result(fw)); +} + +/* Test: loop_*() helpers transparently fall back to blocking I/O when called outside any + * fiber context. */ +TEST(loop_read_write_fallback) { + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC)); + + ASSERT_OK(loop_write(pipefd[1], "fallback", STRLEN("fallback"))); + + char buf[16]; + ssize_t n = loop_read(pipefd[0], buf, STRLEN("fallback"), /* do_poll= */ true); + ASSERT_OK_EQ(n, (ssize_t) STRLEN("fallback")); + ASSERT_EQ(memcmp(buf, "fallback", STRLEN("fallback")), 0); +} + +DEFINE_TEST_MAIN(LOG_DEBUG); From 2f9959648fa0ad4b00052a04480b53f7a5829869 Mon Sep 17 00:00:00 2001 From: Daan De Meyer Date: Mon, 23 Mar 2026 10:15:27 +0100 Subject: [PATCH 4/8] sd-event: suspend instead of blocking when sd_event_run() runs on a fiber MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit sd_event_run() blocks the calling thread on the event loop's epoll fd until something happens. When the caller is a fiber, that's the wrong behaviour: blocking the thread also stalls every other fiber and the outer event loop driving them. The most common way to hit this is a fiber that creates its own inner event loop (e.g. a server-style fiber that wants to dispatch its own sources independently of whatever loop the test or supervising fiber is running on) — with the existing implementation the inner sd_event_run() would hold the thread while the outer scheduler should be free to advance other fibers. Add an event_run_suspend() variant in sd-event/event-future.c that performs the same prepare/wait/dispatch dance, but when the fast path finds nothing ready it (a) creates an IO future watching the inner event loop's epoll fd on the *outer* event loop, (b) optionally creates a time future for the timeout, and (c) suspends the fiber. When either future fires the fiber is resumed and the prepare/wait/dispatch sequence runs once more to actually dispatch what's pending. sd_event_run() checks sd_fiber_is_running() and delegates to this variant when on a fiber; profile_delays accounting is intentionally skipped on that path since the underlying prepare/wait/dispatch primitives already account for themselves. --- src/libsystemd/meson.build | 1 + src/libsystemd/sd-event/event-future.c | 72 ++++ src/libsystemd/sd-event/event-future.h | 2 + src/libsystemd/sd-event/event-util.h | 3 + src/libsystemd/sd-event/sd-event.c | 13 +- src/libsystemd/sd-event/test-event-future.c | 358 ++++++++++++++++++++ 6 files changed, 446 insertions(+), 3 deletions(-) create mode 100644 src/libsystemd/sd-event/test-event-future.c diff --git a/src/libsystemd/meson.build b/src/libsystemd/meson.build index 91a78155d2..4e9fd28d44 100644 --- a/src/libsystemd/meson.build +++ b/src/libsystemd/meson.build @@ -190,6 +190,7 @@ simple_tests += files( 'sd-bus/test-bus-vtable.c', 'sd-device/test-device-util.c', 'sd-device/test-sd-device-monitor.c', + 'sd-event/test-event-future.c', 'sd-future/test-fiber.c', 'sd-future/test-fiber-io.c', 'sd-future/test-fiber-ops.c', diff --git a/src/libsystemd/sd-event/event-future.c b/src/libsystemd/sd-event/event-future.c index 8c9960fcdd..4e0a0a87fc 100644 --- a/src/libsystemd/sd-event/event-future.c +++ b/src/libsystemd/sd-event/event-future.c @@ -6,6 +6,7 @@ #include "alloc-util.h" #include "errno-util.h" #include "event-future.h" +#include "event-util.h" #include "fd-util.h" typedef struct IoFuture { @@ -234,3 +235,74 @@ int future_new_time(sd_event *e, clockid_t clock, uint64_t usec, uint64_t accura int future_new_time_relative(sd_event *e, clockid_t clock, uint64_t usec, uint64_t accuracy, int result, sd_future **ret) { return future_new_time_internal(sd_event_add_time_relative, e, clock, usec, accuracy, result, ret); } + +int event_run_suspend(sd_event *e, uint64_t timeout) { + sd_event *outer = sd_fiber_get_event(); + int r; + + assert(e); + assert(sd_fiber_is_running()); + assert(outer); + assert(e != outer); + + /* Make sure that none of the preparation callbacks ends up freeing the event source under our feet */ + PROTECT_EVENT(e); + + r = sd_event_prepare(e); + if (r < 0) + return r; + if (r == 0) { + r = sd_event_wait(e, 0); + if (r < 0) + return r; + } + if (r > 0) { + r = sd_event_dispatch(e); + if (r < 0) + return r; + + return 1; + } + + if (timeout == 0) + return 0; + + int fd = sd_event_get_fd(e); + if (fd < 0) + return fd; + + _cleanup_(sd_future_cancel_wait_unrefp) sd_future *io = NULL; + r = future_new_io(outer, fd, EPOLLIN, &io); + if (r < 0) + return r; + + _cleanup_(sd_future_cancel_wait_unrefp) sd_future *timer = NULL; + if (timeout != USEC_INFINITY) { + r = future_new_time_relative( + outer, + CLOCK_MONOTONIC, + timeout, + /* accuracy= */ 1, + /* result= */ 0, + &timer); + if (r < 0) + return r; + } + + r = sd_fiber_suspend(); + if (r < 0) + return r; + + r = sd_event_prepare(e); + if (r == 0) + r = sd_event_wait(e, 0); + if (r > 0) { + r = sd_event_dispatch(e); + if (r < 0) + return r; + + return 1; + } + + return r; +} diff --git a/src/libsystemd/sd-event/event-future.h b/src/libsystemd/sd-event/event-future.h index 3bc275e7b7..83d5939d6b 100644 --- a/src/libsystemd/sd-event/event-future.h +++ b/src/libsystemd/sd-event/event-future.h @@ -6,3 +6,5 @@ int future_new_io(sd_event *e, int fd, uint32_t events, sd_future **ret); int future_new_time(sd_event *e, clockid_t clock, uint64_t usec, uint64_t accuracy, int result, sd_future **ret); int future_new_time_relative(sd_event *e, clockid_t clock, uint64_t usec, uint64_t accuracy, int result, sd_future **ret); + +int event_run_suspend(sd_event *e, uint64_t timeout); diff --git a/src/libsystemd/sd-event/event-util.h b/src/libsystemd/sd-event/event-util.h index dc3b3ed70f..ce213b9c9e 100644 --- a/src/libsystemd/sd-event/event-util.h +++ b/src/libsystemd/sd-event/event-util.h @@ -5,6 +5,9 @@ #include "sd-forward.h" +#define PROTECT_EVENT(e) \ + _unused_ _cleanup_(sd_event_unrefp) sd_event *_ref = sd_event_ref(e); + extern const struct hash_ops event_source_hash_ops; int event_reset_time( diff --git a/src/libsystemd/sd-event/sd-event.c b/src/libsystemd/sd-event/sd-event.c index 8c419fad30..4935ae1b4a 100644 --- a/src/libsystemd/sd-event/sd-event.c +++ b/src/libsystemd/sd-event/sd-event.c @@ -10,12 +10,15 @@ #include "sd-daemon.h" #include "sd-event.h" +#include "sd-future.h" #include "sd-id128.h" #include "sd-messages.h" #include "alloc-util.h" #include "errno-util.h" +#include "event-future.h" #include "event-source.h" +#include "event-util.h" #include "fd-util.h" #include "format-util.h" #include "glyph-util.h" @@ -474,9 +477,6 @@ _public_ sd_event* sd_event_unref(sd_event *e) { return event_free(e); } -#define PROTECT_EVENT(e) \ - _unused_ _cleanup_(sd_event_unrefp) sd_event *_ref = sd_event_ref(e); - _public_ sd_event_source* sd_event_source_disable_unref(sd_event_source *s) { int r; @@ -4932,6 +4932,13 @@ _public_ int sd_event_run(sd_event *e, uint64_t timeout) { assert_return(e->state != SD_EVENT_FINISHED, -ESTALE); assert_return(e->state == SD_EVENT_INITIAL, -EBUSY); + /* When running on a fiber, delegate to the suspending implementation. Note that the + * profile_delays accounting below is intentionally skipped on that path: the suspending variant + * drives the event loop via sd_event_prepare()/sd_event_wait()/sd_event_dispatch() itself, which + * are the same primitives profile_delays tracks when called directly. */ + if (sd_fiber_is_running()) + return event_run_suspend(e, timeout); + if (e->profile_delays && e->last_run_usec != 0) { usec_t this_run; unsigned l; diff --git a/src/libsystemd/sd-event/test-event-future.c b/src/libsystemd/sd-event/test-event-future.c new file mode 100644 index 0000000000..754daf0df2 --- /dev/null +++ b/src/libsystemd/sd-event/test-event-future.c @@ -0,0 +1,358 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ + +#include +#include + +#include "sd-event.h" +#include "sd-future.h" + +#include "fd-util.h" +#include "tests.h" +#include "time-util.h" + +static int timer_callback(sd_event_source *s, uint64_t usec, void *userdata) { + int *count = ASSERT_PTR(userdata); + int r; + + (*count)++; + + r = sd_event_source_set_time_relative(s, 5 * USEC_PER_MSEC); + if (r < 0) + return r; + + if (sd_fiber_is_running() && *count >= 3) + return sd_event_exit(sd_event_source_get_event(s), 0); + + return 0; +} + +static int event_run_fiber_func(void *userdata) { + _cleanup_(sd_event_unrefp) sd_event *inner = NULL; + _cleanup_(sd_event_source_unrefp) sd_event_source *inner_timer = NULL; + int r; + + /* Create inner event loop from within the fiber */ + r = sd_event_new(&inner); + if (r < 0) + return r; + + /* Add a timer to the inner event loop that fires every 5ms */ + r = sd_event_add_time_relative(inner, &inner_timer, CLOCK_MONOTONIC, + 5 * USEC_PER_MSEC, 0, timer_callback, + userdata); + if (r < 0) + return r; + + r = sd_event_source_set_enabled(inner_timer, SD_EVENT_ON); + if (r < 0) + return r; + + return sd_event_loop(inner); +} + +TEST(sd_event_loop_fiber) { + /* Create outer event loop for the fiber scheduler */ + _cleanup_(sd_event_unrefp) sd_event *outer = NULL; + ASSERT_OK(sd_event_new(&outer)); + ASSERT_OK(sd_event_set_exit_on_idle(outer, true)); + + /* Add a timer to the outer event loop that fires every 5ms */ + _cleanup_(sd_event_source_unrefp) sd_event_source *outer_timer = NULL; + int outer_timer_count = 0; + ASSERT_OK(sd_event_add_time_relative(outer, &outer_timer, CLOCK_MONOTONIC, + 5 * USEC_PER_MSEC, 0, timer_callback, + &outer_timer_count)); + + /* Create a fiber that will create and run the inner event loop */ + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + int inner_timer_count = 0; + ASSERT_OK(sd_fiber_new(outer, "event-runner", event_run_fiber_func, &inner_timer_count, /* destroy= */ NULL, &f)); + + /* Run the outer event loop */ + ASSERT_OK(sd_event_loop(outer)); + + /* Fiber should have completed successfully */ + ASSERT_OK(sd_future_result(f)); + + /* Both timers should have fired at least once */ + ASSERT_EQ(inner_timer_count, 3); + ASSERT_GT(outer_timer_count, 0); +} + +static int event_run_fiber_timeout_func(void *userdata) { + _cleanup_(sd_event_unrefp) sd_event *inner = NULL; + int r; + + /* Create inner event loop from within the fiber */ + r = sd_event_new(&inner); + if (r < 0) + return r; + + /* Run with a short timeout - should timeout since there are no events */ + return sd_event_run(inner, 10 * USEC_PER_MSEC); +} + +TEST(sd_event_run_fiber_timeout) { + /* Create outer event loop for the fiber scheduler */ + _cleanup_(sd_event_unrefp) sd_event *outer = NULL; + ASSERT_OK(sd_event_new(&outer)); + ASSERT_OK(sd_event_set_exit_on_idle(outer, true)); + + /* Create a fiber that will run sd_event_run() with timeout */ + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(outer, "event-timeout", event_run_fiber_timeout_func, NULL, /* destroy= */ NULL, &f)); + + /* Run the outer event loop */ + ASSERT_OK(sd_event_loop(outer)); + + /* Fiber should have completed successfully (timeout returns 0) */ + ASSERT_OK_ZERO(sd_future_result(f)); +} + +/* Test: sd_event_run() with zero timeout returns immediately */ +static int sd_event_run_zero_timeout_fiber(void *userdata) { + _cleanup_(sd_event_unrefp) sd_event *inner = NULL; + int r; + + r = sd_event_new(&inner); + if (r < 0) + return r; + + /* With zero timeout on an empty event loop, should return 0 immediately */ + r = sd_event_run(inner, 0); + if (r != 0) + return r < 0 ? r : -EIO; + + return 0; +} + +TEST(sd_event_run_zero_timeout) { + _cleanup_(sd_event_unrefp) sd_event *outer = NULL; + ASSERT_OK(sd_event_new(&outer)); + ASSERT_OK(sd_event_set_exit_on_idle(outer, true)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(outer, "run-suspend-zero", sd_event_run_zero_timeout_fiber, NULL, /* destroy= */ NULL, &f)); + + ASSERT_OK(sd_event_loop(outer)); + ASSERT_OK_ZERO(sd_future_result(f)); +} + +/* Test: sd_event_run() dispatches immediately pending IO */ +static int io_callback(sd_event_source *s, int fd, uint32_t revents, void *userdata) { + int *counter = ASSERT_PTR(userdata); + char buf[64]; + + (*counter)++; + + /* Drain the fd */ + (void) read(fd, buf, sizeof(buf)); + + return sd_event_exit(sd_event_source_get_event(s), 0); +} + +static int sd_event_run_immediate_fiber(void *userdata) { + int *pipefd = ASSERT_PTR(userdata); + _cleanup_(sd_event_unrefp) sd_event *inner = NULL; + _cleanup_(sd_event_source_unrefp) sd_event_source *source = NULL; + int counter = 0, r; + + r = sd_event_new(&inner); + if (r < 0) + return r; + + /* Add IO source watching the read end of the pipe */ + r = sd_event_add_io(inner, &source, pipefd[0], EPOLLIN, io_callback, &counter); + if (r < 0) + return r; + + /* Data is already available on the pipe (written before fiber started), so + * sd_event_run() should dispatch immediately without suspending */ + r = sd_event_run(inner, USEC_INFINITY); + if (r < 0) + return r; + + /* The IO callback should have fired */ + if (counter != 1) + return -EIO; + + return 0; +} + +TEST(sd_event_run_immediate) { + _cleanup_(sd_event_unrefp) sd_event *outer = NULL; + ASSERT_OK(sd_event_new(&outer)); + ASSERT_OK(sd_event_set_exit_on_idle(outer, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC | O_NONBLOCK)); + + /* Write data before starting the fiber so it's immediately available */ + ASSERT_OK_EQ_ERRNO(write(pipefd[1], "X", 1), 1); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(outer, "run-suspend-immediate", sd_event_run_immediate_fiber, pipefd, /* destroy= */ NULL, &f)); + + ASSERT_OK(sd_event_loop(outer)); + ASSERT_OK_ZERO(sd_future_result(f)); +} + +/* Test: sd_event_run() with IO arriving during suspension */ +static int sd_event_run_io_fiber(void *userdata) { + int *pipefd = ASSERT_PTR(userdata); + _cleanup_(sd_event_unrefp) sd_event *inner = NULL; + _cleanup_(sd_event_source_unrefp) sd_event_source *source = NULL; + int counter = 0, r; + + r = sd_event_new(&inner); + if (r < 0) + return r; + + r = sd_event_add_io(inner, &source, pipefd[0], EPOLLIN, io_callback, &counter); + if (r < 0) + return r; + + /* No data available yet, so this will suspend the fiber until IO arrives */ + r = sd_event_run(inner, USEC_INFINITY); + if (r < 0) + return r; + + if (counter != 1) + return -EIO; + + return 0; +} + +TEST(sd_event_run_io) { + _cleanup_(sd_event_unrefp) sd_event *outer = NULL; + ASSERT_OK(sd_event_new(&outer)); + ASSERT_OK(sd_event_set_exit_on_idle(outer, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC | O_NONBLOCK)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(outer, "run-suspend-io", sd_event_run_io_fiber, pipefd, /* destroy= */ NULL, &f)); + + /* First iteration: fiber runs, adds IO source, suspends because no data */ + ASSERT_OK_POSITIVE(sd_event_run(outer, 0)); + + /* Write data to the pipe to wake the inner event loop */ + ASSERT_OK_EQ_ERRNO(write(pipefd[1], "Y", 1), 1); + + /* Complete: fiber resumes, dispatches IO, finishes */ + ASSERT_OK(sd_event_loop(outer)); + ASSERT_OK_ZERO(sd_future_result(f)); +} + +/* Test: event_run called in a loop keeps event loop state consistent. + * This is a regression test for a bug where error paths after sd_event_prepare() + * could leave the inner event loop stuck in SD_EVENT_ARMED state. */ +static int sd_event_run_loop_fiber(void *userdata) { + int *pipefd = ASSERT_PTR(userdata); + _cleanup_(sd_event_unrefp) sd_event *inner = NULL; + _cleanup_(sd_event_source_unrefp) sd_event_source *source = NULL; + int counter = 0, r; + + r = sd_event_new(&inner); + if (r < 0) + return r; + + r = sd_event_add_io(inner, &source, pipefd[0], EPOLLIN, io_callback, &counter); + if (r < 0) + return r; + + /* Call sd_event_run() multiple times with short timeouts. + * Each call should leave the inner event loop in a clean state for the next call. */ + for (int i = 0; i < 5; i++) { + r = sd_event_run(inner, 10 * USEC_PER_MSEC); + if (r < 0) + return r; + if (r > 0) + break; + } + + /* After multiple timeouts, the event loop should still be usable. + * Write data and do one more run to verify. */ + if (counter == 0) { + /* Data wasn't written yet, do a final run with longer timeout */ + r = sd_event_run(inner, USEC_INFINITY); + if (r < 0) + return r; + } + + if (counter != 1) + return -EIO; + + return 0; +} + +TEST(sd_event_run_loop) { + _cleanup_(sd_event_unrefp) sd_event *outer = NULL; + ASSERT_OK(sd_event_new(&outer)); + ASSERT_OK(sd_event_set_exit_on_idle(outer, true)); + + _cleanup_close_pair_ int pipefd[2] = EBADF_PAIR; + ASSERT_OK_ERRNO(pipe2(pipefd, O_CLOEXEC | O_NONBLOCK)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(outer, "run-suspend-loop", sd_event_run_loop_fiber, pipefd, /* destroy= */ NULL, &f)); + + /* Let the fiber run through a few timeout iterations */ + for (int i = 0; i < 10; i++) + ASSERT_OK(sd_event_run(outer, 50 * USEC_PER_MSEC)); + + /* Write data to unblock the fiber */ + ASSERT_OK_EQ_ERRNO(write(pipefd[1], "Z", 1), 1); + + ASSERT_OK(sd_event_loop(outer)); + ASSERT_OK_ZERO(sd_future_result(f)); +} + +/* Test: sd_event_run() with an inner timer that fires during suspension */ +static int inner_timer_handler(sd_event_source *s, uint64_t usec, void *userdata) { + int *counter = ASSERT_PTR(userdata); + (*counter)++; + return sd_event_exit(sd_event_source_get_event(s), 0); +} + +static int sd_event_run_timer_fiber(void *userdata) { + _cleanup_(sd_event_unrefp) sd_event *inner = NULL; + _cleanup_(sd_event_source_unrefp) sd_event_source *source = NULL; + int counter = 0, r; + + r = sd_event_new(&inner); + if (r < 0) + return r; + + /* Add a timer that fires after 10ms */ + r = sd_event_add_time_relative(inner, &source, CLOCK_MONOTONIC, + 10 * USEC_PER_MSEC, 0, inner_timer_handler, + &counter); + if (r < 0) + return r; + + /* Should suspend, then resume when the timer fires */ + r = sd_event_run(inner, USEC_INFINITY); + if (r < 0) + return r; + + if (counter != 1) + return -EIO; + + return 0; +} + +TEST(sd_event_run_timer) { + _cleanup_(sd_event_unrefp) sd_event *outer = NULL; + ASSERT_OK(sd_event_new(&outer)); + ASSERT_OK(sd_event_set_exit_on_idle(outer, true)); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + ASSERT_OK(sd_fiber_new(outer, "run-suspend-timer", sd_event_run_timer_fiber, NULL, /* destroy= */ NULL, &f)); + + ASSERT_OK(sd_event_loop(outer)); + ASSERT_OK_ZERO(sd_future_result(f)); +} + +DEFINE_TEST_MAIN(LOG_DEBUG); From 9727f6536c5890c3a20e45de1355791e8ed523d4 Mon Sep 17 00:00:00 2001 From: Daan De Meyer Date: Mon, 11 May 2026 16:27:34 +0200 Subject: [PATCH 5/8] sd-bus: make sd-bus fiber-aware MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two changes to teach sd-bus how to behave when called from a fiber, in order of increasing depth: 2. sd_bus_call() now redirects to a new bus_call_suspend() helper when the caller is a fiber whose event loop is the same one the bus is attached to. The plain bus_poll() path serializes all bus traffic on the slot's reply (only one method call can be in flight per sd_bus*), which would defeat the point of running multiple fibers against one bus. bus_call_suspend() builds on the async sd-bus API: it wraps the call in a new BusFuture (sd-bus/bus-future.{c,h}) that resolves when the reply or method-error arrives, lets the fiber await that future, and surfaces the reply to the caller via future_get_bus_reply(). Because the futures live on the event loop rather than a per-bus slot, multiple fibers can drive concurrent method calls against the same bus. 3. A new private SD_BUS_VTABLE_METHOD_FIBER flag dispatches a vtable method handler on its own fiber, so handlers are free to use sd_bus_call() against the same bus, sd_fiber_sleep(), loop_read(), etc. without stalling the event loop for other connections or handlers. The flag stays out of sd-bus-vtable.h (its bit value is reserved there to prevent collisions) — the fiber runtime is a systemd-internal implementation detail. Lifecycle of fiber-dispatched handlers is tracked on the bus itself: a new bus->fiber_futures set holds a ref to each in-flight handler. bus_enter_closing() cancels every entry and process_closing() returns with the bus still in CLOSING state until the set drains, so we can be sure no fiber handler outlives the bus. bus_fiber_resolved() removes the entry on completion. bus_free()'s assert(set_isempty()) makes the invariant load-bearing. Note that plain sd_bus_call() already works correctly on a fiber as it calls ppoll_usec() which has already been modified to suspend when running on a fiber. To exercise these changes the existing thread-based client/server sd-bus tests (test-bus-chat, test-bus-objects, test-bus-peersockaddr, test-bus-server, test-bus-watch-bind) are migrated to fibers, and a new test-bus-fiber is added that covers SD_BUS_VTABLE_METHOD_FIBER — including handlers that issue nested sd_bus_call() on the same bus, the cancel-on-close path, and concurrent dispatches across multiple fibers. --- src/libsystemd/meson.build | 2 + src/libsystemd/sd-bus/bus-future.c | 135 ++++++++++++ src/libsystemd/sd-bus/bus-future.h | 14 ++ src/libsystemd/sd-bus/bus-internal.h | 13 ++ src/libsystemd/sd-bus/bus-objects.c | 109 +++++++++ src/libsystemd/sd-bus/sd-bus.c | 50 ++++- src/libsystemd/sd-bus/test-bus-chat.c | 51 +++-- src/libsystemd/sd-bus/test-bus-fiber.c | 207 ++++++++++++++++++ src/libsystemd/sd-bus/test-bus-objects.c | 62 ++---- src/libsystemd/sd-bus/test-bus-peersockaddr.c | 33 +-- src/libsystemd/sd-bus/test-bus-server.c | 48 ++-- src/libsystemd/sd-bus/test-bus-watch-bind.c | 102 +++++---- src/systemd/sd-bus-vtable.h | 1 + 13 files changed, 675 insertions(+), 152 deletions(-) create mode 100644 src/libsystemd/sd-bus/bus-future.c create mode 100644 src/libsystemd/sd-bus/bus-future.h create mode 100644 src/libsystemd/sd-bus/test-bus-fiber.c diff --git a/src/libsystemd/meson.build b/src/libsystemd/meson.build index 4e9fd28d44..0cecea3b0d 100644 --- a/src/libsystemd/meson.build +++ b/src/libsystemd/meson.build @@ -49,6 +49,7 @@ sd_bus_sources = files( 'sd-bus/bus-dump.c', 'sd-bus/bus-dump-json.c', 'sd-bus/bus-error.c', + 'sd-bus/bus-future.c', 'sd-bus/bus-internal.c', 'sd-bus/bus-introspect.c', 'sd-bus/bus-kernel.c', @@ -185,6 +186,7 @@ libsystemd_pc = custom_target( simple_tests += files( 'sd-bus/test-bus-creds.c', + 'sd-bus/test-bus-fiber.c', 'sd-bus/test-bus-introspect.c', 'sd-bus/test-bus-match.c', 'sd-bus/test-bus-vtable.c', diff --git a/src/libsystemd/sd-bus/bus-future.c b/src/libsystemd/sd-bus/bus-future.c new file mode 100644 index 0000000000..d7037b3f15 --- /dev/null +++ b/src/libsystemd/sd-bus/bus-future.c @@ -0,0 +1,135 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ + +#include "sd-bus.h" +#include "sd-future.h" + +#include "alloc-util.h" +#include "bus-future.h" +#include "bus-internal.h" +#include "bus-message.h" + +typedef struct BusFuture { + sd_bus_slot *slot; + sd_bus_message *reply; +} BusFuture; + +static void* bus_future_alloc(void) { + return new0(BusFuture, 1); +} + +static void bus_future_free(sd_future *f) { + BusFuture *bf = ASSERT_PTR(sd_future_get_private(f)); + sd_bus_slot_unref(bf->slot); + sd_bus_message_unref(bf->reply); + free(bf); +} + +static int bus_future_cancel(sd_future *f) { + BusFuture *bf = ASSERT_PTR(sd_future_get_private(ASSERT_PTR(f))); + + bf->slot = sd_bus_slot_unref(bf->slot); + return sd_future_resolve(f, -ECANCELED); +} + +static const sd_future_ops bus_future_ops = { + .size = sizeof(sd_future_ops), + .alloc = bus_future_alloc, + .free = bus_future_free, + .cancel = bus_future_cancel, +}; + +static int bus_future_handler(sd_bus_message *m, void *userdata, sd_bus_error *reterr_error) { + sd_future *f = ASSERT_PTR(userdata); + BusFuture *bf = ASSERT_PTR(sd_future_get_private(f)); + int r; + + /* Resolve with 0 on a success reply and -errno (derived from the D-Bus error name) on a + * method error reply, so a caller awaiting the future learns about call failures from the + * resolution value alone. The reply itself is always stashed in bf->reply so + * future_get_bus_reply() can hand back the detailed sd_bus_error (name + message) on + * top of the bare errno. Cancellation surfaces as -ECANCELED via bus_future_cancel(), + * with bf->reply left NULL — callers can distinguish "got an error reply" from "no reply + * will arrive" by whether future_get_bus_reply() can produce a message. */ + bf->slot = sd_bus_slot_unref(bf->slot); + bf->reply = sd_bus_message_ref(m); + + r = sd_bus_message_is_method_error(m, NULL) ? -sd_bus_message_get_errno(m) : 0; + return sd_future_resolve(f, r); +} + +int bus_call_future(sd_bus *bus, sd_bus_message *m, uint64_t usec, sd_future **ret) { + int r; + + assert(bus); + assert(m); + assert(ret); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + r = sd_future_new(&bus_future_ops, &f); + if (r < 0) + return r; + + BusFuture *bf = sd_future_get_private(f); + + r = sd_bus_call_async(bus, &bf->slot, m, bus_future_handler, f, usec); + if (r < 0) + return r; + + *ret = TAKE_PTR(f); + return 0; +} + +int future_get_bus_reply(sd_future *f, sd_bus_error *reterr_error, sd_bus_message **ret_reply) { + BusFuture *bf = ASSERT_PTR(sd_future_get_private(ASSERT_PTR(f))); + sd_bus_message *reply = ASSERT_PTR(bf->reply); + + assert(sd_future_get_ops(f) == &bus_future_ops); + assert(sd_future_state(f) == SD_FUTURE_RESOLVED); + + if (sd_bus_message_is_method_error(reply, NULL)) { + if (reterr_error) + return sd_bus_error_copy(reterr_error, sd_bus_message_get_error(reply)); + return -sd_bus_message_get_errno(reply); + } + + if (reply->n_fds > 0 && !sd_bus_message_get_bus(reply)->accept_fd) + return sd_bus_error_set(reterr_error, SD_BUS_ERROR_INCONSISTENT_MESSAGE, + "Reply message contained file descriptors which I couldn't accept. Sorry."); + + if (reterr_error) + *reterr_error = SD_BUS_ERROR_NULL; + if (ret_reply) + *ret_reply = sd_bus_message_ref(reply); + + return 1; +} + +int bus_call_suspend( + sd_bus *bus, + sd_bus_message *m, + uint64_t usec, + sd_bus_error *reterr_error, + sd_bus_message **ret_reply) { + + int r; + + assert(bus); + assert(m); + assert(sd_fiber_is_running()); + + _cleanup_(sd_future_cancel_wait_unrefp) sd_future *f = NULL; + r = bus_call_future(bus, m, usec, &f); + if (r < 0) + return sd_bus_error_set_errno(reterr_error, r); + + r = sd_fiber_suspend(); + + /* If the future isn't resolved, the suspend was interrupted before a reply arrived (fiber + * cancelled, fiber-wide SD_FIBER_TIMEOUT scope expired, …). There's no reply to extract, + * so surface the resume error directly. When the future is resolved, future_get_bus_reply() + * recovers either the reply or the detailed sd_bus_error from the error reply. */ + if (sd_future_state(f) != SD_FUTURE_RESOLVED) + return sd_bus_error_set_errno(reterr_error, r); + + return future_get_bus_reply(f, reterr_error, ret_reply); +} diff --git a/src/libsystemd/sd-bus/bus-future.h b/src/libsystemd/sd-bus/bus-future.h new file mode 100644 index 0000000000..ec9bd80b15 --- /dev/null +++ b/src/libsystemd/sd-bus/bus-future.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +#pragma once + +#include "sd-forward.h" + +int bus_call_future(sd_bus *bus, sd_bus_message *m, uint64_t usec, sd_future **ret); +int future_get_bus_reply(sd_future *f, sd_bus_error *reterr_error, sd_bus_message **ret_reply); + +int bus_call_suspend( + sd_bus *bus, + sd_bus_message *m, + uint64_t usec, + sd_bus_error *reterr_error, + sd_bus_message **ret_reply); diff --git a/src/libsystemd/sd-bus/bus-internal.h b/src/libsystemd/sd-bus/bus-internal.h index 19a3b67d12..3a52f738d6 100644 --- a/src/libsystemd/sd-bus/bus-internal.h +++ b/src/libsystemd/sd-bus/bus-internal.h @@ -17,6 +17,13 @@ #define DEFAULT_SYSTEM_BUS_ADDRESS "unix:path=/run/dbus/system_bus_socket" #define DEFAULT_USER_BUS_ADDRESS_FMT "unix:path=%s/bus" +/* Private vtable flag: dispatch the method handler on its own fiber, so it can use suspending + * primitives (sd_bus_call() on a fiber, sd_fiber_sleep(), loop_read_suspend(), ...) without + * blocking the event loop for other connections or method calls. Kept out of the public + * sd-bus-vtable.h so the fiber runtime stays an implementation detail of systemd. The bit value is + * reserved in sd-bus-vtable.h to make sure it never collides with a future public flag. */ +#define SD_BUS_VTABLE_METHOD_FIBER (UINT64_C(1) << 10) + typedef struct BusReplyCallback { sd_bus_message_handler_t callback; usec_t timeout_usec; /* this is a relative timeout until we reach the BUS_HELLO state, and an absolute one right after */ @@ -222,6 +229,12 @@ typedef struct sd_bus { Set *vtable_methods; Set *vtable_properties; + /* Futures for outstanding SD_BUS_VTABLE_METHOD_FIBER dispatches. Entries are added as the + * dispatcher spawns each fiber and removed when the fiber resolves. On bus_enter_closing() + * we cancel everything in here and then wait in process_closing() until the set drains, + * before tearing down the rest of the bus. */ + Set *fiber_futures; + union sockaddr_union sockaddr; socklen_t sockaddr_size; diff --git a/src/libsystemd/sd-bus/bus-objects.c b/src/libsystemd/sd-bus/bus-objects.c index 83ba3a5239..795cc68837 100644 --- a/src/libsystemd/sd-bus/bus-objects.c +++ b/src/libsystemd/sd-bus/bus-objects.c @@ -3,6 +3,7 @@ #include #include "sd-bus.h" +#include "sd-future.h" #include "alloc-util.h" #include "bus-internal.h" @@ -337,6 +338,67 @@ static int check_access(sd_bus *bus, sd_bus_message *m, BusVTableMember *c, sd_b return sd_bus_error_setf(reterr_error, SD_BUS_ERROR_ACCESS_DENIED, "Access to %s.%s() not permitted.", c->interface, c->member); } +typedef struct BusFiberData { + sd_bus *bus; + sd_bus_message *message; + sd_bus_slot *slot; + sd_bus_message_handler_t handler; + void *userdata; +} BusFiberData; + +static BusFiberData* bus_fiber_data_free(BusFiberData *d) { + if (!d) + return NULL; + + sd_bus_slot_unref(d->slot); + sd_bus_message_unref(d->message); + sd_bus_unref(d->bus); + return mfree(d); +} + +DEFINE_TRIVIAL_CLEANUP_FUNC(BusFiberData*, bus_fiber_data_free); + +static void bus_fiber_data_destroy(void *userdata) { + bus_fiber_data_free(userdata); +} + +static void bus_fiber_future_unref(void *p) { + sd_future_unref(p); +} + +DEFINE_PRIVATE_HASH_OPS_WITH_KEY_DESTRUCTOR( + bus_fiber_future_hash_ops, + void, + trivial_hash_func, + trivial_compare_func, + bus_fiber_future_unref); + +static int bus_fiber_resolved(sd_future *f) { + sd_bus *bus = ASSERT_PTR(sd_future_get_userdata(f)); + + assert_se(set_remove(bus->fiber_futures, f) == f); + sd_future_unref(f); + return 0; +} + +static int bus_fiber_entry(void *userdata) { + BusFiberData *d = ASSERT_PTR(userdata); + _cleanup_(sd_bus_error_free) sd_bus_error error = SD_BUS_ERROR_NULL; + int r; + + /* Note: unlike the synchronous dispatch path, we deliberately do NOT set + * bus->current_slot/handler/userdata around the callback. Those fields track the slot of the + * message currently being dispatched inline and must be NULL at each entry into + * bus_process_internal(). Because a fiber handler can yield and let the event loop dispatch + * other messages before it resumes, leaving current_slot non-NULL across yields would trip + * that invariant. Fiber handlers receive their slot's userdata via the handler argument, so + * sd_bus_get_current_slot()/handler()/userdata() simply aren't meaningful inside them — the + * handler should use the message/userdata parameters directly instead. */ + r = d->handler(d->message, d->userdata, &error); + + return bus_maybe_reply_error(d->message, r, &error); +} + static int method_callbacks_run( sd_bus *bus, sd_bus_message *m, @@ -407,6 +469,53 @@ static int method_callbacks_run( slot = container_of(c->parent, sd_bus_slot, node_vtable); + if (FLAGS_SET(c->vtable->flags, SD_BUS_VTABLE_METHOD_FIBER)) { + /* A fiber-dispatched method requires an event loop to spawn the fiber on. + * By the time a method call actually arrives the bus is running, so the + * event loop should already be attached — if not, the caller set up the bus + * wrong and there's no meaningful recovery. */ + assert(bus->event); + + _cleanup_(bus_fiber_data_freep) BusFiberData *d = new(BusFiberData, 1); + if (!d) + return -ENOMEM; + + *d = (BusFiberData) { + .bus = sd_bus_ref(bus), + .message = sd_bus_message_ref(m), + .slot = sd_bus_slot_ref(slot), + .handler = c->vtable->x.method.handler, + .userdata = u, + }; + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + r = sd_fiber_new(bus->event, c->member, bus_fiber_entry, d, bus_fiber_data_destroy, &f); + if (r < 0) + return bus_maybe_reply_error(m, r, NULL); + + /* The fiber now owns d via bus_fiber_data_destroy. Drop our cleanup before any + * further fallible calls, so a later failure unwinding f doesn't double-free d. */ + TAKE_PTR(d); + + r = set_ensure_put(&bus->fiber_futures, &bus_fiber_future_hash_ops, f); + if (r < 0) + return bus_maybe_reply_error(m, r, NULL); + assert(r > 0); + + /* Track the future on the bus so shutdown can cancel it and wait for it. */ + r = sd_future_set_callback(f, bus_fiber_resolved, bus); + if (r < 0) { + /* TAKE_PTR(f) hasn't run yet, so our cleanup attribute still owns the + * ref; set_remove() returns the raw pointer without firing the hash_ops + * destructor, and the cleanup will unref f on return. */ + assert_se(set_remove(bus->fiber_futures, f) == f); + return bus_maybe_reply_error(m, r, NULL); + } + + TAKE_PTR(f); + return 1; + } + bus->current_slot = sd_bus_slot_ref(slot); bus->current_handler = c->vtable->x.method.handler; bus->current_userdata = u; diff --git a/src/libsystemd/sd-bus/sd-bus.c b/src/libsystemd/sd-bus/sd-bus.c index 27f788d995..e44c439fad 100644 --- a/src/libsystemd/sd-bus/sd-bus.c +++ b/src/libsystemd/sd-bus/sd-bus.c @@ -10,12 +10,14 @@ #include "sd-bus.h" #include "sd-event.h" +#include "sd-future.h" #include "af-list.h" #include "alloc-util.h" #include "bus-container.h" #include "bus-control.h" #include "bus-error.h" +#include "bus-future.h" #include "bus-internal.h" #include "bus-kernel.h" #include "bus-label.h" @@ -222,6 +224,12 @@ static sd_bus* bus_free(sd_bus *b) { ordered_hashmap_free(b->reply_callbacks); prioq_free(b->reply_callbacks_prioq); + /* Outstanding fiber handlers pin the bus via their BusFiberData ref, so by the time refcount + * reaches zero and bus_free() runs, every fiber has already resolved and removed itself from + * this set. */ + assert(set_isempty(b->fiber_futures)); + set_free(b->fiber_futures); + assert(b->match_callbacks.type == BUS_MATCH_ROOT); bus_match_free(&b->match_callbacks); @@ -1809,6 +1817,9 @@ _public_ sd_bus* sd_bus_flush_close_unref(sd_bus *bus) { } void bus_enter_closing(sd_bus *bus, int exit_code) { + sd_future *f; + int r; + assert(bus); if (!IN_SET(bus->state, BUS_WATCH_BIND, BUS_OPENING, BUS_AUTHENTICATING, BUS_HELLO, BUS_RUNNING)) @@ -1816,6 +1827,19 @@ void bus_enter_closing(sd_bus *bus, int exit_code) { bus_set_state(bus, BUS_CLOSING); bus->exit_code = exit_code; + + /* Cancel all outstanding fiber-dispatched method handlers. Most cancellations are scheduled + * asynchronously (fibers resolve with -ECANCELED the next time they run), but a fiber still + * in FIBER_STATE_INITIAL resolves synchronously, which fires bus_fiber_resolved() and + * removes f from this set mid-iteration. That's safe because SET_FOREACH permits removal of + * exactly the current entry — see the assertion in hashmap_iterate_entry(). Either way this + * doesn't block here: process_closing() waits for the fiber_futures set to drain before it + * continues tearing down the rest of the bus. */ + SET_FOREACH(f, bus->fiber_futures) { + r = sd_future_cancel(f); + if (r < 0) + log_debug_errno(r, "Failed to cancel outstanding fiber method handler, ignoring: %m"); + } } /* Define manually so we can add the PID check */ @@ -2388,23 +2412,30 @@ _public_ int sd_bus_call( sd_bus_error *reterr_error, sd_bus_message **ret_reply) { - _cleanup_(sd_bus_message_unrefp) sd_bus_message *m = sd_bus_message_ref(_m); usec_t timeout; uint64_t cookie; size_t i; int r; - bus_assert_return(m, -EINVAL, reterr_error); - bus_assert_return(m->header->type == SD_BUS_MESSAGE_METHOD_CALL, -EINVAL, reterr_error); - bus_assert_return(!(m->header->flags & BUS_MESSAGE_NO_REPLY_EXPECTED), -EINVAL, reterr_error); + bus_assert_return(_m, -EINVAL, reterr_error); + bus_assert_return(_m->header->type == SD_BUS_MESSAGE_METHOD_CALL, -EINVAL, reterr_error); + bus_assert_return(!(_m->header->flags & BUS_MESSAGE_NO_REPLY_EXPECTED), -EINVAL, reterr_error); bus_assert_return(!bus_error_is_dirty(reterr_error), -EINVAL, reterr_error); if (bus) assert_return(bus = bus_resolve(bus), -ENOPKG); else - assert_return(bus = m->bus, -ENOTCONN); + assert_return(bus = _m->bus, -ENOTCONN); bus_assert_return(!bus_origin_changed(bus), -ECHILD, reterr_error); + /* If the current fiber and the bus share their event loop, we can use sd_bus_call_suspend() + * instead which does an async method call. This allows multiple invocations of sd_bus_call() to + * happen across multiple fibers at once. */ + if (sd_fiber_is_running() && bus->event == sd_fiber_get_event()) + return bus_call_suspend(bus, _m, usec, reterr_error, ret_reply); + + _cleanup_(sd_bus_message_unrefp) sd_bus_message *m = sd_bus_message_ref(_m); + if (!BUS_IS_OPEN(bus->state)) { r = -ENOTCONN; goto fail; @@ -3177,7 +3208,14 @@ static int process_closing(sd_bus *bus, sd_bus_message **ret) { assert(bus); assert(bus->state == BUS_CLOSING); - /* First, fail all outstanding method calls */ + /* Wait for any still-running fiber method handlers to finish unwinding their cancellation + * before tearing down the rest of the bus. bus_enter_closing() scheduled the cancel; each + * fiber resolves asynchronously and bus_fiber_resolved() removes it from the set. Returning + * 1 here keeps the bus in CLOSING state so the event loop drives the fibers to completion. */ + if (!set_isempty(bus->fiber_futures)) + return 1; + + /* Then, fail all outstanding method calls */ c = ordered_hashmap_first(bus->reply_callbacks); if (c) return process_closing_reply_callback(bus, c); diff --git a/src/libsystemd/sd-bus/test-bus-chat.c b/src/libsystemd/sd-bus/test-bus-chat.c index 1f358ccd33..d6f4860c41 100644 --- a/src/libsystemd/sd-bus/test-bus-chat.c +++ b/src/libsystemd/sd-bus/test-bus-chat.c @@ -1,11 +1,11 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ #include -#include #include #include #include "sd-bus.h" +#include "sd-future.h" #include "alloc-util.h" #include "bus-error.h" @@ -102,7 +102,8 @@ static int server_init(sd_bus **ret) { return 0; } -static int server(sd_bus *bus) { +static int server(void *userdata) { + sd_bus *bus = ASSERT_PTR(userdata); bool client1_gone = false, client2_gone = false; int r; @@ -178,7 +179,9 @@ static int server(sd_bus *bus) { client2_gone = true; } else if (sd_bus_message_is_method_call(m, "org.freedesktop.systemd.test", "Slow")) { - sleep(1); + r = sd_fiber_sleep(1 * USEC_PER_SEC); + if (r < 0) + return r; r = sd_bus_reply_method_return(m, NULL); if (r < 0) @@ -194,10 +197,10 @@ static int server(sd_bus *bus) { log_info("Received fd=%d", fd); - if (write(fd, &x, 1) < 0) { - r = log_error_errno(errno, "Failed to write to fd: %m"); + ssize_t n = sd_fiber_write(fd, &x, 1); + if (n < 0) { safe_close(fd); - return r; + return log_error_errno(n, "Failed to write to fd: %m"); } r = sd_bus_reply_method_return(m, NULL); @@ -217,7 +220,7 @@ static int server(sd_bus *bus) { return 0; } -static void* client1(void *p) { +static int client1(void *userdata) { _cleanup_(sd_bus_message_unrefp) sd_bus_message *reply = NULL; _cleanup_(sd_bus_flush_close_unrefp) sd_bus *bus = NULL; _cleanup_(sd_bus_error_free) sd_bus_error error = SD_BUS_ERROR_NULL; @@ -277,9 +280,9 @@ static void* client1(void *p) { goto finish; } - errno = 0; - if (read(pp[0], &x, 1) <= 0) { - log_error("Failed to read from pipe: %s", STRERROR_OR_EOF(errno)); + ssize_t n = sd_fiber_read(pp[0], &x, 1); + if (n <= 0) { + log_error("Failed to read from pipe: %s", STRERROR_OR_EOF(n)); goto finish; } @@ -303,7 +306,7 @@ finish: } - return INT_TO_PTR(r); + return r; } static int quit_callback(sd_bus_message *m, void *userdata, sd_bus_error *ret_error) { @@ -315,7 +318,7 @@ static int quit_callback(sd_bus_message *m, void *userdata, sd_bus_error *ret_er return 1; } -static void* client2(void *p) { +static int client2(void *userdata) { _cleanup_(sd_bus_message_unrefp) sd_bus_message *m = NULL, *reply = NULL; _cleanup_(sd_bus_flush_close_unrefp) sd_bus *bus = NULL; _cleanup_(sd_bus_error_free) sd_bus_error error = SD_BUS_ERROR_NULL; @@ -494,7 +497,7 @@ finish: (void) sd_bus_send(bus, q, NULL); } - return INT_TO_PTR(r); + return r; } static ino_t get_inode(int fd) { @@ -626,9 +629,9 @@ TEST(ctrunc) { } TEST(chat) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + _cleanup_(sd_future_unrefp) sd_future *f_server = NULL, *f_client1 = NULL, *f_client2 = NULL; _cleanup_(sd_bus_flush_close_unrefp) sd_bus *bus = NULL; - pthread_t c1, c2; - void *p; int r; test_setup_logging(LOG_INFO); @@ -639,16 +642,18 @@ TEST(chat) { log_info("Initialized..."); - ASSERT_OK(-pthread_create(&c1, NULL, client1, NULL)); - ASSERT_OK(-pthread_create(&c2, NULL, client2, NULL)); + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); - r = server(bus); + ASSERT_OK(sd_fiber_new(e, "client-1", client1, NULL, /* destroy= */ NULL, &f_client1)); + ASSERT_OK(sd_fiber_new(e, "client-2", client2, NULL, /* destroy= */ NULL, &f_client2)); + ASSERT_OK(sd_fiber_new(e, "server", server, bus, /* destroy= */ NULL, &f_server)); - ASSERT_OK(-pthread_join(c1, &p)); - ASSERT_OK(PTR_TO_INT(p)); - ASSERT_OK(-pthread_join(c2, &p)); - ASSERT_OK(PTR_TO_INT(p)); - ASSERT_OK(r); + ASSERT_OK(sd_event_loop(e)); + + ASSERT_OK(sd_future_result(f_client1)); + ASSERT_OK(sd_future_result(f_client2)); + ASSERT_OK(sd_future_result(f_server)); } DEFINE_TEST_MAIN(LOG_INFO); diff --git a/src/libsystemd/sd-bus/test-bus-fiber.c b/src/libsystemd/sd-bus/test-bus-fiber.c new file mode 100644 index 0000000000..2cadf1dc17 --- /dev/null +++ b/src/libsystemd/sd-bus/test-bus-fiber.c @@ -0,0 +1,207 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ + +#include + +#include "sd-bus.h" +#include "sd-event.h" +#include "sd-future.h" + +#include "bus-internal.h" +#include "tests.h" +#include "time-util.h" + +typedef struct Context { + /* Counters for the concurrency check: every Concurrent invocation bumps in_flight on entry + * and drops it on exit, and tracks the maximum observed concurrency. If fiber dispatch + * works, two overlapping client calls must both be inside the handler at the same time, + * giving a max of at least 2. */ + int in_flight; + int max_in_flight; + sd_future *waiter; +} Context; + +static int method_concurrent(sd_bus_message *m, void *userdata, sd_bus_error *reterr_error) { + Context *c = ASSERT_PTR(userdata); + + ASSERT_OK_POSITIVE(sd_fiber_is_running()); + + c->in_flight++; + if (c->in_flight > c->max_in_flight) + c->max_in_flight = c->in_flight; + + /* Synchronize on the peer instead of sleeping: the first handler to enter stashes its + * fiber and suspends; the second resumes it and then yields, so the first runs to + * completion (sending its reply) before the second proceeds. Both are therefore in + * flight at the same time regardless of how long any individual dispatch takes, and + * the reply order matches the request order. */ + if (c->in_flight < 2) { + assert(!c->waiter); + c->waiter = sd_fiber_get_current(); + ASSERT_OK(sd_fiber_suspend()); + } else { + ASSERT_OK(sd_fiber_resume(TAKE_PTR(c->waiter), 0)); + ASSERT_OK(sd_fiber_yield()); + } + + c->in_flight--; + + return sd_bus_reply_method_return(m, NULL); +} + +static int method_fail_errno(sd_bus_message *m, void *userdata, sd_bus_error *reterr_error) { + ASSERT_OK_POSITIVE(sd_fiber_is_running()); + + /* Yielding first exercises the deferred-error path in the fiber entry: the handler returns + * a negative errno after suspending, and bus_maybe_reply_error() must still turn that into + * a matching sd_bus error reply. */ + ASSERT_OK(sd_fiber_sleep(1 * USEC_PER_MSEC)); + + return -EACCES; +} + +static int method_fail_error(sd_bus_message *m, void *userdata, sd_bus_error *reterr_error) { + ASSERT_OK_POSITIVE(sd_fiber_is_running()); + + ASSERT_OK(sd_fiber_sleep(1 * USEC_PER_MSEC)); + + return sd_bus_error_set(reterr_error, SD_BUS_ERROR_INVALID_ARGS, "bad arguments from fiber"); +} + +static const sd_bus_vtable vtable[] = { + SD_BUS_VTABLE_START(0), + SD_BUS_METHOD("Concurrent", NULL, NULL, method_concurrent, + SD_BUS_VTABLE_UNPRIVILEGED|SD_BUS_VTABLE_METHOD_FIBER), + SD_BUS_METHOD("FailErrno", NULL, NULL, method_fail_errno, + SD_BUS_VTABLE_UNPRIVILEGED|SD_BUS_VTABLE_METHOD_FIBER), + SD_BUS_METHOD("FailError", NULL, NULL, method_fail_error, + SD_BUS_VTABLE_UNPRIVILEGED|SD_BUS_VTABLE_METHOD_FIBER), + SD_BUS_VTABLE_END, +}; + +typedef struct Setup { + int fds[2]; + Context *c; +} Setup; + +static int attach_pair(Setup *s, sd_bus **ret_server, sd_bus **ret_client) { + _cleanup_(sd_bus_flush_close_unrefp) sd_bus *server = NULL, *client = NULL; + sd_id128_t id; + + assert(ret_server); + assert(ret_client); + + ASSERT_OK(sd_id128_randomize(&id)); + ASSERT_OK(sd_bus_new(&server)); + ASSERT_OK(sd_bus_set_description(server, "server")); + ASSERT_OK(sd_bus_set_fd(server, s->fds[0], s->fds[0])); + ASSERT_OK(sd_bus_set_server(server, true, id)); + ASSERT_OK(sd_bus_attach_event(server, sd_fiber_get_event(), 0)); + ASSERT_OK(sd_bus_add_object_vtable(server, NULL, "/test", "test.Fiber", vtable, s->c)); + ASSERT_OK(sd_bus_start(server)); + + ASSERT_OK(sd_bus_new(&client)); + ASSERT_OK(sd_bus_set_description(client, "client")); + ASSERT_OK(sd_bus_set_fd(client, s->fds[1], s->fds[1])); + ASSERT_OK(sd_bus_attach_event(client, sd_fiber_get_event(), 0)); + ASSERT_OK(sd_bus_start(client)); + + *ret_server = TAKE_PTR(server); + *ret_client = TAKE_PTR(client); + return 0; +} + +static int call_concurrent_fiber(void *userdata) { + sd_bus *client = ASSERT_PTR(userdata); + + /* A plain suspending sd_bus_call() — on a fiber this goes through sd_bus_call_suspend() + * which multiplexes onto the single client connection, so multiple caller fibers can have + * calls in flight at the same time. */ + return sd_bus_call_method(client, NULL, "/test", "test.Fiber", "Concurrent", + NULL, NULL, NULL); +} + +static int concurrency_fiber(void *userdata) { + Setup *s = ASSERT_PTR(userdata); + _cleanup_(sd_bus_flush_close_unrefp) sd_bus *server = NULL, *client = NULL; + _cleanup_(sd_future_cancel_wait_unrefp) sd_future *f_a = NULL, *f_b = NULL; + + ASSERT_OK(attach_pair(s, &server, &client)); + + /* Two concurrent calls on the shared client bus. Each lands in method_concurrent which + * blocks on the peer; if fiber dispatch works the second is entered while the first is + * suspended, so max_in_flight on the context reaches 2. */ + ASSERT_OK(sd_fiber_new(sd_fiber_get_event(), "call-a", call_concurrent_fiber, client, + /* destroy= */ NULL, &f_a)); + ASSERT_OK(sd_fiber_new(sd_fiber_get_event(), "call-b", call_concurrent_fiber, client, + /* destroy= */ NULL, &f_b)); + + ASSERT_OK(sd_fiber_await(f_a)); + ASSERT_OK(sd_fiber_await(f_b)); + + ASSERT_OK(sd_future_result(f_a)); + ASSERT_OK(sd_future_result(f_b)); + return 0; +} + +TEST(fiber_method_concurrency) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + Context c = {}; + Setup s = { .c = &c }; + + ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM, 0, s.fds)); + + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + ASSERT_OK(sd_fiber_new(e, "concurrency", concurrency_fiber, &s, /* destroy= */ NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + + ASSERT_OK(sd_future_result(f)); + ASSERT_GE(c.max_in_flight, 2); +} + +static int errors_fiber(void *userdata) { + Setup *s = ASSERT_PTR(userdata); + _cleanup_(sd_bus_flush_close_unrefp) sd_bus *server = NULL, *client = NULL; + + ASSERT_OK(attach_pair(s, &server, &client)); + + /* A fiber handler that returns a negative errno gets turned into a matching sd_bus error + * reply (bus_maybe_reply_error → sd_bus_reply_method_errno). */ + _cleanup_(sd_bus_error_free) sd_bus_error e1 = SD_BUS_ERROR_NULL; + ASSERT_ERROR(sd_bus_call_method(client, NULL, "/test", "test.Fiber", "FailErrno", + &e1, NULL, NULL), + EACCES); + ASSERT_TRUE(sd_bus_error_has_name(&e1, SD_BUS_ERROR_ACCESS_DENIED)); + + /* A fiber handler that populates sd_bus_error directly propagates both name and message. */ + _cleanup_(sd_bus_error_free) sd_bus_error e2 = SD_BUS_ERROR_NULL; + ASSERT_FAIL(sd_bus_call_method(client, NULL, "/test", "test.Fiber", "FailError", + &e2, NULL, NULL)); + ASSERT_TRUE(sd_bus_error_has_name(&e2, SD_BUS_ERROR_INVALID_ARGS)); + ASSERT_STREQ(e2.message, "bad arguments from fiber"); + + return 0; +} + +TEST(fiber_method_errors) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + Context c = {}; + Setup s = { .c = &c }; + + ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM, 0, s.fds)); + + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + ASSERT_OK(sd_fiber_new(e, "errors", errors_fiber, &s, /* destroy= */ NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + + ASSERT_OK(sd_future_result(f)); +} + +DEFINE_TEST_MAIN(LOG_DEBUG); diff --git a/src/libsystemd/sd-bus/test-bus-objects.c b/src/libsystemd/sd-bus/test-bus-objects.c index 4ad60f0d58..ac33086a6f 100644 --- a/src/libsystemd/sd-bus/test-bus-objects.c +++ b/src/libsystemd/sd-bus/test-bus-objects.c @@ -1,8 +1,7 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ -#include - #include "sd-bus.h" +#include "sd-future.h" #include "alloc-util.h" #include "bus-internal.h" @@ -211,9 +210,9 @@ static int enumerator3_callback(sd_bus *bus, const char *path, void *userdata, c return 1; } -static void* server(void *p) { - struct context *c = p; - sd_bus *bus = NULL; +static int server(void *userdata) { + struct context *c = ASSERT_PTR(userdata); + _cleanup_(sd_bus_flush_close_unrefp) sd_bus *bus = NULL; sd_id128_t id; int r; @@ -242,36 +241,25 @@ static void* server(void *p) { log_error("Loop!"); r = sd_bus_process(bus, NULL); - if (r < 0) { - log_error_errno(r, "Failed to process requests: %m"); - goto fail; - } + if (r < 0) + return log_error_errno(r, "Failed to process requests: %m"); if (r == 0) { r = sd_bus_wait(bus, UINT64_MAX); - if (r < 0) { - log_error_errno(r, "Failed to wait: %m"); - goto fail; - } + if (r < 0) + return log_error_errno(r, "Failed to wait: %m"); continue; } } - r = 0; - -fail: - if (bus) { - sd_bus_flush(bus); - sd_bus_unref(bus); - } - - return INT_TO_PTR(r); + return 0; } -static int client(struct context *c) { +static int client(void *p) { + struct context *c = ASSERT_PTR(p); _cleanup_(sd_bus_message_unrefp) sd_bus_message *reply = NULL; - _cleanup_(sd_bus_unrefp) sd_bus *bus = NULL; + _cleanup_(sd_bus_flush_close_unrefp) sd_bus *bus = NULL; _cleanup_(sd_bus_error_free) sd_bus_error error = SD_BUS_ERROR_NULL; _cleanup_strv_free_ char **lines = NULL; const char *s; @@ -575,16 +563,13 @@ static int client(struct context *c) { ASSERT_OK(sd_bus_call_method(bus, "org.freedesktop.systemd.test", "/foo", "org.freedesktop.systemd.test", "Exit", &error, NULL, NULL)); - sd_bus_flush(bus); - return 0; } int main(int argc, char *argv[]) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + _cleanup_(sd_future_unrefp) sd_future *f_server = NULL, *f_client = NULL; struct context c = {}; - pthread_t s; - void *p; - int r, q; test_setup_logging(LOG_DEBUG); @@ -593,21 +578,16 @@ int main(int argc, char *argv[]) { ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM, 0, c.fds)); - r = pthread_create(&s, NULL, server, &c); - if (r != 0) - return -r; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); - r = client(&c); + ASSERT_OK(sd_fiber_new(e, "server", server, &c, /* destroy= */ NULL, &f_server)); + ASSERT_OK(sd_fiber_new(e, "client", client, &c, /* destroy= */ NULL, &f_client)); - q = pthread_join(s, &p); - if (q != 0) - return -q; + ASSERT_OK(sd_event_loop(e)); - if (r < 0) - return r; - - if (PTR_TO_INT(p) < 0) - return PTR_TO_INT(p); + ASSERT_OK(sd_future_result(f_server)); + ASSERT_OK(sd_future_result(f_client)); free(c.something); free(c.automatic_string_property); diff --git a/src/libsystemd/sd-bus/test-bus-peersockaddr.c b/src/libsystemd/sd-bus/test-bus-peersockaddr.c index 2cac35dde4..bee76c9b10 100644 --- a/src/libsystemd/sd-bus/test-bus-peersockaddr.c +++ b/src/libsystemd/sd-bus/test-bus-peersockaddr.c @@ -1,9 +1,9 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ -#include #include #include "sd-bus.h" +#include "sd-future.h" #include "bus-dump.h" #include "fd-util.h" @@ -38,9 +38,9 @@ static bool gid_list_same(const gid_t *a, size_t n, const gid_t *b, size_t m) { gid_list_contained(b, m, a, n); } -static void* server(void *p) { +static int server(void *userdata) { _cleanup_(sd_bus_flush_close_unrefp) sd_bus *bus = NULL; - _cleanup_close_ int listen_fd = PTR_TO_INT(p), fd = -EBADF; + _cleanup_close_ int listen_fd = PTR_TO_INT(userdata), fd = -EBADF; _cleanup_(sd_bus_creds_unrefp) sd_bus_creds *c = NULL; _cleanup_free_ char *our_comm = NULL; sd_id128_t id; @@ -48,7 +48,7 @@ static void* server(void *p) { ASSERT_OK(sd_id128_randomize(&id)); - ASSERT_OK_ERRNO(fd = accept4(listen_fd, NULL, NULL, SOCK_CLOEXEC|SOCK_NONBLOCK)); + ASSERT_OK(fd = sd_fiber_accept(listen_fd, NULL, NULL, SOCK_CLOEXEC|SOCK_NONBLOCK)); ASSERT_OK(sd_bus_new(&bus)); ASSERT_OK(sd_bus_set_fd(bus, fd, fd)); @@ -114,17 +114,18 @@ static void* server(void *p) { } } - return NULL; + return 0; } -static void* client(void *p) { +static int client(void *userdata) { _cleanup_(sd_bus_flush_close_unrefp) sd_bus *bus = NULL; _cleanup_(sd_bus_message_unrefp) sd_bus_message *reply = NULL; const char *z; ASSERT_OK(sd_bus_new(&bus)); ASSERT_OK(sd_bus_set_description(bus, "wuffwuff")); - ASSERT_OK(sd_bus_set_address(bus, p)); + ASSERT_OK(sd_bus_set_address(bus, userdata)); + ASSERT_OK(sd_bus_attach_event(bus, sd_fiber_get_event(), 0)); ASSERT_OK(sd_bus_start(bus)); ASSERT_OK(sd_bus_call_method(bus, "foo.foo", "/foo", "foo.foo", "Foo", NULL, &reply, "s", "foo")); @@ -132,17 +133,18 @@ static void* client(void *p) { ASSERT_OK(sd_bus_message_read(reply, "s", &z)); ASSERT_STREQ(z, "bar"); - return NULL; + return 0; } TEST(description) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + _cleanup_(sd_future_unrefp) sd_future *f_server = NULL, *f_client = NULL; _cleanup_free_ char *a = NULL; _cleanup_close_ int fd = -EBADF; union sockaddr_union sa = { .un.sun_family = AF_UNIX, }; socklen_t salen; - pthread_t s, c; ASSERT_OK_ERRNO(fd = socket(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0)); ASSERT_OK_ERRNO(bind(fd, &sa.sa, offsetof(struct sockaddr_un, sun_path))); /* force auto-bind */ @@ -155,13 +157,18 @@ TEST(description) { ASSERT_OK(asprintf(&a, "unix:abstract=%s", sa.un.sun_path + 1)); - ASSERT_OK(-pthread_create(&s, NULL, server, INT_TO_PTR(fd))); + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); + + ASSERT_OK(sd_fiber_new(e, "server", server, INT_TO_PTR(fd), /* destroy= */ NULL, &f_server)); TAKE_FD(fd); - ASSERT_OK(-pthread_create(&c, NULL, client, a)); + ASSERT_OK(sd_fiber_new(e, "client", client, a, /* destroy= */ NULL, &f_client)); - ASSERT_OK(-pthread_join(s, NULL)); - ASSERT_OK(-pthread_join(c, NULL)); + ASSERT_OK(sd_event_loop(e)); + + ASSERT_OK(sd_future_result(f_server)); + ASSERT_OK(sd_future_result(f_client)); } DEFINE_TEST_MAIN(LOG_INFO); diff --git a/src/libsystemd/sd-bus/test-bus-server.c b/src/libsystemd/sd-bus/test-bus-server.c index 989d2bf10d..1edcec858f 100644 --- a/src/libsystemd/sd-bus/test-bus-server.c +++ b/src/libsystemd/sd-bus/test-bus-server.c @@ -1,10 +1,12 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ -#include #include #include "sd-bus.h" +#include "sd-event.h" +#include "sd-future.h" +#include "errno-util.h" #include "log.h" #include "memory-util.h" #include "string-util.h" @@ -20,7 +22,8 @@ struct context { bool server_anonymous_auth; }; -static int _server(struct context *c) { +static int server(void *userdata) { + struct context *c = ASSERT_PTR(userdata); _cleanup_(sd_bus_flush_close_unrefp) sd_bus *bus = NULL; sd_id128_t id; bool quit = false; @@ -29,6 +32,7 @@ static int _server(struct context *c) { ASSERT_OK(sd_id128_randomize(&id)); ASSERT_OK(sd_bus_new(&bus)); + ASSERT_OK(sd_bus_set_description(bus, "server")); ASSERT_OK(sd_bus_set_fd(bus, c->fds[0], c->fds[0])); ASSERT_OK(sd_bus_set_server(bus, 1, id)); ASSERT_OK(sd_bus_set_anonymous(bus, c->server_anonymous_auth)); @@ -74,17 +78,16 @@ static int _server(struct context *c) { return 0; } -static void* server(void *p) { - return INT_TO_PTR(_server(p)); -} - -static int client(struct context *c) { +static int client(void *userdata) { + struct context *c = ASSERT_PTR(userdata); _cleanup_(sd_bus_message_unrefp) sd_bus_message *m = NULL, *reply = NULL; - _cleanup_(sd_bus_unrefp) sd_bus *bus = NULL; + _cleanup_(sd_bus_flush_close_unrefp) sd_bus *bus = NULL; _cleanup_(sd_bus_error_free) sd_bus_error error = SD_BUS_ERROR_NULL; ASSERT_OK(sd_bus_new(&bus)); + ASSERT_OK(sd_bus_set_description(bus, "client")); ASSERT_OK(sd_bus_set_fd(bus, c->fds[1], c->fds[1])); + ASSERT_OK(sd_bus_attach_event(bus, sd_fiber_get_event(), 0)); ASSERT_OK(sd_bus_negotiate_fds(bus, c->client_negotiate_unix_fds)); ASSERT_OK(sd_bus_set_anonymous(bus, c->client_anonymous_auth)); ASSERT_OK(sd_bus_start(bus)); @@ -103,10 +106,10 @@ static int client(struct context *c) { static int test_one(bool client_negotiate_unix_fds, bool server_negotiate_unix_fds, bool client_anonymous_auth, bool server_anonymous_auth) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + _cleanup_(sd_future_unrefp) sd_future *f_server = NULL, *f_client = NULL; struct context c; - pthread_t s; - void *p; - int r, q; + int r = 0; zero(c); @@ -117,23 +120,18 @@ static int test_one(bool client_negotiate_unix_fds, bool server_negotiate_unix_f c.client_anonymous_auth = client_anonymous_auth; c.server_anonymous_auth = server_anonymous_auth; - r = pthread_create(&s, NULL, server, &c); - if (r != 0) - return -r; + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); - r = client(&c); + ASSERT_OK(sd_fiber_new(e, "server", server, &c, /* destroy= */ NULL, &f_server)); + ASSERT_OK(sd_fiber_new(e, "client", client, &c, /* destroy= */ NULL, &f_client)); - q = pthread_join(s, &p); - if (q != 0) - return -q; + ASSERT_OK(sd_event_loop(e)); - if (r < 0) - return r; + RET_GATHER(r, sd_future_result(f_client)); + RET_GATHER(r, sd_future_result(f_server)); - if (PTR_TO_INT(p) < 0) - return PTR_TO_INT(p); - - return 0; + return r; } int main(int argc, char *argv[]) { @@ -145,7 +143,7 @@ int main(int argc, char *argv[]) { ASSERT_OK(test_one(false, false, false, false)); ASSERT_OK(test_one(true, true, true, true)); ASSERT_OK(test_one(true, true, false, true)); - ASSERT_ERROR(test_one(true, true, true, false), EPERM); + ASSERT_ERROR(test_one(true, true, true, false), EACCES); return EXIT_SUCCESS; } diff --git a/src/libsystemd/sd-bus/test-bus-watch-bind.c b/src/libsystemd/sd-bus/test-bus-watch-bind.c index 1bf4ee7017..6561633b8a 100644 --- a/src/libsystemd/sd-bus/test-bus-watch-bind.c +++ b/src/libsystemd/sd-bus/test-bus-watch-bind.c @@ -1,10 +1,8 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ -#include -#include - #include "sd-bus.h" #include "sd-event.h" +#include "sd-future.h" #include "sd-id128.h" #include "alloc-util.h" @@ -44,33 +42,33 @@ static const sd_bus_vtable vtable[] = { SD_BUS_VTABLE_END, }; -static void* thread_server(void *p) { +static int server(void *userdata) { _cleanup_free_ char *suffixed = NULL, *suffixed_basename = NULL, *suffixed2 = NULL, *d = NULL; _cleanup_close_ int fd = -EBADF; union sockaddr_union u; - const char *path = p; + const char *path = ASSERT_PTR(userdata); int r; log_debug("Initializing server"); /* Let's play some games, by slowly creating the socket directory, and renaming it in the middle */ - usleep_safe(100 * USEC_PER_MSEC); + ASSERT_OK(sd_fiber_sleep(100 * USEC_PER_MSEC)); ASSERT_OK(mkdir_parents(path, 0755)); - usleep_safe(100 * USEC_PER_MSEC); + ASSERT_OK(sd_fiber_sleep(100 * USEC_PER_MSEC)); ASSERT_OK(path_extract_directory(path, &d)); ASSERT_OK(asprintf(&suffixed, "%s.%" PRIx64, d, random_u64())); ASSERT_OK_ERRNO(rename(d, suffixed)); - usleep_safe(100 * USEC_PER_MSEC); + ASSERT_OK(sd_fiber_sleep(100 * USEC_PER_MSEC)); ASSERT_OK(asprintf(&suffixed2, "%s.%" PRIx64, d, random_u64())); ASSERT_OK_ERRNO(symlink(suffixed2, d)); - usleep_safe(100 * USEC_PER_MSEC); + ASSERT_OK(sd_fiber_sleep(100 * USEC_PER_MSEC)); ASSERT_OK(path_extract_filename(suffixed, &suffixed_basename)); ASSERT_OK_ERRNO(symlink(suffixed_basename, suffixed2)); - usleep_safe(100 * USEC_PER_MSEC); + ASSERT_OK(sd_fiber_sleep(100 * USEC_PER_MSEC)); socklen_t sa_len; r = sockaddr_un_set_path(&u.un, path); @@ -81,13 +79,13 @@ static void* thread_server(void *p) { ASSERT_OK_ERRNO(fd); ASSERT_OK_ERRNO(bind(fd, &u.sa, sa_len)); - usleep_safe(100 * USEC_PER_MSEC); + ASSERT_OK(sd_fiber_sleep(100 * USEC_PER_MSEC)); ASSERT_OK_ERRNO(listen(fd, SOMAXCONN_DELUXE)); - usleep_safe(100 * USEC_PER_MSEC); + ASSERT_OK(sd_fiber_sleep(100 * USEC_PER_MSEC)); ASSERT_OK(touch(path)); - usleep_safe(100 * USEC_PER_MSEC); + ASSERT_OK(sd_fiber_sleep(100 * USEC_PER_MSEC)); log_debug("Initialized server"); @@ -101,8 +99,7 @@ static void* thread_server(void *p) { ASSERT_OK(sd_event_new(&event)); - bus_fd = accept4(fd, NULL, NULL, SOCK_NONBLOCK|SOCK_CLOEXEC); - ASSERT_OK_ERRNO(bus_fd); + ASSERT_OK(bus_fd = sd_fiber_accept(fd, NULL, NULL, SOCK_NONBLOCK|SOCK_CLOEXEC)); log_debug("Accepted server connection"); @@ -129,13 +126,13 @@ static void* thread_server(void *p) { log_debug("Server done"); - return NULL; + return 0; } -static void* thread_client1(void *p) { +static int client1(void *userdata) { _cleanup_(sd_bus_error_free) sd_bus_error error = SD_BUS_ERROR_NULL; _cleanup_(sd_bus_flush_close_unrefp) sd_bus *bus = NULL; - const char *path = p, *t; + const char *path = ASSERT_PTR(userdata), *t; log_debug("Initializing client1"); @@ -151,59 +148,65 @@ static void* thread_client1(void *p) { log_debug("Client1 done"); - return NULL; -} - -static int client2_callback(sd_bus_message *m, void *userdata, sd_bus_error *ret_error) { - ASSERT_OK_ZERO(sd_bus_message_is_method_error(m, NULL)); - ASSERT_OK(sd_event_exit(sd_bus_get_event(sd_bus_message_get_bus(m)), 0)); return 0; } -static void* thread_client2(void *p) { +static int client2(void *userdata) { _cleanup_(sd_bus_flush_close_unrefp) sd_bus *bus = NULL; - _cleanup_(sd_event_unrefp) sd_event *event = NULL; - const char *path = p, *t; + const char *path = ASSERT_PTR(userdata), *t; log_debug("Initializing client2"); - ASSERT_OK(sd_event_new(&event)); ASSERT_OK(sd_bus_new(&bus)); ASSERT_OK(sd_bus_set_description(bus, "client2")); t = strjoina("unix:path=", path); ASSERT_OK(sd_bus_set_address(bus, t)); ASSERT_OK(sd_bus_set_watch_bind(bus, true)); - ASSERT_OK(sd_bus_attach_event(bus, event, 0)); + ASSERT_OK(sd_bus_attach_event(bus, sd_fiber_get_event(), 0)); ASSERT_OK(sd_bus_start(bus)); - ASSERT_OK(sd_bus_call_method_async(bus, NULL, "foo.bar", "/foo", "foo.TestInterface", "Foobar", client2_callback, NULL, NULL)); - - ASSERT_OK(sd_event_loop(event)); + _cleanup_(sd_bus_message_unrefp) sd_bus_message *m = NULL; + ASSERT_OK(sd_bus_call_method(bus, "foo.bar", "/foo", "foo.TestInterface", "Foobar", NULL, &m, NULL)); + ASSERT_OK_ZERO(sd_bus_message_is_method_error(m, NULL)); log_debug("Client2 done"); - return NULL; + return 0; } -static void request_exit(const char *path) { +typedef struct RequestExitArgs { + const char *path; + sd_future *client1; + sd_future *client2; +} RequestExitArgs; + +static int request_exit(void *userdata) { _cleanup_(sd_bus_flush_close_unrefp) sd_bus *bus = NULL; + RequestExitArgs *args = ASSERT_PTR(userdata); const char *t; + /* Wait for all client fibers to complete before requesting exit */ + ASSERT_OK(sd_fiber_await(args->client1)); + ASSERT_OK(sd_fiber_await(args->client2)); + ASSERT_OK(sd_bus_new(&bus)); - t = strjoina("unix:path=", path); + t = strjoina("unix:path=", args->path); ASSERT_OK(sd_bus_set_address(bus, t)); ASSERT_OK(sd_bus_set_watch_bind(bus, true)); ASSERT_OK(sd_bus_set_description(bus, "request-exit")); ASSERT_OK(sd_bus_start(bus)); ASSERT_OK(sd_bus_call_method(bus, "foo.bar", "/foo", "foo.TestInterface", "Exit", NULL, NULL, NULL)); + + return 0; } int main(int argc, char *argv[]) { _cleanup_(rm_rf_physical_and_freep) char *d = NULL; - pthread_t server, client1, client2; + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + _cleanup_(sd_future_unrefp) sd_future *f_server = NULL, *f_client1 = NULL, *f_client2 = NULL, *f_exit = NULL; char *path; test_setup_logging(LOG_DEBUG); @@ -214,16 +217,27 @@ int main(int argc, char *argv[]) { path = strjoina(d, "/this/is/a/socket"); - ASSERT_OK(-pthread_create(&server, NULL, thread_server, path)); - ASSERT_OK(-pthread_create(&client1, NULL, thread_client1, path)); - ASSERT_OK(-pthread_create(&client2, NULL, thread_client2, path)); + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); - ASSERT_OK(-pthread_join(client1, NULL)); - ASSERT_OK(-pthread_join(client2, NULL)); + ASSERT_OK(sd_fiber_new(e, "server", server, path, /* destroy= */ NULL, &f_server)); - request_exit(path); + ASSERT_OK(sd_fiber_new(e, "client-1", client1, path, /* destroy= */ NULL, &f_client1)); + ASSERT_OK(sd_fiber_new(e, "client-2", client2, path, /* destroy= */ NULL, &f_client2)); - ASSERT_OK(-pthread_join(server, NULL)); + RequestExitArgs args = { + .path = path, + .client1 = f_client1, + .client2 = f_client2, + }; + ASSERT_OK(sd_fiber_new(e, "request-exit", request_exit, &args, /* destroy= */ NULL, &f_exit)); - return 0; + ASSERT_OK(sd_event_loop(e)); + + ASSERT_OK(sd_future_result(f_client1)); + ASSERT_OK(sd_future_result(f_client2)); + ASSERT_OK(sd_future_result(f_exit)); + ASSERT_OK(sd_future_result(f_server)); + + return EXIT_SUCCESS; } diff --git a/src/systemd/sd-bus-vtable.h b/src/systemd/sd-bus-vtable.h index 5c11ca8ae5..036bda3fe4 100644 --- a/src/systemd/sd-bus-vtable.h +++ b/src/systemd/sd-bus-vtable.h @@ -44,6 +44,7 @@ __extension__ enum { SD_BUS_VTABLE_PROPERTY_EXPLICIT = 1ULL << 7, SD_BUS_VTABLE_SENSITIVE = 1ULL << 8, /* covers both directions: method call + reply */ SD_BUS_VTABLE_ABSOLUTE_OFFSET = 1ULL << 9, + /* Bit 10 is reserved for the private SD_BUS_VTABLE_METHOD_FIBER flag (see bus-internal.h). */ _SD_BUS_VTABLE_CAPABILITY_MASK = 0xFFFFULL << 40 }; From 14605a317cbb3bc8eae3214ab03b927e66a74950 Mon Sep 17 00:00:00 2001 From: Daan De Meyer Date: Tue, 14 Apr 2026 08:54:49 +0000 Subject: [PATCH 6/8] sd-varlink: make sd-varlink fiber-aware MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add varlink_server_bind_fiber() and varlink_server_bind_fiber_many() in varlink-util.{c,h} for registering a method handler that should run on a dedicated fiber per dispatch. The fiber-bound methods live in a separate s->fiber_methods map alongside the regular s->methods; bind_internal()/bind_many_internal() are factored out so the regular and fiber bind variants share their parsing/insertion code. Registering the same method in both maps is rejected because the dispatcher consults the regular map first and would otherwise silently shadow the fiber binding. varlink_dispatch_fiber() builds a VarlinkFiberData (refs to the connection, parameters, and method name), spawns a fiber via sd_fiber_new(), and makes the future floating so the fiber self-manages its lifetime — neither the dispatcher nor the connection has to track it. The fiber's priority is set to one below the connection's quit event source so that on graceful shutdown the fiber's exit handler fires (and runs its cleanup) before varlink's quit_callback() closes the connection underneath it; this is what lets a fiber-bound handler reply or flush its sentinel on a still-open connection during shutdown. The connection state transitions are reordered so they happen before the fiber spawn rather than after the synchronous callback returns: the fiber runs after dispatch has already moved past PROCESSING, which matches the behaviour expected for a deferred reply (the fiber may either reply immediately, or stash the connection and reply later, in which case the post-callback logic treats it as a PENDING_METHOD). Note that all the synchronous varlink APIs (sd_varlink_call() and friends) already behave properly when on a fiber because they call json_stream_wait() which calls ppoll_usec() which we already fixed to suspend when called from a fiber. The client/server varlink tests are migrated to fibers (threads → mock server fibers on the same event loop) to exercise the new paths. --- src/libsystemd/sd-varlink/sd-varlink.c | 301 ++++++++++++--- src/libsystemd/sd-varlink/test-varlink.c | 362 ++++++++++++++----- src/libsystemd/sd-varlink/varlink-internal.h | 3 +- src/libsystemd/sd-varlink/varlink-util.h | 4 + 4 files changed, 529 insertions(+), 141 deletions(-) diff --git a/src/libsystemd/sd-varlink/sd-varlink.c b/src/libsystemd/sd-varlink/sd-varlink.c index 8e43e38800..af9dd30973 100644 --- a/src/libsystemd/sd-varlink/sd-varlink.c +++ b/src/libsystemd/sd-varlink/sd-varlink.c @@ -6,6 +6,7 @@ #include "sd-daemon.h" #include "sd-event.h" +#include "sd-future.h" #include "sd-varlink.h" #include "alloc-util.h" @@ -37,6 +38,7 @@ #include "varlink-internal.h" #include "varlink-io.systemd.h" #include "varlink-org.varlink.service.h" +#include "varlink-util.h" #define VARLINK_DEFAULT_CONNECTIONS_MAX 4096U #define VARLINK_DEFAULT_CONNECTIONS_PER_UID_MAX 128U @@ -956,6 +958,178 @@ static int generic_method_get_interface_description( SD_JSON_BUILD_PAIR_STRING("description", text)); } +static int varlink_dispatch_sentinel(sd_varlink *v) { + int r; + + assert(v); + assert(v->sentinel); + + if (v->previous) { + r = json_stream_enqueue_full(&v->stream, v->previous, v->previous_fds, v->n_previous_fds); + if (r >= 0) { + v->previous = sd_json_variant_unref(v->previous); + v->previous_fds = mfree(v->previous_fds); + v->n_previous_fds = 0; + /* Mirror sd_varlink_reply()'s post-enqueue state machine: PENDING_* means we're + * outside the dispatch stack frame (e.g. called from varlink_fiber_entry after + * the fiber returned), so we go straight to IDLE_SERVER ourselves. PROCESSING_* + * means we're inside varlink_dispatch_method(), which will transition us. */ + if (IN_SET(v->state, VARLINK_PENDING_METHOD, VARLINK_PENDING_METHOD_MORE)) { + varlink_clear_current(v); + varlink_set_state(v, VARLINK_IDLE_SERVER); + } else + varlink_set_state(v, VARLINK_PROCESSED_METHOD); + } + + return r; + } + + char *sentinel = TAKE_PTR(v->sentinel); + + /* Propagate the sentinel to the client if one was configured and no replies were enqueued by + * the callback. */ + if (sentinel == POINTER_MAX) + r = sd_varlink_reply(v, NULL); + else { + r = sd_varlink_error(v, sentinel, NULL); + /* sd_varlink_error() deliberately returns a negative + * errno mapped from the error id on success (so method + * callbacks can `return sd_varlink_error(...);` to + * enqueue a reply and propagate a matching errno in one + * go). For sentinel dispatch we don't care about that + * mapping — the reply is either enqueued or not, which + * we detect via the state transition instead. */ + if (IN_SET(v->state, VARLINK_PROCESSED_METHOD, VARLINK_IDLE_SERVER)) + r = 0; + } + + if (sentinel != POINTER_MAX) + free(sentinel); + + return r; +} + +typedef struct VarlinkFiberData { + sd_varlink *link; + sd_json_variant *parameters; + sd_varlink_method_flags_t flags; + void *userdata; + sd_varlink_method_t callback; +} VarlinkFiberData; + +static VarlinkFiberData* varlink_fiber_data_free(VarlinkFiberData *d) { + if (!d) + return NULL; + + sd_json_variant_unref(d->parameters); + sd_varlink_unref(d->link); + return mfree(d); +} + +DEFINE_TRIVIAL_CLEANUP_FUNC(VarlinkFiberData*, varlink_fiber_data_free); + +static void varlink_fiber_data_destroy(void *userdata) { + varlink_fiber_data_free(userdata); +} + +static int varlink_fiber_entry(void *userdata) { + VarlinkFiberData *d = ASSERT_PTR(userdata); + sd_varlink *v = d->link; + int r; + + r = d->callback(v, d->parameters, d->flags, d->userdata); + + /* The fiber runs after varlink_dispatch_method() has already transitioned the state from + * VARLINK_PROCESSING_METHOD{,_MORE} to VARLINK_PENDING_METHOD{,_MORE}, so that's what we match + * here to decide whether the call still needs a reply. Any other state (e.g. IDLE_SERVER after + * the callback replied, or DISCONNECTED after sd_varlink_close()) means no fixup is needed. */ + if (!IN_SET(v->state, VARLINK_PENDING_METHOD, VARLINK_PENDING_METHOD_MORE)) + return r; + + if (r < 0) { + varlink_log_errno(v, r, "Fiber returned error: %m"); + + /* Propagate error to the client if the method call remains unanswered. */ + r = sd_varlink_error_errno(v, r); + } else if (v->sentinel) { + r = varlink_dispatch_sentinel(v); + if (r < 0) + varlink_log_errno(v, r, "Failed to process sentinel: %m"); + } else if (v->n_ref <= 2) { + /* Bare minimum refs (server + fiber data) means the connection wasn't stashed + * to reply later, so the fiber was supposed to reply itself but didn't. */ + r = varlink_log_errno(v, SYNTHETIC_ERRNO(EPROTO), + "Fiber returned without enqueuing a reply or stashing connection, failing."); + goto fail; + } else + r = 0; + + /* If we didn't manage to enqueue a response, then fail the connection completely. */ + if (r < 0 && IN_SET(v->state, VARLINK_PENDING_METHOD, VARLINK_PENDING_METHOD_MORE)) + goto fail; + + return r; + +fail: + varlink_set_state(v, VARLINK_PROCESSING_FAILURE); + varlink_dispatch_local_error(v, SD_VARLINK_ERROR_PROTOCOL); + sd_varlink_close(v); + + return r; +} + +static int varlink_dispatch_fiber(sd_varlink *v, const char *method, sd_varlink_method_t callback, sd_json_variant *parameters, sd_varlink_method_flags_t flags) { + int r; + + assert(v); + assert(v->server); + assert(method); + assert(callback); + + if (!v->server->event) + return varlink_log_errno(v, SYNTHETIC_ERRNO(EDEADLK), + "Cannot dispatch fiber method without event loop."); + + _cleanup_(varlink_fiber_data_freep) VarlinkFiberData *d = new(VarlinkFiberData, 1); + if (!d) + return log_oom_debug(); + + *d = (VarlinkFiberData) { + .link = sd_varlink_ref(v), + .parameters = sd_json_variant_ref(parameters), + .flags = flags, + .userdata = v->userdata, + .callback = callback, + }; + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + r = sd_fiber_new(v->server->event, method, varlink_fiber_entry, d, varlink_fiber_data_destroy, &f); + if (r < 0) + return r; + + TAKE_PTR(d); /* The fiber owns the data now. */ + + /* Run the fiber at a higher priority than the connection's quit event source, so that on event + * loop exit the fiber's exit source (which cancels it and drives its cleanup) fires before + * varlink's quit_callback closes the connection. This lets a fiber handler reply with an error + * or flush its sentinel on a still-open connection during graceful shutdown. */ + int64_t priority; + r = sd_event_source_get_priority(v->quit_event_source, &priority); + if (r < 0) + return r; + + r = sd_future_set_priority(f, priority > INT64_MIN ? priority - 1 : priority); + if (r < 0) + return r; + + /* Hand the future's lifetime over to the event loop: it'll auto-unref on resolve. */ + r = sd_fiber_set_floating(f, true); + if (r < 0) + return r; + + return 0; +} + static int varlink_dispatch_method(sd_varlink *v) { _cleanup_(sd_json_variant_unrefp) sd_json_variant *parameters = NULL; sd_varlink_method_flags_t flags = 0; @@ -1053,7 +1227,13 @@ static int varlink_dispatch_method(sd_varlink *v) { v->protocol_upgrade || FLAGS_SET(v->server->flags, SD_VARLINK_SERVER_UPGRADABLE)); /* First consult user supplied method implementations */ + bool is_fiber = false; callback = hashmap_get(v->server->methods, method); + if (!callback) { + callback = hashmap_get(v->server->fiber_methods, method); + if (callback) + is_fiber = true; + } if (!callback) { if (streq(method, "org.varlink.service.GetInfo")) callback = generic_method_get_info; @@ -1105,7 +1285,13 @@ static int varlink_dispatch_method(sd_varlink *v) { } if (!invalid) { - r = callback(v, parameters, flags, v->userdata); + if (is_fiber) + /* Spawn a fiber to run the callback. The VarlinkFiberData takes a ref on the + * connection (bumping n_ref above 2), so the post-callback logic below treats + * this as a deferred reply and moves state to PENDING_METHOD. */ + r = varlink_dispatch_fiber(v, method, callback, parameters, flags); + else + r = callback(v, parameters, flags, v->userdata); if (VARLINK_STATE_WANTS_REPLY(v->state)) { if (r < 0) { varlink_log_errno(v, r, "Callback for '%s' returned error: %m", method); @@ -1114,37 +1300,7 @@ static int varlink_dispatch_method(sd_varlink *v) { * if the method call remains unanswered. */ r = sd_varlink_error_errno(v, r); } else if (v->sentinel) { - if (v->previous) { - r = json_stream_enqueue_full(&v->stream, v->previous, v->previous_fds, v->n_previous_fds); - if (r >= 0) { - v->previous = sd_json_variant_unref(v->previous); - v->previous_fds = mfree(v->previous_fds); - v->n_previous_fds = 0; - varlink_set_state(v, VARLINK_PROCESSED_METHOD); - } - } else { - char *sentinel = TAKE_PTR(v->sentinel); - - /* Propagate the sentinel to the client if one was configured - * and no replies were enqueued by the callback. */ - if (sentinel == POINTER_MAX) - r = sd_varlink_reply(v, NULL); - else { - r = sd_varlink_error(v, sentinel, NULL); - /* sd_varlink_error() deliberately returns a negative - * errno mapped from the error id on success (so method - * callbacks can `return sd_varlink_error(...);` to - * enqueue a reply and propagate a matching errno in one - * go). For sentinel dispatch we don't care about that - * mapping — the reply is either enqueued or not, which - * we detect via the state transition instead. */ - if (v->state == VARLINK_PROCESSED_METHOD) - r = 0; - } - - if (sentinel != POINTER_MAX) - free(sentinel); - } + r = varlink_dispatch_sentinel(v); if (r < 0) varlink_log_errno(v, r, "Failed to process sentinel for method '%s': %m", method); } else { @@ -2596,8 +2752,12 @@ _public_ int sd_varlink_set_sentinel(sd_varlink *v, const char *error_id) { if (v->state == VARLINK_PROCESSING_METHOD_ONEWAY) return 0; - /* This has to be called during a callback, and not after it has exited. */ - assert_return(IN_SET(v->state, VARLINK_PROCESSING_METHOD, VARLINK_PROCESSING_METHOD_MORE), + /* This has to be called during a callback, and not after it has exited. The PENDING states + * apply to fiber callbacks, which run after varlink_dispatch_method() has already transitioned + * the state from PROCESSING to PENDING. */ + assert_return(IN_SET(v->state, + VARLINK_PROCESSING_METHOD, VARLINK_PROCESSING_METHOD_MORE, + VARLINK_PENDING_METHOD, VARLINK_PENDING_METHOD_MORE), -EUCLEAN); char *s = NULL; @@ -2899,7 +3059,11 @@ static sd_varlink_server* varlink_server_destroy(sd_varlink_server *s) { while ((m = hashmap_steal_first_key(s->methods))) free(m); + while ((m = hashmap_steal_first_key(s->fiber_methods))) + free(m); + hashmap_free(s->methods); + hashmap_free(s->fiber_methods); hashmap_free(s->interfaces); hashmap_free(s->symbols); hashmap_free(s->by_uid); @@ -3590,23 +3754,32 @@ static bool varlink_symbol_in_interface(const char *method, const char *interfac return !strchr(p+1, '.'); } -_public_ int sd_varlink_server_bind_method(sd_varlink_server *s, const char *method, sd_varlink_method_t callback) { +static int varlink_server_bind_internal(sd_varlink_server *s, Hashmap **methods, const char *method, sd_varlink_method_t callback) { _cleanup_free_ char *m = NULL; int r; - assert_return(s, -EINVAL); - assert_return(method, -EINVAL); - assert_return(callback, -EINVAL); + assert(s); + assert(methods); + assert(method); + assert(callback); if (varlink_symbol_in_interface(method, "org.varlink.service") || varlink_symbol_in_interface(method, "io.systemd")) return varlink_server_log_errno(s, SYNTHETIC_ERRNO(EEXIST), "Cannot bind server to '%s'.", method); + /* Refuse to register the same method in both the regular and fiber method maps: the dispatcher + * always consults methods first and would silently ignore a shadowed fiber_methods entry (or vice + * versa), hiding the misconfiguration. */ + Hashmap *other = methods == &s->methods ? s->fiber_methods : s->methods; + if (hashmap_contains(other, method)) + return varlink_server_log_errno(s, SYNTHETIC_ERRNO(EEXIST), + "Method '%s' is already bound in the other method map.", method); + m = strdup(method); if (!m) return log_oom_debug(); - r = hashmap_ensure_put(&s->methods, &string_hash_ops, m, callback); + r = hashmap_ensure_put(methods, &string_hash_ops, m, callback); if (r == -ENOMEM) return log_oom_debug(); if (r < 0) @@ -3617,13 +3790,12 @@ _public_ int sd_varlink_server_bind_method(sd_varlink_server *s, const char *met return 0; } -_public_ int sd_varlink_server_bind_method_many_internal(sd_varlink_server *s, ...) { - va_list ap; +static int varlink_server_bind_many_internal(sd_varlink_server *s, Hashmap **methods, va_list ap) { int r = 0; - assert_return(s, -EINVAL); + assert(s); + assert(methods); - va_start(ap, s); for (;;) { sd_varlink_method_t callback; const char *method; @@ -3634,10 +3806,51 @@ _public_ int sd_varlink_server_bind_method_many_internal(sd_varlink_server *s, . callback = va_arg(ap, sd_varlink_method_t); - r = sd_varlink_server_bind_method(s, method, callback); + r = varlink_server_bind_internal(s, methods, method, callback); if (r < 0) break; } + + return r; +} + +_public_ int sd_varlink_server_bind_method(sd_varlink_server *s, const char *method, sd_varlink_method_t callback) { + assert_return(s, -EINVAL); + assert_return(method, -EINVAL); + assert_return(callback, -EINVAL); + + return varlink_server_bind_internal(s, &s->methods, method, callback); +} + +_public_ int sd_varlink_server_bind_method_many_internal(sd_varlink_server *s, ...) { + va_list ap; + int r; + + assert_return(s, -EINVAL); + + va_start(ap, s); + r = varlink_server_bind_many_internal(s, &s->methods, ap); + va_end(ap); + + return r; +} + +int varlink_server_bind_fiber(sd_varlink_server *s, const char *method, sd_varlink_method_t callback) { + assert_return(s, -EINVAL); + assert_return(method, -EINVAL); + assert_return(callback, -EINVAL); + + return varlink_server_bind_internal(s, &s->fiber_methods, method, callback); +} + +int varlink_server_bind_fiber_many_internal(sd_varlink_server *s, ...) { + va_list ap; + int r; + + assert_return(s, -EINVAL); + + va_start(ap, s); + r = varlink_server_bind_many_internal(s, &s->fiber_methods, ap); va_end(ap); return r; diff --git a/src/libsystemd/sd-varlink/test-varlink.c b/src/libsystemd/sd-varlink/test-varlink.c index cca859c2f6..31b7cb46f9 100644 --- a/src/libsystemd/sd-varlink/test-varlink.c +++ b/src/libsystemd/sd-varlink/test-varlink.c @@ -2,13 +2,13 @@ #include #include -#include #include #include #include #include #include "sd-event.h" +#include "sd-future.h" #include "sd-json.h" #include "sd-varlink.h" @@ -216,7 +216,12 @@ static void flood_test(const char *address) { /* Block the main event loop while we flood */ ASSERT_OK_EQ_ERRNO(write(block_write_fd, &x, sizeof(x)), (ssize_t) sizeof(x)); - ASSERT_OK(sd_event_default(&e)); + /* Create a fresh event loop for the flood test — we can't reuse the default event because the + * main test (and the fiber we're running in) is already running it, and sd_event_loop() asserts + * the event is in the INITIAL state. Exit-on-idle so the nested loop terminates once the + * overload reply has been received and all other work is quiesced. */ + ASSERT_OK(sd_event_new(&e)); + ASSERT_OK(sd_event_set_exit_on_idle(e, true)); /* Flood the server with connections */ ASSERT_NOT_NULL(connections = new0(sd_varlink*, OVERLOAD_CONNECTIONS)); @@ -251,7 +256,7 @@ static void flood_test(const char *address) { connections[k] = sd_varlink_unref(connections[k]); } -static void *thread(void *arg) { +static int client_fiber(void *arg) { _cleanup_(sd_varlink_flush_close_unrefp) sd_varlink *c = NULL; _cleanup_(sd_json_variant_unrefp) sd_json_variant *i = NULL; _cleanup_(sd_json_variant_unrefp) sd_json_variant *wrong = NULL; @@ -263,7 +268,7 @@ static void *thread(void *arg) { SD_JSON_BUILD_PAIR_INTEGER("b", 99)))); ASSERT_OK(sd_varlink_connect_address(&c, arg)); - ASSERT_OK(sd_varlink_set_description(c, "thread-client")); + ASSERT_OK(sd_varlink_set_description(c, "fiber-client")); ASSERT_OK(sd_varlink_set_allow_fd_passing_input(c, true)); ASSERT_OK(sd_varlink_set_allow_fd_passing_output(c, true)); @@ -321,7 +326,7 @@ static void *thread(void *arg) { ASSERT_OK(sd_varlink_send(c, "io.test.Done", NULL)); - return NULL; + return 0; } static int block_fd_handler(sd_event_source *s, int fd, uint32_t revents, void *userdata) { @@ -348,8 +353,8 @@ TEST(chat) { _cleanup_(rm_rf_physical_and_freep) char *tmpdir = NULL; _cleanup_(sd_json_variant_unrefp) sd_json_variant *v = NULL; _cleanup_(sd_event_unrefp) sd_event *e = NULL; + _cleanup_(sd_future_unrefp) sd_future *f = NULL; _cleanup_close_pair_ int block_fds[2] = EBADF_PAIR; - pthread_t t; const char *sp; ASSERT_OK(mkdtemp_malloc("/tmp/varlink-test-XXXXXX", &tmpdir)); @@ -388,11 +393,11 @@ TEST(chat) { ASSERT_OK(sd_varlink_attach_event(c, e, 0)); - ASSERT_OK(-pthread_create(&t, NULL, thread, (void*) sp)); + ASSERT_OK(sd_fiber_new(e, "client", client_fiber, (void*) sp, /* destroy= */ NULL, &f)); ASSERT_OK(sd_event_loop(e)); - ASSERT_OK(-pthread_join(t, NULL)); + ASSERT_OK(sd_future_result(f)); } static int method_invalid(sd_varlink *link, sd_json_variant *parameters, sd_varlink_method_flags_t flags, void *userdata) { @@ -618,6 +623,173 @@ TEST(sentinel_oneway) { ASSERT_OK(sd_event_loop(e)); } +static int method_fiber_sentinel_error(sd_varlink *link, sd_json_variant *parameters, sd_varlink_method_flags_t flags, void *userdata) { + /* Set an error sentinel from a fiber callback and return without sending a reply. The sentinel + * error should still be propagated by the fiber's post-callback logic, even though the varlink + * state has already been transitioned to VARLINK_PENDING_METHOD by the time the fiber runs. */ + ASSERT_OK(sd_varlink_set_sentinel(link, "io.test.SentinelError")); + return 0; +} + +TEST(fiber_sentinel_error) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_default(&e)); + + _cleanup_(sd_varlink_server_unrefp) sd_varlink_server *s = NULL; + ASSERT_OK(sd_varlink_server_new(&s, 0)); + + ASSERT_OK(sd_varlink_server_attach_event(s, e, 0)); + + ASSERT_OK(varlink_server_bind_fiber(s, "io.test.FiberSentinelError", method_fiber_sentinel_error)); + + int connfd[2]; + ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM|SOCK_NONBLOCK|SOCK_CLOEXEC, 0, connfd)); + ASSERT_OK(sd_varlink_server_add_connection(s, connfd[0], /* ret= */ NULL)); + + _cleanup_(sd_varlink_unrefp) sd_varlink *c = NULL; + ASSERT_OK(sd_varlink_connect_fd(&c, connfd[1])); + + ASSERT_OK(sd_varlink_attach_event(c, e, 0)); + + ASSERT_OK(sd_varlink_bind_reply(c, reply_sentinel_error)); + + ASSERT_OK(sd_varlink_invoke(c, "io.test.FiberSentinelError", /* parameters= */ NULL)); + + ASSERT_OK(sd_event_loop(e)); +} + +static int method_fiber_errno(sd_varlink *link, sd_json_variant *parameters, sd_varlink_method_flags_t flags, void *userdata) { + /* Return a negative errno without sending a reply. The fiber's post-callback logic should + * convert this into a SD_VARLINK_ERROR_SYSTEM reply. */ + return -ENOSYS; +} + +static int reply_fiber_errno(sd_varlink *link, sd_json_variant *parameters, const char *error_id, sd_varlink_reply_flags_t flags, void *userdata) { + ASSERT_STREQ(error_id, SD_VARLINK_ERROR_SYSTEM); + ASSERT_EQ(sd_json_variant_integer(sd_json_variant_by_key(parameters, "errno")), ENOSYS); + ASSERT_OK(sd_event_exit(sd_varlink_get_event(link), EXIT_SUCCESS)); + return 0; +} + +TEST(fiber_errno) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_default(&e)); + + _cleanup_(sd_varlink_server_unrefp) sd_varlink_server *s = NULL; + ASSERT_OK(sd_varlink_server_new(&s, 0)); + + ASSERT_OK(sd_varlink_server_attach_event(s, e, 0)); + + ASSERT_OK(varlink_server_bind_fiber(s, "io.test.FiberErrno", method_fiber_errno)); + + int connfd[2]; + ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM|SOCK_NONBLOCK|SOCK_CLOEXEC, 0, connfd)); + ASSERT_OK(sd_varlink_server_add_connection(s, connfd[0], /* ret= */ NULL)); + + _cleanup_(sd_varlink_unrefp) sd_varlink *c = NULL; + ASSERT_OK(sd_varlink_connect_fd(&c, connfd[1])); + + ASSERT_OK(sd_varlink_attach_event(c, e, 0)); + + ASSERT_OK(sd_varlink_bind_reply(c, reply_fiber_errno)); + + ASSERT_OK(sd_varlink_invoke(c, "io.test.FiberErrno", /* parameters= */ NULL)); + + ASSERT_OK(sd_event_loop(e)); +} + +static int method_fiber_no_reply(sd_varlink *link, sd_json_variant *parameters, sd_varlink_method_flags_t flags, void *userdata) { + /* Return success without replying and without stashing a ref. The fiber's post-callback + * logic should detect this and fail the connection. */ + return 0; +} + +static int reply_fiber_no_reply(sd_varlink *link, sd_json_variant *parameters, const char *error_id, sd_varlink_reply_flags_t flags, void *userdata) { + ASSERT_STREQ(error_id, SD_VARLINK_ERROR_DISCONNECTED); + ASSERT_OK(sd_event_exit(sd_varlink_get_event(link), EXIT_SUCCESS)); + return 0; +} + +TEST(fiber_no_reply) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_default(&e)); + + _cleanup_(sd_varlink_server_unrefp) sd_varlink_server *s = NULL; + ASSERT_OK(sd_varlink_server_new(&s, 0)); + + ASSERT_OK(sd_varlink_server_attach_event(s, e, 0)); + + ASSERT_OK(varlink_server_bind_fiber(s, "io.test.FiberNoReply", method_fiber_no_reply)); + + int connfd[2]; + ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM|SOCK_NONBLOCK|SOCK_CLOEXEC, 0, connfd)); + ASSERT_OK(sd_varlink_server_add_connection(s, connfd[0], /* ret= */ NULL)); + + _cleanup_(sd_varlink_unrefp) sd_varlink *c = NULL; + ASSERT_OK(sd_varlink_connect_fd(&c, connfd[1])); + + ASSERT_OK(sd_varlink_attach_event(c, e, 0)); + + ASSERT_OK(sd_varlink_bind_reply(c, reply_fiber_no_reply)); + + ASSERT_OK(sd_varlink_invoke(c, "io.test.FiberNoReply", /* parameters= */ NULL)); + + ASSERT_OK(sd_event_loop(e)); +} + +static int fiber_stashed_deferred_reply(sd_event_source *s, void *userdata) { + _cleanup_(sd_varlink_unrefp) sd_varlink *link = ASSERT_PTR(userdata); + + sd_event_source_disable_unref(s); + ASSERT_OK(sd_varlink_replybo(link, SD_JSON_BUILD_PAIR_STRING("result", "stashed"))); + return 0; +} + +static int method_fiber_stash(sd_varlink *link, sd_json_variant *parameters, sd_varlink_method_flags_t flags, void *userdata) { + /* Stash a ref on the connection so n_ref > 2 when the fiber returns, and reply later from a + * deferred event source. The fiber's post-callback logic should see the extra ref and treat + * this as a valid deferred-reply case instead of failing the connection. */ + sd_event_source *source; + + ASSERT_OK(sd_event_add_defer(sd_varlink_get_event(link), &source, fiber_stashed_deferred_reply, sd_varlink_ref(link))); + ASSERT_OK(sd_event_source_set_enabled(source, SD_EVENT_ONESHOT)); + return 0; +} + +static int reply_fiber_stash(sd_varlink *link, sd_json_variant *parameters, const char *error_id, sd_varlink_reply_flags_t flags, void *userdata) { + ASSERT_NULL(error_id); + ASSERT_STREQ(sd_json_variant_string(sd_json_variant_by_key(parameters, "result")), "stashed"); + ASSERT_OK(sd_event_exit(sd_varlink_get_event(link), EXIT_SUCCESS)); + return 0; +} + +TEST(fiber_stash) { + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + ASSERT_OK(sd_event_default(&e)); + + _cleanup_(sd_varlink_server_unrefp) sd_varlink_server *s = NULL; + ASSERT_OK(sd_varlink_server_new(&s, 0)); + + ASSERT_OK(sd_varlink_server_attach_event(s, e, 0)); + + ASSERT_OK(varlink_server_bind_fiber(s, "io.test.FiberStash", method_fiber_stash)); + + int connfd[2]; + ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM|SOCK_NONBLOCK|SOCK_CLOEXEC, 0, connfd)); + ASSERT_OK(sd_varlink_server_add_connection(s, connfd[0], /* ret= */ NULL)); + + _cleanup_(sd_varlink_unrefp) sd_varlink *c = NULL; + ASSERT_OK(sd_varlink_connect_fd(&c, connfd[1])); + + ASSERT_OK(sd_varlink_attach_event(c, e, 0)); + + ASSERT_OK(sd_varlink_bind_reply(c, reply_fiber_stash)); + + ASSERT_OK(sd_varlink_invoke(c, "io.test.FiberStash", /* parameters= */ NULL)); + + ASSERT_OK(sd_event_loop(e)); +} + static int method_with_fd_sentinel(sd_varlink *link, sd_json_variant *parameters, sd_varlink_method_flags_t flags, void *userdata) { _cleanup_close_ int fd1 = -EBADF, fd2 = -EBADF; @@ -768,8 +940,9 @@ static int method_upgrade(sd_varlink *link, sd_json_variant *parameters, sd_varl if (r < 0) return r; - /* After upgrade, do raw I/O: read until EOF, reverse, write back. - * The client shuts down its write side after sending, so we get a clean EOF. */ + /* After upgrade, do raw I/O: read until the client shuts down its write side (giving us a clean + * EOF), reverse what we got, and write it back. Use suspending I/O so other fibers (the client) + * can make progress while we're waiting on the socket. */ char buf[64] = {}; ssize_t n = ASSERT_OK(loop_read(input_fd, buf, sizeof(buf) - 1, /* do_poll= */ true)); ASSERT_GT(n, 0); @@ -789,12 +962,10 @@ static int method_upgrade_without_flag(sd_varlink *link, sd_json_variant *parame /* Calling reply_and_upgrade without the client requesting it should fail with -EPROTO */ ASSERT_ERROR(sd_varlink_reply_and_upgrade(link, /* parameters= */ NULL, &input_fd, &output_fd), EPROTO); - sd_event_exit(sd_varlink_get_event(link), EXIT_SUCCESS); - return sd_varlink_reply(link, /* parameters= */ NULL); } -static void *upgrade_thread(void *arg) { +static int upgrade_client_fiber(void *arg) { _cleanup_(sd_varlink_flush_close_unrefp) sd_varlink *c = NULL; _cleanup_close_ int input_fd = -EBADF, output_fd = -EBADF; sd_json_variant *o = NULL; @@ -827,14 +998,15 @@ static void *upgrade_thread(void *arg) { ASSERT_OK(sd_varlink_call(c2, "io.test.UpgradeWithoutFlag", /* parameters= */ NULL, &o, &error_id)); ASSERT_NULL(error_id); - return NULL; + ASSERT_OK(sd_event_exit(sd_fiber_get_event(), EXIT_SUCCESS)); + return 0; } TEST(upgrade) { _cleanup_(sd_varlink_server_unrefp) sd_varlink_server *s = NULL; _cleanup_(rm_rf_physical_and_freep) char *tmpdir = NULL; _cleanup_(sd_event_unrefp) sd_event *e = NULL; - pthread_t t; + _cleanup_(sd_future_unrefp) sd_future *f = NULL; const char *sp; ASSERT_OK(mkdtemp_malloc("/tmp/varlink-test-XXXXXX", &tmpdir)); @@ -844,31 +1016,23 @@ TEST(upgrade) { ASSERT_OK(sd_varlink_server_new(&s, SD_VARLINK_SERVER_UPGRADABLE)); ASSERT_OK(sd_varlink_server_set_description(s, "upgrade-server")); - ASSERT_OK(sd_varlink_server_bind_method(s, "io.test.Upgrade", method_upgrade)); + /* The method does raw I/O on the upgraded socket — bind it as a fiber method so it can + * suspend on loop_read()/loop_write() and the client fiber can make progress concurrently. */ + ASSERT_OK(varlink_server_bind_fiber(s, "io.test.Upgrade", method_upgrade)); ASSERT_OK(sd_varlink_server_bind_method(s, "io.test.UpgradeWithoutFlag", method_upgrade_without_flag)); ASSERT_OK(sd_varlink_server_listen_address(s, sp, 0600)); ASSERT_OK(sd_varlink_server_attach_event(s, e, 0)); - ASSERT_OK(-pthread_create(&t, NULL, upgrade_thread, (void*) sp)); + ASSERT_OK(sd_fiber_new(e, "upgrade-client", upgrade_client_fiber, (void*) sp, /* destroy= */ NULL, &f)); - /* Run the event loop until no more connections (the thread will disconnect when done) */ + /* Run the event loop. Exits on idle once the client fiber completes and all server connections + * have been torn down. */ ASSERT_OK(sd_event_loop(e)); - ASSERT_OK(-pthread_join(t, NULL)); + ASSERT_OK(sd_future_result(f)); } -static int method_upgrade_and_exit(sd_varlink *link, sd_json_variant *parameters, sd_varlink_method_flags_t flags, void *userdata) { - sd_event *event = ASSERT_PTR(userdata); - - int r = method_upgrade(link, parameters, flags, /* userdata= */ NULL); - - /* Exit the event loop after the upgrade is handled. We can't use sd_varlink_get_event() - * here because the connection is already disconnected after reply_and_upgrade. */ - (void) sd_event_exit(event, r < 0 ? r : EXIT_SUCCESS); - return r; -} - -static void *upgrade_pipelining_thread(void *arg) { +static int upgrade_pipelining_client_fiber(void *arg) { union sockaddr_union sa = {}; _cleanup_close_ int fd = -EBADF; @@ -895,8 +1059,8 @@ static void *upgrade_pipelining_thread(void *arg) { /* Shut down write side so server's method_upgrade sees EOF after raw payload */ ASSERT_OK_ERRNO(shutdown(fd, SHUT_WR)); - /* Read everything: upgrade reply (JSON + \0) + reversed raw payload. The server closes - * the connection after writing, so loop_read() reads until EOF and gets it all. */ + /* Read everything: upgrade reply (JSON + \0) + reversed raw payload. The server closes the + * connection after writing, so loop_read() reads until EOF and gets it all. */ char buf[256] = {}; ssize_t n = ASSERT_OK(loop_read(fd, buf, sizeof(buf) - 1, /* do_poll= */ true)); ASSERT_GT(n, 0); @@ -911,14 +1075,15 @@ static void *upgrade_pipelining_thread(void *arg) { ASSERT_EQ(raw_size, strlen(raw_payload)); ASSERT_STREQ(strndupa_safe(raw, raw_size), "!denilepiP"); - return NULL; + ASSERT_OK(sd_event_exit(sd_fiber_get_event(), EXIT_SUCCESS)); + return 0; } TEST(upgrade_pipelining) { _cleanup_(sd_varlink_server_unrefp) sd_varlink_server *s = NULL; _cleanup_(rm_rf_physical_and_freep) char *tmpdir = NULL; _cleanup_(sd_event_unrefp) sd_event *e = NULL; - pthread_t t; + _cleanup_(sd_future_unrefp) sd_future *f = NULL; const char *sp; ASSERT_OK(mkdtemp_malloc("/tmp/varlink-test-XXXXXX", &tmpdir)); @@ -926,25 +1091,23 @@ TEST(upgrade_pipelining) { ASSERT_OK(sd_event_new(&e)); - ASSERT_OK(sd_varlink_server_new(&s, SD_VARLINK_SERVER_UPGRADABLE|SD_VARLINK_SERVER_INHERIT_USERDATA)); + ASSERT_OK(sd_varlink_server_new(&s, SD_VARLINK_SERVER_UPGRADABLE)); ASSERT_OK(sd_varlink_server_set_description(s, "upgrade-pipelining-server")); - ASSERT_OK(sd_varlink_server_bind_method(s, "io.test.Upgrade", method_upgrade_and_exit)); + /* method_upgrade does raw I/O on the upgraded socket, so bind as a fiber method. */ + ASSERT_OK(varlink_server_bind_fiber(s, "io.test.Upgrade", method_upgrade)); ASSERT_OK(sd_varlink_server_listen_address(s, sp, 0600)); ASSERT_OK(sd_varlink_server_attach_event(s, e, 0)); - sd_varlink_server_set_userdata(s, e); - ASSERT_OK(-pthread_create(&t, NULL, upgrade_pipelining_thread, (void*) sp)); + ASSERT_OK(sd_fiber_new(e, "upgrade-pipelining-client", upgrade_pipelining_client_fiber, (void*) sp, /* destroy= */ NULL, &f)); ASSERT_OK(sd_event_loop(e)); - ASSERT_OK(-pthread_join(t, NULL)); + ASSERT_OK(sd_future_result(f)); } typedef struct ExecDirServer { sd_varlink_server *server; - sd_event *event; const char *name; - pthread_t thread; } ExecDirServer; static int method_execute_dir_ping(sd_varlink *link, sd_json_variant *parameters, sd_varlink_method_flags_t flags, void *userdata) { @@ -953,20 +1116,6 @@ static int method_execute_dir_ping(sd_varlink *link, sd_json_variant *parameters return sd_varlink_replybo(link, SD_JSON_BUILD_PAIR_STRING("name", srv->name)); } -static void on_execute_dir_disconnect(sd_varlink_server *s, sd_varlink *link, void *userdata) { - ExecDirServer *srv = ASSERT_PTR(userdata); - - /* Only one client (from varlink_execute_directory()) connects per server — once it's gone, we're done. */ - ASSERT_OK(sd_event_exit(srv->event, 0)); -} - -static void *execute_dir_server_thread(void *arg) { - ExecDirServer *srv = arg; - - ASSERT_OK(sd_event_loop(srv->event)); - return NULL; -} - static int execute_dir_reply(sd_varlink *link, sd_json_variant *parameters, const char *error_id, sd_varlink_reply_flags_t flags, void *userdata) { size_t *count = ASSERT_PTR(userdata); @@ -977,51 +1126,28 @@ static int execute_dir_reply(sd_varlink *link, sd_json_variant *parameters, cons return 0; } -TEST(execute_directory) { - _cleanup_(rm_rf_physical_and_freep) char *tmpdir = NULL; - static const char * const names[] = { "alpha", "beta", "gamma" }; - ExecDirServer servers[ELEMENTSOF(names)] = {}; - size_t reply_count = 0; +typedef struct ExecDirClientArgs { + const char *tmpdir; + size_t n_servers; + size_t *reply_count; +} ExecDirClientArgs; - ASSERT_OK(mkdtemp_malloc("/tmp/varlink-execdir-XXXXXX", &tmpdir)); - - for (size_t i = 0; i < ELEMENTSOF(names); i++) { - ExecDirServer *eds = servers + i; - servers[i].name = names[i]; - - _cleanup_free_ char *j = ASSERT_PTR(path_join(tmpdir, names[i])); - - ASSERT_OK(sd_event_new(&eds->event)); - ASSERT_OK(varlink_server_new(&eds->server, - SD_VARLINK_SERVER_INHERIT_USERDATA, - eds)); - ASSERT_OK(sd_varlink_server_bind_method(eds->server, "io.test.ExecDirPing", method_execute_dir_ping)); - ASSERT_OK(sd_varlink_server_bind_disconnect(eds->server, on_execute_dir_disconnect)); - ASSERT_OK(sd_varlink_server_listen_address(eds->server, j, 0600)); - ASSERT_OK(sd_varlink_server_attach_event(eds->server, eds->event, 0)); - - ASSERT_OK(-pthread_create(&eds->thread, NULL, execute_dir_server_thread, eds)); - } +static int execute_dir_client_fiber(void *arg) { + ExecDirClientArgs *a = ASSERT_PTR(arg); ASSERT_OK_EQ(varlink_execute_directory( - tmpdir, + a->tmpdir, "io.test.ExecDirPing", /* parameters= */ NULL, /* more= */ false, /* timeout_usec= */ USEC_INFINITY, execute_dir_reply, - &reply_count), (ssize_t) ELEMENTSOF(names)); - ASSERT_EQ(reply_count, ELEMENTSOF(names)); - - FOREACH_ELEMENT(eds, servers) { - ASSERT_OK(-pthread_join(eds->thread, NULL)); - eds->server = sd_varlink_server_unref(eds->server); - eds->event = sd_event_unref(eds->event); - } + a->reply_count), (ssize_t) a->n_servers); + ASSERT_EQ(*a->reply_count, a->n_servers); /* Calling the helper against a non-existent directory must fail. */ _cleanup_free_ char *nope = NULL; - ASSERT_OK(asprintf(&nope, "%s/does-not-exist", tmpdir)); + ASSERT_OK(asprintf(&nope, "%s/does-not-exist", a->tmpdir)); ASSERT_FAIL(varlink_execute_directory( nope, "io.test.ExecDirPing", @@ -1029,13 +1155,13 @@ TEST(execute_directory) { /* more= */ false, /* timeout_usec= */ USEC_INFINITY, execute_dir_reply, - &reply_count)); + a->reply_count)); /* An empty directory must simply return 0 and not invoke the reply callback. */ - _cleanup_free_ char *empty = ASSERT_PTR(path_join(tmpdir, "empty")); + _cleanup_free_ char *empty = ASSERT_PTR(path_join(a->tmpdir, "empty")); ASSERT_OK_ERRNO(mkdir(empty, 0755)); - size_t count_before = reply_count; + size_t count_before = *a->reply_count; ASSERT_OK_ZERO(varlink_execute_directory( empty, "io.test.ExecDirPing", @@ -1043,8 +1169,52 @@ TEST(execute_directory) { /* more= */ false, /* timeout_usec= */ USEC_INFINITY, execute_dir_reply, - &reply_count)); - ASSERT_EQ(reply_count, count_before); + a->reply_count)); + ASSERT_EQ(*a->reply_count, count_before); + + ASSERT_OK(sd_event_exit(sd_fiber_get_event(), EXIT_SUCCESS)); + return 0; +} + +TEST(execute_directory) { + _cleanup_(rm_rf_physical_and_freep) char *tmpdir = NULL; + _cleanup_(sd_event_unrefp) sd_event *e = NULL; + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + static const char * const names[] = { "alpha", "beta", "gamma" }; + ExecDirServer servers[ELEMENTSOF(names)] = {}; + size_t reply_count = 0; + + ASSERT_OK(mkdtemp_malloc("/tmp/varlink-execdir-XXXXXX", &tmpdir)); + + ASSERT_OK(sd_event_new(&e)); + + for (size_t i = 0; i < ELEMENTSOF(names); i++) { + ExecDirServer *eds = servers + i; + servers[i].name = names[i]; + + _cleanup_free_ char *j = ASSERT_PTR(path_join(tmpdir, names[i])); + + ASSERT_OK(varlink_server_new(&eds->server, + SD_VARLINK_SERVER_INHERIT_USERDATA, + eds)); + ASSERT_OK(sd_varlink_server_bind_method(eds->server, "io.test.ExecDirPing", method_execute_dir_ping)); + ASSERT_OK(sd_varlink_server_listen_address(eds->server, j, 0600)); + ASSERT_OK(sd_varlink_server_attach_event(eds->server, e, 0)); + } + + ExecDirClientArgs args = { + .tmpdir = tmpdir, + .n_servers = ELEMENTSOF(names), + .reply_count = &reply_count, + }; + ASSERT_OK(sd_fiber_new(e, "execute-dir-client", execute_dir_client_fiber, &args, NULL, &f)); + + ASSERT_OK(sd_event_loop(e)); + + ASSERT_OK(sd_future_result(f)); + + FOREACH_ELEMENT(eds, servers) + eds->server = sd_varlink_server_unref(eds->server); } #define CTRUNC_N_FDS 64U @@ -1075,7 +1245,7 @@ static int reply_ctrunc(sd_varlink *link, sd_json_variant *parameters, const cha TEST(ctrunc) { int r; - + _cleanup_(sd_event_unrefp) sd_event *e = NULL; ASSERT_OK(sd_event_default(&e)); diff --git a/src/libsystemd/sd-varlink/varlink-internal.h b/src/libsystemd/sd-varlink/varlink-internal.h index 32d6d5983a..beec5be42c 100644 --- a/src/libsystemd/sd-varlink/varlink-internal.h +++ b/src/libsystemd/sd-varlink/varlink-internal.h @@ -135,7 +135,8 @@ typedef struct sd_varlink_server { LIST_HEAD(VarlinkServerSocket, sockets); - Hashmap *methods; /* Fully qualified symbol name of a method → VarlinkMethod */ + Hashmap *methods; /* Fully qualified symbol name of a method → sd_varlink_method_t */ + Hashmap *fiber_methods; /* Fully qualified symbol name of a fiber method → sd_varlink_method_t */ Hashmap *interfaces; /* Fully qualified interface name → VarlinkInterface* */ Hashmap *symbols; /* Fully qualified symbol name of method/error → VarlinkSymbol* */ sd_varlink_connect_t connect_callback; diff --git a/src/libsystemd/sd-varlink/varlink-util.h b/src/libsystemd/sd-varlink/varlink-util.h index d6ecb03c54..d5765ca2c7 100644 --- a/src/libsystemd/sd-varlink/varlink-util.h +++ b/src/libsystemd/sd-varlink/varlink-util.h @@ -19,6 +19,10 @@ int varlink_many_notifyb(Set *s, ...); int varlink_many_reply(Set *s, sd_json_variant *parameters); int varlink_many_error(Set *s, const char *error_id, sd_json_variant *parameters); +int varlink_server_bind_fiber(sd_varlink_server *s, const char *method, sd_varlink_method_t callback); +int varlink_server_bind_fiber_many_internal(sd_varlink_server *s, ...); +#define varlink_server_bind_fiber_many(s, ...) varlink_server_bind_fiber_many_internal(s, __VA_ARGS__, NULL) + int varlink_set_info_systemd(sd_varlink_server *server); int varlink_server_new( From 37679d79a8f4f18b5796f5904bdc47da4c9f29ad Mon Sep 17 00:00:00 2001 From: Daan De Meyer Date: Fri, 24 Apr 2026 09:49:02 +0000 Subject: [PATCH 7/8] qmp-client: add fiber-aware call paths The synchronous qmp_client_call() pumps the event loop until its reply arrives, pinning the parsed reply on c->current so it can hand out borrowed pointers to the caller. That model only fits one in-flight sync call: a second qmp_client_call() on the same client clears c->current before issuing its own send, invalidating the first caller's borrowed pointers. On a single-threaded event loop that was fine, but with fibers two concurrent calls on the same client can interleave through the pump (json_stream_wait() suspends the running fiber) and trample each other. To fix this, make qmp_client_call() detect when it's running on a fiber whose event loop matches the client and transparently delegate to qmp_client_call_suspend(), which makes use of a new QmpFuture to allow multiple concurrent calls to qmp_client_call(). To make this work concurrently, we also change qmp_client_call() to hand out references and copies of errors so that we don't have to store the borrowed pointers we hand out in the QmpClient struct. --- src/shared/qmp-client.c | 233 +++++++++++++++++++++++++++++++++---- src/shared/qmp-client.h | 16 ++- src/test/test-qmp-client.c | 4 +- 3 files changed, 224 insertions(+), 29 deletions(-) diff --git a/src/shared/qmp-client.c b/src/shared/qmp-client.c index 41b0c6dd57..8de903de1a 100644 --- a/src/shared/qmp-client.c +++ b/src/shared/qmp-client.c @@ -1,6 +1,7 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ #include "sd-event.h" +#include "sd-future.h" #include "sd-json.h" #include "alloc-util.h" @@ -226,19 +227,23 @@ static int qmp_extract_response_id(sd_json_variant *v, uint64_t *ret) { return 1; } -/* Returns 0 on success (ret_result = "return" value), -EIO on QMP error (reterr_desc set). */ -static int qmp_parse_response(sd_json_variant *v, sd_json_variant **ret_result, const char **reterr_desc) { +/* Returns 0 on success (ret_result = freshly reffed "return" value), -EIO on QMP error + * (ret_error_desc set to a freshly allocated string). Caller owns both outputs. */ +static int qmp_parse_response(sd_json_variant *v, sd_json_variant **ret_result, char **reterr_error_desc) { const char *desc; desc = qmp_extract_error_description(v); if (desc) { - if (reterr_desc) - *reterr_desc = desc; + if (reterr_error_desc) { + *reterr_error_desc = strdup(desc); + if (!*reterr_error_desc) + return -ENOMEM; + } return -EIO; } if (ret_result) - *ret_result = sd_json_variant_by_key(v, "return"); + *ret_result = sd_json_variant_ref(sd_json_variant_by_key(v, "return")); return 0; } @@ -273,8 +278,8 @@ static int qmp_client_build_command( /* Route c->current to event callback or matching async slot. Returns 1 on dispatch. */ static int qmp_client_dispatch(QmpClient *c) { - sd_json_variant *result = NULL; - const char *desc = NULL; + _cleanup_(sd_json_variant_unrefp) sd_json_variant *result = NULL; + _cleanup_free_ char *desc = NULL; uint64_t id; int error, r; @@ -318,8 +323,8 @@ static int qmp_client_dispatch(QmpClient *c) { } /* Synchronous slot (no callback): leave c->current pinned so qmp_client_call() can - * pick up the reply and hand out borrowed pointers into it. The sync caller owns a - * ref on the slot and detects completion by observing slot->client turning NULL. */ + * pick the reply up after its pump loop. The sync caller owns a ref on the slot and + * detects completion by observing slot->client turning NULL. */ if (!slot->callback) { qmp_slot_disconnect(slot, /* unref= */ true); return 1; @@ -574,6 +579,10 @@ static void qmp_client_clear(QmpClient *c) { qmp_client_detach_event(c); qmp_client_clear_current(c); json_stream_done(&c->stream); + /* qmp_client_handle_disconnect() above drained every entry via qmp_client_fail_pending(); + * the set is borrow-only for non-floating slots, so set_free() can't safely run a + * destructor over leftovers — enforce the drain invariant instead. */ + assert(set_isempty(c->slots)); c->slots = set_free(c->slots); } @@ -745,7 +754,7 @@ DEFINE_TRIVIAL_CLEANUP_FUNC(QmpClientArgs*, qmp_client_args_close_fds); /* Shared send path for qmp_client_invoke() and qmp_client_call(). A NULL callback registers * a "synchronous" slot: dispatch_reply leaves c->current pinned on match instead of invoking - * a callback, so qmp_client_call() can hand out borrowed pointers into the reply. If ret_slot + * a callback, so qmp_client_call() can pick the reply up after its pump loop. If ret_slot * is NULL the slot is allocated as floating (owned by c->slots); otherwise a reference is * handed back to the caller. */ static int qmp_client_send( @@ -810,21 +819,193 @@ int qmp_client_invoke( return qmp_client_send(c, command, args, callback, userdata, ret_slot); } +typedef struct QmpFuture { + QmpSlot *slot; /* owned, non-floating; NULL once disconnected */ + sd_json_variant *result; + char *error_desc; +} QmpFuture; + +static void* qmp_future_alloc(void) { + return new0(QmpFuture, 1); +} + +static void qmp_future_free(sd_future *f) { + QmpFuture *qf = ASSERT_PTR(sd_future_get_private(ASSERT_PTR(f))); + qmp_slot_unref(qf->slot); + sd_json_variant_unref(qf->result); + free(qf->error_desc); + free(qf); +} + +static int qmp_future_cancel(sd_future *f) { + QmpFuture *qf = ASSERT_PTR(sd_future_get_private(ASSERT_PTR(f))); + + /* Drop the pending slot so dispatch_reply won't try to fire our callback (and touch + * freed memory) when the reply eventually arrives. */ + qf->slot = qmp_slot_unref(qf->slot); + return sd_future_resolve(f, -ECANCELED); +} + +static const sd_future_ops qmp_call_future_ops = { + .size = sizeof(sd_future_ops), + .alloc = qmp_future_alloc, + .free = qmp_future_free, + .cancel = qmp_future_cancel, +}; + +static int qmp_future_callback( + QmpClient *c, + sd_json_variant *result, + const char *desc, + int error, + void *userdata) { + + sd_future *f = ASSERT_PTR(userdata); + QmpFuture *qf = ASSERT_PTR(sd_future_get_private(f)); + int r; + + assert(result || desc || error); + + if (result) + qf->result = sd_json_variant_ref(result); + if (desc) { + qf->error_desc = strdup(desc); + if (!qf->error_desc) + /* No usable reply payload to surface — propagate as transport-style + * failure so suspend() / sd_future_result() see the OOM. */ + return sd_future_resolve(f, -ENOMEM); + } + + /* Resolve with 0 on a success reply and -EIO on a QMP-level error (matching the synchronous + * path's errno for the desc-without-ret_error_desc case), so a caller awaiting the future + * learns about call failures from the resolution value alone. The reply payload itself + * (result or error_desc) is always stashed on the QmpFuture so future_get_qmp_reply() can + * hand the description string back on top of the bare -EIO. With no reply at all (transport + * failure, disconnect), resolve with the propagated transport errno; cancellation surfaces + * as -ECANCELED via qmp_future_cancel(). */ + if (result) + r = 0; + else if (desc) + r = -EIO; + else + r = error; + + return sd_future_resolve(f, r); +} + +int qmp_client_call_future( + QmpClient *c, + const char *command, + QmpClientArgs *args, + sd_future **ret) { + + int r; + + assert(c); + assert(command); + assert(ret); + + _cleanup_(sd_future_unrefp) sd_future *f = NULL; + r = sd_future_new(&qmp_call_future_ops, &f); + if (r < 0) + return r; + + QmpFuture *qf = sd_future_get_private(f); + + r = qmp_client_send(c, command, args, qmp_future_callback, f, &qf->slot); + if (r < 0) + return r; + + *ret = TAKE_PTR(f); + return 0; +} + +/* Extract the reply from a resolved qmp_client_call_future(). Returns 1 on success (with + * *ret_result a fresh reference the caller unrefs), -EIO on a QMP-level error (with the detail + * description copied into *ret_error_desc when the caller passed one to receive it), and the + * future's negative resume errno when no reply landed at all (transport failure / cancellation). + */ +int future_get_qmp_reply(sd_future *f, sd_json_variant **ret_result, char **reterr_error_desc) { + assert(f); + assert(sd_future_get_ops(f) == &qmp_call_future_ops); + assert(sd_future_state(f) == SD_FUTURE_RESOLVED); + + QmpFuture *qf = ASSERT_PTR(sd_future_get_private(f)); + + /* No reply at all: transport failure or cancellation — surface the future result. */ + if (!qf->result && !qf->error_desc) + return sd_future_result(f); + + if (qf->error_desc) { + if (reterr_error_desc) { + char *desc = strdup(qf->error_desc); + if (!desc) + return -ENOMEM; + *reterr_error_desc = desc; + } + return -EIO; + } + + if (reterr_error_desc) + *reterr_error_desc = NULL; + if (ret_result) + *ret_result = sd_json_variant_ref(qf->result); + + return 1; +} + +static int qmp_client_call_suspend( + QmpClient *c, + const char *command, + QmpClientArgs *args, + sd_json_variant **ret_result, + char **ret_error_desc) { + + int r; + + assert(c); + assert(command); + assert(sd_fiber_is_running()); + + _cleanup_(sd_future_cancel_wait_unrefp) sd_future *call = NULL; + r = qmp_client_call_future(c, command, args, &call); + if (r < 0) + return r; + + r = sd_fiber_suspend(); + + /* If the future isn't resolved, the suspend was interrupted before a reply arrived (fiber + * cancelled, fiber-wide SD_FIBER_TIMEOUT scope expired, …). There's no reply to extract, + * so surface the resume error directly. When the future is resolved, future_get_qmp_reply() + * already encodes success (1), QMP-level error (-EIO with the desc captured if asked for), + * and no-reply (negative future result) — pass it through. */ + if (sd_future_state(call) != SD_FUTURE_RESOLVED) + return r; + + return future_get_qmp_reply(call, ret_result, ret_error_desc); +} + int qmp_client_call( QmpClient *c, const char *command, QmpClientArgs *args, sd_json_variant **ret_result, - const char **ret_error_desc) { + char **reterr_error_desc) { - _cleanup_(qmp_slot_unrefp) QmpSlot *slot = NULL; + _cleanup_(sd_json_variant_unrefp) sd_json_variant *result = NULL; + _cleanup_free_ char *desc = NULL; int r; assert_return(c, -EINVAL); assert_return(command, -EINVAL); - /* Drop any reply pinned by a previous qmp_client_call() before we pin a new one. */ - qmp_client_clear_current(c); + /* If we're on a fiber sharing the QMP client's event loop, use the async + suspend path so + * multiple concurrent qmp_client_call() invocations across fibers don't deadlock each other + * on the process+wait pump. */ + if (sd_fiber_is_running() && qmp_client_get_event(c) == sd_fiber_get_event()) + return qmp_client_call_suspend(c, command, args, ret_result, reterr_error_desc); + + _cleanup_(qmp_slot_unrefp) QmpSlot *slot = NULL; /* NULL callback marks this as a synchronous slot: dispatch_reply matches on id like * any other slot (so stray unknown-id replies still get logged and dropped), but @@ -855,18 +1036,24 @@ int qmp_client_call( return r; } - sd_json_variant *result = NULL; - const char *desc = NULL; - int error = qmp_parse_response(c->current, &result, &desc); + _cleanup_(sd_json_variant_unrefp) sd_json_variant *current = TAKE_PTR(c->current); + r = qmp_parse_response(current, &result, &desc); + if (r < 0 && r != -EIO) + return r; - /* If caller doesn't ask for the error string, surface the error as the return code. */ - if (!ret_error_desc && error < 0) - return error; + /* QMP-level error: copy the description into *ret_error_desc when the caller asked for it, + * and surface the failure via the return value (matching qmp_client_call_suspend() / + * sd_bus_call()'s "negative on error" convention). */ + if (desc) { + if (reterr_error_desc) + *reterr_error_desc = TAKE_PTR(desc); + return -EIO; + } if (ret_result) - *ret_result = result; - if (ret_error_desc) - *ret_error_desc = desc; + *ret_result = TAKE_PTR(result); + if (reterr_error_desc) + *reterr_error_desc = NULL; return 1; } diff --git a/src/shared/qmp-client.h b/src/shared/qmp-client.h index 7dcd53355d..0bf435384c 100644 --- a/src/shared/qmp-client.h +++ b/src/shared/qmp-client.h @@ -68,15 +68,23 @@ int qmp_client_invoke( qmp_command_callback_t callback, void *userdata); -/* Synchronous send + receive. Pumps the event loop until the reply arrives. *ret_result and - * *ret_error_desc are borrowed pointers into the last reply, valid until the next - * qmp_client_call(). Same contract as sd_varlink_call(). */ int qmp_client_call( QmpClient *client, const char *command, QmpClientArgs *args, sd_json_variant **ret_result, - const char **ret_error_desc); + char **reterr_error_desc); + +int qmp_client_call_future( + QmpClient *client, + const char *command, + QmpClientArgs *args, + sd_future **ret); + +int future_get_qmp_reply( + sd_future *f, + sd_json_variant **ret_result, + char **reterr_error_desc); void qmp_client_bind_event(QmpClient *c, qmp_event_callback_t callback, void *userdata); void qmp_client_bind_disconnect(QmpClient *c, qmp_disconnect_callback_t callback, void *userdata); diff --git a/src/test/test-qmp-client.c b/src/test/test-qmp-client.c index befee02484..e5e7ed3b73 100644 --- a/src/test/test-qmp-client.c +++ b/src/test/test-qmp-client.c @@ -646,7 +646,7 @@ TEST(qmp_client_call) { /* Successful call: borrowed result pointer is valid until the next call. */ sd_json_variant *result = NULL; - const char *error_desc = NULL; + _cleanup_free_ char *error_desc = NULL; ASSERT_EQ(qmp_client_call(client, "query-status", NULL, &result, &error_desc), 1); ASSERT_NULL(error_desc); ASSERT_NOT_NULL(result); @@ -658,7 +658,7 @@ TEST(qmp_client_call) { /* QMP error with ret_error_desc provided: returns 1, result NULL, desc set. */ result = (sd_json_variant*) 0x1; /* poison to catch lack-of-write */ - error_desc = NULL; + free(error_desc); ASSERT_EQ(qmp_client_call(client, "stop", NULL, &result, &error_desc), 1); ASSERT_NULL(result); ASSERT_STREQ(error_desc, "not running"); From 742733440fc5e5f30624e3aee27c76d311809261 Mon Sep 17 00:00:00 2001 From: Daan De Meyer Date: Fri, 24 Apr 2026 09:49:10 +0000 Subject: [PATCH 8/8] test-qmp-client: run mock QMP servers as fibers on the shared event loop The mock servers used to be driven out-of-band: each test created a socketpair, forked a child, ran a hand-coded request/response script against the raw fd, and sent SIGTERM to tear it down. That worked but required pidref/process-util/signal plumbing in every test, two distinct execution contexts that couldn't share state, and a JsonStream attached to the mock side that pretended to be event-loop-driven while actually being driven manually via blocking reads. Now that JsonStream suspends when on a fiber, the mocks can live inside the same process and event loop as the client. Each mock is rewritten as an sd-fiber that runs alongside the client fiber: so the mock fiber yields on I/O and the event loop schedules the client in the meantime. Both sides progress cooperatively, no fork/SIGTERM/PID tracking, no manual phase tracking. Two cleanups fall out of the rewrite: - A QMP_TEST(name, mock_fn) { ... } macro encapsulates the per-test scaffolding (event loop, socketpair, mock fiber spawn, exit-on-idle shim) and injects an already-connected QmpClient *client into the test body. Each test now reads as a flat sequence of qmp_client_call() invocations against that client. - Repeated mock command/reply scripting is factored into mock_qmp_expect(), mock_qmp_reply(), mock_qmp_expect_and_reply(), mock_qmp_handshake(), and mock_qmp_query_status_running(). The greeting JSON is built with sd_json_buildo() instead of being parsed from a literal. The file shrinks from 756 to 494 lines, mostly through deletions. --- src/test/test-qmp-client.c | 949 ++++++++++++++----------------------- 1 file changed, 348 insertions(+), 601 deletions(-) diff --git a/src/test/test-qmp-client.c b/src/test/test-qmp-client.c index e5e7ed3b73..948122720e 100644 --- a/src/test/test-qmp-client.c +++ b/src/test/test-qmp-client.c @@ -1,33 +1,27 @@ /* SPDX-License-Identifier: LGPL-2.1-or-later */ -#include -#include +#include #include #include "sd-event.h" +#include "sd-future.h" #include "sd-json.h" #include "errno-util.h" #include "fd-util.h" #include "json-stream.h" -#include "pidref.h" -#include "process-util.h" #include "qmp-client.h" #include "string-util.h" #include "tests.h" -/* Mock QMP server: runs in the child process of a fork, communicates via one end of a socketpair. - * Uses JsonStream as the transport so framing (CRLF delimiter, message queuing, SCM_RIGHTS) is - * handled the same way as on the client side — individual recv() syscalls may coalesce multiple - * messages, and the parser must re-emit each one on its own. */ +/* Mock QMP server runs as an sd-fiber alongside the client on the same event loop. Its + * JsonStream uses the suspending json_stream_wait()/json_stream_flush() helpers, so the mock + * fiber yields whenever it's blocked on I/O and the client makes progress in the meantime. */ -/* We drive the stream manually via read/parse/wait; always report READING so json_stream_wait() - * asks for POLLIN. */ static JsonStreamPhase mock_qmp_phase(void *userdata) { return JSON_STREAM_PHASE_READING; } -/* Never reached — we don't wire the mock stream up to sd-event — but required at init. */ static int mock_qmp_dispatch(void *userdata) { return 0; } @@ -43,9 +37,6 @@ static void mock_qmp_init(JsonStream *s, int fd) { ASSERT_OK(json_stream_connect_fd_pair(s, fd, fd)); } -/* Read one complete JSON message, blocking until available. Handles the case where multiple - * client messages arrived coalesced into a single recv(): the parser walks the input buffer - * one CRLF-delimited message at a time. */ static void mock_qmp_recv(JsonStream *s, sd_json_variant **ret) { int r; @@ -62,144 +53,139 @@ static void mock_qmp_recv(JsonStream *s, sd_json_variant **ret) { } } -/* Enqueue one JSON variant and block until it has been fully written. */ static void mock_qmp_send(JsonStream *s, sd_json_variant *v) { ASSERT_OK(json_stream_enqueue(s, v)); ASSERT_OK(json_stream_flush(s)); } -/* Parse a literal JSON string and send it. Used for fixed greetings and unsolicited events. */ -static void mock_qmp_send_literal(JsonStream *s, const char *msg) { +static void mock_qmp_send_greeting(JsonStream *s) { _cleanup_(sd_json_variant_unrefp) sd_json_variant *v = NULL; - ASSERT_OK(sd_json_parse(msg, 0, &v, NULL, NULL)); + ASSERT_OK(sd_json_buildo(&v, + SD_JSON_BUILD_PAIR("QMP", SD_JSON_BUILD_OBJECT( + SD_JSON_BUILD_PAIR("version", SD_JSON_BUILD_OBJECT( + SD_JSON_BUILD_PAIR("qemu", SD_JSON_BUILD_OBJECT( + SD_JSON_BUILD_PAIR_UNSIGNED("micro", 0), + SD_JSON_BUILD_PAIR_UNSIGNED("minor", 2), + SD_JSON_BUILD_PAIR_UNSIGNED("major", 9))))), + SD_JSON_BUILD_PAIR("capabilities", SD_JSON_BUILD_STRV(STRV_MAKE("oob"))))))); mock_qmp_send(s, v); } -/* Read a command from the client, verify it contains the expected command name, and send a - * reply carrying the same id. If reply_data is NULL, an empty return object is sent. */ +/* Receive one command, assert it matches `expected_command`, return its id (borrowed from *cmd). */ +static sd_json_variant* mock_qmp_expect(JsonStream *s, const char *expected_command, sd_json_variant **cmd) { + mock_qmp_recv(s, cmd); + sd_json_variant *execute = ASSERT_NOT_NULL(sd_json_variant_by_key(*cmd, "execute")); + ASSERT_STREQ(sd_json_variant_string(execute), expected_command); + return ASSERT_NOT_NULL(sd_json_variant_by_key(*cmd, "id")); +} + +/* Send a reply for a previously-received command id. Passing NULL reply_data sends {}. */ +static void mock_qmp_reply(JsonStream *s, sd_json_variant *id, sd_json_variant *reply_data) { + _cleanup_(sd_json_variant_unrefp) sd_json_variant *empty = NULL, *response = NULL; + + if (!reply_data) { + ASSERT_OK(sd_json_build(&empty, SD_JSON_BUILD_EMPTY_OBJECT)); + reply_data = empty; + } + + ASSERT_OK(sd_json_buildo(&response, + SD_JSON_BUILD_PAIR("return", SD_JSON_BUILD_VARIANT(reply_data)), + SD_JSON_BUILD_PAIR("id", SD_JSON_BUILD_VARIANT(id)))); + + mock_qmp_send(s, response); +} + static void mock_qmp_expect_and_reply(JsonStream *s, const char *expected_command, sd_json_variant *reply_data) { - _cleanup_(sd_json_variant_unrefp) sd_json_variant *cmd = NULL, *reply_obj = NULL, *response = NULL; - - mock_qmp_recv(s, &cmd); - - sd_json_variant *execute = ASSERT_NOT_NULL(sd_json_variant_by_key(cmd, "execute")); - ASSERT_STREQ(sd_json_variant_string(execute), expected_command); - - sd_json_variant *id = ASSERT_NOT_NULL(sd_json_variant_by_key(cmd, "id")); - - if (!reply_data) - ASSERT_OK(sd_json_variant_new_object(&reply_obj, NULL, 0)); - - ASSERT_OK(sd_json_buildo( - &response, - SD_JSON_BUILD_PAIR("return", SD_JSON_BUILD_VARIANT(reply_data ?: reply_obj)), - SD_JSON_BUILD_PAIR("id", SD_JSON_BUILD_VARIANT(id)))); - - mock_qmp_send(s, response); + _cleanup_(sd_json_variant_unrefp) sd_json_variant *cmd = NULL; + mock_qmp_reply(s, mock_qmp_expect(s, expected_command, &cmd), reply_data); } -/* Same shape as mock_qmp_expect_and_reply() but replies with a QMP error object. */ static void mock_qmp_expect_and_reply_error(JsonStream *s, const char *expected_command, const char *error_desc) { - _cleanup_(sd_json_variant_unrefp) sd_json_variant *cmd = NULL, *error_obj = NULL, *response = NULL; + _cleanup_(sd_json_variant_unrefp) sd_json_variant *cmd = NULL, *response = NULL; + sd_json_variant *id = mock_qmp_expect(s, expected_command, &cmd); - mock_qmp_recv(s, &cmd); - - sd_json_variant *execute = ASSERT_NOT_NULL(sd_json_variant_by_key(cmd, "execute")); - ASSERT_STREQ(sd_json_variant_string(execute), expected_command); - - sd_json_variant *id = ASSERT_NOT_NULL(sd_json_variant_by_key(cmd, "id")); - - ASSERT_OK(sd_json_buildo( - &error_obj, + ASSERT_OK(sd_json_buildo(&response, + SD_JSON_BUILD_PAIR("error", SD_JSON_BUILD_OBJECT( SD_JSON_BUILD_PAIR_STRING("class", "GenericError"), - SD_JSON_BUILD_PAIR_STRING("desc", error_desc))); - - ASSERT_OK(sd_json_buildo( - &response, - SD_JSON_BUILD_PAIR("error", SD_JSON_BUILD_VARIANT(error_obj)), - SD_JSON_BUILD_PAIR("id", SD_JSON_BUILD_VARIANT(id)))); + SD_JSON_BUILD_PAIR_STRING("desc", error_desc))), + SD_JSON_BUILD_PAIR("id", SD_JSON_BUILD_VARIANT(id)))); mock_qmp_send(s, response); } -static _noreturn_ void mock_qmp_server(int fd) { +static void mock_qmp_handshake(JsonStream *s) { + mock_qmp_send_greeting(s); + mock_qmp_expect_and_reply(s, "qmp_capabilities", NULL); +} + +/* Reply to query-status with a running=true/status="running" payload. */ +static void mock_qmp_query_status_running(JsonStream *s) { + _cleanup_(sd_json_variant_unrefp) sd_json_variant *v = NULL; + + ASSERT_OK(sd_json_buildo(&v, + SD_JSON_BUILD_PAIR_BOOLEAN("running", true), + SD_JSON_BUILD_PAIR_STRING("status", "running"))); + mock_qmp_expect_and_reply(s, "query-status", v); +} + +/* Drive a mock+client pair on a single event loop. The client fiber runs as userdata=client, + * the mock fiber as userdata=fd (the server-side socket). */ +static void run_qmp_test(sd_fiber_func_t mock_fn, sd_fiber_func_t client_fn) { + _cleanup_(sd_event_unrefp) sd_event *event = NULL; + _cleanup_(sd_future_unrefp) sd_future *client_f = NULL, *mock_f = NULL; + _cleanup_(qmp_client_unrefp) QmpClient *client = NULL; + _cleanup_close_pair_ int qmp_fds[2] = EBADF_PAIR; + + ASSERT_OK(sd_event_new(&event)); + ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0, qmp_fds)); + + ASSERT_OK(qmp_client_connect_fd(&client, TAKE_FD(qmp_fds[0]))); + ASSERT_OK(qmp_client_attach_event(client, event, SD_EVENT_PRIORITY_NORMAL)); + + ASSERT_OK(sd_fiber_new(event, "mock", mock_fn, FD_TO_PTR(TAKE_FD(qmp_fds[1])), NULL, &mock_f)); + ASSERT_OK(sd_fiber_new(event, "client", client_fn, client, NULL, &client_f)); + + ASSERT_OK(sd_event_loop(event)); + ASSERT_OK(sd_future_result(client_f)); + ASSERT_OK(sd_future_result(mock_f)); +} + +/* Define a test whose body runs as the client fiber on an event loop shared with `mock_fn`. + * The body receives `QmpClient *client` as its argument. */ +#define QMP_TEST(name, mock_fn) \ + static int test_##name##_body(QmpClient *client); \ + static int test_##name##_fiber(void *userdata) { \ + int r = test_##name##_body(userdata); \ + ASSERT_OK(sd_event_exit(sd_fiber_get_event(), 0)); \ + return r; \ + } \ + TEST(name) { \ + run_qmp_test(mock_fn, test_##name##_fiber); \ + } \ + static int test_##name##_body(QmpClient *client) + +static int mock_qmp_basic_fiber(void *userdata) { _cleanup_(json_stream_done) JsonStream s = {}; - _cleanup_(sd_json_variant_unrefp) sd_json_variant *status_return = NULL; + _cleanup_(sd_json_variant_unrefp) sd_json_variant *stop_event = NULL; - mock_qmp_init(&s, fd); + mock_qmp_init(&s, PTR_TO_FD(userdata)); + mock_qmp_handshake(&s); - /* Send QMP greeting */ - mock_qmp_send_literal(&s, - "{\"QMP\": {\"version\": {\"qemu\": {\"micro\": 0, \"minor\": 2, \"major\": 9}}, \"capabilities\": [\"oob\"]}}"); - - /* Accept qmp_capabilities */ - mock_qmp_expect_and_reply(&s, "qmp_capabilities", NULL); - - /* Accept query-status, reply with running state */ - ASSERT_OK(sd_json_buildo( - &status_return, - SD_JSON_BUILD_PAIR_BOOLEAN("running", true), - SD_JSON_BUILD_PAIR_STRING("status", "running"))); - mock_qmp_expect_and_reply(&s, "query-status", status_return); - - /* Accept stop */ + mock_qmp_query_status_running(&s); mock_qmp_expect_and_reply(&s, "stop", NULL); - /* Send a STOP event */ - mock_qmp_send_literal(&s, - "{\"event\": \"STOP\", \"timestamp\": {\"seconds\": 1234, \"microseconds\": 5678}}"); + ASSERT_OK(sd_json_buildo(&stop_event, + SD_JSON_BUILD_PAIR_STRING("event", "STOP"), + SD_JSON_BUILD_PAIR("timestamp", SD_JSON_BUILD_OBJECT( + SD_JSON_BUILD_PAIR_UNSIGNED("seconds", 1234), + SD_JSON_BUILD_PAIR_UNSIGNED("microseconds", 5678))))); + mock_qmp_send(&s, stop_event); - /* Accept cont */ mock_qmp_expect_and_reply(&s, "cont", NULL); - - /* json_stream_done() on cleanup closes our fd and signals EOF. */ - _exit(EXIT_SUCCESS); -} - -/* Test helper: tracks an async QMP command result and signals completion. */ -typedef struct { - sd_json_variant *result; - char *error_desc; - int error; - bool done; -} QmpTestResult; - -static int on_test_result( - QmpClient *client, - sd_json_variant *result, - const char *error_desc, - int error, - void *userdata) { - - QmpTestResult *t = ASSERT_PTR(userdata); - - t->error = error; - if (result) - t->result = sd_json_variant_ref(result); - if (error_desc) - t->error_desc = strdup(error_desc); - t->done = true; return 0; } -/* Run the event loop until the test result callback fires. */ -static void qmp_test_wait(sd_event *event, QmpTestResult *t) { - assert(event); - assert(t); - - while (!t->done) - ASSERT_OK(sd_event_run(event, UINT64_MAX)); -} - -static void qmp_test_result_done(QmpTestResult *t) { - assert(t); - - sd_json_variant_unref(t->result); - free(t->error_desc); - *t = (QmpTestResult) {}; -} - static int test_event_callback( QmpClient *client, const char *event, @@ -208,516 +194,283 @@ static int test_event_callback( bool *event_received = ASSERT_PTR(userdata); - /* We may also receive a synthetic SHUTDOWN event when the mock server closes the connection; - * only validate the STOP event we actually care about. */ + /* Ignore the synthetic SHUTDOWN emitted when the mock closes the connection. */ if (streq(event, "STOP")) *event_received = true; return 0; } -TEST(qmp_client_basic) { - _cleanup_(qmp_client_unrefp) QmpClient *client = NULL; - _cleanup_(sd_event_unrefp) sd_event *event = NULL; - _cleanup_(pidref_done) PidRef pid = PIDREF_NULL; - QmpTestResult t = {}; - sd_json_variant *running, *status; - int qmp_fds[2]; - int r; - - ASSERT_OK(sd_event_new(&event)); - - ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0, qmp_fds)); - - r = ASSERT_OK(pidref_safe_fork("(mock-qmp)", FORK_DEATHSIG_SIGKILL|FORK_LOG, &pid)); - - if (r == 0) { - safe_close(qmp_fds[0]); - mock_qmp_server(qmp_fds[1]); - } - - safe_close(qmp_fds[1]); - - /* Connect then attach to event loop — handshake completes transparently - * inside the first call()/invoke(). */ - ASSERT_OK(qmp_client_connect_fd(&client, qmp_fds[0])); - ASSERT_OK(qmp_client_attach_event(client, event, SD_EVENT_PRIORITY_NORMAL)); - - /* Set event callback to catch STOP event during cont */ +QMP_TEST(qmp_client_basic, mock_qmp_basic_fiber) { + _cleanup_(sd_json_variant_unrefp) sd_json_variant *result = NULL; + _cleanup_free_ char *error_desc = NULL; bool event_received = false; + qmp_client_bind_event(client, test_event_callback, &event_received); - /* Execute query-status */ - ASSERT_OK(qmp_client_invoke(client, /* ret_slot= */ NULL, "query-status", NULL, on_test_result, &t)); - qmp_test_wait(event, &t); - ASSERT_EQ(t.error, 0); - ASSERT_NOT_NULL(t.result); - - running = ASSERT_NOT_NULL(sd_json_variant_by_key(t.result, "running")); - ASSERT_TRUE(sd_json_variant_boolean(running)); - - status = ASSERT_NOT_NULL(sd_json_variant_by_key(t.result, "status")); - ASSERT_STREQ(sd_json_variant_string(status), "running"); - - qmp_test_result_done(&t); - - /* Execute stop */ - ASSERT_OK(qmp_client_invoke(client, /* ret_slot= */ NULL, "stop", NULL, on_test_result, &t)); - qmp_test_wait(event, &t); - ASSERT_EQ(t.error, 0); - qmp_test_result_done(&t); - - /* Execute cont -- the STOP event should be dispatched by the IO callback */ - ASSERT_OK(qmp_client_invoke(client, /* ret_slot= */ NULL, "cont", NULL, on_test_result, &t)); - qmp_test_wait(event, &t); - ASSERT_EQ(t.error, 0); - qmp_test_result_done(&t); - - /* Verify the STOP event was received */ - ASSERT_TRUE(event_received); - - /* Wait for child and verify clean exit */ - siginfo_t si = {}; - ASSERT_OK(pidref_wait_for_terminate(&pid, &si)); - ASSERT_EQ(si.si_code, CLD_EXITED); - ASSERT_EQ(si.si_status, EXIT_SUCCESS); -} - -TEST(qmp_client_eof) { - _cleanup_(qmp_client_unrefp) QmpClient *client = NULL; - _cleanup_(sd_event_unrefp) sd_event *event = NULL; - _cleanup_(pidref_done) PidRef pid = PIDREF_NULL; - QmpTestResult t = {}; - int qmp_fds[2]; - int r; - - ASSERT_OK(sd_event_new(&event)); - ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0, qmp_fds)); - - r = ASSERT_OK(pidref_safe_fork("(mock-qmp-eof)", FORK_DEATHSIG_SIGKILL|FORK_LOG, &pid)); - - if (r == 0) { - _cleanup_(json_stream_done) JsonStream s = {}; - - safe_close(qmp_fds[0]); - mock_qmp_init(&s, qmp_fds[1]); - - /* Send greeting and accept capabilities, then die */ - mock_qmp_send_literal(&s, - "{\"QMP\": {\"version\": {\"qemu\": {\"micro\": 0, \"minor\": 0, \"major\": 9}}, \"capabilities\": []}}"); - - mock_qmp_expect_and_reply(&s, "qmp_capabilities", NULL); - - /* _exit() closes our fd via kernel teardown, signalling EOF to the peer. */ - _exit(EXIT_SUCCESS); - } - - safe_close(qmp_fds[1]); - - ASSERT_OK(qmp_client_connect_fd(&client, qmp_fds[0])); - ASSERT_OK(qmp_client_attach_event(client, event, SD_EVENT_PRIORITY_NORMAL)); - - /* Executing a command should fail with a disconnect error because the server - * closed. The handshake may succeed or fail inside invoke() — either way the - * invoke itself or the async callback should report a disconnect. */ - r = qmp_client_invoke(client, /* ret_slot= */ NULL, "query-status", NULL, on_test_result, &t); - if (r < 0) - ASSERT_TRUE(ERRNO_IS_NEG_DISCONNECT(r)); - else { - qmp_test_wait(event, &t); - ASSERT_TRUE(ERRNO_IS_NEG_DISCONNECT(t.error)); - qmp_test_result_done(&t); - } - - siginfo_t si = {}; - ASSERT_OK(pidref_wait_for_terminate(&pid, &si)); - ASSERT_EQ(si.si_code, CLD_EXITED); - ASSERT_EQ(si.si_status, EXIT_SUCCESS); -} - -/* Mock QMP server for the fd-passing test. Drives the wire dance: - * greeting → recv qmp_capabilities → reply → recv add-fd → reply - * Asserts that exactly one SCM_RIGHTS fd arrives total across the two recvs. We can't - * require the fd to come attached to add-fd specifically: AF_UNIX coalesces the client's - * non-SCM cap sendmsg forward into the SCM-bearing add-fd sendmsg, so the fd may surface - * with either recv depending on kernel scheduling. QEMU's FIFO fd queue doesn't care. */ -static _noreturn_ void mock_qmp_server_fd(int fd) { - _cleanup_(json_stream_done) JsonStream s = {}; - _cleanup_(sd_json_variant_unrefp) sd_json_variant *cap_cmd = NULL, - *addfd_cmd = NULL, - *cap_reply = NULL, - *addfd_return = NULL, - *addfd_reply = NULL; - - mock_qmp_init(&s, fd); - ASSERT_OK(json_stream_set_allow_fd_passing_input(&s, true, /* with_sockopt= */ true)); - - /* Greeting */ - mock_qmp_send_literal(&s, - "{\"QMP\": {\"version\": {\"qemu\": {\"micro\": 0, \"minor\": 0, \"major\": 9}}, \"capabilities\": []}}"); - - /* Receive qmp_capabilities (may or may not carry the fd depending on coalescing). */ - mock_qmp_recv(&s, &cap_cmd); - size_t n_fds_total = json_stream_get_n_input_fds(&s); - ASSERT_STREQ(sd_json_variant_string(sd_json_variant_by_key(cap_cmd, "execute")), "qmp_capabilities"); - json_stream_close_input_fds(&s); - - sd_json_variant *cap_id = ASSERT_NOT_NULL(sd_json_variant_by_key(cap_cmd, "id")); - ASSERT_OK(sd_json_buildo( - &cap_reply, - SD_JSON_BUILD_PAIR("return", SD_JSON_BUILD_EMPTY_OBJECT), - SD_JSON_BUILD_PAIR("id", SD_JSON_BUILD_VARIANT(cap_id)))); - mock_qmp_send(&s, cap_reply); - - /* Receive add-fd (fd may already have been consumed with cap's recv). */ - mock_qmp_recv(&s, &addfd_cmd); - n_fds_total += json_stream_get_n_input_fds(&s); - ASSERT_STREQ(sd_json_variant_string(sd_json_variant_by_key(addfd_cmd, "execute")), "add-fd"); - json_stream_close_input_fds(&s); - - ASSERT_EQ(n_fds_total, (size_t) 1); - - sd_json_variant *addfd_id = ASSERT_NOT_NULL(sd_json_variant_by_key(addfd_cmd, "id")); - ASSERT_OK(sd_json_buildo( - &addfd_return, - SD_JSON_BUILD_PAIR_UNSIGNED("fdset-id", 0), - SD_JSON_BUILD_PAIR_UNSIGNED("fd", 42))); - ASSERT_OK(sd_json_buildo( - &addfd_reply, - SD_JSON_BUILD_PAIR("return", SD_JSON_BUILD_VARIANT(addfd_return)), - SD_JSON_BUILD_PAIR("id", SD_JSON_BUILD_VARIANT(addfd_id)))); - mock_qmp_send(&s, addfd_reply); - - _exit(EXIT_SUCCESS); -} - -/* End-to-end fd-passing through qmp_client_invoke() with QMP_CLIENT_ARGS_FD(): open a real - * fd, send add-fd, confirm the mock received a single SCM_RIGHTS fd and replied successfully. */ -TEST(qmp_client_invoke_with_fd) { - _cleanup_(qmp_client_unrefp) QmpClient *client = NULL; - _cleanup_(sd_event_unrefp) sd_event *event = NULL; - _cleanup_(pidref_done) PidRef pid = PIDREF_NULL; - _cleanup_(sd_json_variant_unrefp) sd_json_variant *args = NULL; - _cleanup_close_ int fd_to_pass = -EBADF; - QmpTestResult t = {}; - int qmp_fds[2]; - int r; - - ASSERT_OK(sd_event_new(&event)); - ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0, qmp_fds)); - - r = ASSERT_OK(pidref_safe_fork("(mock-qmp-fd)", FORK_DEATHSIG_SIGKILL|FORK_LOG, &pid)); - - if (r == 0) { - safe_close(qmp_fds[0]); - mock_qmp_server_fd(qmp_fds[1]); - } - - safe_close(qmp_fds[1]); - - /* Open a real fd to pass — /dev/null is universally available. */ - fd_to_pass = open("/dev/null", O_RDWR|O_CLOEXEC); - ASSERT_OK(fd_to_pass); - - ASSERT_OK(qmp_client_connect_fd(&client, qmp_fds[0])); - ASSERT_OK(qmp_client_attach_event(client, event, SD_EVENT_PRIORITY_NORMAL)); - - ASSERT_OK(sd_json_buildo(&args, SD_JSON_BUILD_PAIR_UNSIGNED("fdset-id", 0))); - - ASSERT_OK(qmp_client_invoke(client, /* ret_slot= */ NULL, "add-fd", - QMP_CLIENT_ARGS_FD(args, TAKE_FD(fd_to_pass)), - on_test_result, &t)); - - qmp_test_wait(event, &t); - ASSERT_EQ(t.error, 0); - ASSERT_NOT_NULL(t.result); - qmp_test_result_done(&t); - - /* Wait for the mock. If its fd-count assertion tripped, si.si_status is non-zero. */ - siginfo_t si = {}; - ASSERT_OK(pidref_wait_for_terminate(&pid, &si)); - ASSERT_EQ(si.si_code, CLD_EXITED); - ASSERT_EQ(si.si_status, EXIT_SUCCESS); -} - -/* Regression: the caller-supplied fds — already TAKE_FD()'d through QMP_CLIENT_ARGS_FD() — - * must never leak, regardless of whether the invoke reaches the wire. Verified here via a - * dead peer: invoke enqueues (non-blocking), the queue item owns the fd, and client teardown - * must close it. */ -TEST(qmp_client_invoke_failure_closes_fds) { - _cleanup_(sd_json_variant_unrefp) sd_json_variant *args = NULL; - _cleanup_close_ int fd_to_pass = -EBADF; - QmpClient *client = NULL; - QmpTestResult t = {}; - int qmp_fds[2]; - int saved_fd_value; - - ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0, qmp_fds)); - - /* Close the peer end immediately so any write attempt sees EPIPE. */ - safe_close(qmp_fds[1]); - - fd_to_pass = open("/dev/null", O_RDWR|O_CLOEXEC); - ASSERT_OK(fd_to_pass); - saved_fd_value = fd_to_pass; /* remember the int value for the closed-check */ - - ASSERT_OK(sd_json_buildo(&args, SD_JSON_BUILD_PAIR_UNSIGNED("fdset-id", 0))); - ASSERT_OK(qmp_client_connect_fd(&client, qmp_fds[0])); - - /* invoke no longer blocks on the handshake — it just enqueues. The fd is now - * owned by the underlying JsonStream output queue. */ - ASSERT_OK(qmp_client_invoke(client, /* ret_slot= */ NULL, "add-fd", - QMP_CLIENT_ARGS_FD(args, TAKE_FD(fd_to_pass)), - on_test_result, &t)); - ASSERT_EQ(fd_to_pass, -EBADF); /* TAKE_FD cleared our local handle */ - - /* The fd is still open here (held in JsonStream's queue). */ - ASSERT_OK_ERRNO(fcntl(saved_fd_value, F_GETFD)); - - /* Client teardown (json_stream_done) must close queued output fds, otherwise the - * saved fd number would still be valid. */ - client = qmp_client_unref(client); - ASSERT_EQ(fcntl(saved_fd_value, F_GETFD), -1); - ASSERT_EQ(errno, EBADF); -} - -/* Mock for the slot lifecycle + cancel tests: greets, accepts capabilities, then accepts - * query-status and stop, replying with dummy returns. A cancelled query-status still gets - * sent on the wire (cancel merely removes the pending slot), so the server must be prepared - * to read and reply to it. */ -static _noreturn_ void mock_qmp_server_slot(int fd) { - _cleanup_(json_stream_done) JsonStream s = {}; - _cleanup_(sd_json_variant_unrefp) sd_json_variant *status_return = NULL; - - mock_qmp_init(&s, fd); - - mock_qmp_send_literal(&s, - "{\"QMP\": {\"version\": {\"qemu\": {\"micro\": 0, \"minor\": 0, \"major\": 9}}, \"capabilities\": []}}"); - - mock_qmp_expect_and_reply(&s, "qmp_capabilities", NULL); - - ASSERT_OK(sd_json_buildo( - &status_return, - SD_JSON_BUILD_PAIR_BOOLEAN("running", true), - SD_JSON_BUILD_PAIR_STRING("status", "running"))); - mock_qmp_expect_and_reply(&s, "query-status", status_return); - - mock_qmp_expect_and_reply(&s, "stop", NULL); - - _exit(EXIT_SUCCESS); -} - -/* Verify that when qmp_client_invoke() returns a slot, qmp_slot_get_client() tracks the - * connection state: the client pointer is reported while the call is in flight, and flipped - * back to NULL once the reply has been dispatched. The caller must still be able to drop its - * ref safely after that. */ -TEST(qmp_client_invoke_slot_lifecycle) { - _cleanup_(qmp_client_unrefp) QmpClient *client = NULL; - _cleanup_(sd_event_unrefp) sd_event *event = NULL; - _cleanup_(pidref_done_sigkill_wait) PidRef pid = PIDREF_NULL; - _cleanup_(qmp_slot_unrefp) QmpSlot *slot = NULL; - QmpTestResult t = {}; - int qmp_fds[2]; - int r; - - ASSERT_OK(sd_event_new(&event)); - ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0, qmp_fds)); - - r = ASSERT_OK(pidref_safe_fork("(mock-qmp-slot-life)", FORK_DEATHSIG_SIGKILL|FORK_LOG, &pid)); - if (r == 0) { - safe_close(qmp_fds[0]); - mock_qmp_server_slot(qmp_fds[1]); - } - safe_close(qmp_fds[1]); - - ASSERT_OK(qmp_client_connect_fd(&client, qmp_fds[0])); - ASSERT_OK(qmp_client_attach_event(client, event, SD_EVENT_PRIORITY_NORMAL)); - - ASSERT_OK(qmp_client_invoke(client, &slot, "query-status", NULL, on_test_result, &t)); - - /* While in flight the slot still references its client. */ - ASSERT_NOT_NULL(slot); - ASSERT_PTR_EQ(qmp_slot_get_client(slot), client); - - qmp_test_wait(event, &t); - ASSERT_EQ(t.error, 0); - ASSERT_NOT_NULL(t.result); - - /* Once dispatched, the slot is disconnected from the client but still owned by us. */ - ASSERT_NULL(qmp_slot_get_client(slot)); - - qmp_test_result_done(&t); - - /* Drop our ref explicitly (out of order w.r.t. cleanup) to exercise the - * already-disconnected path in qmp_slot_free(). */ - slot = qmp_slot_unref(slot); - ASSERT_NULL(slot); -} - -/* Verify that dropping the only reference on a pending slot before the reply arrives cancels - * the callback. The command is already enqueued on the stream at that point, so the server - * still sees it and replies — but the reply lands on an unknown id and is discarded. */ -TEST(qmp_client_invoke_slot_cancel) { - _cleanup_(qmp_client_unrefp) QmpClient *client = NULL; - _cleanup_(pidref_done_sigkill_wait) PidRef pid = PIDREF_NULL; - QmpTestResult t_cancelled = {}; - QmpSlot *slot = NULL; - int qmp_fds[2]; - int r; - - ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0, qmp_fds)); - - r = ASSERT_OK(pidref_safe_fork("(mock-qmp-slot-cancel)", FORK_DEATHSIG_SIGKILL|FORK_LOG, &pid)); - if (r == 0) { - safe_close(qmp_fds[0]); - mock_qmp_server_slot(qmp_fds[1]); - } - safe_close(qmp_fds[1]); - - /* Drive without an event loop so the subsequent qmp_client_call() owns all pumping; - * it serializes write→read round-trips, which avoids the mock server seeing the - * cancelled query-status and the follow-up stop concatenated into a single recv(). */ - ASSERT_OK(qmp_client_connect_fd(&client, qmp_fds[0])); - - ASSERT_OK(qmp_client_invoke(client, &slot, "query-status", NULL, on_test_result, &t_cancelled)); - ASSERT_NOT_NULL(slot); - - /* Drop our sole ref → slot disconnects itself from the client's pending set. The - * enqueued query-status is still on the wire; when its reply arrives, dispatch_reply - * won't find a matching slot and will log-and-discard it. */ - slot = qmp_slot_unref(slot); - ASSERT_NULL(slot); - - /* Synchronous call drives its own process+wait pump: it first drains the already- - * enqueued query-status write, consumes (and discards) its reply, then sends stop - * and waits for that reply. Any improper fire of the cancelled callback would have - * happened during that process() pass. */ - ASSERT_EQ(qmp_client_call(client, "stop", NULL, NULL, NULL), 1); - - /* The cancelled callback must never have fired. */ - ASSERT_FALSE(t_cancelled.done); - ASSERT_NULL(t_cancelled.result); - ASSERT_NULL(t_cancelled.error_desc); -} - -/* Drives a small wire dance for the sync call test: greeting, capabilities, one successful - * command reply, and two error replies (one for the ret_error_desc path, one for the -EIO - * path). */ -static _noreturn_ void mock_qmp_server_call(int fd) { - _cleanup_(json_stream_done) JsonStream s = {}; - _cleanup_(sd_json_variant_unrefp) sd_json_variant *status_return = NULL; - - mock_qmp_init(&s, fd); - - mock_qmp_send_literal(&s, - "{\"QMP\": {\"version\": {\"qemu\": {\"micro\": 0, \"minor\": 0, \"major\": 9}}, \"capabilities\": []}}"); - - mock_qmp_expect_and_reply(&s, "qmp_capabilities", NULL); - - ASSERT_OK(sd_json_buildo( - &status_return, - SD_JSON_BUILD_PAIR_BOOLEAN("running", true), - SD_JSON_BUILD_PAIR_STRING("status", "running"))); - mock_qmp_expect_and_reply(&s, "query-status", status_return); - - mock_qmp_expect_and_reply_error(&s, "stop", "not running"); - mock_qmp_expect_and_reply_error(&s, "stop", "still not running"); - - _exit(EXIT_SUCCESS); -} - -TEST(qmp_client_call) { - _cleanup_(qmp_client_unrefp) QmpClient *client = NULL; - _cleanup_(pidref_done_sigkill_wait) PidRef pid = PIDREF_NULL; - int qmp_fds[2]; - int r; - - ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0, qmp_fds)); - - r = ASSERT_OK(pidref_safe_fork("(mock-qmp-call)", FORK_DEATHSIG_SIGKILL|FORK_LOG, &pid)); - if (r == 0) { - safe_close(qmp_fds[0]); - mock_qmp_server_call(qmp_fds[1]); - } - safe_close(qmp_fds[1]); - - /* qmp_client_call() drives its own process()+wait() pump, so no event loop needed. */ - ASSERT_OK(qmp_client_connect_fd(&client, qmp_fds[0])); - - /* Successful call: borrowed result pointer is valid until the next call. */ - sd_json_variant *result = NULL; - _cleanup_free_ char *error_desc = NULL; - ASSERT_EQ(qmp_client_call(client, "query-status", NULL, &result, &error_desc), 1); + ASSERT_OK_POSITIVE(qmp_client_call(client, "query-status", NULL, &result, &error_desc)); ASSERT_NULL(error_desc); - ASSERT_NOT_NULL(result); sd_json_variant *running = ASSERT_NOT_NULL(sd_json_variant_by_key(result, "running")); ASSERT_TRUE(sd_json_variant_boolean(running)); sd_json_variant *status = ASSERT_NOT_NULL(sd_json_variant_by_key(result, "status")); ASSERT_STREQ(sd_json_variant_string(status), "running"); - /* QMP error with ret_error_desc provided: returns 1, result NULL, desc set. */ - result = (sd_json_variant*) 0x1; /* poison to catch lack-of-write */ - free(error_desc); - ASSERT_EQ(qmp_client_call(client, "stop", NULL, &result, &error_desc), 1); + ASSERT_OK_POSITIVE(qmp_client_call(client, "stop", NULL, NULL, NULL)); + ASSERT_OK_POSITIVE(qmp_client_call(client, "cont", NULL, NULL, NULL)); + + ASSERT_TRUE(event_received); + return 0; +} + +static int mock_qmp_eof_fiber(void *userdata) { + _cleanup_(json_stream_done) JsonStream s = {}; + + mock_qmp_init(&s, PTR_TO_FD(userdata)); + mock_qmp_handshake(&s); + /* Return; _cleanup_ closes the fd → client sees EOF. */ + return 0; +} + +QMP_TEST(qmp_client_eof, mock_qmp_eof_fiber) { + int r = qmp_client_call(client, "query-status", NULL, NULL, NULL); + ASSERT_TRUE(ERRNO_IS_NEG_DISCONNECT(r)); + return 0; +} + +static int mock_qmp_call_fiber(void *userdata) { + _cleanup_(json_stream_done) JsonStream s = {}; + + mock_qmp_init(&s, PTR_TO_FD(userdata)); + mock_qmp_handshake(&s); + + mock_qmp_query_status_running(&s); + mock_qmp_expect_and_reply_error(&s, "stop", "not running"); + mock_qmp_expect_and_reply_error(&s, "stop", "still not running"); + return 0; +} + +QMP_TEST(qmp_client_call, mock_qmp_call_fiber) { + _cleanup_(sd_future_cancel_wait_unrefp) sd_future *f = NULL; + _cleanup_(sd_json_variant_unrefp) sd_json_variant *result = NULL; + _cleanup_free_ char *error_desc = NULL; + + /* Exercise qmp_client_call_future() + sd_fiber_await() + future_get_qmp_reply() + * directly — success path. */ + ASSERT_OK(qmp_client_call_future(client, "query-status", NULL, &f)); + ASSERT_OK(sd_fiber_await(f)); + ASSERT_OK(sd_future_result(f)); + ASSERT_OK(future_get_qmp_reply(f, &result, &error_desc)); + + ASSERT_NULL(error_desc); + sd_json_variant *running = ASSERT_NOT_NULL(sd_json_variant_by_key(result, "running")); + ASSERT_TRUE(sd_json_variant_boolean(running)); + sd_json_variant *status = ASSERT_NOT_NULL(sd_json_variant_by_key(result, "status")); + ASSERT_STREQ(sd_json_variant_string(status), "running"); + + /* QMP-level error: future resolves with -EIO, sd_fiber_await() returns -EIO, and + * future_get_qmp_reply() also returns -EIO (mirroring future_get_bus_reply()) — with the + * detailed description captured via error_desc on top, and result left NULL. */ + f = sd_future_unref(f); + result = sd_json_variant_unref(result); + error_desc = mfree(error_desc); + + ASSERT_OK(qmp_client_call_future(client, "stop", NULL, &f)); + ASSERT_ERROR(sd_fiber_await(f), EIO); + ASSERT_ERROR(sd_future_result(f), EIO); + ASSERT_ERROR(future_get_qmp_reply(f, &result, &error_desc), EIO); + ASSERT_NULL(result); ASSERT_STREQ(error_desc, "not running"); - /* QMP error without ret_error_desc: surfaces as -EIO. */ - ASSERT_EQ(qmp_client_call(client, "stop", NULL, NULL, NULL), -EIO); + /* qmp_client_call() also surfaces QMP errors as -EIO, regardless of whether the caller + * passed ret_error_desc. */ + ASSERT_ERROR(qmp_client_call(client, "stop", NULL, NULL, NULL), EIO); + return 0; } -/* Server variant for the sync-call disconnect test: greets, accepts capabilities, reads one - * command without replying, then closes the socket so the client sees EOF mid-wait. */ -static _noreturn_ void mock_qmp_server_call_disconnect(int fd) { +static int mock_qmp_call_disconnect_fiber(void *userdata) { _cleanup_(json_stream_done) JsonStream s = {}; _cleanup_(sd_json_variant_unrefp) sd_json_variant *stop_cmd = NULL; - mock_qmp_init(&s, fd); + mock_qmp_init(&s, PTR_TO_FD(userdata)); + mock_qmp_handshake(&s); - mock_qmp_send_literal(&s, - "{\"QMP\": {\"version\": {\"qemu\": {\"micro\": 0, \"minor\": 0, \"major\": 9}}, \"capabilities\": []}}"); - - mock_qmp_expect_and_reply(&s, "qmp_capabilities", NULL); - - /* Consume the stop command but don't reply — json_stream_done() on cleanup closes - * our fd, triggering EOF while the client is blocked in qmp_client_call()'s - * process+wait pump. */ + /* Consume the stop command but don't reply — cleanup closes the fd and the client + * sees a disconnect while suspended. */ mock_qmp_recv(&s, &stop_cmd); - - _exit(EXIT_SUCCESS); + return 0; } -TEST(qmp_client_call_disconnect) { - _cleanup_(qmp_client_unrefp) QmpClient *client = NULL; - _cleanup_(pidref_done_sigkill_wait) PidRef pid = PIDREF_NULL; - int qmp_fds[2]; - int r; +QMP_TEST(qmp_client_call_disconnect, mock_qmp_call_disconnect_fiber) { + int r = qmp_client_call(client, "stop", NULL, NULL, NULL); + ASSERT_TRUE(ERRNO_IS_NEG_DISCONNECT(r)); + return 0; +} + +static int mock_qmp_fd_fiber(void *userdata) { + _cleanup_(json_stream_done) JsonStream s = {}; + _cleanup_(sd_json_variant_unrefp) sd_json_variant *cap_cmd = NULL, *addfd_cmd = NULL, + *addfd_return = NULL; + + mock_qmp_init(&s, PTR_TO_FD(userdata)); + ASSERT_OK(json_stream_set_allow_fd_passing_input(&s, true, true)); + + mock_qmp_send_greeting(&s); + + /* The fd may ride with either command depending on AF_UNIX coalescing; count across both. */ + sd_json_variant *cap_id = mock_qmp_expect(&s, "qmp_capabilities", &cap_cmd); + size_t n_fds_total = json_stream_get_n_input_fds(&s); + json_stream_close_input_fds(&s); + mock_qmp_reply(&s, cap_id, NULL); + + sd_json_variant *addfd_id = mock_qmp_expect(&s, "add-fd", &addfd_cmd); + n_fds_total += json_stream_get_n_input_fds(&s); + json_stream_close_input_fds(&s); + ASSERT_EQ(n_fds_total, (size_t) 1); + + ASSERT_OK(sd_json_buildo(&addfd_return, + SD_JSON_BUILD_PAIR_UNSIGNED("fdset-id", 0), + SD_JSON_BUILD_PAIR_UNSIGNED("fd", 42))); + mock_qmp_reply(&s, addfd_id, addfd_return); + return 0; +} + +QMP_TEST(qmp_client_invoke_with_fd, mock_qmp_fd_fiber) { + _cleanup_(sd_json_variant_unrefp) sd_json_variant *args = NULL; + _cleanup_close_ int fd_to_pass = -EBADF; + _cleanup_(sd_json_variant_unrefp) sd_json_variant *result = NULL; + + fd_to_pass = ASSERT_OK_ERRNO(eventfd(0, EFD_CLOEXEC)); + + ASSERT_OK(sd_json_buildo(&args, SD_JSON_BUILD_PAIR_UNSIGNED("fdset-id", 0))); + + ASSERT_OK_POSITIVE(qmp_client_call(client, "add-fd", + QMP_CLIENT_ARGS_FD(args, TAKE_FD(fd_to_pass)), + &result, NULL)); + ASSERT_NOT_NULL(result); + return 0; +} + +static int on_dead_peer_reply( + QmpClient *client, + sd_json_variant *result, + const char *error_desc, + int error, + void *userdata) { + + bool *fired = ASSERT_PTR(userdata); + + /* Peer was closed before the write hit the wire; expect a disconnect. */ + ASSERT_TRUE(ERRNO_IS_NEG_DISCONNECT(error)); + *fired = true; + return 0; +} + +/* Verify caller-supplied fds passed through QMP_CLIENT_ARGS_FD() are closed on client teardown + * even when the peer is already dead: invoke enqueues, the queue item owns the fd, unref closes. */ +TEST(qmp_client_invoke_failure_closes_fds) { + _cleanup_(sd_json_variant_unrefp) sd_json_variant *args = NULL; + _cleanup_close_ int fd_to_pass = -EBADF; + QmpClient *client = NULL; + _cleanup_close_pair_ int qmp_fds[2] = EBADF_PAIR; + int saved_fd_value; + bool callback_fired = false; ASSERT_OK_ERRNO(socketpair(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0, qmp_fds)); + qmp_fds[1] = safe_close(qmp_fds[1]); - r = ASSERT_OK(pidref_safe_fork("(mock-qmp-call-disc)", FORK_DEATHSIG_SIGKILL|FORK_LOG, &pid)); - if (r == 0) { - safe_close(qmp_fds[0]); - mock_qmp_server_call_disconnect(qmp_fds[1]); - } - safe_close(qmp_fds[1]); + fd_to_pass = ASSERT_OK_ERRNO(eventfd(0, EFD_CLOEXEC)); + saved_fd_value = fd_to_pass; - ASSERT_OK(qmp_client_connect_fd(&client, qmp_fds[0])); + ASSERT_OK(sd_json_buildo(&args, SD_JSON_BUILD_PAIR_UNSIGNED("fdset-id", 0))); + ASSERT_OK(qmp_client_connect_fd(&client, TAKE_FD(qmp_fds[0]))); - /* The server reads our stop command and closes without replying. qmp_client_call() - * is driving its own pump, so it must notice the EOF, transition to DISCONNECTED, - * and return a disconnect error rather than hanging. */ - r = qmp_client_call(client, "stop", NULL, NULL, NULL); - ASSERT_TRUE(r < 0); - ASSERT_TRUE(ERRNO_IS_NEG_DISCONNECT(r)); + ASSERT_OK(qmp_client_invoke(client, NULL, "add-fd", + QMP_CLIENT_ARGS_FD(args, TAKE_FD(fd_to_pass)), + on_dead_peer_reply, &callback_fired)); + ASSERT_EQ(fd_to_pass, -EBADF); + ASSERT_OK_ERRNO(fcntl(saved_fd_value, F_GETFD)); + + client = qmp_client_unref(client); + ASSERT_ERROR_ERRNO(fcntl(saved_fd_value, F_GETFD), EBADF); + ASSERT_TRUE(callback_fired); +} + +/* Shared mock for the two slot tests: the follow-up stop is what drives the event loop long + * enough to dispatch the query-status reply. */ +static int mock_qmp_slot_fiber(void *userdata) { + _cleanup_(json_stream_done) JsonStream s = {}; + + mock_qmp_init(&s, PTR_TO_FD(userdata)); + mock_qmp_handshake(&s); + + mock_qmp_query_status_running(&s); + mock_qmp_expect_and_reply(&s, "stop", NULL); + return 0; +} + +static int nop_callback( + QmpClient *client, + sd_json_variant *result, + const char *error_desc, + int error, + void *userdata) { + + return 0; +} + +/* Tripwire for the cancel test: if it fires, the cancel didn't do its job. */ +static int tripwire_callback( + QmpClient *client, + sd_json_variant *result, + const char *error_desc, + int error, + void *userdata) { + + bool *fired = ASSERT_PTR(userdata); + *fired = true; + return 0; +} + +QMP_TEST(qmp_client_invoke_slot_lifecycle, mock_qmp_slot_fiber) { + _cleanup_(qmp_slot_unrefp) QmpSlot *slot = NULL; + + ASSERT_OK(qmp_client_invoke(client, &slot, "query-status", NULL, nop_callback, NULL)); + ASSERT_PTR_EQ(qmp_slot_get_client(slot), client); + + /* Drive the loop via a follow-up stop; its suspending call lets both replies dispatch. */ + ASSERT_OK_POSITIVE(qmp_client_call(client, "stop", NULL, NULL, NULL)); + + /* After dispatch the slot is disconnected from the client but still owned by us. */ + ASSERT_NULL(qmp_slot_get_client(slot)); + + /* Explicit out-of-order unref exercises the already-disconnected path in qmp_slot_free(). */ + slot = qmp_slot_unref(slot); + return 0; +} + +QMP_TEST(qmp_client_invoke_slot_cancel, mock_qmp_slot_fiber) { + QmpSlot *slot = NULL; + bool fired = false; + + ASSERT_OK(qmp_client_invoke(client, &slot, "query-status", NULL, tripwire_callback, &fired)); + + /* Drop our sole ref → slot disconnects from the client's pending set. The enqueued + * query-status is still on the wire; its reply lands on an unknown id and is discarded. */ + slot = qmp_slot_unref(slot); + + ASSERT_OK_POSITIVE(qmp_client_call(client, "stop", NULL, NULL, NULL)); + + ASSERT_FALSE(fired); + return 0; } TEST(qmp_schema_has_member) { _cleanup_(sd_json_variant_unrefp) sd_json_variant *schema = NULL; - /* QEMU introspection uses opaque numeric type ids ("0", "1", ...) — only member names are - * the actual QAPI strings. Verify we walk all object entries and find the member by name. */ + /* QEMU introspection uses opaque numeric type ids ("0", "1", ...); only member names + * are the real QAPI strings. Verify we walk all object entries to find members by name. */ ASSERT_OK(sd_json_build(&schema, SD_JSON_BUILD_ARRAY( SD_JSON_BUILD_OBJECT( @@ -747,10 +500,4 @@ TEST(qmp_schema_has_member) { ASSERT_FALSE(qmp_schema_has_member(NULL, "discard-no-unref")); } -static int intro(void) { - /* Ignore SIGPIPE so that write() to a closed socket returns EPIPE instead of killing us */ - ASSERT_TRUE(signal(SIGPIPE, SIG_IGN) != SIG_ERR); - return 0; -} - -DEFINE_TEST_MAIN_FULL(LOG_DEBUG, intro, NULL); +DEFINE_TEST_MAIN(LOG_DEBUG);