Define a new io.systemd.SysUpdate.Notify Varlink interface with a single
OnCompletedUpdate() method, and after sysupdate successfully installs an
update, invoke that method on every socket linked into
/run/systemd/sysupdate/notify/ via varlink_execute_directory(). This
gives other components a hook to react to applied updates (e.g. recompute
a TPM policy, link a freshly downloaded kernel, refresh extensions).
The notification carries the component name, the installed version and the
list of updated resources (transfer id + on-disk path). Subscribers are
free to ignore the parameters and just treat the call as a trigger.
Setting SYSTEMD_SYSUPDATE_FORCE_NOTIFY=1 forces the notification to be sent
even when no update was applied (in which case no resource list is included),
so follow-up work can be triggered unconditionally.
Fixes: #35988
This adds an operation equivalent to "systemd-sysupdate cleanup" after
an update completed (regardless if that update was entirely successful
or not). This ensures that any orphaned files are automatically cleaned
up, if they are not referenced by any transfer file's patterns anymore.
Follow-up for: d82e256bb9
Let's make sure we keep track of any file we drop into the system via a
database in /var/. This database is implemented based on symlinks, i.e.
reuses the fs as a simple database. Given the database most likely will
have <= 10 entries only (as we store *patterns* of installed file paths in
them, not the file paths themselves), this should be very efficient.
For implementation details see comments at top of
src/sysupdate/sysupdate-cleanup.c.
Just some refactoring.
The code is slightly updated, for example it now uses string_is_safe().
But mostly this just splits out code from one large file to a smaller
one.
Move the --reboot/--component= rejection into parse_argv() alongside the
other cross-option checks, and tighten TEST-72 to assert the specific
guard message rather than merely a non-zero exit.
The `pending` and `reboot` verbs, as well as the `--reboot` switch, compare
the newest installed version against the booted OS version (IMAGE_VERSION= from
os-release). When a component is selected via --component=, this compares the
component's version against the unrelated host OS version, which by design live
in separate version spaces. The result is arbitrary reboot decisions: depending
on the relative version strings sysupdate would either always or never reboot.
Refuse the combination with a clear error instead of silently performing a
bogus comparison. Correctly tracking a per-component booted version is left as a
future feature.
Fixes: https://github.com/systemd/systemd/issues/42330
unhexmem_full() ignores whitespace, so a 64-byte manifest digest field
can decode to fewer than 32 bytes. Reject that while parsing instead.
[ 83.883087] TEST-72-SYSUPDATE.sh[5995]: systemd-sysupdate: ../src/src/sysupdate/sysupdate-resource.c:581: resource_load_from_web: Assertion `h.iov_len == sizeof(instance->metadata.sha256sum)' failed.
Follow-up for 43cc7a3ef4
Assisted-by: kres (claude-opus-4-7)
Co-developed-by: Claude Opus 4.8 <noreply@anthropic.com>
Currently almost all the dlopens happen in libbasic or libshared code,
so the ELF dlopen notes all end up in libsystemd-shared. Many
distributions split this library and various binaries in separate
packages, and the library ends up with soft-dependencies, even though
many binaries are either completely useless or do not work at all with
the dlopen dependency. This also makes it impossible to know which
executable uses which dlopen dependency without inspecting the source
code.
If someone only wants to add the soft dependencies from libshared they
can just avoid parsing the executable binaries. By design the code in
libbasic/libshared still does the stamping too, at lower priorities, so
that libsystemd-shared will always list all the optional dependencies,
and if one wants to build a minimal system by default, they can just
parse libsystemd-shared dlopen notes, and ignore the individual
executables. But for many distribution the current setup is insufficient
and requires adding a ton of manual library dependencies, as many
executables become effectively broken or useless without the dlopen
dependencies installed (eg: resolved fails to start without libssl,
repart can do basically nothing without blkid, etc).
Add a new set of DLOPEN_<LIB> macros that wrap the dlopen_lib and also
pull in the ELF note voodoo, so that the callers get their ELF binaries
stamped too. Convert a bunch of callers to use the macro, and use
`required` dependencies for the callers that do not work without the
dlopen library being available.
The one caveat is that, in order to avoid duplicating the exact same
note in a binary due to multiple call sites, some `asm` voodoo is done
instead of the previous bare-C section-creating macro. The drawback of
this approach is that if `--gc-sections` is used to link the binary (as
we do), then binutils >= 2.36 is required for the `SHF_GNU_RETAIN` flag.
This effectively cuts off CentOS 9, so what I did here is adding an
override in meson to detect missing support, and drop `SHF_GNU_RETAIN`.
The build works, but on CentOS 9 there's no dlopen ELF notes anymore.
Given it's just that version, and it goes EOL next year, that seems ok
to me. The alternative is to drop the usage of `--gc-sections` on CentOS
9, or to accept duplicated notes everywhere, and both seem worse.
End result:
```
$ readelf --notes build/systemd-executor
Displaying notes found in: .note.gnu.property
Owner Data size Description
GNU 0x00000010 NT_GNU_PROPERTY_TYPE_0
Properties: x86 ISA needed: x86-64-baseline
Displaying notes found in: .note.gnu.build-id
Owner Data size Description
GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring)
Build ID: 8a0c3db54adb79ae54e1432255011aa4ab583742
Displaying notes found in: .note.ABI-tag
Owner Data size Description
GNU 0x00000010 NT_GNU_ABI_TAG (ABI version tag)
OS: Linux, ABI: 3.2.0
Displaying notes found in: .note.dlopen
Owner Data size Description
FDO 0x0000006b FDO_DLOPEN_METADATA
Dlopen Metadata: [{"feature":"pam","description":"Support for LinuxPAM","priority":"recommended","soname":["libpam.so.0"]}]
FDO 0x0000007c FDO_DLOPEN_METADATA
Dlopen Metadata: [{"feature":"seccomp","description":"Support for Seccomp Sandboxes","priority":"recommended","soname":["libseccomp.so.2"]}]
FDO 0x00000090 FDO_DLOPEN_METADATA
Dlopen Metadata: [{"feature":"bpf","description":"Support firewalling and sandboxing with BPF","priority":"recommended","soname":["libbpf.so.1","libbpf.so.0"]}]
FDO 0x000000a0 FDO_DLOPEN_METADATA
Dlopen Metadata: [{"feature":"cryptsetup","description":"Support for disk encryption, integrity, and authentication","priority":"recommended","soname":["libcryptsetup.so.12"]}]
FDO 0x00000078 FDO_DLOPEN_METADATA
Dlopen Metadata: [{"feature":"mount","description":"Support for mount enumeration","priority":"recommended","soname":["libmount.so.1"]}]
$ readelf --notes build/systemd
Displaying notes found in: .note.gnu.property
Owner Data size Description
GNU 0x00000010 NT_GNU_PROPERTY_TYPE_0
Properties: x86 ISA needed: x86-64-baseline
Displaying notes found in: .note.gnu.build-id
Owner Data size Description
GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring)
Build ID: dcd4568842e32da3e71be27db3def51c6b459994
Displaying notes found in: .note.ABI-tag
Owner Data size Description
GNU 0x00000010 NT_GNU_ABI_TAG (ABI version tag)
OS: Linux, ABI: 3.2.0
Displaying notes found in: .note.dlopen
Owner Data size Description
FDO 0x00000075 FDO_DLOPEN_METADATA
Dlopen Metadata: [{"feature":"mount","description":"Support for mount enumeration","priority":"required","soname":["libmount.so.1"]}]
FDO 0x00000072 FDO_DLOPEN_METADATA
Dlopen Metadata: [{"feature":"selinux","description":"Support for SELinux","priority":"recommended","soname":["libselinux.so.1"]}]
```
Switch the executable-owned call sites from the plain dlopen_<lib>()
helpers to the new DLOPEN_<LIB>() macros, so that each binary that loads
an optional library at runtime now carries its own .note.dlopen note
instead of relying solely on libsystemd-shared. Each call site passes a
priority reflecting whether it hard-depends on the library (required) or
degrades gracefully without it (recommended).
Co-developed-by: Claude Opus 4.8 <noreply@anthropic.com>
`sysupdate --json=short components` lists the components known to
sysupdate; these are the components which something like `updatectl`
will try to update.
The `default` component represents the host, and is meant to be listed
if transfer definitions exist in (for example) `/etc/sysupdate.d`
corresponding to the host OS. This then corresponds to `TARGET_HOST` in
`updatectl` and causes it to try updating that target.
The logic for working out whether the `default` component was present
essentially boiled down to “does `{/run,/etc,/usr/lib}/sysupdate.d`
exist”, and it didn’t check whether a `.transfer` or `.conf` file
actually existed in the config directory.
This is quite the corner case, but becomes more evident on systems where
sysupdate is being used to update a portable service but not the main
OS. At that point, if `/etc/sysupdate.d` exists empty (for some reason),
`updatectl` falls over because it starts trying to update the host OS
without any configuration to do so.
So, modify `sysupdate` to more fully load the available configuration
when listing components, and query it a bit more deeply to check whether
a default component exists.
If `sysupdate` is called with various command line arguments to affect
how its configuration is loaded, do *not* say that a default component
exists, as these arguments essentially anull the possibility of a
default being used in that process.
Add an integration test based on the reproducer provided by the issue
reporter. This test has been tested to fail if the changes to
`sysupdate.c` aren’t applied — if so, the second call to `sysupdate
components` would return
`{"default":true,"components":["some-component"]}`.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Fixes: https://github.com/systemd/systemd/issues/41501
`sysupdate --json=short components` lists the components known to
sysupdate; these are the components which something like `updatectl`
will try to update.
The `default` component represents the host, and is meant to be listed
if transfer definitions exist in (for example) `/etc/sysupdate.d`
corresponding to the host OS. This then corresponds to `TARGET_HOST` in
`updatectl` and causes it to try updating that target.
The logic for working out whether the `default` component was present
essentially boiled down to “does `{/run,/etc,/usr/lib}/sysupdate.d`
exist”, and it didn’t check whether a `.transfer` or `.conf` file
actually existed in the config directory.
This is quite the corner case, but becomes more evident on systems where
sysupdate is being used to update a portable service but not the main
OS. At that point, if `/etc/sysupdate.d` exists empty (for some reason),
`updatectl` falls over because it starts trying to update the host OS
without any configuration to do so.
So, modify `sysupdate` to more fully load the available configuration
when listing components, and query it a bit more deeply to check whether
a default component exists.
If `sysupdate` is called with various command line arguments to affect
how its configuration is loaded, do *not* say that a default component
exists, as these arguments essentially anull the possibility of a
default being used in that process.
Add an integration test based on the reproducer provided by the issue
reporter. This test has been tested to fail if the changes to
`sysupdate.c` aren’t applied — if so, the second call to `sysupdate
components` would return
`{"default":true,"components":["some-component"]}`.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Fixes: https://github.com/systemd/systemd/issues/41501
Optionally prevent `context_read_definitions()` erroring out if zero
transfer definitions were found.
This commit makes no functional changes (the flag is always passed to
calls to `context_make_offline()` for the moment), but the new flag will be
used in the following commit.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Helps: https://github.com/systemd/systemd/issues/41501
This commit makes no functional changes, but the new flags will be
used and extended in the following commit. We need a flags variable to
avoid having two bool arguments, which would be confusing.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Helps: https://github.com/systemd/systemd/issues/41501
This allows us to have a separate, more permissive, policy for
cancelling ongoing sysupdate jobs. The new default policy for
cancellation actions is to allow them for the active user, without admin
authentication, because typically the user can just pull the plug on the
computer to cancel a job anyway.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Fixes: https://github.com/systemd/systemd/issues/38568
This allows us to have a separate, more permissive, policy for
cancelling ongoing sysupdate jobs. The new default policy for
cancellation actions is to allow them for the active user, without admin
authentication, because typically the user can just pull the plug on the
computer to cancel a job anyway.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Fixes: https://github.com/systemd/systemd/issues/38568
While commit 83c1e8ff5f added support for interactive polkit
authentication in `updatectl`, some users might want to disable that for
some use cases; so add the standard `--no-ask-password` argument.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Helps: https://github.com/systemd/systemd/issues/37412
- **bpf-util: rename from bpf-dlopen, unify version-specific symbol
handling**
- **cryptsetup: dlopen libcryptsetup in tokens**
- **tree-wide: dlopen libpam in pam plugins**
- **test-bus-marshal: dlopen() glib and libdbus instead of linking
directly**
- **lock-util: Simplify timeout for lock_generic_with_timeout()**
- **color-util: simplify hsv_to_rgb, fix rgb_to_hsv negative-hue wrap**
- **tree-wide: Use our own macros instead of fabs()/fmax()/fmin()**
- **locale-util: dlopen() libintl instead of linking against it**
- **home: Use log2u64() over log2()**
- **meson: drop libdl, threads, and librt dependencies**
- **libc: Use dlsym() from a constructor instead of weak symbols**
- **libc: Make sure C23 versions of strtol(), sscanf() are not used**
Our baseline glibc is 2.34, which merged libdl, libpthread (the
dependency('threads') target), and librt into libc. Empty .so/.a stubs
remain for backward compatibility with old binaries, but new builds
resolve dl_*, pthread_*, mq_*, timer_*, etc. directly from libc.
On musl the same libraries are likewise empty stubs.
Drop the libdl, threads, and librt entries from every meson.build, and
remove the now-stale 'Libs.private: -lrt -pthread' from libudev.pc.in
since both flags resolve to empty link-time stubs on glibc 2.34+ and
musl.
Verified with readelf -d that libsystemd.so, libudev.so, and systemd no
longer carry DT_NEEDED entries for libdl/libpthread/librt.
This was missing in the CurrentSymlink= creation path, and leads to
partially-broken update installs.
Signed-off-by: Aleksa Sarai <aleksa@amutable.com>
An error snuck into the pattern parsing of the `MatchPattern` key in the
sysupdate transfer files. If there's two files "part1-v2.raw", and
"part1-v2.raw.tar" in the source folder, and MatchPattern="part1-@v.raw",
sysupdate will incorrectly choose "part1-v2.raw.tar" instead of
"part1-v2.raw".
While the pattern matching works perfectly fine, after the full pattern
is successfully matched to the string, we don't ensure that the string
actually ends when the pattern just did.
This means we can end up choosing a wrong file for the update, if the
filename/path happens to start with the same MatchPattern.
Fix it by ensuring the string ends after our match pattern ended.
I am really not a fan of full code lines passed to macros as parameters.
Let's get rid of the 3rd parameter of FOREACH_OPTION() hence:
1. Let's return errors just as a regular value (though a negative one),
that can be handled via a OPTION_ERROR case statement for the switch.
This normalizes handling of the error, just like any other event
returned by the option parser.
2. In order to avoid exploding the amount of boilerplate in each use
(that just propagates the error on OPTION_ERROR), let's then
introduce an explicit FOREACH_OPTION_OR_RETURN(), that returns from
the calling function on its own (and makes that clear in the name).
Together this cleans up, normalizes the logic and shortens the code.
It was requested to make the 'c', 'opt', and 'arg' params the same, i.e.
defined through the FOREACH_OPTION macro. But we can't do that easily,
because 'c' was defined in the for loop definition, and we can only
define variables of the same type in that way. Also, in some cases we
need only 'c', in other cases with need 'c' and 'arg, in some cases 'c'
and 'opt', and in other cases all three. We'd need to either
conditionalize or mark those variables with _unused_ to deal with
compiler warnings. But a different approach works quite nicely: add
state.opt and state.arg to show the current option and it's argument.
(The short names are picked on purpose to reduce verbosity since those
are used a lot.)
If an update is partially downloaded and the user tries to update again,
`updatectl` can’t currently do anything (it doesn’t yet support resuming
downloads). At the moment, though, it’ll return success as if the system
was up to date, even though it isn’t up to date.
Instead, print a more helpful error message telling the user to try
vacuuming the partial version and trying again.
I decided not to make it automatically vacuum the partial version, as
that seems like a way to get into a nasty retry loop if, for example,
the checksum provided by the server doesn’t match that of the downloaded
file (which is one way to trigger this code path).
Add an integration test which simulates this failure by corrupting the
`SHA256SUMS` file, trying to download an update, and then working
through the recovery steps.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Fixes: https://github.com/systemd/systemd/issues/41502
Previously we prevented partial and pending versions from being
vacuumed. But until we support resuming downloads, there’s nothing else
which can be done with a partial version except to vacuum it and try
again.
Accordingly, allow partial versions (but not pending versions) to be
vacuumed.
This behaviour can be changed again once resuming downloads is supported
— at that point I expect we’ll want to try resuming the partial download
rather than throwing it all away and trying again.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Helps: https://github.com/systemd/systemd/issues/41502
Previously we allowed a pending version to be the candidate — but if
there are no better choices, then we might as well allow a partial
version to be candidate as well.
The alternative is having no update candidate when a new version is
partially installed (i.e. downloaded but not moved into place). This
would mean that an update which is interrupted then needs to be re-run
with an explicit version number to progress, rather than being able to
be re-run without a version number (as it was in the first place).
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Helps: https://github.com/systemd/systemd/issues/41502
While a resource Instance can either be partial or pending, but not
both; an UpdateSet (which potentially comprises several Instances) can
be both partial *and* pending if it contains Instances in both those
states.
Amend a few bits of internal code to allow that in situations which were
previously overlooked.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Until now OpenSSL was linked into every binary and library that needed
cryptography, pulling libcrypto (and, for resolved, libssl) into the
address space of services that never touch them at runtime. This commit
moves all OpenSSL usage behind the same dlopen helper pattern that we
already use for other optional libraries (libpam, libseccomp, libxz, …)
so libcrypto/libssl are only loaded on demand.
The bulk of the work lives in src/shared/crypto-util.{c,h} (libcrypto)
and src/shared/ssl-util.{c,h} (libssl), which replace the previous
src/shared/openssl-util.{c,h}:
- crypto-util.{c,h} declares every libcrypto function we call via
DLSYM_PROTOTYPE() and resolves them inside dlopen_libcrypto().
- ssl-util.{c,h} holds the libssl-specific DLSYM_PROTOTYPEs,
dlopen_libssl(), and the SSL_freep cleanup helper, so translation
units that only need libcrypto do not pull in libssl declarations.
- Callers refer to the symbols through sym_* aliases rather than the
original names.
- Convenience macros that used to be provided by the OpenSSL headers
(OPENSSL_free, BN_num_bytes, the sk_TYPE_* helpers, …) are
reimplemented as sym_* wrappers so no code path needs to fall back
to the linker-resolved symbols.
- All _cleanup_ helpers are redefined in terms of the sym_* variants
(EVP_PKEY_freep, X509_freep, BIO_freep, …) so cleanup attributes
keep working without pulling in libcrypto symbols at link time.
- The public crypto-util.c entry points (openssl_pubkey_from_pem,
openssl_digest_many, openssl_hmac_many, openssl_cipher_many,
kdf_ss_derive, kdf_kb_hmac_derive, rsa_* / ecc_* helpers,
pubkey_fingerprint, digest_and_sign, pkcs7_new, x509_fingerprint,
openssl_extract_public_key, pkey_generate_volume_keys, the load_*
helpers, …) now call dlopen_libcrypto() at entry before touching any
sym_* pointer.
The call sites across the tree have been converted to call
dlopen_libcrypto()/dlopen_libssl() at the appropriate entry point
before their first sym_* use, and to use sym_* variants throughout:
- bootctl, sbsign, measure, pcrlock, pcrextend, tpm2-setup, repart,
cryptsetup, cryptenroll, homectl, homed, homework, keyutil,
sysupdate, creds, import, dissect-image, pe-binary, pkcs11-util,
pkcs7-util, tpm2-util, creds-util. resolved additionally dlopens
libssl for DoT.
The meson build files are updated to depend on libopenssl_cflags (a
new partial dependency that exposes include paths and compile flags
only, not the linker flags) instead of libopenssl for every target that
previously linked against OpenSSL. Nothing links against libcrypto or
libssl directly anymore.
A new src/sbsign/authenticode.c hosts the Authenticode ASN.1 type
definitions that used to live inline in sbsign.c. The OpenSSL
ASN1_SEQUENCE / ASN1_CHOICE / IMPLEMENT_ASN1_FUNCTIONS macros expand to
code that references libcrypto symbols directly, so to keep this
translation unit unlinked from libcrypto we redirect ASN1_item_* to
the sym_* variants via #define and wrap the ASN1_*_it() getters (which
appear as constant function pointers in static initializers) in small
trampoline functions that forward to the sym_* pointers at runtime.
test-dlopen-so gains assertions for dlopen_libcrypto and dlopen_libssl
so the dlopen contract is exercised in CI, and the openssl-specific
test was renamed from test-openssl.c to test-crypto-util.c to match
the new header naming.
If the resource code is recursing, it’s possible for one iteration to
set a partial flag, and then a recursive iteration to set a pending flag
(or vice-versa). It doesn’t make sense to have both set at the same time
for a specific instance, so make sure to clear the other flag when
setting one of them.
Add some assertions to make this invariant clearer and easier to debug
if it fails.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Every dlopen_xxx() helper now takes an int log_level argument. It is
passed through to dlopen_many_sym_or_warn() (which in turn propagates it
to dlopen_verbose() for the library-not-installed case), and is used by
the fallback stub when support for the library is not compiled in to
emit a "<lib> support is not compiled in." message at the caller's level.
Callers pass LOG_DEBUG when gracefully degrading, or a higher level when
the failure should surface, and no longer need to log redundantly at the
call site.
As part of this, dlopen_bpf_full() (which already took a log_level) is
merged into dlopen_bpf() rather than keeping both.
The static inline fallbacks used to live in the headers, which required
pulling log.h in from every header that declared a dlopen_xxx(). Move
them into the .c files instead: the declaration is always outside the
#if HAVE_XXX block, the impl sits outside the outer #if HAVE_XXX wrap
with its own internal #if HAVE_XXX/#else/#endif, and apparmor-util.c,
idn-util.c, libmount-util.c and pam-util.c are now always compiled so
they can host their stubs.
Convert fdisk-util to the dlopen pattern used by other optional shared
libraries in libshared. Declare the libfdisk API entry points with
DLSYM_PROTOTYPE, resolve them in a dlopen_fdisk() helper, and call the
sym_* wrappers from the homework, sysupdate and repart binaries that
use them.
With this in place fdisk-util can live in libshared itself, linked only
against libfdisk's headers (via libfdisk_cflags). The libshared_fdisk
convenience library and the libfdisk link dependency on systemd-homework,
systemd-sysupdate, systemd-repart and systemd-repart.standalone go away.
Also add a dlopen_fdisk() check to test-dlopen-so.
`READY=1` is already correctly emitted when acquiring an update, but was
forgotten to be emitted when subsequently installing that update.
That meant that the state tracking code in `sysupdated` and hence
`updatectl` could not properly report the progress of the install
operation, and hence printed “Already up-to-date” after a successful
update installation, rather than “Done”.
Add a test to catch this in future.
Signed-off-by: Philip Withnall <pwithnall@gnome.org>
Fixes: https://github.com/systemd/systemd/issues/41502
The reworks the ESP/XBOOTLDR logic to pin the ESP/XBOOTLDR via an fd,
and return that as optional return parameter.
So far we only pinned the parent dir of the ESP/XBOOTLDR, which was
useful when verifying that ESP/XBOOTLDR is actually a mount point by
comparing mount ids. This however became obsolete with
a98a6eb95c. Hence, let's clean this up,
and pin the inode we really care about and return it.
As soon as machinectl list-images has an mstack entry updatectl fails
because systemd-sysupdated crashes with an assertion failing because
the mstack case was not handled.
For now mstack is not supported as image for sysupdate to operate on
and we can skip it.
Fixes https://github.com/systemd/systemd/issues/41649
The updated parsing_hacks.h file uncovered a bunch of extra
things that the check-pointer-deref coccinelle script flags.
This commit fixes them to make the tree check-pointer-deref clean.