RootImageOptions=/ExtensionImages=/MountImages= all support custom
mount options, but mountfsd does not support it. Add varlink
parameters to allow callers to specify mount options so that
those directives can work as expected. Require additional privs via
polkit.
This is in preparation for later changes. The 1500+ line switch statement was very
hard to work with and had quite a bit of duplicated code. This is a rework that
splits outs parts into functions. I split this up into a bunch of commits so that it
can be reasonably reviewed. When viewed as one giant commit, diff just shows
a giant block of removed code and a giant block of added code.
In some cases I wasn't sure why one or the other of the identity arrays is used…
I preserved existing behaviour in all cases.
When creating ext2/ext3/ext4 filesystems, mke2fs generates a random
hash_seed for htree directory indexing. This causes non-reproducible
images even when SOURCE_DATE_EPOCH is set and the same filesystem UUID
is used.
Set the hash_seed explicitly to match the filesystem UUID, ensuring
that repeated builds with the same seed produce bit-for-bit identical
images.
Also add a test case in TEST-58-REPART to verify ext4 reproducibility
by creating the same partition twice and comparing the results.
See https://vdwaa.nl/mkosi-reproducible-arch-images.html
I used claude ai to help me with this change.
RootImageOptions=/ExtensionImages=/MountImages= all support
custom mount options, use the new mountfsd parameters to
configure them if they are specified.
This requires additioanl privileges via polkit due to security
implications of mount options, so document an example policy
that allows to use the nosuid mount option.
Currently mount options are handled in such a way that the first
definition for a given partition wins, and documented as such.
Change them so that they behave like other options, and the
last specified wins.
Applies to RootImageOptions=, MountImages= and ExtensionImages=.
Switch from a linked list to an array indexed by the partition
specifier to store them.
As announced by a few releases now, finally drop support for
sysvinit scripts.
Keep rc-local generator for now, as it's really a distinct
feature even though from the same era.
Otherwise, if the VM is unexpectedly rebooted, then `importctl --user pull-tar`
may fail as the file may already exist.
```
[ 123.351751] TEST-13-NSPAWN.sh[3946]: + run0 -u testuser importctl --user pull-tar file:///var/tmp/image-tar/kurps.tar.gz nurps --verify=checksum -m
[ 123.541603] TEST-13-NSPAWN.sh[4311]: Enqueued transfer job 3. Press C-c to continue download in background.
[ 123.552456] TEST-13-NSPAWN.sh[4311]: Pulling 'file:///var/tmp/image-tar/kurps.tar.gz', saving as 'nurps'.
[ 123.552788] TEST-13-NSPAWN.sh[4311]: Operating on image directory '/home/testuser/.local/state/machines'.
[ 123.819942] TEST-13-NSPAWN.sh[4311]: Got 1% of file:///var/tmp/image-tar/kurps.tar.gz.
[ 124.156557] TEST-13-NSPAWN.sh[4311]: * shutting down connection #0
[ 124.156896] TEST-13-NSPAWN.sh[4311]: * Could not open file /var/tmp/image-tar/kurps.tar.gz.sha256
[ 124.157223] TEST-13-NSPAWN.sh[4311]: * closing connection #-1
[ 124.159198] TEST-13-NSPAWN.sh[4311]: * Could not open file /var/tmp/image-tar/kurps.nspawn
[ 124.159493] TEST-13-NSPAWN.sh[4311]: * closing connection #-1
[ 124.159818] TEST-13-NSPAWN.sh[4311]: Acquired 68.5M.
[ 124.160395] TEST-13-NSPAWN.sh[4311]: Download of file:///var/tmp/image-tar/kurps.tar.gz complete.
[ 124.160664] TEST-13-NSPAWN.sh[4311]: Transfer failed: Could not read a file:// file
[ 124.160923] TEST-13-NSPAWN.sh[4311]: Settings file could not be retrieved, proceeding without.
[ 124.404733] TEST-13-NSPAWN.sh[4311]: * shutting down connection #1
[ 124.405162] TEST-13-NSPAWN.sh[4311]: Acquired 79B.
[ 124.406170] TEST-13-NSPAWN.sh[4311]: Download of file:///var/tmp/image-tar/SHA256SUMS complete.
[ 124.406734] TEST-13-NSPAWN.sh[4311]: SHA256 checksum of file:///var/tmp/image-tar/kurps.tar.gz is valid.
[ 124.455446] TEST-13-NSPAWN.sh[4311]: Failed to rename to final image name to /home/testuser/.local/state/machines/.tar-file:\x2f\x2f\x2fvar\x2ftmp\x2fimage-tar\x2fkurps\x2etar\x2egz: File exists
[ 124.457251] TEST-13-NSPAWN.sh[4311]: Exiting.
```
Workaround for issue #38240.
The verity user certs weren't looked up in the given --root= for
systemd-sysext which made it fail to set up extensions with a strict
image policy.
Look up verity user certs from inside the --root= when we operate on
images in it. The main use case where this matters is when the initrd
sets up the extensions for the final system and thus systemd-sysext
should do the same thing as it would do in the final system.
The test passes but for two minutes the cleanup function just logs:
Failed to inspect home: No home for user homedsshtest known
Failed to inspect home: No home for user homedsshtest known
Failed to inspect home: No home for user homedsshtest known
...
Do not wait for the user to become inactive or remove the user if 'homectl
inspect' doesn't find the user. This brings a successfull run from 176 s
to 58 s here.
With a service with DynamicUser= with static user or group, e.g.,
```
$ systemd-run -p DynamicUser=yes -p Group=disk sleep infinity
```
previously the lookup by name and ID through DBus/Varlink are inconsistent:
```
$ busctl call org.freedesktop.systemd1 /org/freedesktop/systemd1 org.freedesktop.systemd1.Manager LookupDynamicUserByUID "u" 6
Call failed: Dynamic user ID 6 does not exist.
$ busctl call org.freedesktop.systemd1 /org/freedesktop/systemd1 org.freedesktop.systemd1.Manager LookupDynamicUserByName "s" disk
u 6
$ userdbctl group 6
Group name: disk
Disposition: system
GID: 6
Passwords: 1
Service: io.systemd.NameServiceSwitch
$ userdbctl group disk
Group name: disk
Disposition: dynamic
GID: 6
Description: Dynamic Group
Service: io.systemd.DynamicUser
```
With this change, the results of these methods are consistent.
Fixes#40228.
MultiPathRoute= can now specify device-only nexthops without a gateway
address, e.g. MultiPathRoute=@wg0. This enables ECMP configurations over
interfaces that don't use gateway addresses, such as WireGuard tunnels.
The syntax is extended from "address[@device] [weight]" to
"[address]@device [weight]". The address is now optional, but at least
one of gateway or device must be specified. The @ symbol must still be
present for device-only routes, making the syntax unambiguous: @wg0
specifies a device, while a bare IP address specifies a gateway.
Device-only nexthops are only available for IPv4 routes. Device-only
multipath routes for IPv6 are not supported by the kernel's netlink
interface and will be rejected with a warning.
This change is fully backwards compatible. All existing configurations
continue to work unchanged, as they always included a gateway address.
Closes#39699.
Allow CPUSchedulingPolicy to be set to "ext". SCHED_EXT is a new
scheduling policy in Linux v6.12 that allows processes to be scheduled
using custom BPF schedulers instead of the default in-kernel ones.
Selectively setting the SCHED_EXT policy is useful for systems running
in "partial mode" where not all processes are run using a custom
scheduler.
Fallback to SCHED_OTHER and print an error message for systems where
SCHED_EXT isn't available.
Loading images is, generally speaking, the slowest part of sd-executor
when spawning a service. This is due to multiple factors. dm-verity is
obviously a big part of the cost, but dissecting in general via libblkid
also can take a lot of time, due to probing the images and their filesystems.
A performance test doing service restarts in a row shows these
results, ran on a production system (low power and slow ARM64 SOC) with
a real production service, show the following service interruption intervals:
Count | P50 (ms) | P90 (ms) | P95 (ms) | P99 (ms) | P99.9 (ms)
507 | 1,532 | 1,726 | 2,548 | 4,112 | 4,592
One iteration is 507 restarts in this case, but this has ran hundreds
of times and the results are always in line within margin of error.
This also holds true for metrics from live systems, same numbers.
Between 1.0s and 1.2s can be attributed by profiling to the time needed
for the service code itself to start up and sd_notify, the rest is spent
inside systemd's code.
This means there is currently a tradeoff for services - either use secure
images, or make restarting fast. Downtime of services is a very important
metric, as for many cases this directly translates to outages, total or
partial (blackouot or greyout).
In order to facilitate using secure images without downsides, skip the
slow dissect steps (probing, loop devices, etc) when the configured
image is a single filesystem dm-verity image with a policy that pins it
to a single filesystem, and an already existing and open dm-verity device
can be found and reused.
This allows orchestrators to pre-open images on download, before restarting
services, to minimize downtimes.
Useful to extract a certificate from a hardware token to a file, for
example in mkosi to ship the certificate from a hardware token in
/usr/lib/verity.d in an image
Authenticated disk encryption is experimentally supported by cryptsetup
since v2.0.0 and allows for automatic dm-integrity setup for LUKS
devices. Add support for the mode to systemd-repart. The PR adds support
for `cryptsetup luksFormat --integrity` to systemd-repart and
"encryptedwithintegrity" dissection policy.
Limitations:
- No discard, online-only mode for repart.
Closes#39250
Useful to extract a certificate from a hardware token to a file, for
example in mkosi to ship the certificate from a hardware token in
/usr/lib/verity.d in an image
LUKS2 supports built-in integrity checking which may come very handy to
mitigate partial rollback attacks on the storage when only some specific
parts are restored to some old encrypted state. Specific use-cases like
Confidential VMs may want to mandate the usage of feature e.g. on the root
volume. Introduce "encryptedwithintegrity" image policy to support that.
Note, due to the current libcryptsetup limitations, checking whether the
feature is enabled or not for the 'file' case (e.g. DDI image as a raw file)
requires setting up a loop device. To avoid that and keep dissect fully
functional when working unpriviliged, implement a minimal custom LUKS header
parser.
* Do not interleave root_directory_as_fd with stdio fields
* Do not use different serialization key for different modes
pointing to same path
* Escape stdio file paths (as per
9be46b1da8)
There is currently no way to figure out a rate limit was hit on a unit,
as the last result is stripped in order to keep reporting the first
result, which is useful in case of a watchdog failure, which is the
reason why it was changed as such.
But rate limiting is also an important information to provide to
users, so allow the Result property to reflect it when it
happens.
PCR 16 is intended for debugging purposes, and is the only PCR that can
be reset by software. Let's prefer that for testing.
PCR 11 has a purpose (i.e. kernel is measured there on UKI systems, as
are the boot phases), hence let's not use that for testing.
That should remove any interference of boot time ordering and testing
our tpm code, as we will write to PCR 16 only, and reset it before we
rely on it.
This likely, kind of fixes#39582, but the issue mixes two things, PCR
11 issues on real-life hardware, and PCR 11 isues in our artificial
tests. Only the latter is addressed by this PR, the other issue needs
more debugging, but without logs we cannot do much.
Heran YangandJian Wen
Yu Watanabe
Luca Boccassi
Zbigniew Jędrzejewski-Szmek
Luca Boccassi
ZauberNerd
Usama Arif
Cathy Hu
Kai Lueke
Luiz Amaral
Daan De Meyer
Matt Fleming
Lennart Poettering
Mike Yuan
Vitaly Kuznetsov