Handle the case when all the arguments are passed in through a
configuration file:
$ cat ukify.conf
[UKI]
Linux = /boot/vmlinuz-linux
Initrd = /boot/initramfs-linux.img
Before:
$ src/ukify/ukify.py --config ukify.conf build
Traceback (most recent call last):
File "/root/systemd/src/ukify/ukify.py", line 1604, in <module>
main()
File "/root/systemd/src/ukify/ukify.py", line 1590, in main
opts = parse_args()
^^^^^^^^^^^^
File "/root/systemd/src/ukify/ukify.py", line 1584, in parse_args
apply_config(opts)
File "/root/systemd/src/ukify/ukify.py", line 1431, in apply_config
item.apply_config(namespace, section_name, group, key, value)
File "/root/systemd/src/ukify/ukify.py", line 1123, in apply_config
self.config_push(namespace, group, dest, value)
File "/root/systemd/src/ukify/ukify.py", line 1019, in config_list_prepend
setattr(namespace, dest, value + old)
~~~~~~^~~~~
TypeError: can only concatenate list (not "NoneType") to list
After:
$ src/ukify/ukify.py --config ukify.conf build
Kernel version not specified, starting autodetection 😖.
Found uname version: 6.4.7-arch1-3
Wrote unsigned vmlinuz-linux.unsigned.efi
Resolves: #28688
Same scenario as with libsystemd - ldd might use unprefixed RPATH, and
we install our own stuff into the image unconditionally anyway.
Also, bail out early if we hit a missing DSO with a possibly helpful
message.
Since not all fields of the dispatch table are mandatory we might end up
reading garbage if the optional field is not set.
In CIs we were lucky enough (or probably unfortunate enough) that the
garbage was printable, but in other cases the `show-server-state` verb
would just fail with:
$ resolvectl show-server-state
Failed to print table: Invalid argument
Follow-up to bc837621a3.
If we don't have TPM support then `alg` is NULL and passing this to
table_new() means we'd get a table with only two columns instead of
three, leading up to a very confusing output:
$ build/systemd-analyze pcrs
System lacks full TPM2 support, not showing PCR state.
NR NAME
0 platform-code
- 1
platform-config -
2 external-code
- 3
external-config -
4 boot-loader-code
- 5
boot-loader-config -
6 -
- 7
...
Let's name the header in this case with a simple dash, as it's going
to be hidden anyway, to make the table nice again:
$ build/systemd-analyze pcrs
System lacks full TPM2 support, not showing PCR state.
NR NAME
0 platform-code
1 platform-config
2 external-code
3 external-config
4 boot-loader-code
5 boot-loader-config
6 -
7 secure-boot-policy
...
To avoid apt complaining:
+ apt-get -y install clang-15 lldb-15 lld-15 clangd-15
Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
python3-lldb-14 : Conflicts: python3-lldb-x.y
python3-lldb-15 : Conflicts: python3-lldb-x.y
E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.
Otherwise bad things happen:
systemd-networkd[3140]: wlan0: Saved new link: ifindex=5, iftype=ETHER(1), kind=n/a
systemd-networkd[3140]: Assertion '(_error) != 0' failed at src/network/networkd-wifi.c:119, function manager_genl_process_nl80211_config(). Aborting.
Resolves: #28534
Since 7c764d4 we bind mount certain directories during switch root
instead of moving the mount directly, and for /run we do this without
MS_REC. This, unfortunately, leaves all mounts under /run behind
in the old root, which breaks certain use cases.
See: https://github.com/systemd/systemd/issues/28452
This fixes regression introduced in 5a0c810462 with which all requests
for link properties ended up with EINVAL as we kept hitting
the signature_is_single() assert in sd_bus_get_property().
It's been a while since we introduced Differential ShellCheck and it
proved to be quite useful (and in some ways even better than the shellcheck
run by super-linter). So, to have only one linter scream at us for not
knowing how to write bash properly, let's drop the super-linter's one in
favor of Differential ShellCheck.
Follow-up for https://github.com/systemd/systemd/pull/24328#pullrequestreview-1074127504
We create the udevadm -> systemd-udevd symlink during the install phase,
so it doesn't exist in the just compiled tree. This worked in CI since
the symlink is manually created there post-build.
Otherwise the get_testdata_dir() call fails if the source tree is under
/root (which is usually the case in CIs).
I got bitten by this after leaving the source tree under /root but moving the
$BUILD_DIR elsewhere. This used to work by accident, as load_testdata_env()
would try to read $BUILD_DIR/systemd-runtest.env, but would fail if the
$BUILD_DIR is also under /root and fall back to SYSTEMD_TEST_DATA
(/lib/systemd/tests/testdata), which usually exist as we install the just built
revision. However, if the $BUILD_DIR is outside of /root we'd read
$BUILD_DIR/systemd-runtest.env which contains
SYSTEMD_TEST_DATA=/path/to/source/tree/test and that source tree is not visible
once we overmount /root with tmpfs making the test fail:
/* test_run_tests_unprivileged */
Successfully forked off '(test-execute-unprivileged)' as PID 10672.
Changing mount flags / (MS_REMOUNT|MS_BIND "")...
Changing mount propagation / (MS_REC|MS_SHARED "")
Mounting tmpfs (tmpfs) on /dev/shm (MS_NOSUID|MS_NODEV "")...
Mounting tmpfs (tmpfs) on /root (MS_NOSUID|MS_NODEV "")...
Mounting tmpfs (tmpfs) on /tmp (MS_NOSUID|MS_NODEV "")...
Mounting tmpfs (tmpfs) on /var/tmp (MS_NOSUID|MS_NODEV "")...
Mounting tmpfs (tmpfs) on /var/lib (MS_NOSUID|MS_NODEV "")...
Mounting tmpfs (tmpfs) on /run/test-execute-unit-dir (MS_NOSUID|MS_NODEV "")...
ERROR: $SYSTEMD_TEST_DATA directory [/root/systemd/test] not accessible: No such file or directory
Assertion 'get_testdata_dir("test-execute/", &unit_dir) >= 0' failed at src/test/test-execute.c:1306, function prepare_ns(). Aborting.
(test-execute-unprivileged) terminated by signal ABRT.
So the multiple-machines tests run last to avoid a race where the
checked signal would get ignored while the stub init is still processing
the previous signals:
[ 17.380417] testsuite-13.sh[376]: + machinectl reboot long-running long-running long-running
[ 17.389888] systemd-nspawn[495]: ++ touch /reboot
[ 17.390904] testsuite-13.sh[376]: + rm -f /var/lib/machines/long-running/trap
[ 17.393937] testsuite-13.sh[376]: + machinectl kill --signal=SIGTRAP --kill-whom=leader long-running
[ 17.408905] testsuite-13.sh[376]: + timeout 10 bash -c 'while ! test -e /var/lib/machines/long-running/trap; do sleep .5; done'
[ 27.413210] testsuite-13.sh[376]: + at_exit
...
[ 27.993376] testsuite-13.sh[373]: + echo 'Subtest /usr/lib/systemd/tests/testdata/units/testsuite-13.machinectl.sh failed'
Introduce a manual test tool that creates a journal, corrupts it by
flipping bits at given offsets, and then attempts to write to the journal.
In ideal case we should handle this gracefully without any crash or
memory corruption.
When reexecuting system let's put our arguments carrying deserialization
info first followed by any existing arguments to make sure they get
parsed in case we get weird stuff from the kernel cmdline (like --).
See: https://github.com/systemd/systemd/issues/28184
So we're able to detect memory leaks in our NSS modules.
An example after introducing a memory leak in nss-myhostname.c:
testsuite-71.sh[2881]: =================================================================
testsuite-71.sh[2881]: ==2880==ERROR: LeakSanitizer: detected memory leaks
testsuite-71.sh[2881]: Direct leak of 2 byte(s) in 1 object(s) allocated from:
testsuite-71.sh[2881]: #0 0x7fa28907243b in strdup (/usr/lib64/libasan.so.8.0.0+0x7243b)
testsuite-71.sh[2881]: #1 0x7fa286a7bc10 in gethostname_full ../src/basic/hostname-util.c:67
testsuite-71.sh[2881]: #2 0x7fa286a74af9 in gethostname_malloc ../src/basic/hostname-util.h:24
testsuite-71.sh[2881]: #3 0x7fa286a756f4 in _nss_myhostname_gethostbyname4_r ../src/nss-myhostname/nss-myhostname.c:79
testsuite-71.sh[2881]: #4 0x7fa288f17588 in getaddrinfo (/lib64/libc.so.6+0xf4588)
testsuite-71.sh[2881]: #5 0x7fa2890a4d93 in __interceptor_getaddrinfo.part.0 (/usr/lib64/libasan.so.8.0.0+0xa4d93)
testsuite-71.sh[2881]: #6 0x55a54b2b7159 in ahosts_keys_int.part.0 (/usr/bin/getent.orig+0x4159)
testsuite-71.sh[2881]: SUMMARY: AddressSanitizer: 2 byte(s) leaked in 1 allocation(s).
As hitting an ASan/UBSan error in PID1 results in a crash (and a kernel
panic when running under qemu), we usually lose the stack trace which
makes debugging quite painful. Let's mitigate this by forwarding the
stack trace to multiple places - namely to a file and the syslog.
To suppress a warning on journal-remote startup:
systemd-journal-remote[691]: microhttpd:
MHD_OPTION_EXTERNAL_LOGGER is not the first option specified for
the daemon. Some messages may be printed by the standard MHD
logger.
When we call safe_fork() with the first argument set (process name), we
call rename_process() that zeroes out saved argv (that was saved by
save_argc_argv() in the main func defined by DEFINE_MAIN_FUNC()). In this
case this means that with --accept both the target executable name and
its arguments will be empty strings:
```
$ systemd-socket-activate --accept --listen 1111 cat &
Listening on [::]:1111 as 3.
$ curl localhost:1111
Communication attempt on fd 3.
Connection from 127.0.0.1:52948 to [::ffff:127.0.0.1]:1111
Spawned cat (cat) as PID 10576.
Execing ()
Failed to execp (): No such file or directory
Child 10576 died with code 1
curl: (56) Recv failure: Connection reset by peer
```
Let's make a copy of the necessary arguments beforehand and use it
instead to fix this.
To get rid of the warning on gatewayd startup:
microhttpd: MHD_OPTION_EXTERNAL_LOGGER is not the first option
specified for the daemon. Some messages may be printed by the
standard MHD logger.
Let's take a step back and revert back to the original behavior where we
exit on a first failing subtest. The current behavior makes fishing out the
failing test details quite unpleasant, and in certain situations the
journal may even be rotated away so we end up with no actionable logs.
Guard the coverage-related shenanigans from g++ when building the one
C++ unit test we have, so we don't have to make it C++ compatible:
[1573/2109] Compiling C++ object test-bus-vtable-cc.p/src_libsystemd_sd-bus_test-bus-vtable-cc.cc.o
FAILED: test-bus-vtable-cc.p/src_libsystemd_sd-bus_test-bus-vtable-cc.cc.o
ccache c++ -Itest-bus-vtable-cc. ... -c ../src/libsystemd/sd-bus/test-bus-vtable-cc.cc
In file included from <command-line>:
../src/basic/coverage.h:17:15: error: ‘_Noreturn’ does not name a type
17 | static inline _Noreturn void _coverage__exit(int status) {
| ^~~~~~~~~
../src/basic/coverage.h:46:29: error: conflicting declaration of ‘int _coverage_execveat(int, const char*, char* const*, char* const*, int)’ with ‘C’ linkage
46 | #define execveat(d,p,a,e,f) _coverage_execveat(d, p, a, e, f)
| ^~~~~~~~~~~~~~~~~~
../src/basic/coverage.h:34:19: note: previous declaration with ‘C++’ linkage
34 | static inline int _coverage_execveat(
| ^~~~~~~~~~~~~~~~~~
../src/basic/coverage.h:46:29: error: declaration of ‘int _coverage_execveat(int, const char*, char* const*, char* const*, int) noexcept’ has a different exception specifier
46 | #define execveat(d,p,a,e,f) _coverage_execveat(d, p, a, e, f)
| ^~~~~~~~~~~~~~~~~~
../src/basic/coverage.h:34:19: note: from previous declaration ‘int _coverage_execveat(int, const char*, char* const*, char* const*, int)’
34 | static inline int _coverage_execveat(
| ^~~~~~~~~~~~~~~~~~
../src/basic/coverage.h:58:24: error: conflicting declaration of ‘int _coverage_execvpe(const char*, char* const*, char* const*)’ with ‘C’ linkage
58 | #define execvpe(f,a,e) _coverage_execvpe(f, a, e)
| ^~~~~~~~~~~~~~~~~
../src/basic/coverage.h:48:19: note: previous declaration with ‘C++’ linkage
48 | static inline int _coverage_execvpe(
| ^~~~~~~~~~~~~~~~~
../src/basic/coverage.h:58:24: error: declaration of ‘int _coverage_execvpe(const char*, char* const*, char* const*) noexcept’ has a different exception specifier
58 | #define execvpe(f,a,e) _coverage_execvpe(f, a, e)
| ^~~~~~~~~~~~~~~~~
../src/basic/coverage.h:48:19: note: from previous declaration ‘int _coverage_execvpe(const char*, char* const*, char* const*)’
48 | static inline int _coverage_execvpe(
| ^~~~~~~~~~~~~~~~~
[1582/2109] Compiling C object test-event.p/src_libsystemd_sd-event_test-event.c.o
ninja: build stopped: subcommand failed.
The test was originally introduced for Ubuntu CI, as it uses
PREFER_NSPAWN=1, but it was subsequently disabled two years ago [0], so
it was pretty much useless. Let's merge it into TEST-02 and tweak it a
bit to run only certain tests under QEMU when $TEST_PREFER_NSPAWN is
set.
[0] b152adbfa9
The kernel command line may contain newlines which kernel happily
accepts, but we'd ignore everything past the first newline. Let's fix
that by replacing read_one_line_file() with read_full_file().
Frantisek Sumsal
Frantisek Sumsal