Previously we didn't provide any install_callback to
sd_bus_add_match_async() so in case AddMatch() method call timed out we
destroyed the bus connection. This seems overly aggressive and simply
updating the sd_bus_track object accordingly should be enough.
Follow-up for 37ce3fd2b7.
Fixes#32381
It is not easy to understand what happens to a journal file
even with debug logs enabled. Add more dbg messages around operations
started by users to make it possible to follow the flow of operations.
As discussed in https://github.com/systemd/systemd/issues/38399, "ordinary"
systems can have the field table with a large number of values, causing journal
rotation to occur early. For example, audit generates a log of fields:
$ journalctl --fields | rg -c '^_?AUDIT'
114
It seems that the "structured log" capabilities of the journal are being use
more than in the past. Looking at some journal files on my system, it seems
the field hash table field is quite high in many cases:
$ build/test-journal-dump /var/log/journal/*/* | rg 'table fill'
Data hash table fill: 15.1%
Field hash table fill: 69.1%
Data hash table fill: 4.9%
Field hash table fill: 32.4%
Data hash table fill: 10.2%
Field hash table fill: 34.2%
Data hash table fill: 9.9%
Field hash table fill: 37.2%
Data hash table fill: 26.8%
Field hash table fill: 21.9%
Data hash table fill: 35.6%
Field hash table fill: 22.8%
Data hash table fill: 25.5%
Field hash table fill: 54.1%
Data hash table fill: 3.4%
Field hash table fill: 43.8%
Data hash table fill: 75.0%
Field hash table fill: 70.3%
Data hash table fill: 75.0%
Field hash table fill: 63.1%
Data hash table fill: 75.0%
Field hash table fill: 74.2%
Data hash table fill: 35.6%
Field hash table fill: 43.2%
Data hash table fill: 35.5%
Field hash table fill: 75.4%
Data hash table fill: 75.0%
Field hash table fill: 59.8%
Data hash table fill: 75.0%
Field hash table fill: 56.5%
Data hash table fill: 16.9%
Field hash table fill: 76.3%
Data hash table fill: 18.1%
Field hash table fill: 76.9%
Data hash table fill: 75.0%
Field hash table fill: 42.0%
Data hash table fill: 75.0%
Field hash table fill: 22.8%
Data hash table fill: 75.0%
Field hash table fill: 22.8%
Data hash table fill: 75.0%
Field hash table fill: 22.8%
Data hash table fill: 75.0%
Field hash table fill: 22.8%
Data hash table fill: 75.0%
Field hash table fill: 32.1%
Data hash table fill: 75.0%
Field hash table fill: 21.9%
Data hash table fill: 75.0%
Field hash table fill: 21.9%
Data hash table fill: 75.0%
Field hash table fill: 21.9%
Data hash table fill: 75.0%
Field hash table fill: 22.8%
Data hash table fill: 75.0%
Field hash table fill: 22.8%
Data hash table fill: 75.0%
Field hash table fill: 21.9%
Data hash table fill: 75.0%
Field hash table fill: 22.5%
Data hash table fill: 9.6%
Field hash table fill: 53.8%
Data hash table fill: 75.0%
Field hash table fill: 22.2%
Data hash table fill: 75.0%
Field hash table fill: 22.2%
Data hash table fill: 75.0%
Field hash table fill: 22.2%
Data hash table fill: 35.6%
Field hash table fill: 75.1%
Data hash table fill: 33.6%
Field hash table fill: 50.2%
Data hash table fill: 75.0%
Field hash table fill: 26.7%
Data hash table fill: 75.0%
Field hash table fill: 25.8%
Data hash table fill: 75.0%
Field hash table fill: 29.1%
Data hash table fill: 75.0%
Field hash table fill: 25.8%
Data hash table fill: 75.0%
Field hash table fill: 31.8%
Data hash table fill: 75.0%
Field hash table fill: 18.9%
Data hash table fill: 75.0%
Field hash table fill: 22.2%
Data hash table fill: 75.0%
Field hash table fill: 20.1%
Data hash table fill: 75.0%
Field hash table fill: 29.1%
Data hash table fill: 75.0%
Field hash table fill: 30.9%
Data hash table fill: 75.0%
Field hash table fill: 28.5%
Data hash table fill: 75.0%
Field hash table fill: 28.5%
Data hash table fill: 75.0%
Field hash table fill: 25.8%
Data hash table fill: 75.0%
Field hash table fill: 25.2%
Data hash table fill: 75.0%
Field hash table fill: 39.3%
Data hash table fill: 50.2%
Field hash table fill: 75.1%
Data hash table fill: 75.0%
Field hash table fill: 61.9%
Data hash table fill: 75.0%
Field hash table fill: 56.5%
Data hash table fill: 75.0%
Field hash table fill: 58.6%
Data hash table fill: 48.9%
Field hash table fill: 79.6%
Data hash table fill: 75.0%
Field hash table fill: 71.5%
Data hash table fill: 75.0%
Field hash table fill: 60.1%
Data hash table fill: 31.4%
Field hash table fill: 75.7%
Data hash table fill: 27.0%
Field hash table fill: 69.4%
Data hash table fill: 28.9%
Field hash table fill: 76.6%
Data hash table fill: 60.2%
Field hash table fill: 79.9%
Data hash table fill: 8.8%
Field hash table fill: 78.7%
Data hash table fill: 5.8%
Field hash table fill: 61.3%
Data hash table fill: 75.0%
Field hash table fill: 64.0%
Data hash table fill: 61.4%
Field hash table fill: 63.4%
Data hash table fill: 29.7%
Field hash table fill: 61.9%
Data hash table fill: 18.9%
Field hash table fill: 30.9%
Data hash table fill: 1.4%
Field hash table fill: 22.2%
Data hash table fill: 0.4%
Field hash table fill: 13.5%
Data hash table fill: 2.6%
Field hash table fill: 37.5%
Data hash table fill: 1.3%
Field hash table fill: 23.4%
Data hash table fill: 0.6%
Field hash table fill: 15.3%
Data hash table fill: 18.7%
Field hash table fill: 33.9%
Data hash table fill: 7.4%
Field hash table fill: 37.5%
Data hash table fill: 20.2%
Field hash table fill: 44.1%
Data hash table fill: 1.3%
Field hash table fill: 33.0%
Data hash table fill: 75.0%
Field hash table fill: 19.2%
Data hash table fill: 42.2%
Field hash table fill: 23.4%
Data hash table fill: 1.6%
Field hash table fill: 87.1%
Data hash table fill: 0.1%
Field hash table fill: 98.8%
Data hash table fill: 0.2%
Field hash table fill: 128.8%
Data hash table fill: 15.4%
Field hash table fill: 31.2%
Data hash table fill: 7.4%
Field hash table fill: 22.5%
Data hash table fill: 10.5%
Field hash table fill: 38.7%
Data hash table fill: 2.8%
Field hash table fill: 18.0%
Data hash table fill: 1.5%
Field hash table fill: 15.9%
Data hash table fill: 0.0%
Field hash table fill: 7.5%
Data hash table fill: 0.1%
Field hash table fill: 12.0%
Data hash table fill: 0.2%
Field hash table fill: 10.8%
Data hash table fill: 0.2%
Field hash table fill: 15.6%
Data hash table fill: 0.1%
Field hash table fill: 11.7%
Data hash table fill: 0.1%
Field hash table fill: 12.0%
Data hash table fill: 0.0%
Field hash table fill: 6.6%
Data hash table fill: 1.4%
Field hash table fill: 18.0%
Data hash table fill: 0.7%
Field hash table fill: 16.8%
Data hash table fill: 1.1%
Field hash table fill: 18.0%
Data hash table fill: 0.2%
Field hash table fill: 10.8%
Data hash table fill: 0.1%
Field hash table fill: 10.8%
Data hash table fill: 0.4%
Field hash table fill: 11.1%
Since filling of the field hash table to 75% normally causes file rotation,
let's double the default to make rotation happen less often.
We'll use 11kB more for the hash table, which should be fine, considering
that journal files are usually at least 8 MB.
Closes https://github.com/systemd/systemd/issues/38399.
It's easier to think about the size in "objects", not bytes. Let's convert to
bytes at the last moment.
Also drop some of the pointless size suffixes. In general, it's the size of the
variable that matters, not the constant that is written to it.
No functional change.
We had errno_to_name() which works for "known" errnos, and returns NULL for
unknown ones, and then ERRNO_NAME which always returns an answer, possibly just
a number as a string, but requires a helper buffer.
It is possible for the kernel to add a new errno. We recently learned that some
architectures define custom errno names. Or for some function to unexpectedly
return a bogus errno value. In almost all cases it's better to print that value
rather than "n/a" or "(null)". So let's use ERRNO_NAME is most error handling
code. Noteably, our code wasn't very good in handling the potential NULL, so
in various places we could print "(null)". Since this is supposed to be used
most of the time, let's shorten the names to ERRNO_NAME/errno_name.
There are a few places where we don't want to use the fallback path, in
particular for D-Bus error names or when saving the error name. Let's rename
errno_to_name() to errno_name_no_fallback() to make the distinction clearer.
To avoid the following error:
```
test_sd_device_one(/sys/devices/virtual/block/dm-9)
src/libsystemd/sd-device/test-sd-device.c:104: Assertion failed: Expected "sd_device_new_from_syspath(&dev, syspath)" to succeed, but got error: -19/ENODEV
```
This also updates outdates comment about CentOS CI.
Even when we receive an inotify event, there is no relevant event source
exists. In that case, we need to drop the event from the buffer,
otherwise we cannot escape from the loop.
Fixes#38265.
Continuation of #37960.
The same concern as expalined in #37960 exists also in
missing_syscall.h. If we use enough new glibc, a function we want to use
may be already provided by glibc, but our baseline glibc may not. And it
is hard to detect in our daily development.
This moves all prototypes of syscalls to relevant headers, and missing
syscall functions are defined in relevant .c files of libc wrapper. This
way, we can use usual header as is, e.g. when we want to write code with
`move_mount()`, we can simply use sys/mount.h without checking if it is
supported by our baseline glibc.
Then, move syscall definitions to the wrapper, and prototypes are moved
to relevant headers.
This also adds checks for add_key() and request_key(), as one day
glibc may be going to add some of them separatedly.
The check for fspick in meson.build is dropped, as it is currently
unused in our code.
This also moves
- basic/missing_bpf.h -> include/override/linux/bpf.h,
- basic/missing_keyctl.h -> include/override/linux/keyctl.h.
- pass our system include directories to make generators use our libc
wrappers and latest kernel headers,
- include relevant headers in generated gperf file,
- use files() rather than find_program(), as the result of
find_program() cannot be passed to 'input' of custom_target(),
- move generate-bpf-delegate-configs.py to src/core/, as it is only used
by libcore.
These source files uses symbols provided by sys/stat.h, e.g. struct stat,
S_IFREG, S_IFBLK, and so on. Let's explicitly include sys/stat.h where
necessary.
Glibc's fcntl.h includes bits/stat.h, which provides these symbols, so
these symbols can be used without explicitly including sys/stat.h. But,
based on the discussion in #37922, we should explicitly include relevant
headers, and should not rely on the indirect inclusion.
Similar to 4f18ff2e29, but for sys/stat.h.
Currently, when running "systemctl preset-all --root=xxx" in mkosi
to enable/disable units for initrds, the system presets are used.
The problem with this approach is that the system presets are written
for the system, and that is not necessarily ideal for an initrd, but we
still want to use the same packages in the initrd that we install in the
system, so let's introduce a separate directory for initrd presets which
is used to pick up preset files from when we detect that we're configuring
an initrd (by looking for /etc/initrd-release).
We also introduce a systemd preset file for the initrd, which is based on
the system one, except with all the stuff unnecessary for the initrd removed.
Follow-up for dbef4dd4f2. Everything that that
commit says is true, but — at least for me — it wasn't obvious why the code is
correct and we can do fixed-size allocations like new(struct inotify_data, 1).
In sd_event_source.child, we have 5 bools. If we make them each take one byte,
the structure size increases. So let's do that for the three of them, and leave
the other two (less frequently used) squished into the last byte. This allows
more efficient code to be generated, without changing the size of the struct:
$ diff -u <(objdump -S build/src/shared/libsystemd-shared-258.so.0|awk '/^static void source_io_unregister/,/^\}/') \
<(objdump -S build/src/shared/libsystemd-shared-258.so|awk '/^static void source_io_unregister/,/^\}/')
s->io.registered = false;
- 34d46f: 48 8b 45 d8 mov -0x28(%rbp),%rax
- 34d473: 0f b6 90 a4 00 00 00 movzbl 0xa4(%rax),%edx
- 34d47a: 83 e2 fe and $0xfffffffe,%edx
- 34d47d: 88 90 a4 00 00 00 mov %dl,0xa4(%rax)
- 34d483: eb 04 jmp 34d489 <source_io_unregister+0x1ca>
+ 34bffe: 48 8b 45 d8 mov -0x28(%rbp),%rax
+ 34c002: c6 80 a4 00 00 00 00 movb $0x0,0xa4(%rax)
+ 34c009: eb 04 jmp 34c00f <source_io_unregister+0x1be>
return;
We're still discussing whether we want to remove this or not, but
regardless of whether we end up removing it or not, it's something
we add ourselves and as such should not live in our headers that
override glibc headers, so let's move it to fd-util.h.
We use symbols provided by unistd.h without including it. E.g.
open(), close(), read(), write(), access(), symlink(), unlink(), rmdir(),
fsync(), syncfs(), lseek(), ftruncate(), fchown(), dup2(), pipe2(),
getuid(), getgid(), gettid(), getppid(), pipe2(), execv(), _exit(),
environ, STDIN_FILENO, STDOUT_FILENO, STDERR_FILENO, F_OK, and their
friends and variants, so on.
Currently, unistd.h is indirectly included mainly in the following two paths:
- through missing_syscall.h, which is planned to covert to .c file.
- through signal.h -> bits/sigstksz.h, which is new since glibc-2.34.
Note, signal.h is included by sd-eevent.h. So, many source files
indirectly include unistd.h if newer glibc is used.
Currently, our baseline on glibc is 2.31. We need to support glibc older
than 2.34, but unfortunately, we do not have any CI environments with
such old glibc. CIFuzz uses glibc-2.31, but it builds only fuzzers, and
many files are even not compiled.
[1] says:
> Since 0.60.0 the name argument is optional and defaults to the basename of
> the first output
We specify >= 0.62 as the supported version, so drop the duplicate name in all cases
where it is the same as outputs[0], i.e. almost all cases.
[1] https://mesonbuild.com/Reference-manual_functions.html#custom_target
Otherwise, the result may point to outside of the root directory.
This also sets CONF_FILES_REGULAR and CONF_FILES_FILTER_MASKED, as the
callers will call fopen() or friends for the result, thus the enumerated
results must be non-empty regular files.
The check existed for musl. Let's remove it, as we explicitly request glibc.
While removing the check, this also drops generic_mallinfo, introduces
a tiny converter from struct mallinfo to struct mallinfo2 if mallinfo2()
does not exist, and renames mallinfo-util.h to malloc.h.
With this change, we can drop many ifdefs and casts in .c files.
Michal Sekletar
Luca Boccassi
Zbigniew Jędrzejewski-Szmek
Yu Watanabe
DaanDeMeyer
Ubuntu
Mike Yuan
Luca Boccassi
Lennart Poettering