Add a simple fuzzer that verifies our machinery for parsing user records
from JSON works as intended.
The initial corpus was created with the help of Claude, so we have a
bunch of valid user records with as much fields as possible for the
initial corpus.
Previously, when compiled in developer mode, a call into libsystemd with
invalid parameters would result in an abort. This means that it's effectively
impossible to install such libsystemd in a normal system, since various
third-party programs may now abort. A shared library should generally never
abort or exit the calling program.
In python-systemd, the test suite calls into libsystemd, to check if the proper
return values are received and propagated through the Python wrappers.
Obviously with libsystemd compiled from git, the test suite now fails
in a nasty way.
So rework the code to set assert_return_is_critical similarly to how we handle
mempool enablement: the function that returns true is declared as a week
symbol, and we "opt in" by linking a file that provides the function in
libsystemd-shared. Effectively, libsystemd and libudev always have
assert_return_is_critical==false, and our binaries and modules enable it
conditionally.
Let's not leak details from src/shared and src/libsystemd into
src/basic, even though you can't actually do anything useful with
just forward declarations from src/shared.
The sd-forward.h header is put in src/libsystemd/sd-common as we
don't have a directory for shared internal headers for libsystemd
yet.
Let's also rename forward.h to basic-forward.h to keep things
self-explanatory.
All those files are standalone programs that can be executed directly.
Some .py files were marked executable, others weren't, probably accidentally.
Mark them all as executable in preparation for subsequent changes.
We're planning to do code generation based on the systemd varlink
APIs. To simplify this, let's expose the IDL parser, so we can use
it to do code generation instead of having to write our own IDL
parser.
Now that the necessary functions from log.h have been moved to macro.h,
we can stop including log.h in macro.h. This requires modifying source
files all over the tree to include log.h instead.
Various functions in log.h are only used by asserts, and there's
enough assertion related stuff in macro.h to justify a separate header
which also makes it easier to avoid circular dependencies.
Let's introduce assert-util.h and an accompanying fundamental header
and move all the assertion related stuff over there. PROJECT_FILE is
moved over to macro.h.
- set destructors to catalog_hash_ops,
- acquire OrderedHashmap when necessary,
- gracefully handle NULL catalog directories and output stream,
- rename function output arguments,
- add many many assertions,
- use RET_GATHER().
Fixes https://github.com/systemd/systemd/issues/35508.
As reported in the bug, the values were hardcoded for the systemd project.
https://varlink.org/Service lists vendor, product, version, url, and interfaces
as the mandatory parameters, so add an interface to set the first four. The
last field is set automatically based on the registered interfaces as before.
If the values are not filled in, we return empty strings. With NULL,
'varlinkctl info' would say:
(string):1:25: Object field 'vendor' has wrong type null, expected string.
The bootspec util-lib's handling of global addons didn't previously
match the behavior of sd-stub, and this commit corrects that.
First, bootspec didn't load global addons from the XBOOTLDR dir, but the
stub does. So, bootspec now enumerates addons in XBOOTLDR, not just ESP
Second, the stub only loads resources (including addons) from the
partition that it was found on. Thus, we must keep track of which
partition the global addons come from, and which partition each boot
entry comes from. In other words: global addons found on the ESP will
NOT apply to UKIs found in XBOOTLDR, and bootspec now reflects that.
It's time. sd-json was already done earlier in this cycle, let's now
make sd-varlink public too.
This is mostly just a search/replace job of epical proportions.
I left some functions internal (mostly IDL handling), and I turned some
static inline calls into regular calls.
This is preparation for making our Varlink API a public API. Since our
Varlink API is built on top of our JSON API we need to make that public
first (it's a nice API, but JSON APIs there are already enough, this is
purely about the Varlink angle).
I made most of the json.h APIs public, and just placed them in
sd-json.h. Sometimes I wasn't so sure however, since the underlying data
structures would have to be made public too. If in doubt I didn#t risk
it, and moved the relevant API to src/libsystemd/sd-json/json-util.h
instead (without any sd_* symbol prefixes).
This is mostly a giant search/replace patch.
Several test cases intentionally trigger assert_return(). So, to avoid
the entire test fails, this introduces several macros that tentatively
make assert_return() not critical.
varlink_dispatch() is a simple wrapper around json_dispatch() that
returns clean, standards-compliant InvalidParameter error back to
clients, if the specified JSON cannot be parsed properly.
For this json_dispatch() is extended to return the offending field's
name. Because it already has quite a few parameters, I then renamed
json_dispatch() to json_dispatch_full() and made json_dispatch() a
wrapper around it that passes the new argument as NULL. While doing so I
figured we should also get rid of the bad= argument in the short
wrapper, since it's only used in the OCI code.
To simplify the OCI code this adds a second wrapper oci_dispatch()
around json_dispatch_full(), that fills in bad= the way we want.
Net result: instead of one json_dispatch() call there are now:
1. json_dispatch_full() for the fully feature mother of all dispathers.
2. json_dispatch() for the simpler version that you want to use most of
the time.
3. varlink_dispatch() that generates nice Varlink errors
4. oci_dispatch() that does the OCI specific error handling
And that's all there is.
We use it for more than just pipe() arrays. For example also for
socketpair(). Hence let's give it a generic name.
Also add EBADF_TRIPLET to mirror this for things like
stdin/stdout/stderr arrays, which we use a bunch of times.
Even more than with the previous commit, this is not a trivial function
and there's no reason to believe this will actually be inlined nor that
it would be beneficial.
This is preparation for #28891, which adds a bunch more helpers around
"struct iovec", at which point this really deserves its own .c/.h file.
The idea is that we sooner or later can consider "struct iovec" as an
entirely generic mechanism to reference some binary blob, and is the
go-to type for this purpose whenever we need one.
json_append() is a useful wrapper around json_variant_merge(). However,
I think the naming sould be cleaned up a bit of both functions.
I thinker "merge" is the better word than "append", since it does
decidedly more than just append: it replaces existing fields of the same
name, hence "merge" sounds more appropriate. This is as opposed to the
similar operations for arrays, where no such override logic is applied
and we really just append, hence those functions are called "append"
already.
To make clearer that "merge" is about objects, and "append" about
arrays, also include "object" in the name.
Also, include "json_variant" in the name, like we do for almost all
other functions in the JSON API that take a JSON object as primary
input, and hence are kinda object methods.
Finally, let's follow the logic that helpers that combine json_build()
with some other operation get suffixed with "b" like we already have in
some cases.
Hence:
json_variant_merge() → json_variant_merge_object()
json_append() → json_variant_merge_objectb()
This mirrors nicely the existing:
json_variant_append_array()
json_vairant_append_arrayb()
This also drops the variant of json_append() that takes a va_arg
parameter (i.e. json_appendv()). We have no user of that so far, and
given the nature as a helper function only I don#t see that happening,
and if it happens after all it's trivial to bring back.
In case one of the allocations fails.
For example:
AddressSanitizer:DEADLYSIGNAL
=================================================================
==17==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7fb352a476e5 bp 0x7ffe45154850 sp 0x7ffe45154008 T0)
==17==The signal is caused by a READ memory access.
==17==Hint: address points to the zero page.
SCARINESS: 10 (null-deref)
#0 0x7fb352a476e5 (/lib/x86_64-linux-gnu/libc.so.6+0x1886e5) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee)
#1 0x435878 in __interceptor_strlen /src/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc
#2 0x4de1e4 in LLVMFuzzerTestOneInput /work/build/../../src/systemd/src/fuzz/fuzz-calendarspec.c:20:21
#3 0x4deea8 in NaloFuzzerTestOneInput (/build/fuzz-calendarspec+0x4deea8)
#4 0x4fde33 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:611:15
#5 0x4fd61a in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:514:3
#6 0x4fece9 in fuzzer::Fuzzer::MutateAndTestOne() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:757:19
#7 0x4ff9b5 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile> >&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:895:5
#8 0x4eed1f in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:912:6
#9 0x4ef5e8 in LLVMFuzzerRunDriver /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:925:10
#10 0x4df105 in main (/build/fuzz-calendarspec+0x4df105)
#11 0x7fb3528e3082 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24082) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee)
#12 0x41f80d in _start (/build/fuzz-calendarspec+0x41f80d)
Found by Nallocfuzz.
-1 was used everywhere, but -EBADF or -EBADFD started being used in various
places. Let's make things consistent in the new style.
Note that there are two candidates:
EBADF 9 Bad file descriptor
EBADFD 77 File descriptor in bad state
Since we're initializating the fd, we're just assigning a value that means
"no fd yet", so it's just a bad file descriptor, and the first errno fits
better. If instead we had a valid file descriptor that became invalid because
of some operation or state change, the other errno would fit better.
In some places, initialization is dropped if unnecessary.
The man page says nothing about "e". Glibc clearly accepts it without fuss, but
it is meaningless for a memory object (and probably doesn't work). This use is
not portable, so let's avoid it.
The compiler may decide computations like these are not doing anything
and decide to optimize them away. This would defeat the whole fuzzing
exercise. This macro will force the compiler to materialize the value
no matter what. It should be less prone to accidents compared to using
log functions, which would either slow things down or still optimize the
value away (or simply move it into the if branch the log macros create).
The benefit over assert_se would be that no requirement is made on the
value itself. If we are fine getting a string of any size (including
zero), an assert_se would either create a noisy compiler warning about
conditions that would alawys be met or yet again optimize the whole
thing away.
Yu Watanabe
Frantisek Sumsal
Zbigniew Jędrzejewski-Szmek
Daan De Meyer
Jelle van der Waa
Lennart Poettering
Andrii Chubatiuk
Adrian Vovk
Jan Janssen
Li kunyu
David Tardon
Luca Boccassi