Let's raise our supported baseline a bit: CLOCK_BOOTTIME started to work
with timerfd in kernel 3.15 (i.e. back in 2014), let's require support
for it now.
This will raise our baseline only modestly from 3.13 → 3.15.
The meson default for static_library() are:
build_by_default=true, install=false. We never interact with the
static libraries, and we only care about them as a stepping-stone towards
the installable executables or libraries. Thus let's only build them if
they are a dependency of something else we are building.
While at it, let's drop install:false, since this appears to be the default.
This change would have fixed the issue with lib_import_common failing
to build too: we wouldn't attempt to build it.
In practice this changes very little, because we generally only declare static
libraries where there's something in the default target that will make use of
them. But it seems to be a better pattern to set build_by_default to false.
Use a 'convenience library' to do the compilation once and then link the
objects into all the files that need it. Those files are small, so this probably
doesn't matter too much for speed, but has the advantage that we don't get the
same error four times if something goes wrong.
The library is conditionalized in the same way importd itself, because we
cannot build it without the deps.
The variable is not useful outside of the loop (it'll always be null
after the loop is finished), so we can declare it inline in the loop.
This saves one variable declaration and reduces the chances that somebody
tries to use the variable outside of the loop.
For consistency, 'de' is used everywhere for the var name.
This is heavily based on Kevin Kuehler's work, but the logic is also
significantly changed: instead of a straighforward port to openssl, both
versions of the code are kept, and at compile time we pick one or the other.
The code is purposefully kept "dumb" — the idea is that the libgcrypt codepaths
are only temporary and will be removed after everybody upgrades to openssl 3.
Thus, a separate abstraction layer is not introduced. Instead, very simple
ifdefs are used to select one or the other. If we added an abstraction layer,
we'd have to remove it again afterwards, and it don't think it makes sense to
do that for a temporary solution.
Co-authored-by: Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
# Conflicts:
# meson.build
Previously the mkdir_label() family of calls was implemented in
src/shared/mkdir-label.c but its functions partly declared ins
src/shared/label.h and partly in src/basic/mkdir.h (!!). That's weird
(and wrong).
Let's clean this up, and add a proper mkdir-label.h matching the .c
file.
rearrange_stdio() invalidates specified fds even on failure, which means
we should always invalidate the fds we pass in no matter what. Let's
make this explicit by using TAKE_FD() for that everywhere.
Note that in many places we such invalidation doesnt get us much
behaviour-wise, since we don't use the variables anymore later. But
TAKE_FD() in a way is also documentation, it encodes explicitly that the
fds are invalidated here, so I think it's a good thing to always make
this explicit here.
Only if we create a file we know for sure that it is empty and hence our
sparse file logic of skipping over NUL bytes can can work. If we hwoever
are called to write data to some existing file/block device, we must do
regular writes to override everything that might be in place before.
Hence, conditionalize sparse file writing on the write offset not being
configured (which is how we internally distinguish write to existing
file and write to new file)
Previously we only allows http/https urls, let's open this up a bit.
Why? Because it makes testing *so* *much* *easier* as we don't need to
run a HTTP server all the time.
CURL mostly abstracts the differences of http/https away from us, hence
we can get away with very little extra work.
Let's lock things down a bit and now allow curl's weirder protocols to
be used with our use. i.e. stick to http:// + https:// + file:// and
turn everything else off. (Gopher!)
This is cde that interfaces with the network after all, and we better
shouldn't support protocols needlessly that are much less tested.
(Given that HTTP redirects (and other redirects) exist, this should give
us a security benefit, since we will then be sure that noone can forward
us to a weird protocol, which we never tested, and other people test
neither)
This basically does what the previous two commits did for systemd-import
+ systemd-pull but for systemd-import-fs.
This commit is a bit simpler though, as a --direct mode doesn't change
that much. It's mostly about not searching for existing, conflicting
images and not much else.
This does what the previous commit did for systemd-import the same way
for systemd-pull.
It also adds one more thing: the checksum validation is extended, in
addition of doing SHA256SUMS/gpg verification it is now possible to
immediately specify a hash value on the command line that the download
needs to match. This is particularly useful in --direct mode as we can
download/decompress/unpack arbitrary files and check the hash of the
downloaded file on-the-fly.
This reworks/modernizes the tar/raw import logic and adds the following
new features:
- Adds the ability to control btrfs subvol and quota behaviour which was
previously always on via an env var and cmdline arg
- Adds control whether to sync() stuff after writing it, similar via env
var + cmdline arg
- Similar, the QCOW2 unpacking logic that was previously the implied
default may now be controlled via env var + cmdline arg.
- adds a "direct" mode. In this mode, the systemd-import tool can be
used as a simple tool for decompressing/unpacking/installing arbitrary
files, without all the additional meta data and auxiliary resources,
i.e. outside of the immediate disk image context. Via the new
--offset= and --size-max= switches the downloaded data can be written
to specific locations of a file (which is particularly useful to use
the tool to download fs images and write them to a partition location
before actually creating the partition).
We'll later use the latter feature for "sysupdate" concept, where images
can be directly be written to partitions. That way the systemd-import
binary will be used as backend for both "systemd-importd" and
"systemd-sysupdate" and share most of the same code.
When the flag COPY_ALL_XATTRS is set, it causes the complete set of xattrs
to be copied. If the flag is unset, only xattrs from the "user" namespace
are copied.
Fixes#17178.
Let's be paranoid and do something useful if we operate with empty
haystack/needle. This doesn't actually fix anything, as the places as
far as I can see check for non-emptyness already beforehand, but I will
sleep safer at night, if we don't even allow the trap to be fallen in,
ever, even if the code is changed sooner or later.
In general we almost never hit those asserts in production code, so users see
them very rarely, if ever. But either way, we just need something that users
can pass to the developers.
We have quite a few of those asserts, and some have fairly nice messages, but
many are like "WTF?" or "???" or "unexpected something". The error that is
printed includes the file location, and function name. In almost all functions
there's at most one assert, so the function name alone is enough to identify
the failure for a developer. So we don't get much extra from the message, and
we might just as well drop them.
Dropping them makes our code a tiny bit smaller, and most importantly, improves
development experience by making it easy to insert such an assert in the code
without thinking how to phrase the argument.
We recently started making more use of malloc_usable_size() and rely on
it (see the string_erase() story). Given that we don't really support
sytems where malloc_usable_size() cannot be trusted beyond statistics
anyway, let's go fully in and rework GREEDY_REALLOC() on top of it:
instead of passing around and maintaining the currenly allocated size
everywhere, let's just derive it automatically from
malloc_usable_size().
I am mostly after this for the simplicity this brings. It also brings
minor efficiency improvements I guess, but things become so much nicer
to look at if we can avoid these allocation size variables everywhere.
Note that the malloc_usable_size() man page says relying on it wasn't
"good programming practice", but I think it does this for reasons that
don't apply here: the greedy realloc logic specifically doesn't rely on
the returned extra size, beyond the fact that it is equal or larger than
what was requested.
(This commit was supposed to be a quick patch btw, but apparently we use
the greedy realloc stuff quite a bit across the codebase, so this ends
up touching *a*lot* of code.)
Using a enum is all nice and generic, but at this point it seems unlikely that
we'll add further build modes. But having an enum means that we need to include
the header file with the enumeration whenerever the conditional is used. I want
to use the conditional in log.h, which makes it hard to avoid circular imports.
Clean up ignore_signals() + default_signals() + sigaction_many() a bit:
make it unnecessary to explicitly terminate the signal list with -1.
Merge all three calls into a single function that is just called with
slightly different parameters. And eliminate an unnecessary extra
iteration in its inner for() loop.
No change in behaviour.
A good chunk of parse-util.[ch] has been about parsing parts per
hundred/thousand/ten-thousand. Let's split that out into its own file.
No code changes, just some shuffling around.
If the cleanup function returns the appropriate type, use that to reset the
variable. For other functions (usually the foreign ones which return void), add
an explicit value to reset to.
This causes a bit of code churn, but I think it might be worth it. In a
following patch static destructors will be called from a fuzzer, and this
change allows them to be called multiple times. But I think such a change might
help with detecting unitialized code reuse too. We hit various bugs like this,
and things are more obvious when a pointer has been set to NULL.
I was worried whether this change increases text size, but it doesn't seem to:
-Dbuildtype=debug:
before "tree-wide: return NULL from freeing functions":
-rwxrwxr-x 1 zbyszek zbyszek 4117672 Feb 16 14:36 build/libsystemd.so.0.30.0*
-rwxrwxr-x 1 zbyszek zbyszek 4494520 Feb 16 15:06 build/systemd*
after "tree-wide: return NULL from freeing functions":
-rwxrwxr-x 1 zbyszek zbyszek 4117672 Feb 16 14:36 build/libsystemd.so.0.30.0*
-rwxrwxr-x 1 zbyszek zbyszek 4494576 Feb 16 15:10 build/systemd*
now:
-rwxrwxr-x 1 zbyszek zbyszek 4117672 Feb 16 14:36 build/libsystemd.so.0.30.0*
-rwxrwxr-x 1 zbyszek zbyszek 4494640 Feb 16 15:15 build/systemd*
-Dbuildtype=release:
before "tree-wide: return NULL from freeing functions":
-rwxrwxr-x 1 zbyszek zbyszek 5252256 Feb 14 14:47 build-rawhide/libsystemd.so.0.30.0*
-rwxrwxr-x 1 zbyszek zbyszek 1834184 Feb 16 15:09 build-rawhide/systemd*
after "tree-wide: return NULL from freeing functions":
-rwxrwxr-x 1 zbyszek zbyszek 5252256 Feb 14 14:47 build-rawhide/libsystemd.so.0.30.0*
-rwxrwxr-x 1 zbyszek zbyszek 1834184 Feb 16 15:10 build-rawhide/systemd*
now:
-rwxrwxr-x 1 zbyszek zbyszek 5252256 Feb 14 14:47 build-rawhide/libsystemd.so.0.30.0*
-rwxrwxr-x 1 zbyszek zbyszek 1834184 Feb 16 15:16 build-rawhide/systemd*
I would expect that the compiler would be able to elide the setting of a
variable if the variable is never used again. And this seems to be the case:
in optimized builds there is no change in size whatsoever. And the change in
size in unoptimized build is negligible.
Something strange is happening with size of libsystemd: it's bigger in
optimized builds. Something to figure out, but unrelated to this patch.
As suggested in https://github.com/systemd/systemd/pull/11484#issuecomment-775288617.
This does not touch anything exposed in src/systemd. Changing the defines there
would be a compatibility break.
Note that tests are broken after this commit. They will be fixed in the next one.
Let's make sure we still look at the etags reported by http 304 (i.e.
the cache management code). Otherwise we won't properly realize we
already downloaded this before.
This fixes a bug introduced in 6792cbbcf8
The old name originates when this was used to discover "machine" images,
as managed by machined/machinectl. But nowadays this is also used by
portable services and system extensions, hence let's use a more generic
name for this API. Taking inspiration from "dissect-image.[ch]", let's call
this "discover-image.[ch]".
This is pure renaming, no other changes.
