Commit Graph
2867 Commits
Author SHA1 Message Date
Florian Westphal 79dd224093 add ipv6 range element creation test cases
The nft set backend doesn't support network masks, it works with
ranges.  Inputs like dead::/64 thus need to be translated to two
'start' and 'end' elements.

The 'start' element is the first element in the range (i.e., dead::).
The 'stop' element is the first element *past* the range, (dead:0:0100::
in the example).

This adds a few test cases.
2021-01-19 16:38:56 +01:00
Florian Westphal 0e544221c9 firewall-util: add ipv6 support to nftables backend
closely mirrors the existing ipv4 ruleset:
table ip6 io.systemd.nat {
        set masq_saddr {
                type ipv6_addr
                flags interval
        }

        map map_port_ipport {
                type inet_proto . inet_service : ipv6_addr . inet_service
        }

        chain prerouting {
                type nat hook prerouting priority dstnat + 1; policy accept;
                fib daddr type local dnat ip6 addr . port to meta l4proto . th dport map @map_port_ipport
        }

        chain output {
                type nat hook output priority -99; policy accept;
                ip6 daddr != ::1 oif "lo" dnat ip6 addr . port to meta l4proto . th dport map @map_port_ipport
        }

        chain postrouting {
                type nat hook postrouting priority srcnat + 1; policy accept;
                ip6 saddr @masq_saddr masquerade
        }
}

Only difference is the use of ipv6 addresses instead of ipv4 ones.

Currently has no effect: all in-tree callers pass AF_INET exclusively.
Followup patches will make nspawn expose ipv6 too and rework
IPMasquerade option to support both/v4/v6.
2021-01-19 16:22:58 +01:00
Zbigniew Jędrzejewski-Szmek 30927a2484 Allow control characters in environment variable values
So far, we would allow certain control characters (NL since
b4346b9a77, TAB since 6294aa76d8), but not others. Having
other control characters in environment variable *value* is expected and widely
used, for various prompts like $LESS, $LESS_TERMCAP_*, and other similar
variables. The typical environment exported by bash already contains a dozen or
so such variables, so programs need to handle them.

We handle then correctly too, for example in 'systemctl show-environment',
since 804ee07c13. But we would still disallow setting such variables
by the user, in unit file Environment= and in set-environment/import-environment
operations. This is unexpected and confusing and doesn't help with anything
because such variables are present in the environment through other means.

When printing such variables, 'show-environment' escapes all special
characters, so variables with control characters are plainly visible.
In other uses, e.g. 'cat -v' can be used in similar fashion. This would already
need to be done to suppress color codes starting with \[.

Note that we still forbid invalid utf-8 with this patch. (Control characters
are valid, since they are valid 7-bit ascii.) I'm not sure if we should do
that, but since people haven't been actually asking for invalid utf-8, and only
for control characters, and invalid utf-8 causes other issues, I think it's OK
to leave this unchanged.

Fixes #4446, https://gitlab.gnome.org/GNOME/gnome-session/-/issues/45.
2021-01-19 14:18:34 +01:00
Yu Watanabe e025c9dfe3 meson: move test or fuzzer definitions to relevant meson.build in subdirectories 2021-01-19 07:06:32 +09:00
Yu Watanabe 04e7ada0d7 meson: drop unnecessary files from test definitions 2021-01-19 07:06:32 +09:00
Yu Watanabe 74b28f2b36 meson: drop unnecessary libraries from journal related tests 2021-01-19 07:06:32 +09:00
Yu Watanabe 8d40961c3c meson: slightly disentangle code dependencies
But, still sd-id128 is used in src/basic.
2021-01-19 07:06:32 +09:00
Yu Watanabe 0275e918af meson: move source file list for systemd-xdg-autostart-generator and its tests 2021-01-19 07:06:32 +09:00
Yu Watanabe d33b0daa6c meson: use condition argument in test definition 2021-01-19 07:06:32 +09:00
Yu Watanabe 5acb3cabd1 meson: make the second and third elements of tests or fuzzers optional
Then, we can shorten many test definitions.
2021-01-19 07:04:19 +09:00
Yu Watanabe 9aeec01e77 test: move test-umount.c to src/shutdown
Then, we can drop src/shutdown from include directories later.
2021-01-19 07:04:19 +09:00
Yu Watanabe bac11cd667 core: move several source files to src/shared
As, the files are used by many executables, fstab-generator, remount-fs,
machine-id-setup, and etc.
With this change, the total size of the relevant executables and
libshared slightly decreases.

Before:
```
$ ll systemd-fstab-generator systemd-remount-fs systemd-machine-id-setup systemd-shutdown systemd-nspawn systemd src/shared/libsystemd-shared-247.so
-rwxrwxr-x 1 watanabe watanabe 7577800 Jan  5 13:35 src/shared/libsystemd-shared-247.so
-rwxrwxr-x 1 watanabe watanabe 4608360 Jan  5 13:35 systemd
-rwxrwxr-x 1 watanabe watanabe  117240 Jan  5 13:35 systemd-fstab-generator
-rwxrwxr-x 1 watanabe watanabe   61576 Jan  5 13:35 systemd-machine-id-setup
-rwxrwxr-x 1 watanabe watanabe  853080 Jan  5 13:35 systemd-nspawn
-rwxrwxr-x 1 watanabe watanabe   70600 Jan  5 13:35 systemd-remount-fs
-rwxrwxr-x 1 watanabe watanabe  172624 Jan  5 13:35 systemd-shutdown
```
Total: 13461280

After:
```
$ ll systemd-fstab-generator systemd-remount-fs systemd-machine-id-setup systemd-shutdown systemd-nspawn systemd src/shared/libsystemd-shared-247.so
-rwxrwxr-x 1 watanabe watanabe 7658336 Jan  5 13:32 src/shared/libsystemd-shared-247.so
-rwxrwxr-x 1 watanabe watanabe 4523560 Jan  5 13:32 systemd
-rwxrwxr-x 1 watanabe watanabe   78288 Jan  5 13:32 systemd-fstab-generator
-rwxrwxr-x 1 watanabe watanabe   30984 Jan  5 13:32 systemd-machine-id-setup
-rwxrwxr-x 1 watanabe watanabe  840384 Jan  5 13:32 systemd-nspawn
-rwxrwxr-x 1 watanabe watanabe   39104 Jan  5 13:32 systemd-remount-fs
-rwxrwxr-x 1 watanabe watanabe  117160 Jan  5 13:32 systemd-shutdown
```
Total: 13287816
2021-01-19 07:04:19 +09:00
Yu Watanabe 772e0a76f3 test: rewrite test-sd-device-thread.c to depend only on libsystemd
Similarly, this makes test-udev-device-thread.c depend only on libudev.
2021-01-19 07:04:19 +09:00
Yu Watanabe 261fc53b99 test: move tests for libudev into src/libudev 2021-01-19 07:04:19 +09:00
Yu Watanabe e2e7888191 xdg: move tests for xdg-autostart-generator
Then, we can drop src/xdg-autostart-generator from include directories
later.
2021-01-19 07:04:19 +09:00
Yu Watanabe 0703d2f6c4 test: move test-systemd-tmpfiles.py from src/test to test
As the other test scripts, e.g. test-sysusers.sh, are located under
test rather than src/test.
2021-01-19 07:04:19 +09:00
Yu Watanabe 100fd93f55 tmpfiles: move offline-passwd.[ch] to src/tmpfiles
offline-passwd.[ch] are only used by systemd-tmpfiles and the relevant
test. And are not included in libshared. So, it is not suitable to
located under src/shared.
2021-01-19 07:04:18 +09:00
Yu Watanabe 99b9f8fddd sd-journal: move source files for sd-journal to src/libsystemd/sd-journal 2021-01-19 07:04:18 +09:00
Luca Boccassi 5e8deb94c6 core: add DBUS method to bind mount new nodes without service restart
Allow to setup new bind mounts for a service at runtime (via either
DBUS or a new 'systemctl bind' verb) with a new helper that forks into
the unit's mount namespace.
Add a new integration test to cover this.

Useful for zero-downtime addition to services that are running inside
mount namespaces, especially when using RootImage/RootDirectory.

If a service runs with a read-only root, a tmpfs is added on /run
to ensure we can create the airlock directory for incoming mounts
under /run/host/incoming.
2021-01-18 17:24:05 +00:00
Lucas Werkmeister 8d7dab1fda Add truncate: to StandardOutput= etc.
This adds the ability to specify truncate:PATH for StandardOutput= and
StandardError=, similar to the existing append:PATH. The code is mostly
copied from the related append: code. Fixes #8983.
2021-01-15 09:54:50 +01:00
Frantisek Sumsal f1a8fed286 ci: skip test-execute on GH Actions under ASan
It seems to suffer from the same issue as on Travis CI, where the test
randomly fails due to timeouts in its subtests.

See: https://github.com/systemd/systemd/issues/10696#issuecomment-758501797
2021-01-13 21:59:48 +01:00
Luca Boccassi 9ca600e2bf bpf: do not use structured initialization for bpf_attr
It looks like zero'ing the struct is not enough, and with some level
of optimizations there is still non-zero padding left over.
Switch to member-by-member initialization. Also convert all remaining
bpf_attr variables in other files.
2021-01-10 21:16:38 +00:00
Lennart Poettering 84319c5c07 test-xattr-util: don't insist that /usr supports xattrs
(apparently overlayfs has issues with xattrs, hence don't require that
/usr supports xattrs)
2021-01-09 21:31:25 +01:00
Luca Boccassi 28abf5ad34 bpf: zero bpf_attr before initialization
When building with Clang and using structured initialization, the
bpf_attr union is not zero-padded, so the kernel misdetects it as
an unsupported extension.
zero it until Clang's behaviour matches GCC. Do not skip the test
on Github Actions anymore.
2021-01-09 17:35:38 +01:00
Frantisek Sumsal d1b74295e8 test: temporarily skip failing tests on GH Actions 2021-01-08 11:37:01 +01:00
Lennart Poettering c2bc710b24 string-util: imply NULL termination of strextend() argument list
The trailing NULL in the argument list is now implied (similar to
what we already have in place in strjoin()).
2021-01-06 17:24:46 +01:00
Luca Boccassi 8bab802910 basic: add make_mount_point_inode helper
Creates a file or a directory depending on the source path, useful
for creating mount points.
2020-12-21 21:48:13 +01:00
Yu Watanabe 458610429f tree-wide: fix typo 2020-12-18 12:59:29 +09:00
Lennart Poettering a60d5b2f38 test: add tpm2 and fido2 libs to dlopen test 2020-12-17 20:02:24 +01:00
Lennart Poettering 73d874bacd homed: move homectl's recovery key generation/modhex code to src/shared/
This allows us to later reuse the code to generate recovery keys for
traditional LUKS volumes, too and share the code.
2020-12-17 19:57:18 +01:00
Lennart Poettering 95231c7215 test: fix fd_is_mount_point() check
So the currentl and only fd_is_mount_point() check is actually entirely
bogus: it passes "/" as filename argument, but that's not actually a
a valid filename, but an absolute path.

fd_is_mount_point() is written in a way tha the fd refers to a directory
and the specified path is a file directly below it that shall be
checked. The test call actually violated that rule, but still expected
success.

Let's fix this, and check for this explicitly, and refuse it.

Let's extend the test and move it to test-mountpoint-util.c where the
rest of the tests for related calls are placed.

Replaces: #18004
Fixes: #17950
2020-12-17 19:29:24 +01:00
Lennart Poettering a8af734e75 Merge pull request #17026 from fw-strlen/nft_16
add networkd/nspawn nftables backend
2020-12-16 19:18:22 +01:00
Zbigniew Jędrzejewski-Szmek b6fad30665 shared/hostname-setup: add mode where we check what would be set, without doing
This allows the 'unsafe' mark to be removed from the test.
2020-12-16 11:02:17 +01:00
Zbigniew Jędrzejewski-Szmek e2054217d5 Move hostname setup logic to new shared/hostname-setup.[ch]
No functional change, just moving a bunch of things around. Before
we needed a rather complicated setup to test hostname_setup(), because
the code was in src/core/. When things are moved to src/shared/
we can just test it as any function.

The test is still "unsafe" because hostname_setup() may modify the
hostname.
2020-12-16 11:02:15 +01:00
Yu Watanabe a7c52e1ee1 Merge pull request #17968 from yuwata/purge-libudev
udev: do not link with libudev
2020-12-16 12:07:46 +09:00
Florian Westphal 715a70e721 firewall-util: add nftables backend
Idea is to use a static ruleset, added when the first attempt to
add a masquerade or dnat rule is made.

The alternative would be to add the ruleset when the init function is called.
The disadvantage is that this enables connection tracking and NAT in the kernel
(as the ruleset needs this to work), which comes with some overhead that might
not be needed (no nspawn usage and no IPMasquerade option set).

There is no additional dependency on the 'nft' userspace binary or other libraries.
sd-netlinks nfnetlink backend is used to modify the nftables ruleset.

The commit message/comments still use nft syntax since that is what
users will see when they use the nft tool to list the ruleset.

The added initial skeleton (added on first fw_add_masquerade/local_dnat
call) looks like this:

table ip io.systemd.nat {
        set masq_saddr {
                type ipv4_addr
                flags interval
                elements = { 192.168.59.160/28 }
        }

        map map_port_ipport {
                type inet_proto . inet_service : ipv4_addr . inet_service
                elements = { tcp . 2222 : 192.168.59.169 . 22 }
        }

        chain prerouting {
                type nat hook prerouting priority dstnat + 1; policy accept;
                fib daddr type local dnat ip addr . port to meta l4proto . th dport map @map_port_ipport
        }

        chain output {
                type nat hook output priority -99; policy accept;
                ip daddr != 127.0.0.0/8 oif "lo" dnat ip addr . port to meta l4proto . th dport map @map_port_ipport
        }

        chain postrouting {
                type nat hook postrouting priority srcnat + 1; policy accept;
                ip saddr @masq_saddr masquerade
        }
}

Next calls to fw_add_masquerade/add_local_dnat will then only add/delete the
element/mapping to masq_saddr and map_port_ipport, i.e. the ruleset doesn't
change -- only the set/map content does.

Running test-firewall-util with this backend gives following output
on a parallel 'nft monitor':

$ nft monitor
add table ip io.systemd.nat
add chain ip io.systemd.nat prerouting { type nat hook prerouting priority dstnat + 1; policy accept; }
add chain ip io.systemd.nat output { type nat hook output priority -99; policy accept; }
add chain ip io.systemd.nat postrouting { type nat hook postrouting priority srcnat + 1; policy accept; }
add set ip io.systemd.nat masq_saddr { type ipv4_addr; flags interval; }
add map ip io.systemd.nat map_port_ipport { type inet_proto . inet_service : ipv4_addr . inet_service; }
add rule ip io.systemd.nat prerouting fib daddr type local dnat ip addr . port to meta l4proto . th dport map @map_port_ipport
add rule ip io.systemd.nat output ip daddr != 127.0.0.0/8 fib daddr type local dnat ip addr . port to meta l4proto . th dport map @map_port_ipport
add rule ip io.systemd.nat postrouting ip saddr @masq_saddr masquerade
add element ip io.systemd.nat masq_saddr { 10.1.2.3 }
add element ip io.systemd.nat masq_saddr { 10.0.2.0/28 }
delete element ip io.systemd.nat masq_saddr { 10.0.2.0/28 }
delete element ip io.systemd.nat masq_saddr { 10.1.2.3 }
add element ip io.systemd.nat map_port_ipport { tcp . 4711 : 1.2.3.4 . 815 }
delete element ip io.systemd.nat map_port_ipport { tcp . 4711 : 1.2.3.4 . 815 }
add element ip io.systemd.nat map_port_ipport { tcp . 4711 : 1.2.3.5 . 815 }
delete element ip io.systemd.nat map_port_ipport { tcp . 4711 : 1.2.3.5 . 815 }
CTRL-C

Things not implemented/supported:
1. Change monitoring.  The kernel allows userspace to learn about changes
   made by other clients (using nfnetlink notifications). It would be
   possible to detect when e.g. someone removes the systemd nat table.
   This would need more work.  Its also not clear on how to react to
   external changes -- it doesn't seem like a good idea to just auto-undo
   everthing.
2. 'set masq_saddr' doesn't handle overlaps.
   Example:

   fw_add_masquerade(true, AF_INET, "10.0.0.0" , 16);
   fw_add_masquerade(true, AF_INET, "10.0.0.0" , 8); /* fails */

With the iptables backend the second call works, as it adds an
independent iptables rule.

With the nftables backend, the range 10.0.0.0-10.255.255.255 clashes with
the existing range of 10.0.0.0-10.0.255.255 so 2nd add gets rejected by the
kernel.

This will generate an error message from networkd ("Could not enable IP
masquerading: File exists").

To resolve this it would be needed to either keep track of the added elements
and perform range merging when overlaps are detected.

However, the add erquests are done using the configured network on a
device, so no overlaps should occur in normal setups.

IPv6 support is added in a extra changeset.

Fixes: #13307
2020-12-16 01:07:08 +01:00
Florian Westphal 761cf19d7b firewall-util: introduce context structure
for planned nft backend we have three choices:

- open/close a new nfnetlink socket for every operation
- keep a nfnetlink socket open internally
- expose a opaque fw_ctx and stash all internal data here.

Originally I opted for the 2nd option, but during review it was
suggested to avoid static storage duration because of perceived
problems with threaded applications.

This adds fw_ctx and new/free functions, then converts the existing api
and nspawn and networkd to use it.
2020-12-16 00:35:56 +01:00
Florian Westphal 47ed20e1e0 firewall-util: reject NULL source or address with prefixlen 0
Make sure we don't add masquerading rules without a explicitly
specified network range we should be masquerading for.

The only caller aside from test case is
networkd-address.c which never passes a NULL source.

As it also passes the network prefix, that should always be > 0 as well.

This causes expected test failure:
Failed to modify firewall: Invalid argument
Failed to modify firewall: Invalid argument
Failed to modify firewall: Invalid argument
Failed to modify firewall: Protocol not available
Failed to modify firewall: Protocol not available
Failed to modify firewall: Protocol not available
Failed to modify firewall: Protocol not available

The failing test cases are amended to expect failure on
NULL source or prefix instead of success.
2020-12-16 00:35:56 +01:00
Yu Watanabe e4b127e2cc meson: do not link with libudev 2020-12-16 02:31:44 +09:00
Yu Watanabe 1223227f95 udev: move util_resolve_subsys_kernel() to udev-util.c 2020-12-16 02:14:47 +09:00
Yu Watanabe 5953d8b910 udev: move util_replace_whitespace() to udev-util.c 2020-12-16 02:14:47 +09:00
Lennart Poettering 52ef5dd798 hostname-util: flagsify hostname_is_valid(), drop machine_name_is_valid()
Let's clean up hostname_is_valid() a bit: let's turn the second boolean
argument into a more explanatory flags field, and add a flag that
accepts the special name ".host" as valid. This is useful for the
container logic, where the special hostname ".host" refers to the "root
container", i.e. the host system itself, and can be specified at various
places.

let's also get rid of machine_name_is_valid(). It was just an alias,
which is confusing and even more so now that we have the flags param.
2020-12-15 17:59:48 +01:00
Yu Watanabe 8d80f27530 sd-device: make TAGS= property prefixed and suffixed with ":"
The commit 6f3ac0d517 drops the prefix and
suffix in TAGS= property. But there exists several rules that have like
`TAGS=="*:tag:*"`. So, the property must be always prefixed and suffixed
with ":".

Fixes #17930.
2020-12-14 14:04:53 +09:00
Yu Watanabe 1a0bd01529 udev: introduce new OPTIONS="log_level=" udev rule 2020-12-10 12:31:45 +09:00
Lennart Poettering 68fdc72cfe test: add test that dlopen()'s all our weak library deps once
This test should ensure we notice if distros update shared libraries
that broke so name, and we still use the old soname.

(In contrast to what the commit summary says, this currently doesn#t
cover really all such deps, specifically xkbcommon and PCRE are missing,
since they currently aren't loaded from src/shared/. This is stuff to
fix later)
2020-12-09 13:34:40 +01:00
Yu Watanabe 4dbce71787 set: introduce set_strjoin() 2020-12-08 12:28:54 +09:00
Zbigniew Jędrzejewski-Szmek 7e0079f95f Make support for nscd flushing optional
Fedora will deprecate support for nscd in the upcoming release [1] and plans to
drop it in the next one [2]. At that point we might as well build systemd
without that support too, since there'll be nothing to talk too.

[1] https://fedoraproject.org/wiki/Changes/DeprecateNSCD
[2] https://fedoraproject.org/wiki/Changes/RemoveNSCD
2020-12-07 19:46:02 +01:00
Lennart Poettering 6dd16814a5 Merge pull request #17079 from keszybz/late-exec-resolution
Resolve executable paths before execution, use fexecve()
2020-12-03 14:58:20 +01:00
Lennart Poettering 03d8ebb935 Merge pull request #17812 from poettering/systemctl-version-feature-update
build.h: add a bunch of missing features strings
2020-12-03 14:34:08 +01:00
Zbigniew Jędrzejewski-Szmek 91b79ba84e shared/build: make the version string definition less terrible
The BLKID and ELFUTILS strings were present twice. Let's reaarange things so that
each times requires definition in exactly one place.

Also let's sort things a bit:
the "heavy hitters" like PAM/MAC first,
then crypto libs,
then other libs, alphabetically,
compressors,
and external compat integrations.

I think it's useful for users to group similar concepts together to some extent.
For example, when checking what compression is available, it helps a lot to have
them listed together.

FDISK is renamed to LIBFDISK to make it clear that this is about he library and
the executable.
2020-12-03 12:16:02 +01:00