Yu Watanabe
2f08761be4
systemd has two UTF-8 decoders, and the console width path uses the
permissive one.
`utf8_encoded_to_unichar()` performs no overlong check, no
`unichar_is_valid()` check, and still accepts the obsolete five and six
byte forms. Those checks live in the wrapper
`utf8_encoded_valid_unichar()`, which is what `utf8_is_valid()` and
`ellipsize_mem()` use. `utf8_char_console_width()` called the bare
decoder directly.
Checking both against `utf8_is_valid_n()` over all 4,244,897,280 NUL
free byte strings of length one to four, which is the entire UTF-8
encoding space:
| decoder | disagreements with `utf8_is_valid_n()` |
| --- | --- |
| `utf8_encoded_valid_unichar()` | 0 |
| `utf8_encoded_to_unichar()` | 8,835,838, always more permissive |
So `utf8_console_width()` returned a finite width for strings
`utf8_is_valid()` rejects. `format-table.c` measures a cell with
`utf8_console_width()` and, when it does not fit, hands it to
`ellipsize()`, which validates strictly and returns NULL. That is
reported as `-ENOMEM`:
```
lead surrogate valid_utf8=0 utf8_console_width=25 ellipsize=NULL
-> table_format FAILS: Cannot allocate memory
```
A surrogate, overlong, noncharacter or five byte sequence in any cell
failed the whole table with a bogus out of memory error, and only once
that column was narrow enough to need truncation, so whether it happened
depended on the width of the user's terminal. At other widths the
invalid sequence was printed to the terminal instead. Invalid cells are
now refused consistently with `-EINVAL`, which is what the sequences
`utf8_encoded_to_unichar()` already rejected always produced.
The second commit fixes a related disagreement. `ellipsize_mem()` and
`align_string_mem()` open coded character width as `unichar_iswide(c) ?
2 : 1`, while `unichar_console_width()` counts a tab as eight cells.
`format-table.c` uses both: it finds a field too wide with
`utf8_console_width()`, then asks `ellipsize()` to fit that many
columns, and `ellipsize()` thought a tab was one cell, so the result
still overflowed the column that the measurement had just asked it to
fit. A table pinned to `table_set_width(20)` emitted lines of 25, 44 and
39 columns. This uses `unichar_console_width()`, which was split out for
exactly this purpose in 5c0d4b5061.
Both commits add tests that fail before the change and pass after. The
full unit suite is otherwise unaffected.
System and Service Manager
Details
Most documentation is available on systemd's website.
Assorted, older, general information about systemd can be found in the systemd Wiki.
Information about build requirements is provided in the README file.
Consult our NEWS file for information about what's new in the most recent systemd versions.
Please see the Code Map for information about this repository's layout and content.
Please see the Hacking guide for information on how to hack on systemd and test your modifications.
Please see our Contribution Guidelines for more information about filing GitHub Issues and posting GitHub Pull Requests.
When preparing patches for systemd, please follow our Coding Style Guidelines.
If you are looking for support, please contact our mailing list, join our IRC channel #systemd on libera.chat or Matrix channel
Stable branches with backported patches are available in the stable repo.
We have a security bug bounty program sponsored by the Sovereign Tech Fund hosted on YesWeHack
Repositories with distribution packages built from git main are available on OBS, and also repositories with packages built from the latest stable release