diff options
| author | Gao Xiang <xiang@kernel.org> | 2026-09-03 22:28:41 +0800 |
|---|---|---|
| committer | Gao Xiang <xiang@kernel.org> | 2026-09-03 22:49:35 +0800 |
| commit | 82e664cf1219c459c33aae931b222cf951af9cb7 (patch) | |
| tree | 47c3127499c2398ee7ec0676a8e176267cf603a3 /scripts/basic | |
| parent | 617d0d8d199ba1790c94310fd75a22d01c97a8d6 (diff) | |
| download | linux-82e664cf1219c459c33aae931b222cf951af9cb7.tar.gz linux-82e664cf1219c459c33aae931b222cf951af9cb7.zip | |
erofs: disable LZ4 rolling decompression for now
LZ4 rolling decompression [1] was introduced to reduce the memory
footprint of temporary pages:
For many cases, it is needed for users to read small data within
a compressed extent (pcluster), either due to random small read, or
since uptodate folios (typically order-0) cannot be reused for
decompression again since decompression algorithm refills
already-uptodate folios.
Rolling decompression works because LZ4 is LZ77-based and only refers
to the most recent 64 KiB of decompressed data, so in theory only a
bounded rolling window of temporary pages is needed when decompressing.
It can save a lot of temporary memory, e.g.
601,960-byte data can be compressed into a 256k LZ4 compressed extent,
which means it needs 146 extra pages per request in the worst case if
rolling decompression is disabled.
However, the upstream LZ4 implementation is not under EROFS' control:
For example, the literal copy memmove() may still **copy long literals
backward** on x86 based on the address comparison even when the source
and destination ranges do not overlap (IOWs, inline decompression
doesn't need to be considered here). That breaks the rolling assumption
and makes the optimization broken.
Disable it for now to make sure the data correctness first since EROFS
is used everywhere now: The rolling window approach can be revived once
we either ensure that the official LZ4 code always copies forward for
non-overlapping ranges or maintain our own LZ4 implementation in EROFS.
The main impact is a higher runtime memory footprint; However, recent
commit 0f6273ab4637 ("erofs: add a reserved buffer pool for lz4
decompression") helps mitigate this when enabled but it's still not
perfect.
[1] https://www.usenix.org/conference/atc19/presentation/gao
ยง 3.3 Decompression
Reported-by: "Walther, Jens-Uwe" <waltju@amazon.de>
Closes: https://lore.kernel.org/r/BEZP281MB2102E57CD31862B8D958B33DD2AC2@BEZP281MB2102.DEUP281.PROD.OUTLOOK.COM
Fixes: 8e6c8fa9f2e9 ("erofs: enable big pcluster feature")
Cc: Yann Collet <yann.collet.73@gmail.com>
Signed-off-by: Gao Xiang <xiang@kernel.org>
Diffstat (limited to 'scripts/basic')
0 files changed, 0 insertions, 0 deletions
