diff options
| author | Tom Rini <trini@konsulko.com> | 2026-09-01 15:11:37 -0600 |
|---|---|---|
| committer | Tom Rini <trini@konsulko.com> | 2026-09-01 15:11:37 -0600 |
| commit | b55d2b12d3a88d003a9911adc8d96e4921f67ad1 (patch) | |
| tree | f4a3141e99fc081e5ac854387819e17ac2f4a935 | |
| parent | 97626d3f0ae6e7d470ee4048be58ee180b29fca5 (diff) | |
| parent | 6bd934a1f5e9f37579af123d49114aee35607449 (diff) | |
| download | u-boot-b55d2b12d3a88d003a9911adc8d96e4921f67ad1.tar.gz u-boot-b55d2b12d3a88d003a9911adc8d96e4921f67ad1.zip | |
Merge patch series "bootm: size the noload buffer from the compressor header"
Aristo Chen <aristo.chen@canonical.com> says:
This is v2 of "bootm: size the noload decompression buffer from the
compressor header". Tom pushed back on v1
(https://lore.kernel.org/r/20260809042338.63397-2-aristo.chen@canonical.com/)
on two grounds:
1. No concrete problem report driving the change.
2. ~1297 platforms grew by ~170-400 bytes; the change is not
opt-in, so the size cost falls on everyone.
On the first point, Nora Schiffer replied with a concrete use case
(EFI-in-FIT plus padded loaders such as shim, systemd-boot, and
OpenWrt's lzma-loader can produce compression ratios that outrun the
8x heuristic), and mentioned this is on the road map for TQ-Systems
standard BSPs.
On the second point, v2 reworks the implementation to cut the size
cost, measures it across the format and architecture buckets, and
splits the work per format so each decompressor's support can be
taken or dropped on its own.
Background: for a compressed kernel_noload image, bootm_load_os()
sizes the decompression buffer as ALIGN(image_len * 8, SZ_1M). The
8x heuristic works for typical kernels, but any well-compressed
payload can exceed it, and no fixed multiplier is safe against
arbitrarily compressible input.
Each implementation patch adds a small static header-parse helper in
bootm.c (no new public API) and wires it into a size-hint switch;
helper and switch case are only compiled when the matching
decompressor is enabled, so boards that do not build a format pay no
code for it. gzip's ISIZE is a fixed trailer read, lzma's size a
fixed header read, lz4 mirrors ulz4fn()'s frame-header validation,
and zstd asks zstd_get_frame_header(), whose frame-parsing code
already ships with the zstd decompressor. The header-recorded value
is attacker-controlled, so it is capped at CONFIG_SYS_BOOTM_LEN, and
it is only an allocation hint: the decoder stays authoritative
during the actual decompression.
Text size deltas of the u-boot ELF (size(1), distro gcc 13.3 cross
toolchains); data/bss are unchanged everywhere. To make the columns
directly comparable, the v1 column is v1's implementation commit
cherry-picked onto this series' base, so both columns share one
baseline:
board arch decompressors v1 v2
qemu_arm arm gzip +160 +104
qemu-ppce500 powerpc gzip +176 +112
mt7623n_bpir2 arm gzip+lzma +184 +128
qemu_arm64 arm64 gzip+lzma+lz4 +384 +368
qemu-riscv64 riscv64 gzip+lzma+lz4 +332 +352
th1520_lpi4a riscv64 all four +412 +404
am62x_evm_a53 arm64 all four, LTO +0 * +8192 *
qemu-x86 x86 none +108 -2
* am62x_evm_a53's number is dominated by the Cortex-A53 erratum
843419 linker workaround (default-enabled in distro binutils for
aarch64): symbol-level code growth (nm -S) is +392 for v1 and
+452 for v2, but those bytes shift which ADRP instructions land
at the erratum's page offsets, and ld pads each inserted veneer
to a full 4 KiB page. v2 happens to trigger two such pages here;
v1 triggered the same two on its own original base and none on
this one. See the world-build note below.
To see how much each bucket weighs, I configured all 1550 defconfigs
and sorted them by which decompressors they enable next to bootm:
858 gzip only (722 of them arm, essentially the 32-bit boards)
530 gzip+lzma+lz4 (494 arm, mostly arm64, plus 36 riscv)
38 gzip+lzma
31 bootm with no decompressor at all
27 gzip+lzma+lz4+zstd
19 gzip+lz4
15 gzip+zstd
6 other combinations
26 do not link bootm at all
To measure at the same scale as the original objection, I also ran a
full world build (buildman, all 1550 defconfigs, distro plus
kernel.org toolchains, gcc 13.3/14.2) over one branch holding the
base, the v1 implementation, its revert, and this series. 1496
boards built on all four commits with the revert reproducing the
base sizes exactly (44 boards did not build on every commit, and 10
built nondeterministically; both sets were excluded). Of those 1496,
the same 1375 change under either version and the rest are
untouched, including every board without bootm or without a
decompressor:
v1 v2
mean delta over all boards +194 B +166 B
median delta (changed boards) +160 B +96 B
median, 856 gzip-only boards +112 B +80 B
median, 517 gzip+lzma+lz4 boards +392 B +376 B
boards cheaper with v2 - 1222
boards costlier with v2 - 74
The world build also puts the am62x footnote in proportion: 53
boards under v1 and 49 under v2 (28 in both sets), all arm64, show
size(1) jumps of one or two 4 KiB pages in either direction (min
-8192, max +8192). The mechanism is the Cortex-A53 erratum 843419
linker workaround: when a code change shifts which ADRP instructions
land at page offsets 0xff8/0xffc, ld materialises a 16-byte veneer
and pads it to a full 4 KiB page so the page offsets of all
downstream code stay unchanged. Any few-hundred-byte change re-rolls
which boards are affected, in both directions; symbol-level growth
on every such board I checked matches the byte ranges above.
Since Tom noted the higher growth in his run was on multi-algorithm
platforms: building each patch in sequence on a gzip+lzma+lz4 board
(qemu_arm64) and an all-four board (th1520_lpi4a) gives the
per-format cost directly, in bytes:
qemu_arm64 th1520_lpi4a
gzip +112 +86
zstd +0 +62
lz4 +176 +176
lzma +80 +80
(zstd is +0 on qemu_arm64 because that board does not enable it, so
the guard really does compile the helper out.) lz4 is the most
expensive parser because it mirrors ulz4fn()'s frame validation;
zstd is the cheapest because zstd_get_frame_header() already ships
with the decompressor. Since the series is split per format, if the
multi-algorithm cost still looks too high, dropping the lz4 patch
alone would cut the 517-board gzip+lzma+lz4 bucket from a median of
+376 to roughly +200; lz4 images then simply keep the 8x fallback.
In the v1 thread Simon suggested recording the uncompressed size as
a FIT property instead. As discussed there, the two compose: a FIT
property could be layered on top later, with bootm preferring the
property, then the stream header, then the 8x fallback. This series
provides the part that works for every existing image and for the
legacy uImage form of kernel_noload.
Series layout, one decompressor at a time:
1. gzip helper + wiring
2. gzip pytests (lying-header overflow, header-sized, boundary)
3. zstd helper
4. zstd pytest (guarded by requiredtool zstd)
5. lz4 helper
6. lz4 pytest (guarded by requiredtool lz4)
7. lzma helper
8. lzma pytests (real size patched into the header field, plus the
"unknown" size marker fallback; needs no external tool since
Python's lzma module is in the standard library)
Every patch builds in isolation on sandbox_defconfig and
qemu_arm_defconfig; the seven kernel_noload_decomp pytests and the
full test_fit class pass on sandbox.
Link: https://lore.kernel.org/r/20260818132332.324173-1-aristo.chen@canonical.com
| -rw-r--r-- | boot/bootm.c | 144 | ||||
| -rwxr-xr-x | test/py/tests/test_fit.py | 282 |
2 files changed, 400 insertions, 26 deletions
diff --git a/boot/bootm.c b/boot/bootm.c index 3bce8586834..758edeb964c 100644 --- a/boot/bootm.c +++ b/boot/bootm.c @@ -23,7 +23,10 @@ #include <asm/cache.h> #include <asm/global_data.h> #include <asm/io.h> +#include <asm/unaligned.h> #include <linux/sizes.h> +#include <linux/zstd.h> +#include <lzma/LzmaDec.h> #include <tpm-v2.h> #include <tpm_tcg2.h> #if defined(CONFIG_CMD_USB) @@ -638,6 +641,101 @@ static int handle_decomp_error(int comp_type, size_t uncomp_size, #endif #ifndef USE_HOSTCC +#if CONFIG_IS_ENABLED(GZIP) +/* + * Return the gzip stream's uncompressed size from its ISIZE trailer, or + * 0 if the buffer is not a gzip stream. Only the two magic bytes are + * checked, since a fuller validation happens inside gunzip() during + * decompression; the caller uses the return value as a size hint only. + */ +static ulong bootm_gzip_uncompressed_size(const void *src, ulong len) +{ + const u8 *b = src; + + /* Minimum gzip: 10-byte header + 2-byte deflate + 8-byte trailer */ + if (len < 20 || b[0] != 0x1f || b[1] != 0x8b) + return 0; + return get_unaligned_le32(b + len - 4); +} +#endif + +#if CONFIG_IS_ENABLED(LZMA) +/* + * Return the uncompressed size recorded in the lzma stream header, or + * 0 if the buffer is too short or the size field carries the "unknown" + * marker (0xff..ff). The .lzma-alone format keeps the size in a fixed + * 8-byte field right after the 5-byte properties block; nothing else + * is validated since the value is only an allocation hint. + */ +static ulong bootm_lzma_uncompressed_size(const void *src, ulong len) +{ + const u8 *b = src; + u64 usize; + + if (len < LZMA_PROPS_SIZE + 8) + return 0; + usize = get_unaligned_le64(b + LZMA_PROPS_SIZE); + if (usize == U64_MAX || usize > ULONG_MAX) + return 0; + return (ulong)usize; +} +#endif + +#if CONFIG_IS_ENABLED(LZ4) +/* + * Return the lz4 frame's Content_Size, or 0 if the buffer is not an + * lz4 frame or the frame does not carry the size. The header parse + * mirrors ulz4fn()'s validation so we do not accept a stream the + * decoder itself would refuse. + */ +static ulong bootm_lz4_uncompressed_size(const void *src, ulong len) +{ + const u8 *b = src; + u8 flg, version, indep_blocks, has_content_size, bd; + u64 cs; + + if (len < 4 + 2 || get_unaligned_le32(b) != LZ4F_MAGIC) + return 0; + flg = b[4]; + bd = b[5]; + version = (flg >> 6) & 3; + indep_blocks = (flg >> 5) & 1; + has_content_size = (flg >> 3) & 1; + if (version != 1 || !indep_blocks || (flg & 3) || (bd & 0x8f) || + !has_content_size) + return 0; + if (len < 4 + 2 + 8) + return 0; + cs = get_unaligned_le64(b + 6); + return cs > ULONG_MAX ? 0 : (ulong)cs; +} +#endif + +#if CONFIG_IS_ENABLED(ZSTD) +/* + * Return the zstd frame's Frame_Content_Size, or 0 if the header does + * not parse or the size is absent. zstd_get_frame_header() and the + * frame-parsing code behind it are part of the zstd decompressor that + * is already linked into any board with ZSTD enabled, so the call adds + * only the call site. The value is an allocation hint; the decoder + * stays authoritative during the actual decompression. + */ +static ulong bootm_zstd_uncompressed_size(const void *src, ulong len) +{ + zstd_frame_header hdr; + size_t ret; + + ret = zstd_get_frame_header(&hdr, src, len); + if (zstd_is_error(ret) || ret > 0) + return 0; + if (hdr.frameContentSize == ZSTD_CONTENTSIZE_UNKNOWN || + hdr.frameContentSize == ZSTD_CONTENTSIZE_ERROR || + hdr.frameContentSize > ULONG_MAX) + return 0; + return (ulong)hdr.frameContentSize; +} +#endif + static int bootm_load_os(struct bootm_headers *images, int boot_progress) { const struct image_info os = images->os; @@ -654,17 +752,54 @@ static int bootm_load_os(struct bootm_headers *images, int boot_progress) void *load_buf, *image_buf; int err; + image_buf = map_sysmem(os.image_start, image_len); + /* * For a "noload" compressed kernel we need to allocate a buffer large * enough to decompress in to and use that as the load address now. - * Allow up to 8x compression: this comfortably covers what zstd and xz - * achieve on real kernels, with headroom for well-compressed payloads. - * Use an alignment of 2MB since this might help arm64 + * When the compressed stream records its uncompressed size and that + * value is within CONFIG_SYS_BOOTM_LEN, allocate exactly that. + * Otherwise fall back to an 8x multiplier, which comfortably covers + * what zstd and xz achieve on real kernels with headroom for + * well-compressed payloads. Use an alignment of 2MB since this + * might help arm64. */ if (os.type == IH_TYPE_KERNEL_NOLOAD && os.comp != IH_COMP_NONE) { phys_addr_t addr; + ulong hdr_size = 0; - decomp_len = ALIGN(image_len * 8, SZ_1M); + switch (os.comp) { +#if CONFIG_IS_ENABLED(GZIP) + case IH_COMP_GZIP: + hdr_size = bootm_gzip_uncompressed_size(image_buf, + image_len); + break; +#endif +#if CONFIG_IS_ENABLED(LZMA) + case IH_COMP_LZMA: + hdr_size = bootm_lzma_uncompressed_size(image_buf, + image_len); + break; +#endif +#if CONFIG_IS_ENABLED(LZ4) + case IH_COMP_LZ4: + hdr_size = bootm_lz4_uncompressed_size(image_buf, + image_len); + break; +#endif +#if CONFIG_IS_ENABLED(ZSTD) + case IH_COMP_ZSTD: + hdr_size = bootm_zstd_uncompressed_size(image_buf, + image_len); + break; +#endif + default: + break; + } + if (hdr_size && hdr_size <= CONFIG_SYS_BOOTM_LEN) + decomp_len = ALIGN(hdr_size, SZ_1M); + else + decomp_len = ALIGN(image_len * 8, SZ_1M); decomp_limit = BOOTM_DECOMP_LIMIT_PER_IMAGE; err = lmb_alloc_mem(LMB_MEM_ALLOC_ANY, SZ_2M, &addr, decomp_len, LMB_NONE); @@ -679,7 +814,6 @@ static int bootm_load_os(struct bootm_headers *images, int boot_progress) } load_buf = map_sysmem(load, 0); - image_buf = map_sysmem(os.image_start, image_len); err = image_decomp(os.comp, load, os.image_start, os.type, load_buf, image_buf, image_len, decomp_len, &load_end); diff --git a/test/py/tests/test_fit.py b/test/py/tests/test_fit.py index 76adb98e2c5..0edf875a9e1 100755 --- a/test/py/tests/test_fit.py +++ b/test/py/tests/test_fit.py @@ -118,8 +118,10 @@ host save hostfs 0 %(loadables2_addr)x %(loadables2_out)s %(loadables2_size)x ''' # A minimal ITS for a compressed 'kernel_noload' kernel. bootm allocates a -# per-image decompression buffer for this image type, sized as a multiple of -# the compressed length; see the test_fit_kernel_noload_decomp_* tests. +# per-image decompression buffer for this image type, sized either from the +# compressor header (gzip ISIZE, zstd Frame_Content_Size, ...) or as a +# multiple of the compressed length; see the test_fit_kernel_noload_decomp_* +# tests. NOLOAD_ITS = ''' /dts-v1/; @@ -133,7 +135,7 @@ NOLOAD_ITS = ''' type = "kernel_noload"; arch = "sandbox"; os = "linux"; - compression = "gzip"; + compression = "%(compression)s"; load = <0>; entry = <0>; }; @@ -511,14 +513,13 @@ class TestFitImage: + output) @pytest.mark.buildconfigspec('gzip') - def test_fit_kernel_noload_decomp_overflow(self, ubman, fsetup): - """Test that an over-large compressed kernel_noload image is rejected + def test_fit_kernel_noload_decomp_gzip_lying_hdr(self, ubman, fsetup): + """A tampered gzip ISIZE cannot shrink the buffer past the payload - For a compressed 'kernel_noload' kernel, bootm_load_os() allocates a - decompression buffer of ALIGN(image_len * 8, SZ_1M) and must bound the - decompressor by that buffer. A kernel that decompresses to far more - than eight times its compressed size must therefore fail with a - decompression error instead of overflowing the buffer. + bootm_load_os() sizes the kernel_noload decompression buffer from the + gzip ISIZE trailer. That value is attacker-controlled; rewriting + ISIZE to understate the real size must not let decompression overflow + the resulting buffer. """ sz_1m = 1 << 20 @@ -527,23 +528,24 @@ class TestFitImage: # per-image kernel_noload buffer rather than by that global limit. bootm_len = int(ubman.config.buildconfig['config_sys_bootm_len'], 0) - # 4MB of zeros compresses to a few KB, so the decompression buffer - # (ALIGN(image_len * 8, SZ_1M), i.e. 1MB here) ends up far smaller - # than the uncompressed image. decomp_size = 4 * sz_1m + assert decomp_size <= bootm_len, ( + 'Test setup error: uncompressed size (%#x) must be <= ' + 'CONFIG_SYS_BOOTM_LEN (%#x)' % (decomp_size, bootm_len)) kernel = fit_util.make_fname(ubman, 'test-noload-kernel.bin') with open(kernel, 'wb') as fd: fd.write(b'\0' * decomp_size) kernel_gz = self.make_compressed(ubman, kernel) - image_len = self.filesize(kernel_gz) - req_size = (image_len * 8 + sz_1m - 1) // sz_1m * sz_1m - assert req_size < decomp_size <= bootm_len, ( - 'Test setup error: need decomp buffer (%#x) < image (%#x) <= ' - 'CONFIG_SYS_BOOTM_LEN (%#x)' % (req_size, decomp_size, bootm_len)) + # Rewrite gzip ISIZE (the last 4 bytes) to claim a tiny image, so + # bootm allocates ALIGN(<lie>, SZ_1M) = 1 MiB and the real 4 MiB + # decompression has to overrun that buffer. + with open(kernel_gz, 'r+b') as fd: + fd.seek(-4, os.SEEK_END) + fd.write((256).to_bytes(4, 'little')) fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS, - {'kernel': kernel_gz}) + {'kernel': kernel_gz, 'compression': 'gzip'}) fit_addr = fsetup['fit_addr'] ubman.run_command_list([ @@ -563,7 +565,245 @@ class TestFitImage: ubman.restart_uboot() @pytest.mark.buildconfigspec('gzip') - def test_fit_kernel_noload_decomp_boundary(self, ubman, fsetup): + def test_fit_kernel_noload_decomp_gzip_hdr_sized(self, ubman, fsetup): + """A well-compressed kernel_noload image fits when ISIZE is honest + + bootm_load_os() reads gzip ISIZE to size the decompression buffer. + For a well-compressed image whose ratio exceeds the 8x fallback + heuristic (e.g. 6 MiB of zeros gzipping to a few KiB), an ISIZE-sized + buffer is the only way the decompression fits. + """ + sz_1m = 1 << 20 + bootm_len = int(ubman.config.buildconfig['config_sys_bootm_len'], 0) + + # Stay under CONFIG_SYS_BOOTM_LEN so the ISIZE hint isn't rejected as + # bogus; still large enough that image_len * 8 falls well short. + decomp_size = 6 * sz_1m + assert decomp_size <= bootm_len, ( + 'Test setup error: decomp_size (%#x) must be <= ' + 'CONFIG_SYS_BOOTM_LEN (%#x)' % (decomp_size, bootm_len)) + kernel = fit_util.make_fname(ubman, 'test-noload-kernel-hdrsized.bin') + with open(kernel, 'wb') as fd: + fd.write(b'\0' * decomp_size) + kernel_gz = self.make_compressed(ubman, kernel) + + image_len = self.filesize(kernel_gz) + heuristic_bound = (image_len * 8 + sz_1m - 1) // sz_1m * sz_1m + assert heuristic_bound < decomp_size, ( + 'Test setup error: 8x heuristic bound (%#x) must be < uncompressed ' + 'size (%#x); if this fires, the compressor got less effective and ' + 'the test needs a bigger payload' % (heuristic_bound, decomp_size)) + + fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS, + {'kernel': kernel_gz, 'compression': 'gzip'}, + basename='test-noload-hdrsized.fit') + fit_addr = fsetup['fit_addr'] + + # Decompression must succeed: bootm read ISIZE and allocated a big + # enough buffer despite the ratio being past the fallback heuristic. + output = ubman.run_command_list([ + 'host load hostfs 0 %x %s' % (fit_addr, fit), + 'bootm start %x' % fit_addr, + 'bootm loados', + ]) + text = '\n'.join(output) + assert 'Image too large' not in text, ( + 'bootm rejected a well-compressed kernel_noload image whose ' + 'ISIZE trailer records the real uncompressed size: %s' % text) + + @pytest.mark.buildconfigspec('lz4') + @pytest.mark.requiredtool('lz4') + def test_fit_kernel_noload_decomp_lz4_hdr_sized(self, ubman, fsetup): + """A well-compressed lz4 kernel_noload image fits when the frame + header carries the content size. + + Same as test_fit_kernel_noload_decomp_gzip_hdr_sized but for lz4: + the tool must be invoked with --content-size so the frame's FLG + bit is set and bootm can read the size instead of falling back to + the 8x heuristic. + """ + sz_1m = 1 << 20 + bootm_len = int(ubman.config.buildconfig['config_sys_bootm_len'], 0) + + decomp_size = 6 * sz_1m + assert decomp_size <= bootm_len, ( + 'Test setup error: decomp_size (%#x) must be <= ' + 'CONFIG_SYS_BOOTM_LEN (%#x)' % (decomp_size, bootm_len)) + kernel = fit_util.make_fname(ubman, 'test-noload-kernel-lz4.bin') + with open(kernel, 'wb') as fd: + fd.write(b'\0' * decomp_size) + kernel_lz4 = kernel + '.lz4' + utils.run_and_log( + ubman, ['lz4', '--content-size', '-f', kernel, kernel_lz4]) + + image_len = self.filesize(kernel_lz4) + heuristic_bound = (image_len * 8 + sz_1m - 1) // sz_1m * sz_1m + assert heuristic_bound < decomp_size, ( + 'Test setup error: 8x heuristic bound (%#x) must be < uncompressed ' + 'size (%#x); if this fires, lz4 got less effective and the test ' + 'needs a bigger payload' % (heuristic_bound, decomp_size)) + + fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS, + {'kernel': kernel_lz4, 'compression': 'lz4'}, + basename='test-noload-lz4-hdrsized.fit') + fit_addr = fsetup['fit_addr'] + + output = ubman.run_command_list([ + 'host load hostfs 0 %x %s' % (fit_addr, fit), + 'bootm start %x' % fit_addr, + 'bootm loados', + ]) + text = '\n'.join(output) + assert 'Image too large' not in text, ( + 'bootm rejected a well-compressed lz4 kernel_noload image whose ' + 'frame header records the real content size: %s' % text) + + @pytest.mark.buildconfigspec('zstd') + @pytest.mark.requiredtool('zstd') + def test_fit_kernel_noload_decomp_zstd_hdr_sized(self, ubman, fsetup): + """A well-compressed zstd kernel_noload image fits when the frame + header carries Frame_Content_Size. + + Same as test_fit_kernel_noload_decomp_gzip_hdr_sized but for zstd. + The default zstd encoder embeds Frame_Content_Size for a + single-segment frame, so bootm can read it and size the buffer + accordingly. + """ + sz_1m = 1 << 20 + bootm_len = int(ubman.config.buildconfig['config_sys_bootm_len'], 0) + + decomp_size = 6 * sz_1m + assert decomp_size <= bootm_len, ( + 'Test setup error: decomp_size (%#x) must be <= ' + 'CONFIG_SYS_BOOTM_LEN (%#x)' % (decomp_size, bootm_len)) + kernel = fit_util.make_fname(ubman, 'test-noload-kernel-zstd.bin') + with open(kernel, 'wb') as fd: + fd.write(b'\0' * decomp_size) + kernel_zstd = kernel + '.zst' + utils.run_and_log(ubman, ['zstd', '-f', kernel, '-o', kernel_zstd]) + + image_len = self.filesize(kernel_zstd) + heuristic_bound = (image_len * 8 + sz_1m - 1) // sz_1m * sz_1m + assert heuristic_bound < decomp_size, ( + 'Test setup error: 8x heuristic bound (%#x) must be < uncompressed ' + 'size (%#x); if this fires, zstd got less effective and the test ' + 'needs a bigger payload' % (heuristic_bound, decomp_size)) + + fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS, + {'kernel': kernel_zstd, 'compression': 'zstd'}, + basename='test-noload-zstd-hdrsized.fit') + fit_addr = fsetup['fit_addr'] + + output = ubman.run_command_list([ + 'host load hostfs 0 %x %s' % (fit_addr, fit), + 'bootm start %x' % fit_addr, + 'bootm loados', + ]) + text = '\n'.join(output) + assert 'Image too large' not in text, ( + 'bootm rejected a well-compressed zstd kernel_noload image whose ' + 'frame header records the real content size: %s' % text) + + @pytest.mark.buildconfigspec('lzma') + def test_fit_kernel_noload_decomp_lzma_hdr_sized(self, ubman, fsetup): + """A well-compressed lzma kernel_noload image fits when the header + records the real uncompressed size. + + Same as test_fit_kernel_noload_decomp_gzip_hdr_sized but for lzma. + Streaming encoders write the "unknown" marker into the .lzma-alone + size field, so compress with Python's lzma module and patch the + real size into the fixed 8-byte header field, the way LZMA SDK + style encoders record it. + """ + lzma = pytest.importorskip('lzma') + sz_1m = 1 << 20 + bootm_len = int(ubman.config.buildconfig['config_sys_bootm_len'], 0) + + decomp_size = 6 * sz_1m + assert decomp_size <= bootm_len, ( + 'Test setup error: decomp_size (%#x) must be <= ' + 'CONFIG_SYS_BOOTM_LEN (%#x)' % (decomp_size, bootm_len)) + kernel = fit_util.make_fname(ubman, 'test-noload-kernel-lzma.bin') + with open(kernel, 'wb') as fd: + fd.write(b'\0' * decomp_size) + filters = [{'id': lzma.FILTER_LZMA1, 'preset': 6, + 'dict_size': 1 << 20}] + blob = lzma.compress(self.read_file(kernel), + format=lzma.FORMAT_ALONE, filters=filters) + assert blob[5:13] == b'\xff' * 8, ( + 'Test setup error: expected the streaming encoder to write the ' + '"unknown" size marker') + blob = blob[:5] + decomp_size.to_bytes(8, 'little') + blob[13:] + kernel_lzma = kernel + '.lzma' + with open(kernel_lzma, 'wb') as fd: + fd.write(blob) + + image_len = self.filesize(kernel_lzma) + heuristic_bound = (image_len * 8 + sz_1m - 1) // sz_1m * sz_1m + assert heuristic_bound < decomp_size, ( + 'Test setup error: 8x heuristic bound (%#x) must be < uncompressed ' + 'size (%#x); if this fires, lzma got less effective and the test ' + 'needs a bigger payload' % (heuristic_bound, decomp_size)) + + fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS, + {'kernel': kernel_lzma, 'compression': 'lzma'}, + basename='test-noload-lzma-hdrsized.fit') + fit_addr = fsetup['fit_addr'] + + output = ubman.run_command_list([ + 'host load hostfs 0 %x %s' % (fit_addr, fit), + 'bootm start %x' % fit_addr, + 'bootm loados', + ]) + text = '\n'.join(output) + assert 'Image too large' not in text, ( + 'bootm rejected a well-compressed lzma kernel_noload image whose ' + 'header records the real uncompressed size: %s' % text) + + @pytest.mark.buildconfigspec('lzma') + def test_fit_kernel_noload_decomp_lzma_unknown_size(self, ubman, fsetup): + """An lzma stream with the "unknown" size marker falls back cleanly + + Streaming encoders write 0xff..ff into the .lzma-alone size field. + bootm must fall back to the 8x heuristic buffer and still boot the + image. + """ + lzma = pytest.importorskip('lzma') + sz_1m = 1 << 20 + + # Incompressible data keeps the real size well inside the 8x + # fallback buffer. + payload = os.urandom(sz_1m) + kernel = fit_util.make_fname(ubman, 'test-noload-kernel-lzma-unk.bin') + filters = [{'id': lzma.FILTER_LZMA1, 'preset': 6, + 'dict_size': 1 << 20}] + blob = lzma.compress(payload, format=lzma.FORMAT_ALONE, + filters=filters) + assert blob[5:13] == b'\xff' * 8, ( + 'Test setup error: expected the streaming encoder to write the ' + '"unknown" size marker') + kernel_lzma = kernel + '.lzma' + with open(kernel_lzma, 'wb') as fd: + fd.write(blob) + + fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS, + {'kernel': kernel_lzma, 'compression': 'lzma'}, + basename='test-noload-lzma-unk.fit') + fit_addr = fsetup['fit_addr'] + + output = ubman.run_command_list([ + 'host load hostfs 0 %x %s' % (fit_addr, fit), + 'bootm start %x' % fit_addr, + 'bootm loados', + ]) + text = '\n'.join(output) + assert 'Image too large' not in text, ( + 'bootm rejected an lzma kernel_noload image carrying the ' + '"unknown" size marker; the 8x fallback should have covered ' + 'it: %s' % text) + + @pytest.mark.buildconfigspec('gzip') + def test_fit_kernel_noload_decomp_gzip_boundary(self, ubman, fsetup): """Test that decompression succeeds exactly at the buffer limit For a compressed 'kernel_noload' kernel, bootm_load_os() allocates a @@ -589,7 +829,7 @@ class TestFitImage: % (decomp_size, req_size)) fit = fit_util.make_fit(ubman, fsetup['mkimage'], NOLOAD_ITS, - {'kernel': kernel_gz}, + {'kernel': kernel_gz, 'compression': 'gzip'}, basename='test-noload-boundary.fit') fit_addr = fsetup['fit_addr'] |
