[PATCH v2] gunzip: Implement chunked decompression

[Simon Glass]

Hi Marek,

On Wed, 18 Feb 2026 at 17:34, Marek Vasut
<marek.vasut+renesas at mailbox.org> wrote:
>
> The current gzwrite() implementation is limited to 4 GiB compressed
> input buffer size due to struct z_stream_s { uInt avail_in } member,
> which is of type unsigned int. Current gzwrite() implementation sets
> the entire input buffer size as avail_in and performs decompression
> of the whole compressed input buffer in one round, which limits the
> size of input buffer to 4 GiB.
>
> Rework the decompression loop to use chunked approach, and decompress
> the input buffer in up to 4 GiB - 1 kiB avail_in chunks, possibly in
> multiple decompression rounds. This way, the compressed input buffer
> size is limited by gzwrite() function 'len' parameter type, which is
> unsigned long.
>
> In case of sandbox build, include parsing of 'gzwrite_chunk'
> environment variable, so the chunked approach can be thoroughly tested
> with non default chunk size. For non-sandbox builds, the chunk size is
> 4 GiB - 1 kiB.
>
> The gzwrite test case is extended to test various chunk sizes during
> gzwrite decompression test.
>
> Signed-off-by: Marek Vasut <marek.vasut+renesas at mailbox.org>
> ---
> Cc: Alexander Graf <agraf at csgraf.de>
> Cc: Heinrich Schuchardt <xypron.glpk at gmx.de>
> Cc: Ilias Apalodimas <ilias.apalodimas at linaro.org>
> Cc: Jerome Forissier <jerome at forissier.org>
> Cc: Mattijs Korpershoek <mkorpershoek at kernel.org>
> Cc: Neil Armstrong <neil.armstrong at linaro.org>
> Cc: Peng Fan <peng.fan at nxp.com>
> Cc: Quentin Schulz <quentin.schulz at cherry.de>
> Cc: Simon Glass <sjg at chromium.org>
> Cc: Tom Rini <trini at konsulko.com>
> Cc: Yuya Hamamachi <yuya.hamamachi.sx at renesas.com>
> Cc: u-boot at lists.denx.de
> ---
> V2: Rebase on master
> ---
>  lib/gunzip.c     | 76 +++++++++++++++++++++++++++++++++---------------
>  test/cmd/unzip.c | 12 +++++++-
>  2 files changed, 64 insertions(+), 24 deletions(-)
>
> diff --git a/lib/gunzip.c b/lib/gunzip.c
> index 76f3397fced..20cc14f9688 100644
> --- a/lib/gunzip.c
> +++ b/lib/gunzip.c
> @@ -8,8 +8,10 @@
>  #include <command.h>
>  #include <console.h>
>  #include <div64.h>
> +#include <env.h>
>  #include <gzip.h>
>  #include <image.h>
> +#include <linux/sizes.h>
>  #include <malloc.h>
>  #include <memalign.h>
>  #include <u-boot/crc.h>
> @@ -119,7 +121,7 @@ void gzwrite_progress_finish(int returnval,
>  int gzwrite(unsigned char *src, size_t len, struct blk_desc *dev,
>             size_t szwritebuf, off_t startoffs, size_t szexpected)
>  {
> -       int i, flags;
> +       int flags;
>         z_stream s;
>         int r = 0;
>         unsigned char *writebuf;
> @@ -127,13 +129,23 @@ int gzwrite(unsigned char *src, size_t len, struct blk_desc *dev,
>         ulong totalfilled = 0;
>         lbaint_t blksperbuf, outblock;
>         u32 expected_crc;
> -       size_t payload_size;
> +       size_t i, payload_size;
> +       unsigned long blocks_written;
> +       lbaint_t writeblocks;
> +       int numfilled = 0;
>         int iteration = 0;
> -
> -       if (len > 0xffffffff) {
> -               log_err("Input size over 4 GiB in size not supported\n");
> -               return -1;
> -       }
> +       /*
> +        * Allow runtime configuration of decompression chunk on
> +        * sandbox to better cover the chunked decompression
> +        * functionality without having to use > 4 GiB files.
> +        */
> +       const ulong minchunk = 0x400;
> +       const ulong maxchunk = SZ_4G - minchunk;
> +       const ulong chunk =
> +               CONFIG_IS_ENABLED(SANDBOX,
> +                                 (clamp(env_get_ulong("gzwrite_chunk", 10, maxchunk),
> +                                        minchunk, maxchunk)),
> +                                 (maxchunk));
>

As you mentioned in the last version, there is a compression test in
compression.c - could you add a version of this function that takes
maxchunk as an argument and call it from that test? Reading an
environment variable to avoid passing a parameter seems pretty odd to
me :-)

[..]

Regards,
Simon