[PATCH] gunzip: Fix len parameter in function signature

[Marek Vasut]

The only call site of gzwrite() is cmd/unzip.c do_gzwrite(), where
the 'len' parameter passed to gzwrite(..., len, ...) function is of
type unsigned long. This usage is correct, the 'len' parameter is
an unsigned integer, and the gzwrite() function currently supports
input data 'len' of up to 4 GiB - 1 .

The function signature of gzwrite() function in both include/gzip.h
and lib/gunzip.c does however list 'len' as signed integer, which
is not correct, and ultimatelly limits the implementation to only
2 GiB input data 'len' .

Fix this, update gzwrite() function parameter 'len' data type to
unsigned long consistently in include/gzip.h and lib/gunzip.c .

Furthermore, update gzwrite() function 'szwritebuf' parameter in
lib/gunzip.c from 'unsigned long' to 'ulong' to be synchronized
with include/gzip.h .

Since the gzwrite() function currently surely only supports input
data size of 4 GiB - 1, add input data size check. The limitation
comes from the current use of zlib z_stream .avail_in parameter,
to which the gzwrite() function sets the entire input data size,
and which is of unsigned int type, which cannot accept any number
beyond 4 GiB - 1. This limitation will be removed in future commit.

Reported-by: Yuya Hamamachi <yuya.hamamachi.sx at renesas.com>
Signed-off-by: Marek Vasut <marek.vasut+renesas at mailbox.org>
---
Cc: Alexander Graf <agraf at csgraf.de>
Cc: Heinrich Schuchardt <xypron.glpk at gmx.de>
Cc: Ilias Apalodimas <ilias.apalodimas at linaro.org>
Cc: Jerome Forissier <jerome at forissier.org>
Cc: Mattijs Korpershoek <mkorpershoek at kernel.org>
Cc: Neil Armstrong <neil.armstrong at linaro.org>
Cc: Peng Fan <peng.fan at nxp.com>
Cc: Quentin Schulz <quentin.schulz at cherry.de>
Cc: Simon Glass <sjg at chromium.org>
Cc: Tom Rini <trini at konsulko.com>
Cc: Yuya Hamamachi <yuya.hamamachi.sx at renesas.com>
Cc: u-boot at lists.denx.de
---
 include/gzip.h |  4 ++--
 lib/gunzip.c   | 13 ++++++++-----
 2 files changed, 10 insertions(+), 7 deletions(-)

diff --git a/include/gzip.h b/include/gzip.h
index 304002ffc42..5396e3ffec7 100644
--- a/include/gzip.h
+++ b/include/gzip.h
@@ -77,8 +77,8 @@ void gzwrite_progress_finish(int retcode, ulong totalwritten, ulong totalsize,
  *		for files under 4GiB
  * Return: 0 if OK, -1 on error
  */
-int gzwrite(unsigned char *src, int len, struct blk_desc *dev, ulong szwritebuf,
-	    ulong startoffs, ulong szexpected);
+int gzwrite(unsigned char *src, unsigned long len, struct blk_desc *dev,
+	    ulong szwritebuf, ulong startoffs, ulong szexpected);
 
 /**
  * gzip()- Compress data into a buffer using the gzip algorithm
diff --git a/lib/gunzip.c b/lib/gunzip.c
index a05dcde9a75..040450c0e79 100644
--- a/lib/gunzip.c
+++ b/lib/gunzip.c
@@ -116,11 +116,8 @@ void gzwrite_progress_finish(int returnval,
 	}
 }
 
-int gzwrite(unsigned char *src, int len,
-	    struct blk_desc *dev,
-	    unsigned long szwritebuf,
-	    ulong startoffs,
-	    ulong szexpected)
+int gzwrite(unsigned char *src, unsigned long len, struct blk_desc *dev,
+	    ulong szwritebuf, ulong startoffs, ulong szexpected)
 {
 	int i, flags;
 	z_stream s;
@@ -133,6 +130,12 @@ int gzwrite(unsigned char *src, int len,
 	u32 payload_size;
 	int iteration = 0;
 
+	if (len > 0xffffffff) {
+		printf("%s: input size over 4 GiB in size not supported\n",
+		       __func__);
+		return -1;
+	}
+
 	if (!szwritebuf ||
 	    (szwritebuf % dev->blksz) ||
 	    (szwritebuf < dev->blksz)) {
-- 
2.51.0