[PATCH 04/45] rsa: Avoid warning in padding_pss_verify()

[Simon Glass]

With gcc 12 we ge the following warning:

In file included from tools/lib/rsa/rsa-verify.c:1:
lib/rsa/rsa-verify.c:275:11: warning: ‘*db’ may be used uninitialized
  275 |         db[0] &= 0xff >> leftmost_bits;

Check the value of db_len to ensure this cannot happen.

Signed-off-by: Simon Glass <sjg at chromium.org>
---

 lib/rsa/rsa-verify.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/lib/rsa/rsa-verify.c b/lib/rsa/rsa-verify.c
index 1d95cfbdee0..81c39251e59 100644
--- a/lib/rsa/rsa-verify.c
+++ b/lib/rsa/rsa-verify.c
@@ -234,6 +234,11 @@ int padding_pss_verify(struct image_sign_info *info,
 	uint8_t leftmost_mask;
 	struct checksum_algo *checksum = info->checksum;
 
+	if (db_len <= 0) {
+		ret = -EINVAL;
+		goto out;
+	}
+
 	/* first, allocate everything */
 	db_mask = malloc(db_len);
 	db = malloc(db_len);
-- 
2.37.3.998.g577e59143f-goog