[U-Boot] [PATCH v2 2/2] rsa: Fix return value and masked error

[Simon Glass]

On 22 July 2016 at 00:58, Mario Six <mario.six at gdsys.cc> wrote:
> When signing images, we repeatedly call fit_add_file_data() with
> successively increasing size values to include the keys in the DTB.
>
> Unfortunately, if large keys are used (such as 4096 bit RSA keys), this
> process fails sometimes, and mkimage needs to be called repeatedly to
> integrate the keys into the DTB.
>
> This is because fit_add_file_data actually returns the wrong error
> code, and the loop terminates prematurely, instead of trying again with
> a larger size value.
>
> This patch corrects the return value by fixing the return value of
> fdt_add_bignum, fixes a case where an error is masked by a unconditional
> setting of a return value variable, and also removes a error message,
> which is misleading, since we actually allow the function to fail. A
> (hopefully helpful) comment is also added to explain the lack of error
> message.
>
> This is probably related to 1152a05 ("tools: Correct error handling in
> fit_image_process_hash()") and the corresponding error reported here:
>
> https://www.mail-archive.com/u-boot@lists.denx.de/msg217417.html
>
> Signed-off-by: Mario Six <mario.six at gdsys.cc>
> ---
>
> Changes in v2:
>  - Re-formatted the comment block
>
> ---
>  lib/rsa/rsa-sign.c | 12 +++++++-----
>  1 file changed, 7 insertions(+), 5 deletions(-)

Reviewed-by: Simon Glass <sjg at chromium.org>