[BUG] ut lib lib_rsa_verify_valid crashes on qemu_arm if RSA_VERIFY_WITH_PKEY=y

[AKASHI Takahiro]

Heinrich,

On Tue, Jun 09, 2020 at 03:54:44AM +0000, Heinrich Schuchardt wrote:
> Am June 9, 2020 1:42:14 AM UTC schrieb AKASHI Takahiro <takahiro.akashi at linaro.org>:
> >Heinrich,
> >
> >On Mon, Jun 08, 2020 at 11:08:53PM +0200, Heinrich Schuchardt wrote:
> >> Hello Takahiro,
> >> 
> >> when trying to execute command
> >> 
> >> ut lib lib_rsa_verify_valid
> >> 
> >> on qemu_arm_defconfig with CONFIG_UNIT_TEST=y and
> >> CONFIG_RSA_VERIFY_WITH_PKEY=y it crashes in
> >> 
> >> free((void *)prop->modulus) called from
> >> rsa_free_key_prop() called from
> >> rsa_verify_key() called from
> >> rsa_verify_with_pkey().
> >> 
> >> Without CONFIG_RSA_VERIFY_WITH_PKEY=y the problem does not occur.
> >> On qemu_arm64_defconfig the problem does not occur.
> >
> >I can't reproduce your problem on v2020.07-rc4 exactly with
> >qemu_arm64_defconfig + PKEY=y:
> >
> >U-Boot 2020.07-rc4-dirty (Jun 09 2020 - 10:33:30 +0900)
> >
> >...
> >
> >=> ut lib
> >Running 11 lib tests
> >Test: lib_asn1_pkcs7
> >Test: lib_asn1_pkey
> >Test: lib_asn1_x509
> >Test: lib_memcpy
> >Test: lib_memmove
> >Test: lib_memset
> >Test: lib_rsa_verify_invalid
> >Test: lib_rsa_verify_valid
> >Test: lib_test_bin2hex
> >Test: lib_test_hex2bin
> >Test: lib_test_hex_to_bin
> >Failures: 0
> >
> >
> >-Takahiro Akashi
> 
> As said I only see the problem with 32 bit qemu_arm_defconfig.

Okay. I think that the size of rrtmp variable is not good enough
and when it is handed over to br_i32_decode(), it accidentally
destroys (*prop)->modulus.

Heiko's patch:
  https://lists.denx.de/pipermail/u-boot/2020-May/413101.html
will also fix this issue, but I'm not yet confident that
the solution here, doubling max_rsa_size, is a right approach.

-Takahiro Akashi


> Best regards
> 
> Heinrich
> 
> 
> >
> >> Could you, please, have a look.
> >> 
> >> Best regards
> >> 
> >> Heinrich
>