[PATCH v2] Add support for OpenSSL Provider API

[Quentin Schulz]

Hi Eddie,

On 10/27/25 8:58 PM, Eddie Kovsky wrote:
> [You don't often get email from ekovsky at redhat.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
> 
> The Engine API has been deprecated since the release of OpenSSL 3.0. End
> users have been advised to migrate to the new Provider interface.
> Several distributions have already removed support for engines, which is
> preventing U-Boot from being compiled in those environments.
> 

Which ones? How do I reproduce?

> The Kconfig option OPENSSL_NO_DEPRECATED introduces support for the

Please consider renaming this, OpenSSL itself uses OPENSSL_NO_DEPRECATED 
constants for many things. I would recommend simply renaming to 
OPENSSL_NO_ENGINE which is also the symbol OpenSSL is using. If there 
comes a time we have more OPENSSL_NO_ options, we can always have a 
"virtual" symbol called OPENSSL_NO_DEPRECATED which would select them 
all if one wanted for example.

> Provider API while continuing to use the existing Engine API on distros
> shipping older releases of OpenSSL.
> 

One can use org.openssl.engine: as prefix for provider arguments when 
one wants to use an engine still.

> This is based on similar work contributed by Jan Stancek updating Linux
> to use the Provider interface.
> 
>      commit 558bdc45dfb2669e1741384a0c80be9c82fa052c
>      Author: Jan Stancek <jstancek at redhat.com>
>      Date:   Fri Sep 20 19:52:48 2024 +0300
> 
>          sign-file,extract-cert: use pkcs11 provider for OPENSSL MAJOR >= 3
> 
> The changes have been tested with the FIT signature verification vboot
> tests on Fedora 42 and Debian 13. All 30 tests pass with both the legacy
> Engine library installed and with the Provider API.
> 

Are there actually tests using an OpenSSL engine? Because otherwise it's 
simply checking that local keys are still working... which isn't that 
much different from what we currently have with engines when not using 
engines.

I'm implementing FIT images signing with OpenSSL engines, and it'd be 
nice if we could have something that doesn't require changes to support 
providers (or if it does, not in a confusing manner for example).

https://lore.kernel.org/u-boot/20251031-binman-engine-v1-0-c13c1b5dac43@cherry.de/T/#t 
for the v1, I'll soon (next hours or tomorrow) post a v2 and Cc you if 
you don't mind.

[...]

> diff --git a/lib/rsa/Kconfig b/lib/rsa/Kconfig
> index 9033384e60a3..1bf0ac96d598 100644
> --- a/lib/rsa/Kconfig
> +++ b/lib/rsa/Kconfig
> @@ -20,6 +20,13 @@ config SPL_RSA
>          bool "Use RSA Library within SPL"
>          depends on SPL
> 
> +config OPENSSL_NO_DEPRECATED
> +       bool "Build U-Boot without support for OpenSSL Engine"
> +       help
> +         Add support for the OpenSSL Provider API, which is the officially
> +         supported mechanism in OpenSSL 3.x and later releases for accessing
> +         hardware and software cryptography.
> +

mmmm Cannot we use providers for something else than RSA? In which case 
it's a bit odd to have it in lib/rsa/Kconfig (but I have no better 
suggestion).

>   config SPL_RSA_VERIFY
>          bool
>          depends on SPL_RSA

[...]

> @@ -207,6 +247,37 @@ static int rsa_pem_get_priv_key(const char *keydir, const char *name,
>                  return -ENOENT;
>          }
> 
> +#ifdef CONFIG_OPENSSL_NO_DEPRECATED
> +       EVP_PKEY *private_key = NULL;
> +       OSSL_STORE_CTX *store;
> +
> +       if (!OSSL_PROVIDER_try_load(NULL, "pkcs11", true))
> +               ERR(1, "OSSL_PROVIDER_try_load(pkcs11)");
> +       if (!OSSL_PROVIDER_try_load(NULL, "default", true))
> +               ERR(1, "OSSL_PROVIDER_try_load(default)");
> +
> +       store = OSSL_STORE_open(path, NULL, NULL, NULL, NULL);
> +       ERR(!store, "OSSL_STORE_open");
> +
> +       while (!OSSL_STORE_eof(store)) {
> +               OSSL_STORE_INFO *info = OSSL_STORE_load(store);
> +
> +               if (!info) {
> +                       drain_openssl_errors(__LINE__, 0);
> +                       continue;
> +               }
> +               if (OSSL_STORE_INFO_get_type(info) == OSSL_STORE_INFO_PKEY) {
> +                       private_key = OSSL_STORE_INFO_get1_PKEY(info);
> +                       ERR(!private_key, "OSSL_STORE_INFO_get1_PKEY");
> +               }
> +               OSSL_STORE_INFO_free(info);
> +               if (private_key)
> +                       break;
> +       }
> +       OSSL_STORE_close(store);
> +
> +       *evpp = private_key;
> +#else

Wondering if it really makes sense to have the provider API implemented 
as an #ifdef to save like 10 common lines between key-based and 
provider-based implems.

On another topic, my first reading of the code makes me a bit worried by 
the fact that there's seemingly no way to select whether we want to use 
a local key or a provider key. Also, I see "pkcs11" and "default" here, 
but how do I select which provider I want to use (e.g. my custom one). 
If I somehow manage to have a key named the same locally and in one or 
more providers, how do I make sure the proper one is selected? I'm 
wondering whether we should be reusing the engine parameter to select a 
provider?

I have 0 security or crypto background, don't hesitate to be verbose in 
your answer.

Cheers,
Quentin