[PATCH 1/1] tools: fix building with OpenSSL 4.0

[Quentin Schulz]

Hi Heinrich,

On 6/15/26 6:07 PM, Heinrich Schuchardt wrote:
> Building with OpenSSL 4.0 fails:
> 
>      In file included from tools/generated/lib/rsa/rsa-sign.c:1:
>      ./tools/../lib/rsa/rsa-sign.c: In function ‘rsa_engine_get_pub_key’:
>      ./tools/../lib/rsa/rsa-sign.c:115:9: warning:
>      ‘ENGINE_get_id’ is deprecated: ENGINE_get_id API symbol is removed.
>      Define OPENSSL_ENGINE_STUBS to mask linker errors.
>      [-Wdeprecated-declarations]
>        115 |         engine_id = ENGINE_get_id(engine);
>            |         ^~~~~~~~~
> 
>      rsa-sign.c: undefined reference to `ENGINE_load_builtin_engines'
>      rsa-sign.c: undefined reference to `ENGINE_by_id'
>      rsa-sign.c: undefined reference to `ENGINE_init'
>      rsa-sign.c: undefined reference to `ENGINE_set_default_RSA'
>      rsa-sign.c: undefined reference to `ENGINE_ctrl_cmd_string'
>      rsa-sign.c: undefined reference to `ENGINE_free'
>      rsa-sign.c: undefined reference to `ENGINE_finish
> 
> Add -DOPENSSL_ENGINE_STUBS -Wno-deprecated-declarations to the
> rsa-sign.c build flags.
> 

Please no.

We should not build OpenSSL engines when they aren't supported at all 
anymore.

Fedora is already complaining it doesn't build on OpenSSL 3.x anymore 
for them. See 
https://lore.kernel.org/u-boot/20260429180247.83091-1-ekovsky@redhat.com/

The linked patch is not satisfactory though, and I've provided lengthy 
feedback a few times (although often with a big delay).

I'm waiting on Eddie to answer as it's "gentleman agreement" in FOSS 
that the first who posts a patch gets to keep working on the patch until 
it gets satisfactory enough to be merged. I do admit I took a very long 
time a few times to answer so it didn't help with getting this forward. 
I am unsure how much longer we should wait for Eddie if more people are 
getting hit by this issue.

As told on the linked patch, I have local patches (that I need to write 
nice commit logs for) to support OpenSSL providers and remove OpenSSL 
engine support for releases and variants of OpenSSL not supporting 
engines. With (binman) unit tests passing.

Note that this patch here is also not sufficient as, sure, it makes it 
possible to build U-Boot again, but you won't be able to run the test 
suite as we build dummy-rsa-engine.c unconditionally and you'll have the 
same build issue then.

As said in the linked patch, I would welcome an immediate patch that is 
disabling OpenSSL engine support with a big ifdef (still won't fix the 
"dummy-rsa-engine"-based tests but I haven't come up with a way to fix 
this "nicely").

Cheers,
Quentin