[PATCH 1/1] tools: fix building with OpenSSL 4.0

[Peter Robinson]

Hi Quentin,

> >      In file included from tools/generated/lib/rsa/rsa-sign.c:1:
> >      ./tools/../lib/rsa/rsa-sign.c: In function ‘rsa_engine_get_pub_key’:
> >      ./tools/../lib/rsa/rsa-sign.c:115:9: warning:
> >      ‘ENGINE_get_id’ is deprecated: ENGINE_get_id API symbol is removed.
> >      Define OPENSSL_ENGINE_STUBS to mask linker errors.
> >      [-Wdeprecated-declarations]
> >        115 |         engine_id = ENGINE_get_id(engine);
> >            |         ^~~~~~~~~
> >
> >      rsa-sign.c: undefined reference to `ENGINE_load_builtin_engines'
> >      rsa-sign.c: undefined reference to `ENGINE_by_id'
> >      rsa-sign.c: undefined reference to `ENGINE_init'
> >      rsa-sign.c: undefined reference to `ENGINE_set_default_RSA'
> >      rsa-sign.c: undefined reference to `ENGINE_ctrl_cmd_string'
> >      rsa-sign.c: undefined reference to `ENGINE_free'
> >      rsa-sign.c: undefined reference to `ENGINE_finish
> >
> > Add -DOPENSSL_ENGINE_STUBS -Wno-deprecated-declarations to the
> > rsa-sign.c build flags.
> >
>
> Please no.
>
> We should not build OpenSSL engines when they aren't supported at all
> anymore.
>
> Fedora is already complaining it doesn't build on OpenSSL 3.x anymore
> for them. See
> https://lore.kernel.org/u-boot/20260429180247.83091-1-ekovsky@redhat.com/

To be very clear Red Hat != Fedora. I am the U-Boot maintianer in
Fedora, I do not work for Red Hat. That doesn't mean that Red Hat has
related interests.

That aside Fedora's openssl maintainers are employed by Red Hat and
Fedora has been leading on this and engine's are completely disabled
and unavailable in RHEL 10 even thought it shipped with openssl3.

Fedora 45 will ship with openssl 4, it landed in rawhide this week so
I know I already have build issues for the next build I do for U-Boot
2026.07 and that's the entry point for the latest versions of U-Boot
into that Red Hat derived ecosystem.

> The linked patch is not satisfactory though, and I've provided lengthy
> feedback a few times (although often with a big delay).
>
> I'm waiting on Eddie to answer as it's "gentleman agreement" in FOSS
> that the first who posts a patch gets to keep working on the patch until
> it gets satisfactory enough to be merged. I do admit I took a very long
> time a few times to answer so it didn't help with getting this forward.
> I am unsure how much longer we should wait for Eddie if more people are
> getting hit by this issue.
>
> As told on the linked patch, I have local patches (that I need to write
> nice commit logs for) to support OpenSSL providers and remove OpenSSL
> engine support for releases and variants of OpenSSL not supporting
> engines. With (binman) unit tests passing.
>
> Note that this patch here is also not sufficient as, sure, it makes it
> possible to build U-Boot again, but you won't be able to run the test
> suite as we build dummy-rsa-engine.c unconditionally and you'll have the
> same build issue then.
>
> As said in the linked patch, I would welcome an immediate patch that is
> disabling OpenSSL engine support with a big ifdef (still won't fix the
> "dummy-rsa-engine"-based tests but I haven't come up with a way to fix
> this "nicely").
>
> Cheers,
> Quentin