[PATCH v2 02/13] Makefile: Allow LTO to be disabled for a build

[Simon Glass]

Hi Tom,

On Mon, 14 Mar 2022 at 16:42, Tom Rini <trini at konsulko.com> wrote:
>
> On Mon, Mar 14, 2022 at 03:43:05PM -0600, Simon Glass wrote:
> > Hi Tom,
> >
> > On Mon, 14 Mar 2022 at 14:23, Tom Rini <trini at konsulko.com> wrote:
> > >
> > > On Mon, Mar 14, 2022 at 02:18:14PM -0600, Simon Glass wrote:
> > > > Hi Tom,
> > > >
> > > > On Mon, 14 Mar 2022 at 13:45, Tom Rini <trini at konsulko.com> wrote:
> > > > >
> > > > > On Mon, Mar 14, 2022 at 01:21:02PM -0600, Simon Glass wrote:
> > > > > > Hi Tom,
> > > > > >
> > > > > > On Mon, 14 Mar 2022 at 12:29, Tom Rini <trini at konsulko.com> wrote:
> > > > > > >
> > > > > > > On Mon, Mar 14, 2022 at 12:24:42PM -0600, Simon Glass wrote:
> > > > > > > > Hi Tom,
> > > > > > > >
> > > > > > > > On Mon, 14 Mar 2022 at 06:49, Tom Rini <trini at konsulko.com> wrote:
> > > > > > > > >
> > > > > > > > > On Sat, Mar 12, 2022 at 10:58:44AM -0700, Simon Glass wrote:
> > > > > > > > > > Hi Tom,
> > > > > > > > > >
> > > > > > > > > > On Mon, 7 Mar 2022 at 07:33, Tom Rini <trini at konsulko.com> wrote:
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Mar 04, 2022 at 08:42:57AM -0700, Simon Glass wrote:
> > > > > > > > > > >
> > > > > > > > > > > > LTO (Link-Time Optimisation) is an very useful feature which can
> > > > > > > > > > > > significantly reduce the size of U-Boot binaries. So far it has been
> > > > > > > > > > > > made available for selected ARM boards and sandbox.
> > > > > > > > > > > >
> > > > > > > > > > > > However, incremental builds are much slower when LTO is used. For example,
> > > > > > > > > > > > an incremental build of sandbox takes 2.1 seconds on my machine, but 6.7
> > > > > > > > > > > > seconds with LTO enabled.
> > > > > > > > > > > >
> > > > > > > > > > > > Add a LTO_BUILD=n parameter to the build, so it can be disabled during
> > > > > > > > > > > > development if needed, for faster builds.
> > > > > > > > > > > >
> > > > > > > > > > > > Add some documentation about LTO while we are here.
> > > > > > > > > > > >
> > > > > > > > > > > > Signed-off-by: Simon Glass <sjg at chromium.org>
> > > > > > > > > > >
> > > > > > > > > > > We don't need this since you can do:
> > > > > > > > > > > make EXTRA_CFLAGS="-fno-lto" EXTRA_LDFLAGS="-fno-lto"
> > > > > > > > > > > to pass -fno-lto to compile/linking and disable lto and per
> > > > > > > > > > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=46905 this has been working
> > > > > > > > > > > for some time.
> > > > > > > > > >
> > > > > > > > > > Thanks for that, it is a big pain point for me, picking up this patch
> > > > > > > > > > for every series I write. The incremental build time for sandbox goes
> > > > > > > > > > from 3 seconds to 27 seconds on my laptop with LTO, which is
> > > > > > > > > > intolerable.
> > > > > > > > >
> > > > > > > > > Yeah, I noticed it's visible on my laptop, but not at all on my desktop
> > > > > > > > > (i7 vs Ryzen 7).
> > > > > > > > >
> > > > > > > > > > The EXTRA_CFLAGS says it is for 'Backward compatibility' and it still
> > > > > > > > > > does the various LTO things (i.e. it changes the build logic). It
> > > > > > > > >
> > > > > > > > > We're unlikely to move to newer Linux kernel kbuild logic so this isn't
> > > > > > > > > going away, and there's not much in the way of logic that's changed for
> > > > > > > > > LTO that I see.
> > > > > > > > >
> > > > > > > > > > seems odd to me to enable the option and then disable it later in the
> > > > > > > > > > command line. It is therefore not quite equivalent. But it seems to
> > > > > > > > > > work well enough for me fom a small amount of testing. If you are
> > > > > > > > > > really set on not having a special option for it, I can live with it
> > > > > > > > > > for now. I'm also not convinced that my patch entirely removes the LTO
> > > > > > > > > > stuff in a consistent way.
> > > > > > > > >
> > > > > > > > > Yeah, I really don't want to go down the path of overriding CONFIG
> > > > > > > > > options via make/environment logic.  I'm also open to turning off LTO on
> > > > > > > > > sandbox and on with qemu-* so it gets wider CI testing.
> > > > > > > >
> > > > > > > > Yes you did mention that, but the problem is that LTO is very handy
> > > > > > > > with sandbox, to test the strange things that happen. For example, I
> > > > > > > > found the bug where LTO was dropping a linker-list item, using
> > > > > > > > sandbox. We could perhaps make one of the sandbox builds not use LTO,
> > > > > > > > e.g. sandbox_flattree ?
> > > > > > >
> > > > > > > Well, the big issue with LTO+sandbox is that it slows down your
> > > > > > > workflow, so I would think you want the inverse, one platform does
> > > > > > > enable it?
> > > > > >
> > > > > > It slows down all boards that use it, actually, so I normally don't
> > > > > > want it enabled for my IDE. This is not sandbox-specific.
> > > > >
> > > > > The degree of slowdown depends on what you're building on.  I see
> > > > > several seconds of link time on my laptop and nothing on my desktop
> > > > > (which isn't that high end either, it's not an EPYC or anything) when
> > > > > LTO is/isn't enabled.
> > > >
> > > > For me, on 16-core AMD desktop, incremental build
> > > >
> > > > LTO: real 0m7.744s
> > > > No LTO: real 0m3.172s
> > > >
> > > > On 4-core laptop:
> > > >
> > > > No LTO: real 0m2.429s
> > > > LTO: real 0m15.650s
> > > >
> > > > Perhaps the disconnect here is that you just don't see any difference?
> > > > What times are you seeing?
> > >
> > > My 16 core desktop is 14 seconds without, 15 seconds with (full build,
> > > not just the link time) and I don't recall what my laptop was but since
> > > it's longer than the desktop I'm always just ssh'd in there anyhow.
> >
> > Seems very slow. Are you sure that is an incremental build?
>
> No, it's a from-scratch build, which should be the worst case for this.

OK. So I don't do fresh builds very often, mostly they are incremental
(for obvious reasons...I am developing things incrementally). So the
difference is quite dramatic, as you can see.

Does my problem make more sense now?

Regards,
Simon