[PATCH v2 2/2] Makefile: Only build dtc if needed

[Simon Glass]

Hi Tom,

On Tue, 28 Apr 2020 at 09:52, Tom Rini <trini at konsulko.com> wrote:
>
> On Tue, Apr 28, 2020 at 09:41:14AM -0600, Simon Glass wrote:
> > Hi Tom.
> >
> > On Tue, 28 Apr 2020 at 08:19, Tom Rini <trini at konsulko.com> wrote:
> > >
> > > On Mon, Apr 27, 2020 at 04:10:06PM -0700, Vagrant Cascadian wrote:
> > > > On 2020-04-27, Simon Glass wrote:
> > > > > On Sun, 26 Apr 2020 at 18:58, Heinrich Schuchardt <xypron.glpk at gmx.de> wrote:
> > > > >>
> > > > >> Am April 27, 2020 12:29:29 AM UTC schrieb Simon Glass <sjg at chromium.org>:
> > > > >> >At present U-Boot always builds dtc if CONFIG_OF_CONTROL is defined.
> > > > >> >This
> > > > >> >is wasteful when the system already has a suitable version available.
> > > > >> >
> > > > >> >Update the Makefile logic to build dtc only if the version available is
> > > > >> >too old.
> > > > >> >
> > > > >> >This saves about 2.5 seconds of elapsed time on a clean build for me.
> > > > >> >
> > > > >> >- Add a patch to bring back the dtc-version.sh script
> > > > >> >- Update the check to make sure libfdt is available if needed
> > > > >>
> > > > >> U -Boot has been set up to create reproducible builds. With this
> > > > >> patch dtc will have to be made a build dependency to provide
> > > > >> reproducibility. Cf. https://www.debian.org/doc/debian-policy/ch-source.html#reproducibility
> > > > >>
> > > > >> This may require changes in the packaging of U-Boot in Linux
> > > > >> distributions. Nothing to stop this patch, just something to keep in
> > > > >> mind.
> > > > >>
> > > > >> You presume that future versions of dtc will always be backward
> > > > >> compatible with U-Boot. Ok, we do the same for other tools like gcc
> > > > >> too (with surprises at each new major release).
> > > >
> > > > In general when packaging for Debian, the preference is to not use
> > > > embedded code copies if at all possible. This does require paying
> > > > attention to backwards and forwards compatibility issues a bit.
> > > >
> > > > A simple example: The security team in Debian generally likes to fix a
> > > > problem in a single source package, rather than an arbitrary number of
> > > > source packages that happen to share some embedded copy of the code from
> > > > who knows when...
> > > >
> > > > So at least from my perspective, I'd be happy to use the Debian packaged
> > > > dtc (a.k.a. device-tree-compiler), rather than the one embedded in
> > > > u-boot sources.
> > > >
> > > > Silently switching to the embedded copy sounds a little scary; I would
> > > > prefer for that to be explicit... a build flag to specify one way or the
> > > > other and failing is better that being too clever about autodetecting.
> > > >
> > > >
> > > > > Should we disable this check (and always build dtc) when doing a
> > > > > repeatable build? Is that SOURCE_DATE_EPOCH?
> > > >
> > > > And with my Reproducible Builds hat on, builds would ideally *always* be
> > > > reproducible, given the same sources and toolchain... several
> > > > distributions and software projects provide information sufficient to
> > > > reproduce the build environment:
> > > >
> > > >   https://reproducible-builds.org/docs/recording/
> > > >
> > > >
> > > > While SOURCE_DATE_EPOCH is definitely one sign that the builder is
> > > > explicitly attempting to be reproducible; It's a bit of a kludge to try
> > > > and be more reproducible just because SOURCE_DATE_EPOCH is
> > > > set. SOURCE_DATE_EPOCH should really only affect the behavior of date or
> > > > time related things; even better would be to not embded time related
> > >
> > > This is probably one of those cases where we should just continue to act
> > > like the linux kernel and always use and build our own copy of dtc.
> > > Then, when someone convinces the kernel folks to change their ways, we
> > > can adopt that.
> >
> > It seems that Vagrant wants to use the system dtc by default and
> > require an explicit option to use the in-built dtc. I don't think that
> > would work well for most users though.
>
> Right, and this is where I disagree and point to the kernel.  Get that
> changed first.
>
> > It is in my view somewhat mad to build dtc for every one of 1400
> > boards as I do today when running buildman.
>
> This is a different funny case.  Perhaps ccache could be helpful here?
> I think the way it's used in OpenEmbedded, such that you have a cache
> that's more local to what's building vs global cache, could be helpful
> here too.  A ccache instance per CI job / world build could help.  A
> flag to buildman to support that could help, yes?  Thanks!

So not allow using the system dtc? Or are you OK with a build option?

The thing is I would prefer to avoid another level of cache for a
problem that only exists because of our kernel design decision.

As to changing the kernel, I cannot imagine that happening as they
change dtc all the time.

Regards,
Simon