[PATCH v2 2/2] Makefile: Only build dtc if needed

[Tom Rini]

On Tue, Apr 28, 2020 at 09:41:14AM -0600, Simon Glass wrote:
> Hi Tom.
> 
> On Tue, 28 Apr 2020 at 08:19, Tom Rini <trini at konsulko.com> wrote:
> >
> > On Mon, Apr 27, 2020 at 04:10:06PM -0700, Vagrant Cascadian wrote:
> > > On 2020-04-27, Simon Glass wrote:
> > > > On Sun, 26 Apr 2020 at 18:58, Heinrich Schuchardt <xypron.glpk at gmx.de> wrote:
> > > >>
> > > >> Am April 27, 2020 12:29:29 AM UTC schrieb Simon Glass <sjg at chromium.org>:
> > > >> >At present U-Boot always builds dtc if CONFIG_OF_CONTROL is defined.
> > > >> >This
> > > >> >is wasteful when the system already has a suitable version available.
> > > >> >
> > > >> >Update the Makefile logic to build dtc only if the version available is
> > > >> >too old.
> > > >> >
> > > >> >This saves about 2.5 seconds of elapsed time on a clean build for me.
> > > >> >
> > > >> >- Add a patch to bring back the dtc-version.sh script
> > > >> >- Update the check to make sure libfdt is available if needed
> > > >>
> > > >> U -Boot has been set up to create reproducible builds. With this
> > > >> patch dtc will have to be made a build dependency to provide
> > > >> reproducibility. Cf. https://www.debian.org/doc/debian-policy/ch-source.html#reproducibility
> > > >>
> > > >> This may require changes in the packaging of U-Boot in Linux
> > > >> distributions. Nothing to stop this patch, just something to keep in
> > > >> mind.
> > > >>
> > > >> You presume that future versions of dtc will always be backward
> > > >> compatible with U-Boot. Ok, we do the same for other tools like gcc
> > > >> too (with surprises at each new major release).
> > >
> > > In general when packaging for Debian, the preference is to not use
> > > embedded code copies if at all possible. This does require paying
> > > attention to backwards and forwards compatibility issues a bit.
> > >
> > > A simple example: The security team in Debian generally likes to fix a
> > > problem in a single source package, rather than an arbitrary number of
> > > source packages that happen to share some embedded copy of the code from
> > > who knows when...
> > >
> > > So at least from my perspective, I'd be happy to use the Debian packaged
> > > dtc (a.k.a. device-tree-compiler), rather than the one embedded in
> > > u-boot sources.
> > >
> > > Silently switching to the embedded copy sounds a little scary; I would
> > > prefer for that to be explicit... a build flag to specify one way or the
> > > other and failing is better that being too clever about autodetecting.
> > >
> > >
> > > > Should we disable this check (and always build dtc) when doing a
> > > > repeatable build? Is that SOURCE_DATE_EPOCH?
> > >
> > > And with my Reproducible Builds hat on, builds would ideally *always* be
> > > reproducible, given the same sources and toolchain... several
> > > distributions and software projects provide information sufficient to
> > > reproduce the build environment:
> > >
> > >   https://reproducible-builds.org/docs/recording/
> > >
> > >
> > > While SOURCE_DATE_EPOCH is definitely one sign that the builder is
> > > explicitly attempting to be reproducible; It's a bit of a kludge to try
> > > and be more reproducible just because SOURCE_DATE_EPOCH is
> > > set. SOURCE_DATE_EPOCH should really only affect the behavior of date or
> > > time related things; even better would be to not embded time related
> >
> > This is probably one of those cases where we should just continue to act
> > like the linux kernel and always use and build our own copy of dtc.
> > Then, when someone convinces the kernel folks to change their ways, we
> > can adopt that.
> 
> It seems that Vagrant wants to use the system dtc by default and
> require an explicit option to use the in-built dtc. I don't think that
> would work well for most users though.

Right, and this is where I disagree and point to the kernel.  Get that
changed first.

> It is in my view somewhat mad to build dtc for every one of 1400
> boards as I do today when running buildman.

This is a different funny case.  Perhaps ccache could be helpful here?
I think the way it's used in OpenEmbedded, such that you have a cache
that's more local to what's building vs global cache, could be helpful
here too.  A ccache instance per CI job / world build could help.  A
flag to buildman to support that could help, yes?  Thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200428/8a0ea818/attachment.sig>