[PATCH v2 2/2] Makefile: Only build dtc if needed

[Tom Rini]

On Tue, Apr 28, 2020 at 04:40:47PM -0600, Simon Glass wrote:
> Hi Tom,
> 
> On Tue, 28 Apr 2020 at 09:52, Tom Rini <trini at konsulko.com> wrote:
> >
> > On Tue, Apr 28, 2020 at 09:41:14AM -0600, Simon Glass wrote:
> > > Hi Tom.
> > >
> > > On Tue, 28 Apr 2020 at 08:19, Tom Rini <trini at konsulko.com> wrote:
> > > >
> > > > On Mon, Apr 27, 2020 at 04:10:06PM -0700, Vagrant Cascadian wrote:
> > > > > On 2020-04-27, Simon Glass wrote:
> > > > > > On Sun, 26 Apr 2020 at 18:58, Heinrich Schuchardt <xypron.glpk at gmx.de> wrote:
> > > > > >>
> > > > > >> Am April 27, 2020 12:29:29 AM UTC schrieb Simon Glass <sjg at chromium.org>:
> > > > > >> >At present U-Boot always builds dtc if CONFIG_OF_CONTROL is defined.
> > > > > >> >This
> > > > > >> >is wasteful when the system already has a suitable version available.
> > > > > >> >
> > > > > >> >Update the Makefile logic to build dtc only if the version available is
> > > > > >> >too old.
> > > > > >> >
> > > > > >> >This saves about 2.5 seconds of elapsed time on a clean build for me.
> > > > > >> >
> > > > > >> >- Add a patch to bring back the dtc-version.sh script
> > > > > >> >- Update the check to make sure libfdt is available if needed
> > > > > >>
> > > > > >> U -Boot has been set up to create reproducible builds. With this
> > > > > >> patch dtc will have to be made a build dependency to provide
> > > > > >> reproducibility. Cf. https://www.debian.org/doc/debian-policy/ch-source.html#reproducibility
> > > > > >>
> > > > > >> This may require changes in the packaging of U-Boot in Linux
> > > > > >> distributions. Nothing to stop this patch, just something to keep in
> > > > > >> mind.
> > > > > >>
> > > > > >> You presume that future versions of dtc will always be backward
> > > > > >> compatible with U-Boot. Ok, we do the same for other tools like gcc
> > > > > >> too (with surprises at each new major release).
> > > > >
> > > > > In general when packaging for Debian, the preference is to not use
> > > > > embedded code copies if at all possible. This does require paying
> > > > > attention to backwards and forwards compatibility issues a bit.
> > > > >
> > > > > A simple example: The security team in Debian generally likes to fix a
> > > > > problem in a single source package, rather than an arbitrary number of
> > > > > source packages that happen to share some embedded copy of the code from
> > > > > who knows when...
> > > > >
> > > > > So at least from my perspective, I'd be happy to use the Debian packaged
> > > > > dtc (a.k.a. device-tree-compiler), rather than the one embedded in
> > > > > u-boot sources.
> > > > >
> > > > > Silently switching to the embedded copy sounds a little scary; I would
> > > > > prefer for that to be explicit... a build flag to specify one way or the
> > > > > other and failing is better that being too clever about autodetecting.
> > > > >
> > > > >
> > > > > > Should we disable this check (and always build dtc) when doing a
> > > > > > repeatable build? Is that SOURCE_DATE_EPOCH?
> > > > >
> > > > > And with my Reproducible Builds hat on, builds would ideally *always* be
> > > > > reproducible, given the same sources and toolchain... several
> > > > > distributions and software projects provide information sufficient to
> > > > > reproduce the build environment:
> > > > >
> > > > >   https://reproducible-builds.org/docs/recording/
> > > > >
> > > > >
> > > > > While SOURCE_DATE_EPOCH is definitely one sign that the builder is
> > > > > explicitly attempting to be reproducible; It's a bit of a kludge to try
> > > > > and be more reproducible just because SOURCE_DATE_EPOCH is
> > > > > set. SOURCE_DATE_EPOCH should really only affect the behavior of date or
> > > > > time related things; even better would be to not embded time related
> > > >
> > > > This is probably one of those cases where we should just continue to act
> > > > like the linux kernel and always use and build our own copy of dtc.
> > > > Then, when someone convinces the kernel folks to change their ways, we
> > > > can adopt that.
> > >
> > > It seems that Vagrant wants to use the system dtc by default and
> > > require an explicit option to use the in-built dtc. I don't think that
> > > would work well for most users though.
> >
> > Right, and this is where I disagree and point to the kernel.  Get that
> > changed first.
> >
> > > It is in my view somewhat mad to build dtc for every one of 1400
> > > boards as I do today when running buildman.
> >
> > This is a different funny case.  Perhaps ccache could be helpful here?
> > I think the way it's used in OpenEmbedded, such that you have a cache
> > that's more local to what's building vs global cache, could be helpful
> > here too.  A ccache instance per CI job / world build could help.  A
> > flag to buildman to support that could help, yes?  Thanks!
> 
> So not allow using the system dtc? Or are you OK with a build option?

I'd rather not have a build option as that's going to encourage people
to use it, and then that'll lead to problems down the line.

> The thing is I would prefer to avoid another level of cache for a
> problem that only exists because of our kernel design decision.
> 
> As to changing the kernel, I cannot imagine that happening as they
> change dtc all the time.

We really need to stay better in sync with the kernel here too.  Trying
to get more syncs of kbuild/kconfig/dtc/etc with kernel releases is on
my list.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200430/32721f73/attachment.sig>