[PATCH v2 2/2] Makefile: Only build dtc if needed

[Simon Glass]

Hi Tom,

On Thu, 30 Apr 2020 at 09:05, Tom Rini <trini at konsulko.com> wrote:

> On Tue, Apr 28, 2020 at 04:40:47PM -0600, Simon Glass wrote:
> > Hi Tom,
> >
> > On Tue, 28 Apr 2020 at 09:52, Tom Rini <trini at konsulko.com> wrote:
> > >
> > > On Tue, Apr 28, 2020 at 09:41:14AM -0600, Simon Glass wrote:
> > > > Hi Tom.
> > > >
> > > > On Tue, 28 Apr 2020 at 08:19, Tom Rini <trini at konsulko.com> wrote:
> > > > >
> > > > > On Mon, Apr 27, 2020 at 04:10:06PM -0700, Vagrant Cascadian wrote:
> > > > > > On 2020-04-27, Simon Glass wrote:
> > > > > > > On Sun, 26 Apr 2020 at 18:58, Heinrich Schuchardt <
> xypron.glpk at gmx.de> wrote:
> > > > > > >>
> > > > > > >> Am April 27, 2020 12:29:29 AM UTC schrieb Simon Glass <
> sjg at chromium.org>:
> > > > > > >> >At present U-Boot always builds dtc if CONFIG_OF_CONTROL is
> defined.
> > > > > > >> >This
> > > > > > >> >is wasteful when the system already has a suitable version
> available.
> > > > > > >> >
> > > > > > >> >Update the Makefile logic to build dtc only if the version
> available is
> > > > > > >> >too old.
> > > > > > >> >
> > > > > > >> >This saves about 2.5 seconds of elapsed time on a clean
> build for me.
> > > > > > >> >
> > > > > > >> >- Add a patch to bring back the dtc-version.sh script
> > > > > > >> >- Update the check to make sure libfdt is available if needed
> > > > > > >>
> > > > > > >> U -Boot has been set up to create reproducible builds. With
> this
> > > > > > >> patch dtc will have to be made a build dependency to provide
> > > > > > >> reproducibility. Cf.
> https://www.debian.org/doc/debian-policy/ch-source.html#reproducibility
> > > > > > >>
> > > > > > >> This may require changes in the packaging of U-Boot in Linux
> > > > > > >> distributions. Nothing to stop this patch, just something to
> keep in
> > > > > > >> mind.
> > > > > > >>
> > > > > > >> You presume that future versions of dtc will always be
> backward
> > > > > > >> compatible with U-Boot. Ok, we do the same for other tools
> like gcc
> > > > > > >> too (with surprises at each new major release).
> > > > > >
> > > > > > In general when packaging for Debian, the preference is to not
> use
> > > > > > embedded code copies if at all possible. This does require paying
> > > > > > attention to backwards and forwards compatibility issues a bit.
> > > > > >
> > > > > > A simple example: The security team in Debian generally likes to
> fix a
> > > > > > problem in a single source package, rather than an arbitrary
> number of
> > > > > > source packages that happen to share some embedded copy of the
> code from
> > > > > > who knows when...
> > > > > >
> > > > > > So at least from my perspective, I'd be happy to use the Debian
> packaged
> > > > > > dtc (a.k.a. device-tree-compiler), rather than the one embedded
> in
> > > > > > u-boot sources.
> > > > > >
> > > > > > Silently switching to the embedded copy sounds a little scary; I
> would
> > > > > > prefer for that to be explicit... a build flag to specify one
> way or the
> > > > > > other and failing is better that being too clever about
> autodetecting.
> > > > > >
> > > > > >
> > > > > > > Should we disable this check (and always build dtc) when doing
> a
> > > > > > > repeatable build? Is that SOURCE_DATE_EPOCH?
> > > > > >
> > > > > > And with my Reproducible Builds hat on, builds would ideally
> *always* be
> > > > > > reproducible, given the same sources and toolchain... several
> > > > > > distributions and software projects provide information
> sufficient to
> > > > > > reproduce the build environment:
> > > > > >
> > > > > >   https://reproducible-builds.org/docs/recording/
> > > > > >
> > > > > >
> > > > > > While SOURCE_DATE_EPOCH is definitely one sign that the builder
> is
> > > > > > explicitly attempting to be reproducible; It's a bit of a kludge
> to try
> > > > > > and be more reproducible just because SOURCE_DATE_EPOCH is
> > > > > > set. SOURCE_DATE_EPOCH should really only affect the behavior of
> date or
> > > > > > time related things; even better would be to not embded time
> related
> > > > >
> > > > > This is probably one of those cases where we should just continue
> to act
> > > > > like the linux kernel and always use and build our own copy of dtc.
> > > > > Then, when someone convinces the kernel folks to change their
> ways, we
> > > > > can adopt that.
> > > >
> > > > It seems that Vagrant wants to use the system dtc by default and
> > > > require an explicit option to use the in-built dtc. I don't think
> that
> > > > would work well for most users though.
> > >
> > > Right, and this is where I disagree and point to the kernel.  Get that
> > > changed first.
> > >
> > > > It is in my view somewhat mad to build dtc for every one of 1400
> > > > boards as I do today when running buildman.
> > >
> > > This is a different funny case.  Perhaps ccache could be helpful here?
> > > I think the way it's used in OpenEmbedded, such that you have a cache
> > > that's more local to what's building vs global cache, could be helpful
> > > here too.  A ccache instance per CI job / world build could help.  A
> > > flag to buildman to support that could help, yes?  Thanks!
> >
> > So not allow using the system dtc? Or are you OK with a build option?
>
> I'd rather not have a build option as that's going to encourage people
> to use it, and then that'll lead to problems down the line.
>

OK, let's drop this patch then. If I can think of some other way, I'll let
you know.


>
> > The thing is I would prefer to avoid another level of cache for a
> > problem that only exists because of our kernel design decision.
> >
> > As to changing the kernel, I cannot imagine that happening as they
> > change dtc all the time.
>
> We really need to stay better in sync with the kernel here too.  Trying
> to get more syncs of kbuild/kconfig/dtc/etc with kernel releases is on
> my list.
>
> --
> Tom
>
-------------- next part --------------
-----BEGIN PGP SIGNATURE-----

iQGzBAABCgAdFiEEGjx/cOCPqxcHgJu/FHw5/5Y0tywFAl6q6TsACgkQFHw5/5Y0
tyweFwv/Yq/gxrCtv21GHYlvjuYf5Aj1WuF1aiciQo8VpQuBpG2XpXT9ZVBDsoXH
3lTbKWbziBvREJ9kU9LL+dXS+fDwX66qwjFItGeYgOLE+ejuL1bkOcpBmdgKDrkH
MzDIv9W22yNRbh42C3v5OB6+94D4QLbmtCTcSC8iOvIaERmTp1sI6RIat7X6I4yL
VYZAbFKxm6gsSD+sNOGDLo3aQk48h01bNI52egJNXDzedim7xBdob7zgFkePMTSH
g1Pny/fQ9jVqMJAlMPvV0Dp1JAeldiCrNjWF06tR1PNZRyX4wVo+R5NDJL8dRfZr
PRypdsrI0uxbgdkmQykBf1KhgnsAqo28rrc+Wv7hvmAtSXpd7DDaaNB7dCbn53gz
j5XlkBChYH5ufub6+bjA85kGNBEdXXgxjuPqunLiN7uhO7Wve8DAu052l7aLR8Y9
zOpcQnVplyYkWdbFqP6vJMtV6lkLF89rELeIdJ5ifUjig6zdIvoyhuLOUD6g+aS6
aMRpRT8T
=Ropl
-----END PGP SIGNATURE-----