incompatible device trees between u-boot and linux

[François Ozog]

Hi,

Trying to respond to a U-Boot digest manually, please accept apologies
as I don't know how to do it properly...

>On Wed, Aug 25, 2021 at 06:43:23PM +0300, Vladimir Oltean wrote:
>> On Wed, Aug 25, 2021 at 11:24:08AM -0400, Tom Rini wrote:
>> > On Wed, Aug 25, 2021 at 06:12:20PM +0300, Vladimir Oltean wrote:
>> > > On Wed, Aug 25, 2021 at 10:26:10AM -0400, Tom Rini wrote:
>> > > > On Wed, Aug 25, 2021 at 05:18:16PM +0300, Vladimir Oltean wrote:
>> > > > > On Wed, Aug 25, 2021 at 10:00:45AM -0400, Tom Rini wrote:
>> > > > > > On Wed, Aug 25, 2021 at 03:58:10PM +0200, Michael Walle wrote:
>> > > > > >
>> > > > > > > Hi,
>> > > > > > >
>> > > > > > > I noticed that there is a fallback to the u-boot device tree for linux
>> > > > > > > (esp. EFI boot) if no other device tree was found, see [1]. It seems this
>> > > > > > > is working fine for imx devices, for example, where you can just boot a
>> > > > > > > stock installer iso via EFI. It will just work and it is quite a nice
>> > > > > > > feature as a fallback.
>> > > > > > >
>> > > > > > > Now for the layerscape architecture, the ls1028a in my case, things are
>> > > > > > > more difficult because the bindings differ between u-boot and linux - one
>> > > > > > > which comes to mind is DSA and ethernet.
>> > > > > > >
>> > > > > > > Which begs the general question, is it encouraged to have both bindings
>> > > > > > > diverge? To me it seems, that most bindings in u-boot are ad-hoc and there
>> > > > > > > is no real review or alignment but just added as needed, which is ok if
>> > > > > > > they are local to u-boot. But since they are nowadays passed to linux
>> > > > > > > (by default!) I'm not so sure anymore.
>> > > > > > >
>> > > > > > > OTOH The whole structure around a .dts{,i} and -u-boot.dtsi looks like
>> > > > > > > they should (could?) be shared between linux and u-boot.
>> > > > > > >
>> > > > > > > -michael
>> > > > > > >
>> > > > > > > [1]
>> > > > > > > https://elixir.bootlin.com/u-boot/v2021.10-rc2/source/common/board_r.c#L471
>> > > > > >
>> > > > > > The U-Boot device tree is supposed to be able to be passed on to Linux
>> > > > > > and Just Work. The bindings are not supposed to be different between
>> > > > > > the two (except for when we take the binding while it's being hashed out
>> > > > > > upstream BUT THEN RESYNCED).
>> > > > >
>> > > > > You might need to spell that out a bit clearer.
>> > > > >
>> > > > > You are saying that both U-Boot and Linux are allowed to have their own
>> > > > > custom properties (like 'u-boot,dm-spl' for U-Boot, and 'managed = "in-band-status"'
>> > > > > for Linux), as long as the device tree files themselves are in sync, and
>> > > > > the subset of the device tree blob understood by Linux (i.e. the U-Boot
>> > > > > blob sans the U-Boot specifics) is compatible with the Linux DT blob?
>> > > >
>> > > > I don't know what about the Linux example makes it Linux specific. But
>> > > > yes, 'u-boot,dm-spl' is clearly in our namespace and should be ignored
>> > > > by Linux. The whole reason we have the -u-boot.dtsi automatic drop-in
>> > > > logic (as much as it can be used is that device trees are device trees
>> > > ^
>> > > I don't think this parenthesis ever closes...
>> >
>> > Ah, whoops. Should have been "(as much as it can be used)" because it
>> > does get #included instead in some cases, for reasons.
>> >
>> > >
>> > > > and describe the hardware and developers don't need to write a device
>> > > > tree for Linux and a device tree for U-Boot and a device tree for
>> > > > FreeBSD and ... So yes, you're supposed to use the device tree for a
>> > > ^
>> > > so I never get the answer to "the whole reason is...".
>> > >
>> > > > platform and it works here and there and every where.
>> > >
>> > > The fact that only Linux uses it makes it Linux specific.
>> > >
>> > > > > To expand even further on that, it means we should put 'managed = "in-band-status"'
>> > > > > in U-Boot, which is a Linux phylink device tree property, even if U-Boot
>> > > > > does not use phylink?
>> > > >
>> > > > We should be able to drop in the device trees from Linux and use them.
>> > > > Custodians should be re-syncing them periodically. Some are, even.
>> > >
>> > > Are you ready to take up device tree bindings for PTP timers, PCIe root
>> > > complex event collectors, cascaded interrupt controllers, things which
>> > > U-Boot will never ever need to support?
>> > >
>> > > At least in Linux there is a policy to not add device tree nodes that do
>> > > not have drivers. Is the same policy not true for U-Boot? At least your
>> > > ./scripts/checkpatch.pl does have the same "check for DT compatible
>> > > documentation" section as Linux. You might consider removing it if you
>> > > want people to not strip the DTs they submit to U-Boot.
>> > >
>> > > And why do we even maintain the device tree bindings in Linux at all?
>> > > It seems rather counter-productive for both ends to do that, if it is
>> > > expected that the kernel works with DT blobs provided by third parties too,
>> > > and if all third parties need to resync with it (there are other boot
>> > > loaders too beyond U-Boot, and other kernels beyond Linux). Somehow it
>> > > doesn't feel right for the reference to be the Linux kernel. Maybe this
>> > > is something that needs to be brought up with higher-level Linux maintainers.
>> > >
>> > > I have no problem at all with structuring the device tree in the same
>> > > way in U-Boot as in Linux, as long as that proves to not be a foolish
>> > > endeavor.
>> >
>> > DT is ABI is hardware description and OS-agnostic has been the rule for
>> > 10+ years. If that's no longer the case, can someone please tell me?
>>
>> So if Michael's board with DT provided by U-Boot doesn't work for some
>> stupid reason like "Linux expects the pcie node to be under /soc", or
>> "Linux wants all PCIe BARs of a RCIEP ECAM to be spelled out in the
>> 'ranges' property, because it's too dumb to detect them itself", or
>> something like that, I've got no argument against that, let's go ahead
>> and resync U-Boot with Linux.
>>
>> But "DT is ABI is hardware description" is a pretty vague truism that
>> does not actually help here.
>
>I'm saying that because it's what's been said for what feels like 10+
>years. I don't want to think how many countless hours have been spent
>on that point at conferences over the years. It's not even a Linux
>thing. I would swear you can (or could, unless it got broken) take the
>same DTB for some platforms and boot Linux or FreeBSD or some other BSD
>or maybe even VxWorks and it works.
>
>
I cannot agree more.
For historical reasons in the embedded market, the product maker was
the board maker, the distro producer and the firmware producer. That
led to a situation where the Linux kernel developper was stating the
hardware description and that led to...problems. The best analogy I
have to describe how wrong this is: As a French driver, my Device Tree
description about the car is that the steering wheel is on the left
hand side of the car; when entering an English car, I just complain
that the wheel is not where it should be.
Now the value chain is going more "mature" and your may end up booting
a Fedora IoT edition on a bunch of platforms that have been designed
to boot any distro; and the Trusted Applications may come from the OEM
and not the board maker.

Organizing the software value chain along side of the hardware value
chain (Silicon provider, SoM provider, Carrier provider, Integrator,
Car manufacturer for instance) need DT technology and lifecycle
re-boot. (softwsystem up firmware such as Trusted Firmware, boot
firmware such as U-Boot and operating systems)

Some elements of discussion can be found here:
https://linaro.atlassian.net/wiki/spaces/DTE/overview
https://docs.google.com/document/d/1CLkhLRaz_zcCq44DLGmPZQFPbYHOC6nzPowaL0XmRk0/edit?usp=sharing

If we take the https://www.solid-run.com/arm-servers-networking-platforms/honeycomb-workstation/
board as an example to stay on NXP SoCs: what is the ideal way to deal
with device tree should we start from a blank sheet?

A key element are SerDes pins: from a SoC perspective they can be made
PCI lanes or Ethernet lanes. This is chosen by the board vendor. An
operating system should never manage the SerDes, it will see either
the PCI lanes or the Ethernet port (not exactly true yet because of
DPAA2 - but let's keep it high level). It is the responsibility of the
firmware to actually match the SoC configuration with board layout
(PCI slot, PCI device, Ethernet port).

In my mind, the ideal situation is: The board vendor takes the SoC DT
as input and applies fixups (at design time or runtime) to produce the
board DT. The board DT should be independent from the firmware stack
(Coreboot, TF-A+U-Boot, TF-A + LinuxBoot) and the booted OS: it is
just describing the hardware!!!
Firmware elements can still apply fixes at runtime: TF-A role is to
pass detected DRAM, U-Boot may have a parameter (saved as U-Boot
variable) to define active cores and ensures the DT is fixed up to
match that desire, OP-TEE can expose the amount of Secure DRAM it has
carved out through FF-A API.



>> Will you accept device trees with devices for which a driver will never
>> probe in U-Boot,
>
>Yes, I will absolutely take device trees that have devices we don't need
>in U-Boot since the point is, and many SoC vendors are doing (and the
>ones that aren't are, I am not happy about / with) right now.
>
>> and will you remove the checkpatch warnings about those,
>> to not discourage people from submitting them prior to the actual public
>> review?
>
>With respect to checkpatch.pl, maybe I'm just missing the line in
>question? Or maybe it's a kernel-related warning we need to disable in
>our .checkpatch.conf. But I don't want to side-track over this part.
>
>> If a U-Boot driver will be written for a device that does not have a
>> driver yet in Linux, then the Linux driver will be written but with
>> DT bindings incompatible with what was established in U-Boot, will you
>> wake up the U-Boot developer from the grave and ask them to resync the
>> driver to follow Linux? Or will you accept drivers at all for hardware
>> that is not supported by Linux?
>
>What I've said for years (but yes, I've missed changes, maybe the yaml
>dt binding stuff would help so I could make CI fail or at least require
>manual override?) is that U-Boot will take immature bindings but it's on
>the developer to re-sync once the bindings are fully reviewed. This is
>to help with the chicken-and-egg problem. But old bindings are not
>intended to be supported, once it's finalized. That is part of the
>bargain.
>
>> I also think there are various degrees of what it means "to work" with a
>> device tree provided directly by U-Boot. Distros like Arch Linux ARM
>> have a package for device tree blobs, and it is expected that these are
>> updated by the distro, and U-Boot just loads them. As Michael points out,
>> the DT provided by U-Boot is just a fallback. The OS should boot to
>> prompt and have a network connection to set itself up properly again.
>> But we need to draw a harder line on what we _actually_ desire to work
>> beyond that.
>
>Every distribution has a package for device tree binaries, because
>that's how you get a device tree on the still vast majority of platforms
>that don't ship one in-flash (RPis being the modern exception) and to my
>knowledge none of them are happy about having to build and pass and make
>sure the right one is used on a given board at boot. So yes, U-Boot
>being able to pass a device tree on to the next stage is one of those
>things to help everything Just Work and be boring.
>
>--
>Tom