[PATCH v7 01/31] doc: Add documentation about devicetree usage

[Tom Rini]

On Mon, Dec 06, 2021 at 05:11:39PM -0700, Simon Glass wrote:

[snip]
> +Building the devicetree
> +-----------------------
> +
> +U-Boot automatically builds the devicetree for a board, from the
> +`arch/<arch>/dts` directory. The Makefile in those directories has rules for
> +building devicetree files. It is preferable to avoid target-specific rules in
> +those files: i.e. all boards for a particular SoC should be built at once,
> +where practical. Apart from simplifying the Makefile, this helps to efficiently
> +(and immediately) ensure that changes in one board's DT do not break others that
> +are related. Building devicetrees is fast, so performance is seldom a concern
> +here.

In that it makes our Makefile more like the Linux one, yes, this is
fine.  But that's why we're doing it, familiarity with Linux.  Device
trees shouldn't be changing all the time!

> +Overriding the default devicetree
> +---------------------------------
> +
> +When building U-Boot, the `DEVICE_TREE` environment variable allows the
> +default devicetree file to be overridden at build time. This can be useful if
> +modifications have to be made to the in-tree devicetree file, for the benefit
> +of a downstream build system.

This is fine.

> Note that the in-tree devicetree must be
> +sufficient to build and boot, so this is not a way to bypass that requirement.

This doesn't make sense.  Either we're CONFIG_OF_BOARD=n and so yes, the
device tree must work or we're CONFIG_OF_BOARD=y and this will never
ever be used by anything.  We shouldn't even be needing to build it.

> +Modifying the devicetree after building
> +---------------------------------------
> +
> +While it is generally painful and hacky to modify the code or rodata of a
> +program after it is built, in many cases it is useful to do so, e.g. to add
> +configuration information like serial numbers, enabling/disabling features, etc.
> +
> +Devicetree provides a very nice solution to these problems since it is
> +structured data and it is relatively easy to change it, even in binary form
> +(see fdtput).
> +
> +U-Boot takes care that the devicetree is easily accessible after the build
> +process. In fact it is placed in a separate file called `u-boot.dtb`. If the
> +build system wants to modify or replace that file, it can do so. Then all that
> +is needed is to run `binman update` to update the file inside the image. If
> +binman is not used, then `u-boot-nodtb.bin` and the new `u-boot.dtb` can simply
> +be concatenated to achieve the desired result. U-Boot happily copes with the
> +devicetree growing or shrinking.

This can be handy, yes.  But it's also an optional thing because
CONFIG_OF_BOARD=y means we're always going to use that run time
provided tree.  If we're going to support something like this with
CONFIG_OF_BOARD=y we'll need to come up with some solution, probably
doing one of the things you've listed later on as being too complex, but
I believe we already do in at least one case (rcar3 applies an overlay
to dtb passed in from TF-A).

> +The `u-boot.bin` image contains both pieces. While it is possible to locate the
> +devicetree within the image using the signature at the start of the file, this
> +is a bit messy.
> +
> +This is why `CONFIG_OF_SEPARATE` should always be used when building U-Boot.
> +The `CONFIG_OF_EMBED` option embeds the devicetree somewhere in the U-Boot ELF
> +image as rodata, meaning that it is hard to find it and it cannot increase in
> +size.

We should really figure out and fix up / remove CONFIG_OF_EMBED.  But
that needs to be another thread with the relevant board maintainers to
see why they did what they did to start with.

> +When modifying the devicetree, the different cases to consider are as follows:
> +
> +- CONFIG_OF_SEPARATE
> +    This is easy, described above. Just change, replace or rebuild the
> +    devicetree so it suits your needs, then rerun binman or redo the `cat`
> +    operation to join `u-boot-nodtb.bin` and the new `u-boot.dtb`

Yes.

> +- CONFIG_OF_EMBED
> +    This is tricky, since the devicetree cannot easily be located. If the EFL
> +    file is available, then the _dtb_dt_begin and __dtb_dt_end symbols can be
> +    examined to find it. While it is possible to contract the file, it is not
> +    possible to expand the file since that would involve re-linking

We should remove this, I believe.

> +- CONFIG_OF_BOARD
> +    This is a board-specific situation, so needs to be considered on a
> +    case-by-case base.

No.  Or rather, maybe we need to rename this so it's clear that this is
the case where the device tree we use is passed in at run-time in some
manner.

> +Use of U-Boot /config node
> +--------------------------

This should just be a link to the binding documentation, yes?

[snip]
> +Devicetree in another project
> +-----------------------------
> +
> +In some cases U-Boot receive its devicetree at runtime from a program that calls
> +it. For example ARM's Trusted Firmware A (`TF-A`_) may have a devicetree that it
> +passes to U-Boot. This overrides any devicetree build by U-Boot. When packaging
> +the firmware, the U-Boot devicetree may in fact be left out if it can be
> +guaranteed that it will receive one from another project.

Yes.

> +In this case, the devicetree in the other project must track U-Boot's use of
> +device tree, for the following reasons:

Well, no?  That's not right, or at least reads much more confrontational
that needed.  At heart, U-Boot has documented and upstreamed properties,
as part of the appropriate bindings.  They need to be put in to the
device tree in order for features to work.  If they aren't there, they
won't work.  But if they needed to be there for the platform to work,
they'd have been present anyhow.  It's only a gray area in that we have
bindings to upstream still.

[snip]
> +Note that the U-Boot in-tree devicetree source must be sufficient to build and
> +boot, so this is not a way to bypass that requirement.

But this is wrong because we cannot have a static device tee in many
cases that would work.

> +If binman is used, the devicetree source in U-Boot must contain the binman
> +definition so that a valid image can be build. This helps people discover what
> +other firmware components are needed and seek out appropriate documentation.

This is probably a little awkward as I'm not sure if we have systems
that are both CONFIG_OF_BOARD=y and using binman to generate the image
that flashes to hardware.  That may be a little tricky and we need to
see what the examples and usage there is.

> +If verified boot is used, the project must provide a way to inject a public key,
> +certificate or other material into the U-Boot devicetree so that it is available
> +to U-Boot at runtime. See `Signing with U-Boot devicetree`_. This may be
> +through tooling in the project itself or by making use of U-Boot's tooling.

This is fine.

> +Devicetree generated on-the-fly in another project
> +--------------------------------------------------
> +
> +In some rare cases, another project may wish to create a devicetree for U-Boot

We don't need to quantify here, just "In some cases".

> +entirely on-the-fly, then pass it to U-Boot at runtime.

True.

> The only known example
> +of this at the time of writing (2021) is QEMU, for ARM (`QEMU ARM`_) and
> +RISC-V (`QEMU RISC-V`_).

Also Xen, and I'm not sure it's helpful to attempt to enumerate all of
them, just an example is probably fine.

> +In effect, when the board boots, U-Boot is *downstream* of the other project.
> +It is entirely reliant on that project for its correct operation.
> +
> +This does not mean to imply that the other project is creating its own,
> +incompatible devicetree. In fact QEMU generates a valid devicetree which is
> +suitable for both U-Boot and Linux. It is quite normal for a devicetree to be
> +present in flash and be made available to U-Boot at runtime. What matters is
> +where the devicetree comes from. If the other project builds a devicetree for
> +U-Boot then it needs to support adding the things needed by U-Boot features.
> +Without them, for example:

Everything I said above about making device trees that comply with
bindings goes here.  And then that makes this not a special case at all
worth mentioning.  There's nothing special about this case compared to
any other case where we get the device tree at run time.

[snip]
> +Passing the devicetree through to Linux
> +---------------------------------------
> +
> +Ideally U-Boot and Linux use the same devicetree source, even though it is
> +hosted in separate projects. U-Boot adds some extra pieces, such as the
> +`config/` node and tags like `u-boot,dm-spl`. Linux adds some extra pieces, such
> +as `linux,default-trigger` and `linux,code`. This should not interfere with
> +each other.
> +
> +In principle it is possible for U-Boot's control devicetree to be passed to
> +Linux. This is, after all, one of the goals of devicetree and the original
> +Open Firmware project, to have the firmware provide the hardware description to
> +the Operating System.

This reads a bit more confrontational than intended perhaps?

> +For boards where this approach is used, care must be taken. U-Boot typically
> +needs to 'fix up' the devicetree before passing it to Linux, e.g. to add
> +information about the memory map, about which serial console is used, provide
> +the kernel address space layout randomization (KASLR) seed or select whether the
> +console should be silenced for a faster boot.

We should note / explain that this is something we've been doing almost
since DT inception. And then we need to document which hooks / paths
happen at boot the OS stage, and which paths happen sooner and can
change U-Boot as well at run time.

> +Fix-ups involve modifying the devicetree. If the control devicetree is used,
> +that means the control devicetree could be modified, while U-Boot is using it.
> +Removing a device and reinserting it can cause problems if the devicetree offset
> +has changed, for example, since the device will be unable to locates its
> +devicetree properties at the expected devicetree offset, which is a fixed
> +integer.
> +
> +To deal with this, it is recommended to employ one or more of the following
> +approaches:
> +
> +- Make a copy of the devicetree and 'fix up' the copy, leaving the control
> +  devicetree alone
> +- Enable `CONFIG_OF_LIVE` so that U-Boot makes its own copy of the devicetree
> +  during relocation; fixups then happen on the original flat tree
> +- Ensure that fix-ups happen after all loading has happened and U-Boot has
> +  completed image verification
> +
> +In practice,the last point is typically observed, since boot_prep_linux() is
> +called just before jumping to Linux, long after signature verification, for
> +example. But it is important to make sure that this line is not blurred,
> +particularly if untrusted user data is involved.

In addition to what I said above, we should explain which paths are best
for which cases.

> +Devicetree use cases that must be supported
> +-------------------------------------------
> +
> +Regardless of how the devicetree is provided to U-Boot at runtime, various
> +U-Boot features must be fully supported. This section describes some of these
> +features and the implications for other projects.

"must" is pretty strong for some of these cases.  A whole lot of U-Boot
is opt-in if you need it.

> +If U-Boot uses its own in-tree devicetree these features are supported
> +automatically.

This is both confrontational (again, you write vs the binding) and over
simplifies.  We don't automatically know where to shove
u-boot,dm-pre-reloc but it's not likely to be passed in to us yet
because it's not upstreamedd.  But but it's also potentially not needed,
if we can just deal with having something not probe until a bit later.

> +Signing with U-Boot devicetree
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This should link to the documentation for this feature, and is not
mandatory for all platforms.

[snip]
> +Providing the binman image definition
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Not mandatory, and should link to more extensive documentation on how
this works, along with just a brief summary here.

[snip]
> +Protecting the devicetree
> +-------------------------
> +
> +U-Boot relies heavily on devicetree for correct operation. A corrupt or invalid
> +device can cause U-Boot to fail to start, behave incorrectly, crash (e.g. if
> +`CONFIG_OF_LIBFDT_ASSUME_MASK` is adjusted, or fail to boot an Operating System.

This is a true statement, but doesn't describe a feaature.

> +Within U-Boot, the devicetree is as important as any other part of the source
> +code. At ruuntime, the devicetree can be considered to be structured rodata.
> +
> +With secure systems, care must be taken that the devicetree is valid:

With all systems care must be taken, or everything blows up, or is
compromised more easily.  We _do_ attempt to take care of the device
tree via the LMB mechanism, and should mention that here.  And double
check that we're covering it :)

> +- If the code / rodata has a hash or signature, the devicetree should also, if
> +  they are packaged separately.

This should be part of the verification documentation.

> +- If the code / rodata is write-protected when running, the devicetree should be
> +  also. Note that U-Boot relocates its code and devicetree, so this is not as
> +  simple as it sounds. U-Boot must write-protect these items after relocating.

This is a general design / implementation issue, yes.  This should be
more "If you have the ability to protect memory, then .." I think.

> +Why does U-Boot have its nodes and properties?
> +----------------------------------------------

This is the part I think that causes the most objection / confusion in
general as it's written in a way that reads as an argument, to me.  I'd
like to strike the whole section, honestly, as it reads to me as "we
argued a lot on the mailing list, now I'm making my part of that the
documentation".

> +See also :doc:`../devicetree/intro`.
> +
> +There has been pushback at the concept that U-Boot dares have its own nodes and
> +properties in the devicetree.
> +
> +Apart from these nodes and properties, U-Boot uses the same bindings as Linux.
> +A `u-boot.dtsi` file helps to keep U-Boot-specific changes in separate files,
> +making it easier to keep devicetree source files in U-Boot in sync with Linux.

As I said in my other reply, I think the -u-boot.dtsi was a mistake, in
hindsight as it let us not try and upstream bindings / properties for
far too long.  And now we're at this point where platforms are coming
in, wanting to use a run-time provided device tree, and hitting the wall
on what to do exactly about some of our nodes and properties.

> +As a counter-example, the Zephyr OS project takes a different approach. It uses
> +entirely different bindings, in general, making no effort to sync devicetree
> +source files with Linux. U-Boot strives to be compatible with Linux in a number
> +of ways, such as source code style and common APIs, to aid porting of code
> +between the projects. Devicetree is another way where U-Boot and Linux follow a
> +similar approach.

I don't think it's helpful to mention Zephyr at all.  They're doing
exactly what they intended to do and document that they do.  I believe
no one is surprised when they go in to Zephyr-land at what they see.

> +Fundamentally, the idea that U-Boot cannot have its own tags flies in the face
> +of the devicetree specification (see dtspec_), which says:
> +
> +  Nonstandard property names should specify a **unique string prefix**, such as
> +  a stock ticker symbol, identifying the name of the company **or organization**
> +  that defined the property. Examples:
> +
> +  - fsl,channel-fifo-len
> +  - ibm,ppc-interrupt-server#s
> +  - **linux**,network-index
> +
> +It is also fundamentally unbalanced. Linux has many tags of its own (some 36 in
> +version 5.13) and at least one Linux-specific node, even if you ignore things
> +like flash partitions which clearly provide configuration information to Linux.

What we can't have is properties that aren't documented and upstreamed,
and that part of the equation isn't spelled out or emphasized in this
document nearly enough.

> +Practically speaking there are many reasons why U-Boot has its own nodes and
> +properties. Some examples:
> +
> +- Binding every device before relocation even if it won't be used, consumes time
> +  and memory: tags on each node can specify which are needed in SPL or before
> +  relocation. Linux has no such constraints.

Documenting how and why we make this special one-off DTB for SPL needs
to happen.  And then maybe we need to think of something more clever?
I'm not sure.  Because yes, I see this as one of the harder to upstream
properties.  But maybe I'm just over-worrying.

> +- Requiring the full clock tree to be up and running just to get the debug UART
> +  running is inefficient. It is also and self-defeating, since if that much
> +  code is working properly, you probably don't need the debug UART. A devicetree
> +  property to provide the UART input-clock frequency is a simple solution.

This is because of that discussion we had a long time ago about how that
property got dropped from some devices, yes?  I _think_ things have
evolved and as it's a valid property should be able to be in the dts,
upstream, again/still.

> +- U-Boot does not have a user space to provide policy and configuration. It
> +  cannot do what Linux does and run programs and look up filesystems to figure
> +  out how to boot.

This is the /config node, which is being documented / upstreamed and
it's time to poke Rob again.

With that said, replacing this section with something that points to our
upstream bindings would be helpful.

> +Why not have two devicetrees?
> +-----------------------------
> +
> +Setting aside the argument for restricting U-Boot from having its own nodes and
> +properties, another idea proposed is to have two devicetrees, one for the
> +U-Boot-specific bits (here called `special`) and one for everything else (here
> +called `linux`). This would mean that U-Boot would be controlled by two
> +devicetrees, i.e. OF_CONTROL would require/allow two devicetrees in order to
> +work.
> +
> +On the positive side, it might quieten the discussion alluded to in the section
> +above. But there are many negatives to consider and many open questions to
> +resolve.

So, this reads as another argument-by-documentation section, please drop
it.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20211207/47c4aa87/attachment.sig>