[PATCH] doc: Add documentation about devicetree usage

Sat Aug 28 18:45:27 CEST 2021

Hi Heinrich,

On Sat, 28 Aug 2021 at 07:39, Tom Rini <trini at konsulko.com> wrote:
>
> On Sat, Aug 28, 2021 at 03:30:15PM +0200, Heinrich Schuchardt wrote:
> > On 8/28/21 3:01 PM, Tom Rini wrote:
> > > On Sat, Aug 28, 2021 at 02:29:21PM +0200, Heinrich Schuchardt wrote:
> > > > On 8/28/21 5:23 AM, Simon Glass wrote:
> > > > > At present some of the ideas and techniques behind devicetree in U-Boot
> > > > > are assumed, implied or unsaid. Add some documentation to cover how
> > > > > devicetree is build, how it can be modified and the rules about using
> > > > > the various CONFIG_OF_... options.
> > > > >
> > > > > Signed-off-by: Simon Glass <sjg at chromium.org>
> > > > > ---
> > > > >
> > > > >    doc/develop/index.rst              |   1 +
> > > > >    doc/develop/package/devicetree.rst | 315 +++++++++++++++++++++++++++++
> > > > >    doc/develop/package/index.rst      |   1 +
> > > > >    3 files changed, 317 insertions(+)
> > > > >    create mode 100644 doc/develop/package/devicetree.rst
> > > > >
> > > > > diff --git a/doc/develop/index.rst b/doc/develop/index.rst
> > > > > index 83c929babda..d5ad8f9fe53 100644
> > > > > --- a/doc/develop/index.rst
> > > > > +++ b/doc/develop/index.rst
> > > > > @@ -36,6 +36,7 @@ Packaging
> > > > >       :maxdepth: 1
> > > > >
> > > > >       package/index
> > > > > +   package/devicetree
> > > > >
> > > > >    Testing
> > > > >    -------
> > > > > diff --git a/doc/develop/package/devicetree.rst b/doc/develop/package/devicetree.rst
> > > > > new file mode 100644
> > > > > index 00000000000..fccbb182f3e
> > > > > --- /dev/null
> > > > > +++ b/doc/develop/package/devicetree.rst
> > > > > @@ -0,0 +1,315 @@
> > > > > +.. SPDX-License-Identifier: GPL-2.0+
> > > > > +
> > > > > +Updating the devicetree
> > > > > +=======================
> > > > > +
> > > > > +U-Boot uses devicetree for runtime configuration and storing required blobs or
> > > > > +any other information it needs to operate. It is possible to update the
> > > > > +devicetree separately from actually building U-Boot. This provides a good degree
> > > > > +of control and flexibility for firmware that uses U-Boot in conjunction with
> > > > > +other project.
> > > > > +
> > > > > +There are many reasons why it is useful to modify the devicetree after building
> > > > > +it:
> > > > > +
> > > > > +- Configuration can be changed, e.g. which UART to use
> > > > > +- A serial number can be added
> > > > > +- Public keys can be added to allow image verification
> > > > > +- Console output can be changed (e.g. to select serial or vidconsole)
> > > > > +
> > > > > +This section describes how to work with devicetree to accomplish your goals.
> > > > > +
> > > > > +See also :doc:`../devicetree/control` for a basic summary of the available
> > > > > +features.
> > > > > +
> > > > > +
> > > > > +Devicetree source
> > > > > +-----------------
> > > > > +
> > > > > +Every board in U-Boot must include a devicetree sufficient to build and boot
> > > > > +that board on suitable hardware (or emulation). This is specified using the
> > > > > +`CONFIG DEFAULT_DEVICE_TREE` option.
> > > > > +
> > > > > +
> > > > > +Current situation (August 2021)
> > > > > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > +
> > > > > +As an aside, at present U-Boot allows `CONFIG_DEFAULT_DEVICE_TREE` to be empty,
> > > > > +e.g. if `CONFIG_OF_BOARD` or `CONFIG_OF_PRIOR_STAGE` are used. This has
> > > > > +unfortunately created an enormous amount of confusion and some wasted effort.
> > > > > +This was not intended and this bug will be fixed soon. Specifically:
> > > > > +
> > > > > +- `CONFIG_OF_BOARD` was added in rpi_patch_ for Raspberry Pi, which does have
> > > > > +  an in-tree devicetree, but this feature has since been used for boards that
> > > > > +  don't
> > > > > +- `CONFIG_OF_PRIOR_STAGE` was added in bcm_patch_ as part of a larger Broadcom
> > > > > +  change with a tag indicating it only affected one board, so the change in
> > > > > +  behaviour was not noticed at the time. It has since been used by RISC-V qemu
> > > > > +  boards.
> > > > > +
> > > > > +Once this bug is fixed, CONFIG_OF_BOARD and CONFIG_OF_PRIOR_STAGE will override
> > > > > +(at runtime) the devicetree suppled with U-Boot, but will otherwise use
> > > > > +CONFIG_OF_SEPARATE for the in-tree build. So these two will become options,
> > > > > +moving out of the 'choice' in `dts/Kconfig`
> > > > > +
> > > > > +Offending boards are:
> > > > > +
> > > > > +- bcm7260
> > > > > +- bcm7445
> > > > > +- qemu_arm64
> > > > > +- qemu_arm
> > > > > +- qemu-ppce500
> > > > > +- qemu-riscv32
> > > > > +- qemu-riscv32_smode
> > > > > +- qemu-riscv64
> > > > > +- qemu-riscv64_smode
> > > > > +
> > > > > +All of these need to have a devicetree added in-tree. This is targeted to be
> > > > > +fixed in the 2022.01 release.
> > > > > +
> > > > > +
> > > > > +Building the devicetree
> > > > > +-----------------------
> > > > > +
> > > > > +U-Boot automatically builds the devicetree for a board, from the
> > > > > +`arch/<arch>/dts` directory. The Makefile in those directories has rules for
> > > > > +building devicetree files. It is preferable to avoid target-specific rules in
> > > > > +those files: i.e. all boards for a particular SoC should be built at once,
> > > > > +where practical. Apart from simplifying the Makefile, this helps to efficiently
> > > > > +(and immediately) ensure that changes in one board's DT do not break others that
> > > > > +are related. Building devicetrees is fast, so performance is seldom a concern
> > > > > +here.
> > > > > +
> > > > > +
> > > > > +Overriding the default devicetree
> > > > > +---------------------------------
> > > > > +
> > > > > +When building U-Boot, the `DEVICE_TREE` environment variable allows the
> > > > > +default devicetree file to be overridden at build time. This can be useful if
> > > > > +modifications have to be made to the in-tree devicetree file, for the benefit
> > > > > +of a downstream build system. Note that the in-tree devicetree must be
> > > > > +sufficient to build and boot, so this is not a way to bypass that requirement.
> > > > > +
> > > > > +
> > > > > +Modifying the devicetree after building
> > > > > +---------------------------------------
> > > > > +
> > > > > +While it is generally painful and hacky to modify the code or rodata of a
> > > > > +program after it is built, in many cases it is useul to do so, e.g. to add
> > > > > +configuration information like serial numbers, enabling/disabling features, etc.
> > > > > +
> > > > > +Devicetree provides a very nice solution to these problems since it is
> > > > > +structured data and it is relatively easy to change it, even in binary form
> > > > > +(see fdtput).
> > > > > +
> > > > > +U-Boot takes care that the devicetree is easily accessible after the build
> > > > > +process. In fact it is placed in a separate file called `u-boot.dtb`. If the
> > > > > +build system wants to modify or replace that file, it can do so. Then all that
> > > > > +is needed is to run `binman update` to update the file inside the image. If
> > > > > +binman is not used, then `u-boot-nodtb.bin` and the new `u-boot.dtb` can simply
> > > > > +be concatenated to achieve the desired result. U-Boot happily copes with the
> > > > > +devicetree growing or shrinking.
> > > > > +
> > > > > +The `u-boot.bin` image contains both pieces. While it is possible to locate the
> > > > > +devicetree within the image using the signature at the start of the file, this
> > > > > +is a bit messy.
> > > > > +
> > > > > +This is why `CONFIG_OF_SEPARATE` should always be used when building U-Boot.
> > > > > +The `CONFIG_OF_EMBED` option embeds the devicetree somewhere in the U-Boot ELF
> > > > > +image as rodata, meaning that it is hard to find it and it cannot increase in
> > > > > +size.
> > > > > +
> > > > > +When modifying the devicetree, the different cases to consider are as follows:
> > > > > +
> > > > > +- CONFIG_OF_SEPARATE
> > > > > +    This is easy, described above. Just change, replace or rebuild the
> > > > > +    devicetree so it suits your needs, then rerun binman or redo the `cat`
> > > > > +    operation to join `u-boot-nodtb.bin` and the new `u-boot.dtb`
> > > > > +
> > > > > +- CONFIG_OF_EMBD
> > > > > +    This is tricky, since the devicetree cannot easily be located. If the EFL
> > > > > +    file is available, then the _dtb_dt_begin and __dtb_dt_end symbols can be
> > > > > +    examined to find it. While it is possible to contract the file, it is not
> > > > > +    possible to expand the file since that would involve re-linking
> > > > > +
> > > > > +- CONFIG_OF_PRIOR_STAGE
> > > > > +    In this case the devicetree must be modified in the project which provides
> > > > > +    it, as described below
> > > > > +
> > > > > +- CONFIG_OF_BOARD
> > > > > +    This is a board-specific situation, so needs to be considered on a
> > > > > +    case-by-case base. The devicetree must be modified so that the correct
> > > > > +    one is provided to U-Boot. How this is done depends entirely on the
> > > > > +    implementation of this option for the board. It might require injecting the
> > > > > +    changes into a different project somehow using tooling available there, or
> > > > > +    it might involve merging an overlay file at runtime to obtain the desired
> > > > > +    result.
> > > > > +
> > > > > +
> > > > > +Devicetree in another project
> > > > > +-----------------------------
> > > > > +
> > > > > +In some cases U-Boot receive its devicetree at runtime from a program that calls
> > > > > +it. For example ARM's Trusted Firmware A (`TF-A`_) may have a devicetree that it
> > > > > +passes to U-Boot. This overrides any devicetree build by U-Boot. When packaging
> > > > > +the firmware, the U-Boot devicetree may in fact be left out if it can be
> > > > > +guaranteed that it will receive one from another project.
> > > > > +
> > > > > +In this case, the devicetree in the other project must track U-Boot's use of
> > > > > +device tree. It must provide a way to add configuration and other information to
> > > >
> > > > U-Boot does not rule the world. So never ever is this going to happen.
> > > > It is the other way round. U-Boot must consume what the prior bootstage
> > > > delivers. If U-Boot needs extra nodes it has to provide these on its own.
> > > >
> > > > Please, remove this assumption from the document.
> > >
> > > We need to figure out which compatibles we need to push upstream, and
> > > which we need to see if we can solve another way.
> > >
> > > > > +the devicetree for use by U-Boot, such as the /config node. Note that the
> > > > > +U-Boot in-tree devicetree must be sufficient to build and boot, so this is not a
> > > > > +way to bypass that requirement.
> > > > > +
> > > > > +If binman is used, the in-tree U-Boot devicetree must contain the binman
> > > > > +definition so that a valid image can be build.
> > > >
> > > > No clue what an in-tree tree might be. Please, avoid such confusing
> > > > language.
> > >
> > > Would "source tree devicetree" be less confusing?  Or can you suggest an
> > > alternative?
> > >
> > > >
> > > > > +
> > > > > +If verified boot is used, the project must provide a way to inject a public key,
> > > >
> > > > %s/the project/U-Boot/
> > > >
> > > > > +certificate or other material into the U-Boot devicetree so that it is available
> > > > > +to U-Boot at runtime. See `Signing with U-Boot devicetree`_. This may be
> > > > > +through tooling in the project itself or by making use of U-Boot's tooling.
> > > > > +
> > > > > +
> > > > > +Devicetree generated on-the-fly in another project
> > > > > +--------------------------------------------------
> > >
> > > I think this is a confusing topic, and gets things a bit backwards.
> > >
> > > > > +
> > > > > +In some rare cases, another project may wish to create a devicetree for U-Boot
> > > > > +entirely on-the-fly, then pass it to U-Boot at runtime. The only known example
> > > > > +of this at the time of writing (2021) is qemu, for ARM (`QEMU ARM`_) and
> > > > > +RISC-V (`QEMU RISC-V`_).
> > >
> > > What's the difference between QEMU and hardware that ships with a device
> > > tree stored in flash?  In both cases, we need to have the device tree
> > > that's provided be the device tree that works.  Like I was just raising
> > > in another thread, there are not multiple device trees for a given
> > > device, there is the device tree and it works for everyone that needs to
> > > consume a device tree.
> > >
> > > > > +In this case, the devicetree in the other project must track U-Boot's use of
> > > > > +device tree, so that it remains compatible. If a particular version of the
> > > >
> > > > Why? U-Boot must support its internal needs itself!
> > > >
> > > > Don't try to force a bad U-Boot design on other projects.
> > > >
> > > > Please, come up with a concept that makes sense.
> > >
> > > This is the same situation as above, where we need to see about pushing
> > > some changes upstream perhaps.
> > >
> >
> > In U-Boot we currently have a lot of device-tree nodes that are
> > irrelevant for anything but U-Boot, e.g. all those nodes marked as
> > compatible to "u-boot,*". It does not make sense to push these upstream.
> >
> > So if a prior bootstage provides a devicetree, we need a separate
> > devicetree in U-Boot.
> >
> > We can copy the incoming devicetree to a private copy and amend all our
> > U-Boot specific stuff. Then let's use this private copy for
> > $fdt_control_addr and bringing up U-Boot while keeping the original
> > incoming devicetree as $fdt_addr so that we can pass it to the next boot
> > stage.
>
> I'm not sure that's how this works?  The "linux" vendor prefix is for
> Linux specific bindings and are expected to be present in the tree it
> receives, and not fixed up and added at runtime.  But if you can proof
> of concept out what your idea, I suspect that would work to resolve a
> handful of problems all the same, so it would be good to see.

I actually suggested a different approach: copy the DT before fixing
it up. This effectively happens already if OF_LIVE is enabled.

U-Boot obviously cannot 'amend all our U-Boot specific stuff' at
runtime unless that is available somewhere else (at runtime). I'll
address that point in the v2 series.

Should we require the Linux stuff to be in a separate DT also? :-)

Regards,
SImon