[PATCH] doc: Add documentation about devicetree usage

[Heinrich Schuchardt]

On 8/28/21 3:01 PM, Tom Rini wrote:
> On Sat, Aug 28, 2021 at 02:29:21PM +0200, Heinrich Schuchardt wrote:
>> On 8/28/21 5:23 AM, Simon Glass wrote:
>>> At present some of the ideas and techniques behind devicetree in U-Boot
>>> are assumed, implied or unsaid. Add some documentation to cover how
>>> devicetree is build, how it can be modified and the rules about using
>>> the various CONFIG_OF_... options.
>>>
>>> Signed-off-by: Simon Glass <sjg at chromium.org>
>>> ---
>>>
>>>    doc/develop/index.rst              |   1 +
>>>    doc/develop/package/devicetree.rst | 315 +++++++++++++++++++++++++++++
>>>    doc/develop/package/index.rst      |   1 +
>>>    3 files changed, 317 insertions(+)
>>>    create mode 100644 doc/develop/package/devicetree.rst
>>>
>>> diff --git a/doc/develop/index.rst b/doc/develop/index.rst
>>> index 83c929babda..d5ad8f9fe53 100644
>>> --- a/doc/develop/index.rst
>>> +++ b/doc/develop/index.rst
>>> @@ -36,6 +36,7 @@ Packaging
>>>       :maxdepth: 1
>>>
>>>       package/index
>>> +   package/devicetree
>>>
>>>    Testing
>>>    -------
>>> diff --git a/doc/develop/package/devicetree.rst b/doc/develop/package/devicetree.rst
>>> new file mode 100644
>>> index 00000000000..fccbb182f3e
>>> --- /dev/null
>>> +++ b/doc/develop/package/devicetree.rst
>>> @@ -0,0 +1,315 @@
>>> +.. SPDX-License-Identifier: GPL-2.0+
>>> +
>>> +Updating the devicetree
>>> +=======================
>>> +
>>> +U-Boot uses devicetree for runtime configuration and storing required blobs or
>>> +any other information it needs to operate. It is possible to update the
>>> +devicetree separately from actually building U-Boot. This provides a good degree
>>> +of control and flexibility for firmware that uses U-Boot in conjunction with
>>> +other project.
>>> +
>>> +There are many reasons why it is useful to modify the devicetree after building
>>> +it:
>>> +
>>> +- Configuration can be changed, e.g. which UART to use
>>> +- A serial number can be added
>>> +- Public keys can be added to allow image verification
>>> +- Console output can be changed (e.g. to select serial or vidconsole)
>>> +
>>> +This section describes how to work with devicetree to accomplish your goals.
>>> +
>>> +See also :doc:`../devicetree/control` for a basic summary of the available
>>> +features.
>>> +
>>> +
>>> +Devicetree source
>>> +-----------------
>>> +
>>> +Every board in U-Boot must include a devicetree sufficient to build and boot
>>> +that board on suitable hardware (or emulation). This is specified using the
>>> +`CONFIG DEFAULT_DEVICE_TREE` option.
>>> +
>>> +
>>> +Current situation (August 2021)
>>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>> +
>>> +As an aside, at present U-Boot allows `CONFIG_DEFAULT_DEVICE_TREE` to be empty,
>>> +e.g. if `CONFIG_OF_BOARD` or `CONFIG_OF_PRIOR_STAGE` are used. This has
>>> +unfortunately created an enormous amount of confusion and some wasted effort.
>>> +This was not intended and this bug will be fixed soon. Specifically:
>>> +
>>> +- `CONFIG_OF_BOARD` was added in rpi_patch_ for Raspberry Pi, which does have
>>> +  an in-tree devicetree, but this feature has since been used for boards that
>>> +  don't
>>> +- `CONFIG_OF_PRIOR_STAGE` was added in bcm_patch_ as part of a larger Broadcom
>>> +  change with a tag indicating it only affected one board, so the change in
>>> +  behaviour was not noticed at the time. It has since been used by RISC-V qemu
>>> +  boards.
>>> +
>>> +Once this bug is fixed, CONFIG_OF_BOARD and CONFIG_OF_PRIOR_STAGE will override
>>> +(at runtime) the devicetree suppled with U-Boot, but will otherwise use
>>> +CONFIG_OF_SEPARATE for the in-tree build. So these two will become options,
>>> +moving out of the 'choice' in `dts/Kconfig`
>>> +
>>> +Offending boards are:
>>> +
>>> +- bcm7260
>>> +- bcm7445
>>> +- qemu_arm64
>>> +- qemu_arm
>>> +- qemu-ppce500
>>> +- qemu-riscv32
>>> +- qemu-riscv32_smode
>>> +- qemu-riscv64
>>> +- qemu-riscv64_smode
>>> +
>>> +All of these need to have a devicetree added in-tree. This is targeted to be
>>> +fixed in the 2022.01 release.
>>> +
>>> +
>>> +Building the devicetree
>>> +-----------------------
>>> +
>>> +U-Boot automatically builds the devicetree for a board, from the
>>> +`arch/<arch>/dts` directory. The Makefile in those directories has rules for
>>> +building devicetree files. It is preferable to avoid target-specific rules in
>>> +those files: i.e. all boards for a particular SoC should be built at once,
>>> +where practical. Apart from simplifying the Makefile, this helps to efficiently
>>> +(and immediately) ensure that changes in one board's DT do not break others that
>>> +are related. Building devicetrees is fast, so performance is seldom a concern
>>> +here.
>>> +
>>> +
>>> +Overriding the default devicetree
>>> +---------------------------------
>>> +
>>> +When building U-Boot, the `DEVICE_TREE` environment variable allows the
>>> +default devicetree file to be overridden at build time. This can be useful if
>>> +modifications have to be made to the in-tree devicetree file, for the benefit
>>> +of a downstream build system. Note that the in-tree devicetree must be
>>> +sufficient to build and boot, so this is not a way to bypass that requirement.
>>> +
>>> +
>>> +Modifying the devicetree after building
>>> +---------------------------------------
>>> +
>>> +While it is generally painful and hacky to modify the code or rodata of a
>>> +program after it is built, in many cases it is useul to do so, e.g. to add
>>> +configuration information like serial numbers, enabling/disabling features, etc.
>>> +
>>> +Devicetree provides a very nice solution to these problems since it is
>>> +structured data and it is relatively easy to change it, even in binary form
>>> +(see fdtput).
>>> +
>>> +U-Boot takes care that the devicetree is easily accessible after the build
>>> +process. In fact it is placed in a separate file called `u-boot.dtb`. If the
>>> +build system wants to modify or replace that file, it can do so. Then all that
>>> +is needed is to run `binman update` to update the file inside the image. If
>>> +binman is not used, then `u-boot-nodtb.bin` and the new `u-boot.dtb` can simply
>>> +be concatenated to achieve the desired result. U-Boot happily copes with the
>>> +devicetree growing or shrinking.
>>> +
>>> +The `u-boot.bin` image contains both pieces. While it is possible to locate the
>>> +devicetree within the image using the signature at the start of the file, this
>>> +is a bit messy.
>>> +
>>> +This is why `CONFIG_OF_SEPARATE` should always be used when building U-Boot.
>>> +The `CONFIG_OF_EMBED` option embeds the devicetree somewhere in the U-Boot ELF
>>> +image as rodata, meaning that it is hard to find it and it cannot increase in
>>> +size.
>>> +
>>> +When modifying the devicetree, the different cases to consider are as follows:
>>> +
>>> +- CONFIG_OF_SEPARATE
>>> +    This is easy, described above. Just change, replace or rebuild the
>>> +    devicetree so it suits your needs, then rerun binman or redo the `cat`
>>> +    operation to join `u-boot-nodtb.bin` and the new `u-boot.dtb`
>>> +
>>> +- CONFIG_OF_EMBD
>>> +    This is tricky, since the devicetree cannot easily be located. If the EFL
>>> +    file is available, then the _dtb_dt_begin and __dtb_dt_end symbols can be
>>> +    examined to find it. While it is possible to contract the file, it is not
>>> +    possible to expand the file since that would involve re-linking
>>> +
>>> +- CONFIG_OF_PRIOR_STAGE
>>> +    In this case the devicetree must be modified in the project which provides
>>> +    it, as described below
>>> +
>>> +- CONFIG_OF_BOARD
>>> +    This is a board-specific situation, so needs to be considered on a
>>> +    case-by-case base. The devicetree must be modified so that the correct
>>> +    one is provided to U-Boot. How this is done depends entirely on the
>>> +    implementation of this option for the board. It might require injecting the
>>> +    changes into a different project somehow using tooling available there, or
>>> +    it might involve merging an overlay file at runtime to obtain the desired
>>> +    result.
>>> +
>>> +
>>> +Devicetree in another project
>>> +-----------------------------
>>> +
>>> +In some cases U-Boot receive its devicetree at runtime from a program that calls
>>> +it. For example ARM's Trusted Firmware A (`TF-A`_) may have a devicetree that it
>>> +passes to U-Boot. This overrides any devicetree build by U-Boot. When packaging
>>> +the firmware, the U-Boot devicetree may in fact be left out if it can be
>>> +guaranteed that it will receive one from another project.
>>> +
>>> +In this case, the devicetree in the other project must track U-Boot's use of
>>> +device tree. It must provide a way to add configuration and other information to
>>
>> U-Boot does not rule the world. So never ever is this going to happen.
>> It is the other way round. U-Boot must consume what the prior bootstage
>> delivers. If U-Boot needs extra nodes it has to provide these on its own.
>>
>> Please, remove this assumption from the document.
>
> We need to figure out which compatibles we need to push upstream, and
> which we need to see if we can solve another way.
>
>>> +the devicetree for use by U-Boot, such as the /config node. Note that the
>>> +U-Boot in-tree devicetree must be sufficient to build and boot, so this is not a
>>> +way to bypass that requirement.
>>> +
>>> +If binman is used, the in-tree U-Boot devicetree must contain the binman
>>> +definition so that a valid image can be build.
>>
>> No clue what an in-tree tree might be. Please, avoid such confusing
>> language.
>
> Would "source tree devicetree" be less confusing?  Or can you suggest an
> alternative?
>
>>
>>> +
>>> +If verified boot is used, the project must provide a way to inject a public key,
>>
>> %s/the project/U-Boot/
>>
>>> +certificate or other material into the U-Boot devicetree so that it is available
>>> +to U-Boot at runtime. See `Signing with U-Boot devicetree`_. This may be
>>> +through tooling in the project itself or by making use of U-Boot's tooling.
>>> +
>>> +
>>> +Devicetree generated on-the-fly in another project
>>> +--------------------------------------------------
>
> I think this is a confusing topic, and gets things a bit backwards.
>
>>> +
>>> +In some rare cases, another project may wish to create a devicetree for U-Boot
>>> +entirely on-the-fly, then pass it to U-Boot at runtime. The only known example
>>> +of this at the time of writing (2021) is qemu, for ARM (`QEMU ARM`_) and
>>> +RISC-V (`QEMU RISC-V`_).
>
> What's the difference between QEMU and hardware that ships with a device
> tree stored in flash?  In both cases, we need to have the device tree
> that's provided be the device tree that works.  Like I was just raising
> in another thread, there are not multiple device trees for a given
> device, there is the device tree and it works for everyone that needs to
> consume a device tree.
>
>>> +In this case, the devicetree in the other project must track U-Boot's use of
>>> +device tree, so that it remains compatible. If a particular version of the
>>
>> Why? U-Boot must support its internal needs itself!
>>
>> Don't try to force a bad U-Boot design on other projects.
>>
>> Please, come up with a concept that makes sense.
>
> This is the same situation as above, where we need to see about pushing
> some changes upstream perhaps.
>

In U-Boot we currently have a lot of device-tree nodes that are
irrelevant for anything but U-Boot, e.g. all those nodes marked as
compatible to "u-boot,*". It does not make sense to push these upstream.

So if a prior bootstage provides a devicetree, we need a separate
devicetree in U-Boot.

We can copy the incoming devicetree to a private copy and amend all our
U-Boot specific stuff. Then let's use this private copy for
$fdt_control_addr and bringing up U-Boot while keeping the original
incoming devicetree as $fdt_addr so that we can pass it to the next boot
stage.

Best regards

Heinrich