[PATCH RFC] doc: Document address spaces used by U-Boot

[Heinrich Schuchardt]

On 5/17/24 13:45, Jiaxun Yang wrote:
> This serves as a reference for developers on how to handle
> all those address spaces for U-Boot.
>
> Signed-off-by: Jiaxun Yang <jiaxun.yang at flygoat.com>
> ---
> Hi all,
>
> I was trying to clear up long standing myth of virtual
> address vs physical adddress mapping on MIPS by having
> a clear division betweem virt and phys adddress everywhere.
>
> In this process I would like to confirm my understanding
> by writing a document and requesting comments from the
> community.
>
> Note that there are some sysmem APIs mentioned in the document
> are not in U-Boot tree, namely ``sysmem_addr_t`` and sysmem_to_phys
> co. I think they can help with annotating address spaces.
>
> Please kindly comment on the design.
>
> Thanks!

Thanks for looking into this.

> ---
>   doc/develop/address_spaces.rst | 73 ++++++++++++++++++++++++++++++++++++++++++
>   doc/develop/index.rst          |  1 +
>   2 files changed, 74 insertions(+)
>
> diff --git a/doc/develop/address_spaces.rst b/doc/develop/address_spaces.rst
> new file mode 100644
> index 000000000000..05c4a1575f27
> --- /dev/null
> +++ b/doc/develop/address_spaces.rst
> @@ -0,0 +1,73 @@
> +.. SPDX-License-Identifier: GPL-2.0+
> +.. Copyright (c) 2024 Jiaxun Yang <jiaxun.yang at flygoat.com>
> +
> +Address Spaces
> +==============
> +
> +Introduction
> +------------
> +
> +In U-Boot, we have three address spaces that are used to describe a memory
> +address:
> +
> +    - ``virt``: virtual address space
> +    - ``phys``: physical address space
> +    - ``sysmem``: system memory address space
> +
> +On most architectures, we maintain a 1:1:1 mapping between all three address,
> +with the exception of MIPS and sandbox.
> +
> +There are many API misused in existing code base, this document is to clarify
> +the usage of these address spaces and APIs. Hopefully it will help to reduce
> +the confusion and porting effort.
> +
> +``virt``
> +--------
> +
> +This is the address space that is directly used by U-Boot to access memory.
> +Every pointer in U-Boot must in ``virt`` address space. to get a valid ``virt``

%s/must in/must be in/

%s/to get/To get/

> +address from any other address space, you must use relavant mapping functions

%s/relavant/relevant/

> +to ensure it's being tracked. It is recomanded to use ``ulong`` or pointer types
> +to store ``virt`` address.
> +
> +``phys``
> +--------
> +
> +This is the address space that is used to describe the physical memory address.
> +It's used to describe the physical memory address of a device, or the physical
> +memory address of a memory region. Usual places you would see ``phys`` address
> +are addresses comes from Device Tree and some memory related APIs. It is
> +recommended to use ``fdt_addr_t`` or ``phys_addr_t`` to store ``phys`` address.
> +
> +Following APIs are availble to handle ``phys`` address:

%s/availble/available/

> +
> +    - ``map_physmem()``: Create mapping from physical address to virtual address
> +    - ``unmap_physmem()``: Remove mapping created by ``map_physmem()``
> +    - ``virt_to_phys()``: Find physical address mapping for a virtual address
> +    - ``ioremap()``: Create mapping for I/O memory region
> +    - ``iounmap()``: Remove mapping created by ``ioremap()``


These functions do not reprogram the MMU. In fact these functions are
only used to translate between the sandbox' virtual address space and
the host's virtual address space.

Looking at the sandbox code

static inline void *map_sysmem(phys_addr_t paddr, unsigned long len)
{
         return map_physmem(paddr, len, MAP_WRBACK);
}

there seems to be no difference between sysmem and physmem on the sandbox.

> +
> +``sysmem``
> +----------
> +
> +This is the address space that is used by U-Boot internally to describe RAM
> +address. It helps to abstract some architecture differences and provide a

I would say: It is the address space used in the command line interface
and in environment variables.

It should not be used elsewhere. Specifically these sandbox virtual
addresses must not be used in structures that are passed to external
programs like the kernel or EFI binaries.

> +consistent way to managed RAM. Usual places you would see ``sysmem`` address

The CLI addresses are used for memory mapped devices, too. So 'managed
RAM' is a bit confusing here.

> +are addresses comes from ``gd->bd->bi_dram[]``, addresses comes from ``LMB``
> +addresses being used as memory related config optinos, and addresses being

%s/optinos/options/

Relating to gd->bd->bi_dram[] or LMB just adds a layer of confusion to
the reader. I would not refer to specific internal structures in this
document.

> +used as arguments to load commands. It is recommended to use ``sysmem_addr_t``
> +to store ``sysmem`` address.

In most places long is used.

> +
> +It is further used by sandbox to simulate memory, and by MIPS to handle
> +differences between ``virt`` and ``phys`` address spaces.

You have been submitting patches to implement EFI for MIPS.

The EFI specification explicitly requires a 1:1 mapping in the MMU
between. virtual and physical addresses.

In what respect does MIPS not comply to this?

In how far do MIPS addresses shown in the CLI differ from the physical
addresses (i.e. addresses on the electrical address lines)?

CONFIG_ARCH_MAP_SYSMEM seems only to be set on the sandbox and not on
MIPS. We should try to keep it that way.

> +
> +Following APIs are availble to handle ``sysmem`` address:

%s/availble/available/




To summarize. If I have not mistaken anything about MIPS:

All physical devices are identity mapped.

The sandbox uses a virtual address space incorrectly called 'physical'
which is mapped to the virtual address space of the host. This sandbox
virtual address space is used in

* the command line interface
* in the sandbox device-tree
* in environment variable

This allows using the same address values in sandbox tests irrespective
of what part of the host RAM was mapped via mmap().

Best regards

Heinrich

> +
> +    - ``map_sysmem()``: Create mapping from system memory address to virtual address
> +    - ``unmap_sysmem()``: Remove mapping created by ``map_sysmem()``
> +    - ``map_to_sysmem()``: Find system memory address mapping for a virtual address
> +    - ``sysmem_to_phys()``: Find physical address mapping for a system memory address
> +    - ``phys_to_sysmem()``: Find system memory address mapping for a physical address
> +    - ``nomap_sysmem()``: Pass through an address unchanged (For sandbox tracking)
> +    - ``nomap_to_sysmem()``: Pass through an address unchanged
> +
> diff --git a/doc/develop/index.rst b/doc/develop/index.rst
> index f82e148b101c..1b8e11cd2f24 100644
> --- a/doc/develop/index.rst
> +++ b/doc/develop/index.rst
> @@ -27,6 +27,7 @@ Implementation
>   .. toctree::
>      :maxdepth: 1
>
> +   address_spaces
>      directories
>      bloblist
>      bootstd
>
> ---
> base-commit: ad7dce5abd49ef3b5c93da5303e15449c8c162b4
> change-id: 20240517-address-spaces-1c3b9c379b13
>
> Best regards,