bugs with EFI "reserved" memory and LMB?

[Heinrich Schuchardt]

On 27.07.23 13:08, Emanuele Ghidoli wrote:
> Efi loader module have its own memory management that flags as reserved the
> area between ram_top to ram_end (currently where dt reserved-memory is
> falling).  uboot lmb updates reserved-memory by adding these
> efi loader module areas (see lmb_reserve_common->efi_lmb_reserve).
>
>
> On our system (AM62xx SoC, 512MB RAM) we have some reserved-memory specified in
> the device tree (used for comunication with other microcontroller on the SoC,
> OPTEE, ...) falling in the DDR range between 456 and 512 MB.
>
> Normally U-Boot relocate itself at the end of DDR so it overlap this
> reserved-memory and this of course does not work [1].
>
> In this case U-Boot prints the following errors:
>
>    ERROR: reserving fdt memory region failed (addr=9c800000 size=300000)
>    ERROR: reserving fdt memory region failed (addr=9cb00000 size=100000)
>    ...
>    ERROR: reserving fdt memory region failed (addr=9e800000 size=1800000)
>
> because U-Boot reserved area (lmb_flag LMB_NONE) overlaps with the FDT reserve
> memories (lmb_flag LMB_NOMAP).
>
>
> To fix this I moved the U-Boot relocation address, implementing
> board_get_usable_ram_top() and therefore setting gd->ram_top to a lower value
> with no overlap (448 MB).
>
>
> The memory map is:
> +---------------+ 512 MB
> |DT reserved-mem|
> +---------------+ 456 MB
> |free space     |
> +---------------+ 448 MB (ram_top)
> |               |
> |uboot          |
> |               |
> +---------------+ 0 MB
>
> This is working fine (the board is able to boot, no memory overlaps) ...
>
> however, in this configuration U-Boot we still have some errors prints while
> loading linux dtb:
>
>    ERROR: reserving fdt memory region failed (addr=9cb00000 size=100000 flags=4)
>    ERROR: reserving fdt memory region failed (addr=9cc00000 size=e00000 flags=4)
>    ERROR: reserving fdt memory region failed (addr=9da00000 size=100000 flags=4)
>    ERROR: reserving fdt memory region failed (addr=9db00000 size=c00000 flags=4)
>    ERROR: reserving fdt memory region failed (addr=9e780000 size=80000 flags=4)
>    ERROR: reserving fdt memory region failed (addr=9e800000 size=1800000 flags=4)
>
> And the reason of these error is complete different: The efi loader lmb memory
> reservation (lib/lmb.c:efi_lmb_reserve()) is wrongly reserving some area.
>
>
> 1) lib/lmb.c:lmb_reserve_common() reserve these memories:
>    ~415M - 448M u-boot reserved area (lmb_flag LMB_NONE)
>     456M - 512M fdt reserved-memory (lmb_flag LMB_NOMAP)
>
> 2) lib/lmb.c:efi_lmb_reserve() reserve a region just before u-boot reserved
> area. This is not coalesced/merged with the u-boot reserved area because it is
> not contiguous.
>
> This is the reserved-memory array at this point:
>    ~414M - ~414M efi loader reserved area (lmb_flag LMB_NONE)
>    ~415M - 448M u-boot reserved area (lmb_flag LMB_NONE)
>     456M - 512M fdt reserved-memory (lmb_flag LMB_NOMAP)
>
> 3) lib/lmb.c:efi_lmb_reserve() reserve some more areas.
> In the last two steps efi loader add some reserved area that overlap u-boot and fdt
> reserved area BUT they are also contiguous to the "efi loader reserved area".
> The very last step reserve also area between ram_top (448M) to ram_end (512M)
>
> We fall in this condition (where we have overlapping areas!):
>    ~414M - 512M efi loader reserved area (lmb_flag LMB_NONE)
>    ~415M - 448M u-boot reserved area (lmb_flag LMB_NONE)
>     456M - 512M fdt reserved-memory (lmb_flag LMB_NOMAP)
>
> 4) Now, while loading the linux fdt the reserved-memory areas are checked
> toward efi loader reserved area, they overlap, BUT they have different
> lmb_flag. So we get the ERROR print.
>
> Hopefully this is clear, I undestand is not that obvious ...

Thanks Emanuele for reporting the issue.

Could you, please, provide the output of

    efidebug memmap

and of

    bdinfo

Best regards

Heinrich

>
> IMO we have two different bugs:
>   - there is nothing that prevent that coalesced area may overlap other areas
>     (while lmb module expect that there aren't overlapping areas)
>   - efi loader (correctly) reserve between ram_top and ram_end BUT this area is
>     reserved by fdt.
>
> One potential solution could be to override efi_add_known_memory() and set
> ram_top = ram_end, e.g.
>
> -           efi_add_conventional_memory_map(ram_start, ram_end, ram_top);
> +           efi_add_conventional_memory_map(ram_start, ram_end, ram_end);
>
>
> but this does not really seems like something that should ve overridden at the
> board level.
>
> Any suggestion?
>
> Emanuele