bugs with EFI "reserved" memory and LMB?

[Emanuele Ghidoli]

Efi loader module have its own memory management that flags as reserved the
area between ram_top to ram_end (currently where dt reserved-memory is
falling).  uboot lmb updates reserved-memory by adding these
efi loader module areas (see lmb_reserve_common->efi_lmb_reserve).


On our system (AM62xx SoC, 512MB RAM) we have some reserved-memory specified in
the device tree (used for comunication with other microcontroller on the SoC,
OPTEE, ...) falling in the DDR range between 456 and 512 MB.

Normally U-Boot relocate itself at the end of DDR so it overlap this
reserved-memory and this of course does not work [1].

In this case U-Boot prints the following errors:

  ERROR: reserving fdt memory region failed (addr=9c800000 size=300000)
  ERROR: reserving fdt memory region failed (addr=9cb00000 size=100000)
  ...
  ERROR: reserving fdt memory region failed (addr=9e800000 size=1800000)

because U-Boot reserved area (lmb_flag LMB_NONE) overlaps with the FDT reserve
memories (lmb_flag LMB_NOMAP).


To fix this I moved the U-Boot relocation address, implementing
board_get_usable_ram_top() and therefore setting gd->ram_top to a lower value
with no overlap (448 MB).


The memory map is:
+---------------+ 512 MB
|DT reserved-mem|
+---------------+ 456 MB
|free space     |
+---------------+ 448 MB (ram_top)
|               |
|uboot          |
|               |
+---------------+ 0 MB

This is working fine (the board is able to boot, no memory overlaps) ...

however, in this configuration U-Boot we still have some errors prints while
loading linux dtb:

  ERROR: reserving fdt memory region failed (addr=9cb00000 size=100000 flags=4)
  ERROR: reserving fdt memory region failed (addr=9cc00000 size=e00000 flags=4)
  ERROR: reserving fdt memory region failed (addr=9da00000 size=100000 flags=4)
  ERROR: reserving fdt memory region failed (addr=9db00000 size=c00000 flags=4)
  ERROR: reserving fdt memory region failed (addr=9e780000 size=80000 flags=4)
  ERROR: reserving fdt memory region failed (addr=9e800000 size=1800000 flags=4)

And the reason of these error is complete different: The efi loader lmb memory
reservation (lib/lmb.c:efi_lmb_reserve()) is wrongly reserving some area.


1) lib/lmb.c:lmb_reserve_common() reserve these memories:
  ~415M - 448M u-boot reserved area (lmb_flag LMB_NONE)
   456M - 512M fdt reserved-memory (lmb_flag LMB_NOMAP)

2) lib/lmb.c:efi_lmb_reserve() reserve a region just before u-boot reserved
area. This is not coalesced/merged with the u-boot reserved area because it is
not contiguous.

This is the reserved-memory array at this point:
  ~414M - ~414M efi loader reserved area (lmb_flag LMB_NONE)
  ~415M - 448M u-boot reserved area (lmb_flag LMB_NONE)
   456M - 512M fdt reserved-memory (lmb_flag LMB_NOMAP)

3) lib/lmb.c:efi_lmb_reserve() reserve some more areas.
In the last two steps efi loader add some reserved area that overlap u-boot and fdt
reserved area BUT they are also contiguous to the "efi loader reserved area".
The very last step reserve also area between ram_top (448M) to ram_end (512M)

We fall in this condition (where we have overlapping areas!):
  ~414M - 512M efi loader reserved area (lmb_flag LMB_NONE)
  ~415M - 448M u-boot reserved area (lmb_flag LMB_NONE)
   456M - 512M fdt reserved-memory (lmb_flag LMB_NOMAP)

4) Now, while loading the linux fdt the reserved-memory areas are checked
toward efi loader reserved area, they overlap, BUT they have different
lmb_flag. So we get the ERROR print.

Hopefully this is clear, I undestand is not that obvious ...

IMO we have two different bugs:
 - there is nothing that prevent that coalesced area may overlap other areas
   (while lmb module expect that there aren't overlapping areas)
 - efi loader (correctly) reserve between ram_top and ram_end BUT this area is
   reserved by fdt.

One potential solution could be to override efi_add_known_memory() and set
ram_top = ram_end, e.g.

-           efi_add_conventional_memory_map(ram_start, ram_end, ram_top);
+           efi_add_conventional_memory_map(ram_start, ram_end, ram_end);


but this does not really seems like something that should ve overridden at the
board level.

Any suggestion?

Emanuele