bugs with EFI "reserved" memory and LMB?
Emanuele Ghidoli
ghidoliemanuele at gmail.com
Thu Jul 27 14:14:22 CEST 2023
On 27/07/2023 13:21, Heinrich Schuchardt wrote:
> On 27.07.23 13:08, Emanuele Ghidoli wrote:
>> Efi loader module have its own memory management that flags as reserved the
>> area between ram_top to ram_end (currently where dt reserved-memory is
>> falling). uboot lmb updates reserved-memory by adding these
>> efi loader module areas (see lmb_reserve_common->efi_lmb_reserve).
>>
>>
>> On our system (AM62xx SoC, 512MB RAM) we have some reserved-memory specified in
>> the device tree (used for comunication with other microcontroller on the SoC,
>> OPTEE, ...) falling in the DDR range between 456 and 512 MB.
>>
>> Normally U-Boot relocate itself at the end of DDR so it overlap this
>> reserved-memory and this of course does not work [1].
>>
>> In this case U-Boot prints the following errors:
>>
>> ERROR: reserving fdt memory region failed (addr=9c800000 size=300000)
>> ERROR: reserving fdt memory region failed (addr=9cb00000 size=100000)
>> ...
>> ERROR: reserving fdt memory region failed (addr=9e800000 size=1800000)
>>
>> because U-Boot reserved area (lmb_flag LMB_NONE) overlaps with the FDT reserve
>> memories (lmb_flag LMB_NOMAP).
>>
>>
>> To fix this I moved the U-Boot relocation address, implementing
>> board_get_usable_ram_top() and therefore setting gd->ram_top to a lower value
>> with no overlap (448 MB).
>>
>>
>> The memory map is:
>> +---------------+ 512 MB
>> |DT reserved-mem|
>> +---------------+ 456 MB
>> |free space |
>> +---------------+ 448 MB (ram_top)
>> | |
>> |uboot |
>> | |
>> +---------------+ 0 MB
>>
>> This is working fine (the board is able to boot, no memory overlaps) ...
>>
>> however, in this configuration U-Boot we still have some errors prints while
>> loading linux dtb:
>>
>> ERROR: reserving fdt memory region failed (addr=9cb00000 size=100000 flags=4)
>> ERROR: reserving fdt memory region failed (addr=9cc00000 size=e00000 flags=4)
>> ERROR: reserving fdt memory region failed (addr=9da00000 size=100000 flags=4)
>> ERROR: reserving fdt memory region failed (addr=9db00000 size=c00000 flags=4)
>> ERROR: reserving fdt memory region failed (addr=9e780000 size=80000 flags=4)
>> ERROR: reserving fdt memory region failed (addr=9e800000 size=1800000 flags=4)
>>
>> And the reason of these error is complete different: The efi loader lmb memory
>> reservation (lib/lmb.c:efi_lmb_reserve()) is wrongly reserving some area.
>>
>>
>> 1) lib/lmb.c:lmb_reserve_common() reserve these memories:
>> ~415M - 448M u-boot reserved area (lmb_flag LMB_NONE)
>> 456M - 512M fdt reserved-memory (lmb_flag LMB_NOMAP)
>>
>> 2) lib/lmb.c:efi_lmb_reserve() reserve a region just before u-boot reserved
>> area. This is not coalesced/merged with the u-boot reserved area because it is
>> not contiguous.
>>
>> This is the reserved-memory array at this point:
>> ~414M - ~414M efi loader reserved area (lmb_flag LMB_NONE)
>> ~415M - 448M u-boot reserved area (lmb_flag LMB_NONE)
>> 456M - 512M fdt reserved-memory (lmb_flag LMB_NOMAP)
>>
>> 3) lib/lmb.c:efi_lmb_reserve() reserve some more areas.
>> In the last two steps efi loader add some reserved area that overlap u-boot and fdt
>> reserved area BUT they are also contiguous to the "efi loader reserved area".
>> The very last step reserve also area between ram_top (448M) to ram_end (512M)
>>
>> We fall in this condition (where we have overlapping areas!):
>> ~414M - 512M efi loader reserved area (lmb_flag LMB_NONE)
>> ~415M - 448M u-boot reserved area (lmb_flag LMB_NONE)
>> 456M - 512M fdt reserved-memory (lmb_flag LMB_NOMAP)
>>
>> 4) Now, while loading the linux fdt the reserved-memory areas are checked
>> toward efi loader reserved area, they overlap, BUT they have different
>> lmb_flag. So we get the ERROR print.
>>
>> Hopefully this is clear, I undestand is not that obvious ...
>
> Thanks Emanuele for reporting the issue.
>
> Could you, please, provide the output of
>
> efidebug memmap
Verdin AM62 # efidebug memmap
MMC: no card present
No EFI system partition
No EFI system partition
Failed to persist EFI variables
Type Start End Attributes
================ ================ ================ ==========
CONVENTIONAL 0000000080000000-0000000098eec000 WB
BOOT DATA 0000000098eec000-0000000098eee000 WB
RUNTIME DATA 0000000098eee000-0000000098eef000 WB|RT
BOOT DATA 0000000098eef000-0000000098ef2000 WB
RUNTIME DATA 0000000098ef2000-0000000098ef3000 WB|RT
BOOT DATA 0000000098ef3000-0000000098ef4000 WB
RUNTIME DATA 0000000098ef4000-0000000098ef6000 WB|RT
BOOT DATA 0000000098ef6000-0000000098ef7000 WB
RUNTIME DATA 0000000098ef7000-0000000098f07000 WB|RT
BOOT DATA 0000000098f07000-0000000098f10000 WB
BOOT CODE 0000000098f10000-000000009bf20000 WB
RUNTIME CODE 000000009bf20000-000000009bf30000 WB|RT
BOOT CODE 000000009bf30000-000000009c000000 WB
BOOT DATA 000000009c000000-00000000a0000000 WB
>
> and of
>
> bdinfo
With this "debug" patch I hope giving you a better picture:
diff --git a/lib/lmb.c b/lib/lmb.c
index b2c233edb64e..1feb7189936e 100644
--- a/lib/lmb.c
+++ b/lib/lmb.c
@@ -197,6 +197,9 @@ static void lmb_reserve_common(struct lmb *lmb, void *fdt_blob)
if (CONFIG_IS_ENABLED(OF_LIBFDT) && fdt_blob)
boot_fdt_add_mem_rsv_regions(lmb, fdt_blob);
+ printf("%s. Calling lmb_dump_all_force between boot_fdt_add_mem_rsv_regions and efi_lmb_reserve\n", __func__);
+ lmb_dump_all_force(lmb);
+
if (CONFIG_IS_ENABLED(EFI_LOADER))
efi_lmb_reserve(lmb);
}
Verdin AM62 # bdinfo
boot_params = 0x0000000000000000
DRAM bank = 0x0000000000000000
-> start = 0x0000000080000000
-> size = 0x0000000020000000
flashstart = 0x0000000000000000
flashsize = 0x0000000000000000
flashoffset = 0x0000000000000000
baudrate = 115200 bps
relocaddr = 0x000000009bf22000
reloc off = 0x000000001b722000
Build = 64-bit
current eth = ethernet at 8000000port@1
ethaddr = 00:14:2d:e5:73:c1
IP addr = <NULL>
fdt_blob = 0x0000000099f10890
new_fdt = 0x0000000099f10890
fdt_size = 0x000000000000f4e0
multi_dtb_fit= 0x0000000000000000
lmb_reserve_common. Calling lmb_dump_all_force between boot_fdt_add_mem_rsv_regions and efi_lmb_reserve
lmb_dump_all:
memory.cnt = 0x1 / max = 0x10
memory[0] [0x80000000-0x9fffffff], 0x20000000 bytes flags: 0
reserved.cnt = 0x3 / max = 0x10
reserved[0] [0x99f0c280-0x9bffffff], 0x020f3d80 bytes flags: 0
reserved[1] [0x9db00000-0x9e6fffff], 0x00c00000 bytes flags: 4
reserved[2] [0x9e780000-0x9fffffff], 0x01880000 bytes flags: 4
lmb_dump_all:
memory.cnt = 0x1 / max = 0x10
memory[0] [0x80000000-0x9fffffff], 0x20000000 bytes flags: 0
reserved.cnt = 0x4 / max = 0x10
reserved[0] [0x98f06000-0x9fffffff], 0x070fa000 bytes flags: 0
reserved[1] [0x99f0c280-0x9bffffff], 0x020f3d80 bytes flags: 0
reserved[2] [0x9db00000-0x9e6fffff], 0x00c00000 bytes flags: 4
reserved[3] [0x9e780000-0x9fffffff], 0x01880000 bytes flags: 4
devicetree = separate
serial addr = 0x0000000002800000
width = 0x0000000000000000
shift = 0x0000000000000002
offset = 0x0000000000000000
clock = 0x0000000002dc6c00
arch_number = 0x0000000000000000
TLB addr = 0x000000009bff0000
irq_sp = 0x0000000099f10880
sp start = 0x0000000099f10880
Early malloc usage: 2a88 / 8000
>
> Best regards
>
> Heinrich
> >>
>> IMO we have two different bugs:
>> - there is nothing that prevent that coalesced area may overlap other areas
>> (while lmb module expect that there aren't overlapping areas)
>> - efi loader (correctly) reserve between ram_top and ram_end BUT this area is
>> reserved by fdt.
>>
>> One potential solution could be to override efi_add_known_memory() and set
>> ram_top = ram_end, e.g.
>>
>> - efi_add_conventional_memory_map(ram_start, ram_end, ram_top);
>> + efi_add_conventional_memory_map(ram_start, ram_end, ram_end);
>>
>>
>> but this does not really seems like something that should ve overridden at the
>> board level.
>>
>> Any suggestion?
>>
>> Emanuele
>
Very interesting the efidebug command...
Thank you.
Emanuele
More information about the U-Boot
mailing list