[REGRESSION] lmb: add and reserve memory above ram_top

[Sughosh Ganu]

On Mon, 9 Dec 2024 at 12:12, Sughosh Ganu <sughosh.ganu at linaro.org> wrote:
>
> On Sun, 8 Dec 2024 at 06:49, Sam Protsenko <semen.protsenko at linaro.org> wrote:
> >
> > On Thu, Dec 5, 2024 at 11:23 PM Sughosh Ganu <sughosh.ganu at linaro.org> wrote:
> > >
> >
> > [snip]
> >
> > > >     ERROR: reserving fdt memory region failed (addr=bab00000
> > > > size=5500000 flags=2)
> > > >     ERROR: reserving fdt memory region failed (addr=f0000000
> > > > size=200000 flags=4)
> > > >
> > > > It looks like all mentioned error messages are harmless, in a way that
> > > > I'm still able to boot Linux, and U-Boot reserved areas still seem to
> > > > exist in 'bdinfo'. But seeing error messages like that is definitely a
> > > > nuisance by itself, and might be an indicator of some actual issues.
> > > > Do you have any remedy for that by chance? Please let me know if you
> > > > need more debug input from my side.
> > >
> > > This seems to be happening because the board is trying to reserve
> > > memory above ram_top with no-map or no-overwrite attributes. To
> > > confirm, you can print the value of ram_top in setup_dest_addr().
> > >
> > > -sughosh
> > >
> >
> > Thanks for your input, Sughosh! I did a bit of further analysis, and
> > found the mentioned error message is happening on 3 different
> > occasions. All 3 of them are caused by the recent LMB patches merged
> > in U-Boot/master. They are all probably harmless, but a bit
> > misleading. Also, I'm not sure whether they are actually *causing*
> > those errors, or just revealing some already existing issues.
> > Specifically those 3 occurrences are:
>
> Thanks much for your investigation.
>
> >
> > 1. The one reported in this thread. The root cause, like you
> > suggested, is the board trying to reserve the memory above ram_top.
> > It's actually happening in lmb_init(). Details:
> >
> >   - The 'memory' node from my dts:
> >
> >         memory at 80000000 {
> >             device_type = "memory";
> >             reg = <0x0 0x80000000 0x3ab00000>,
> >                   <0x0 0xc0000000 0x40000000>,
> >                   <0x8 0x80000000 0x80000000>;
> >         };
> >
> >   - Reserved memory chunk: (addr=f0000000 size=200000 flags=4).
> >     That's 'ramoops' chunk from the 'reserved-memory' node in my dts.
> >   - ram_top (as seen in lmb_init) = 0xbab0_0000.
> >     That's because my board's dram_init() calls the
> > fdtdec_setup_mem_size_base(), which only takes the first cell from the
> > 'memory' node.
> >     Which is 0x8000_0000 ... 0xbab0_0000.
> >   - So the reserved memory is above ram_top, as you suggested.
> >   - Not sure how to properly fix it though. Maybe I need to "teach"
> > U-Boot to use all 3 chunks from my 'memory' node somehow?
>
> I think there needs to be a check in the related function to not
> reserve memory regions above ram_top. This wasn't showing up earlier
> because of the local nature of the lmb memory map, but is not the case
> anymore. I have a patch which I am testing currently. Will send it out
> in a day or so.

This change should not be needed after your series gets applied.

>
> >   - I was able to work-around it by reworking the 'memory' node in my
> > device tree as described in my first message in this thread:
> >
> >         memory at 80000000 {
> >             device_type = "memory";
> >             reg = <0x0 0x80000000 0x80000000>,
> >                   <0x8 0x80000000 0x80000000>;
> >         };
> >
> > 2. When booting Linux with 'booti' command, there happens the repeated
> > attempt to reserve all 'reserved-memory' chunks, which were already
> > reserved during the board init, which also leads to similar error
> > messages. I addressed that one (hopefully properly) in my series [1].
> > Please take a look. That one is caused by a combination of these two
> > commits:
> >
> >   - 1d9aa4a283da ("lmb: Fix the allocation of overlapping memory areas
> > with !LMB_NONE")
> >   - 827dee587b75 ("fdt: lmb: add reserved regions as no-overwrite")
>
> I will try out your patch series on my side and get back.
>
> >
> > 3. I noticed another similar error message when running sandbox U-Boot on my PC:
> >
> >         $ make sandbox_defconfig
> >         $ make -j32
> >         $ ./u-boot -D
> >         ...
> >         ERROR: reserving fdt memory region failed (addr=fffe000
> > size=2000 flags=2)
> >
> >   Can you please check that one on your U-Boot/master? I was able to
> > narrow it down to commit 6534d26ee9a5 ("lmb: do away with
> > arch_lmb_reserve()"), if that helps.
>
> This is strange. I have not seen this issue with sandbox earlier.
> Might be a regression. Will check. Thanks.

Same here. The issue goes away after this series.

-sughosh

>
> -sughosh
>
> >
> > Thanks!
> >
> > [1] https://lists.denx.de/pipermail/u-boot/2024-December/574308.html