[REGRESSION] lmb: add and reserve memory above ram_top

[Sam Protsenko]

On Mon, Dec 9, 2024 at 5:43 AM Sughosh Ganu <sughosh.ganu at linaro.org> wrote:
>
> On Mon, 9 Dec 2024 at 12:12, Sughosh Ganu <sughosh.ganu at linaro.org> wrote:
> >
> > On Sun, 8 Dec 2024 at 06:49, Sam Protsenko <semen.protsenko at linaro.org> wrote:
> > >
> > > On Thu, Dec 5, 2024 at 11:23 PM Sughosh Ganu <sughosh.ganu at linaro.org> wrote:
> > > >
> > >
> > > [snip]
> > >
> > > > >     ERROR: reserving fdt memory region failed (addr=bab00000
> > > > > size=5500000 flags=2)
> > > > >     ERROR: reserving fdt memory region failed (addr=f0000000
> > > > > size=200000 flags=4)
> > > > >
> > > > > It looks like all mentioned error messages are harmless, in a way that
> > > > > I'm still able to boot Linux, and U-Boot reserved areas still seem to
> > > > > exist in 'bdinfo'. But seeing error messages like that is definitely a
> > > > > nuisance by itself, and might be an indicator of some actual issues.
> > > > > Do you have any remedy for that by chance? Please let me know if you
> > > > > need more debug input from my side.
> > > >
> > > > This seems to be happening because the board is trying to reserve
> > > > memory above ram_top with no-map or no-overwrite attributes. To
> > > > confirm, you can print the value of ram_top in setup_dest_addr().
> > > >
> > > > -sughosh
> > > >
> > >
> > > Thanks for your input, Sughosh! I did a bit of further analysis, and
> > > found the mentioned error message is happening on 3 different
> > > occasions. All 3 of them are caused by the recent LMB patches merged
> > > in U-Boot/master. They are all probably harmless, but a bit
> > > misleading. Also, I'm not sure whether they are actually *causing*
> > > those errors, or just revealing some already existing issues.
> > > Specifically those 3 occurrences are:
> >
> > Thanks much for your investigation.
> >
> > >
> > > 1. The one reported in this thread. The root cause, like you
> > > suggested, is the board trying to reserve the memory above ram_top.
> > > It's actually happening in lmb_init(). Details:
> > >
> > >   - The 'memory' node from my dts:
> > >
> > >         memory at 80000000 {
> > >             device_type = "memory";
> > >             reg = <0x0 0x80000000 0x3ab00000>,
> > >                   <0x0 0xc0000000 0x40000000>,
> > >                   <0x8 0x80000000 0x80000000>;
> > >         };
> > >
> > >   - Reserved memory chunk: (addr=f0000000 size=200000 flags=4).
> > >     That's 'ramoops' chunk from the 'reserved-memory' node in my dts.
> > >   - ram_top (as seen in lmb_init) = 0xbab0_0000.
> > >     That's because my board's dram_init() calls the
> > > fdtdec_setup_mem_size_base(), which only takes the first cell from the
> > > 'memory' node.
> > >     Which is 0x8000_0000 ... 0xbab0_0000.
> > >   - So the reserved memory is above ram_top, as you suggested.
> > >   - Not sure how to properly fix it though. Maybe I need to "teach"
> > > U-Boot to use all 3 chunks from my 'memory' node somehow?
> >
> > I think there needs to be a check in the related function to not
> > reserve memory regions above ram_top. This wasn't showing up earlier
> > because of the local nature of the lmb memory map, but is not the case
> > anymore. I have a patch which I am testing currently. Will send it out
> > in a day or so.
>
> This change should not be needed after your series gets applied.
>
> >
> > >   - I was able to work-around it by reworking the 'memory' node in my
> > > device tree as described in my first message in this thread:
> > >
> > >         memory at 80000000 {
> > >             device_type = "memory";
> > >             reg = <0x0 0x80000000 0x80000000>,
> > >                   <0x8 0x80000000 0x80000000>;
> > >         };
> > >
> > > 2. When booting Linux with 'booti' command, there happens the repeated
> > > attempt to reserve all 'reserved-memory' chunks, which were already
> > > reserved during the board init, which also leads to similar error
> > > messages. I addressed that one (hopefully properly) in my series [1].
> > > Please take a look. That one is caused by a combination of these two
> > > commits:
> > >
> > >   - 1d9aa4a283da ("lmb: Fix the allocation of overlapping memory areas
> > > with !LMB_NONE")
> > >   - 827dee587b75 ("fdt: lmb: add reserved regions as no-overwrite")
> >
> > I will try out your patch series on my side and get back.
> >
> > >
> > > 3. I noticed another similar error message when running sandbox U-Boot on my PC:
> > >
> > >         $ make sandbox_defconfig
> > >         $ make -j32
> > >         $ ./u-boot -D
> > >         ...
> > >         ERROR: reserving fdt memory region failed (addr=fffe000
> > > size=2000 flags=2)
> > >
> > >   Can you please check that one on your U-Boot/master? I was able to
> > > narrow it down to commit 6534d26ee9a5 ("lmb: do away with
> > > arch_lmb_reserve()"), if that helps.
> >
> > This is strange. I have not seen this issue with sandbox earlier.
> > Might be a regression. Will check. Thanks.
>
> Same here. The issue goes away after this series.
>

You are right, I just confirmed it on my end. With Ilias's suggestion,
all 3 error messages are gone. I'll send v2 soon. Thanks for noticing
that!

> -sughosh
>
> >
> > -sughosh
> >
> > >
> > > Thanks!
> > >
> > > [1] https://lists.denx.de/pipermail/u-boot/2024-December/574308.html