[U-Boot] [REGRESSION] tegra124: nyan-big: LPAE not working

[Thierry Reding]

On Mon, Feb 11, 2019 at 10:04:37AM +0000, Tristan Bastian wrote:
> 
> 
> 
> Thierry Reding – Mon, 11. February 2019 10:38
> > On Mon, Feb 11, 2019 at 09:20:33AM +0000, Tristan Bastian wrote:
> > > 
> > > Thierry Reding – Mon, 11. February 2019 9:52
> > > > On Sun, Feb 10, 2019 at 08:53:11PM +0100, Tristan Bastian wrote:
> > > > > Thierry, do you have any news on this?
> > > > > 
> > > > > I don't think, that google is going to push an updated version of
> > coreboot
> > > > > to each nyan-big device..
> > > > > So reverting this patch at least for the nyan devices would be the best
> > I
> > > > > think..
> > > > 
> > > > Yes, I agree. I had a brief chat with Rob Herring about this and he too
> > > > agrees that we should revert it for now. I'll make it a manual revert
> > > > and add a comment to the device tree node that will hopefully avoid any
> > > > future janitorial "cleanups" of this sort.
> > > 
> > > great news! :)
> > > 
> > > > > BTW: I'm now running u-boot natively and it seems like u-boot always has
> > a
> > > > > problem with the memory..
> > > > > If u-boot is used chainloaded to coreboot it is only getting 2GB of
> > memory
> > > > > and running u-boot natively also just gives me 2GB..
> > > > > I've tested that with a kernel with "ARM: tegra: Fix unit_address_vs_reg
> > DTC
> > > > > warnings for /memory" reverted and also on a kernel, before this patch
> > was
> > > > > applied..
> > > > 
> > > > It's possible that U-Boot doesn't support LPAE and therefore may not be
> > > > able to use more than the 2 GiB of memory.
> > > 
> > > So I at least enabled LPAE in u-boot with "CONFIG_ARMV7_LPAE=y" and
> > > this was for some reason also needed to get some output on the
> > > display..
> > > I'm not sure why LPAE needs to activated in u-boot for display output
> > > on the nyan-big..
> > > Without LPAE enabled u-boot was still working, and booted linux, but
> > > u-boot didn't display anything on screen, linux then did..
> > 
> > Yeah, that's surprising. Perhaps without LPAE U-Boot thinks there's not
> > enough memory for the framebuffer? There should be plenty, so maybe
> > there is something else going on here.
> > 
> > > > However, U-Boot should be
> > > > able to tell the kernel exactly how much memory the system has and pass
> > > > that on via device tree. That still does work, right?
> > > 
> > > It seems like this is not working..
> > > And this seems to be the case with both, u-boot chainloaded and
> > > running u-boot natively..
> > > 
> > > I've used these scripts for flashing:
> > > github.com/NVIDIA/tegra-uboot-flasher-scripts
> > > And used the norrin device since it seems like it is the nyan-big dev
> > > board?
> > > But going with the norrin device config should not be the issue here
> > > since the problem also exists when chainloading u-boot, right?
> > 
> > It could be a problem. The memory bank configuration is stored in what's
> > called a BCT along with a bunch of other parameters that define what the
> > memory controller needs to access the given memory chips. So if you've
> > flashed a BCT that's for a board with only 2 GiB of memory you would end
> > up with a system that can't address any more than that. It's somewhat
> > surprising that memory accesses work at all with a BCT that's for
> > different memory chips, but sometimes you can get lucky.
> > 
> > You may want to try reflashing with the right BCT. The simplest would
> > probably be to duplicate the cbootimage configuration for Norrin and
> > substitute the relevant bits by what you have from the Chromebook flash
> > utilities. Looks like the BCT in really the only thing you can replace
> > there. Make sure to replace it with the one that matches your Chromebook
> > and it should give you the right memory bank configuration.
> 
> Is there a way to check if the correct bct for my 4GB model was used?
> What I tried was to edit the existing bct config file of norrin from here:
> https://github.com/NVIDIA/cbootimage-configs/blob/master/tegra124/nvidia/norrin/PM370_Hynix_2GB_H5TC4G63AFR_PBA_924MHz_01212014.bct.cfg
> by replacing all entries for each memory bank with the entries from here:
> https://github.com/coreboot/coreboot/blob/master/src/mainboard/google/nyan_big/bct/sdram-hynix-4GB-792.inc
> 
> I also added the SDRAM[0], SDRAM[1], SDRAM[2], SDRAM[3] each time in front of each line..
> This also booted for me, but also just leaves me with only 2Gb of my 4GB..
> 
> And since this problem also occurs, when chainloading u-boot, this must be a problem with u-boot, I think..

Not necessarily. U-Boot reads the memory size from EMC registers and
those EMC registers are programmed based on the contents of the BCT. So
if the BCT is wrong, then it's going to be wrong irrespective of what
bootloader you use.

That said, I does indeed look as if U-Boot only supports 2 GiB of memory
on 32-bit Tegra. arch/arm/mach-tegra/board.c contains the relevant code.
If you look at dram_init() it uses query_sdram_size() to get the size of
RAM. That will allow to convey up to 4 GiB of RAM, but then the code in
board2.c (see dram_init_banksize()) only uses a single bank on 32-bit
Tegra (CONFIG_PHYS_64BIT is only set for 64-bit Tegra).

According to the comments for dram_init_bank_size() we can't store the
second bank because the .start field is 32 bits wide and therefore can't
store the 0x100000000 start address for the bank.

It sounds to me like ARMV7_LPAE should perhaps select PHYS_64BIT to make
sure all the appropriate variables can store addresses beyond the 32-bit
boundary. It's one thing to not support > 2 GiB of memory in U-Boot, but
it's an entirely different story if we can't even pass > GiB of memory
to the kernel if they exist.

Stephen, do you remember if there was anything preventing us from going
64-bit for phys_addr_t and phys_size_t on LPAE enabled systems?

Cc Tom for visibility and U-Boot since this is a U-Boot bug.

Thierry

> Tristan
> 
> > > Simon Glass used his servo board to enter the tegrarcm mode:
> > > patchwork.ozlabs.org/patch/1019003/
> > > 
> > > I had to erase the internal SPI flash in order to boot into the
> > > tegrarcm mode.. 
> > > Do you have some instructions or internal documentation to tell me, if
> > > some of the unpopulated buttons on the nyan-big mainboard can be used
> > > to enter the tegrarcm state?
> > 
> > You should be able to boot into recovery mode with the "enterrcm"
> > command in U-Boot. I'm not aware of any buttons on the mainboard that
> > can be used for this, unfortunately.
> > 
> > Thierry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20190211/c94ed6dd/attachment.sig>