[U-Boot] [REGRESSION] tegra124: nyan-big: LPAE not working

Tue Feb 12 01:02:27 UTC 2019

Hi there

On Mon, 2019-02-11 at 15:13 -0700, Stephen Warren wrote:
> On 2/11/19 3:48 AM, Thierry Reding wrote:
> > On Mon, Feb 11, 2019 at 10:04:37AM +0000, Tristan Bastian wrote:
> > > 
> > > 
> > > Thierry Reding – Mon, 11. February 2019 10:38
> > > > On Mon, Feb 11, 2019 at 09:20:33AM +0000, Tristan Bastian
> > > > wrote:
> > > > > Thierry Reding – Mon, 11. February 2019 9:52
> > > > > > On Sun, Feb 10, 2019 at 08:53:11PM +0100, Tristan Bastian
> > > > > > wrote:
> > > > > > > Thierry, do you have any news on this?
> > > > > > > 
> > > > > > > I don't think, that google is going to push an updated
> > > > > > > version of
> > > > coreboot
> > > > > > > to each nyan-big device..
> > > > > > > So reverting this patch at least for the nyan devices
> > > > > > > would be the best
> > > > I
> > > > > > > think..
> > > > > > 
> > > > > > Yes, I agree. I had a brief chat with Rob Herring about
> > > > > > this and he too
> > > > > > agrees that we should revert it for now. I'll make it a
> > > > > > manual revert
> > > > > > and add a comment to the device tree node that will
> > > > > > hopefully avoid any
> > > > > > future janitorial "cleanups" of this sort.
> > > > > 
> > > > > great news! :)
> > > > > 
> > > > > > > BTW: I'm now running u-boot natively and it seems like u-
> > > > > > > boot always has
> > > > a
> > > > > > > problem with the memory..
> > > > > > > If u-boot is used chainloaded to coreboot it is only
> > > > > > > getting 2GB of
> > > > memory
> > > > > > > and running u-boot natively also just gives me 2GB..
> > > > > > > I've tested that with a kernel with "ARM: tegra: Fix
> > > > > > > unit_address_vs_reg
> > > > DTC
> > > > > > > warnings for /memory" reverted and also on a kernel,
> > > > > > > before this patch
> > > > was
> > > > > > > applied..
> > > > > > 
> > > > > > It's possible that U-Boot doesn't support LPAE and
> > > > > > therefore may not be
> > > > > > able to use more than the 2 GiB of memory.
> > > > > 
> > > > > So I at least enabled LPAE in u-boot with
> > > > > "CONFIG_ARMV7_LPAE=y" and
> > > > > this was for some reason also needed to get some output on
> > > > > the
> > > > > display..
> > > > > I'm not sure why LPAE needs to activated in u-boot for
> > > > > display output
> > > > > on the nyan-big..
> > > > > Without LPAE enabled u-boot was still working, and booted
> > > > > linux, but
> > > > > u-boot didn't display anything on screen, linux then did..
> > > > 
> > > > Yeah, that's surprising. Perhaps without LPAE U-Boot thinks
> > > > there's not
> > > > enough memory for the framebuffer? There should be plenty, so
> > > > maybe
> > > > there is something else going on here.
> > > > 
> > > > > > However, U-Boot should be
> > > > > > able to tell the kernel exactly how much memory the system
> > > > > > has and pass
> > > > > > that on via device tree. That still does work, right?
> > > > > 
> > > > > It seems like this is not working..
> > > > > And this seems to be the case with both, u-boot chainloaded
> > > > > and
> > > > > running u-boot natively..
> > > > > 
> > > > > I've used these scripts for flashing:
> > > > > github.com/NVIDIA/tegra-uboot-flasher-scripts
> > > > > And used the norrin device since it seems like it is the
> > > > > nyan-big dev
> > > > > board?
> > > > > But going with the norrin device config should not be the
> > > > > issue here
> > > > > since the problem also exists when chainloading u-boot,
> > > > > right?
> > > > 
> > > > It could be a problem. The memory bank configuration is stored
> > > > in what's
> > > > called a BCT along with a bunch of other parameters that define
> > > > what the
> > > > memory controller needs to access the given memory chips. So if
> > > > you've
> > > > flashed a BCT that's for a board with only 2 GiB of memory you
> > > > would end
> > > > up with a system that can't address any more than that. It's
> > > > somewhat
> > > > surprising that memory accesses work at all with a BCT that's
> > > > for
> > > > different memory chips, but sometimes you can get lucky.
> > > > 
> > > > You may want to try reflashing with the right BCT. The simplest
> > > > would
> > > > probably be to duplicate the cbootimage configuration for
> > > > Norrin and
> > > > substitute the relevant bits by what you have from the
> > > > Chromebook flash
> > > > utilities. Looks like the BCT in really the only thing you can
> > > > replace
> > > > there. Make sure to replace it with the one that matches your
> > > > Chromebook
> > > > and it should give you the right memory bank configuration.
> > > 
> > > Is there a way to check if the correct bct for my 4GB model was
> > > used?
> > > What I tried was to edit the existing bct config file of norrin
> > > from here:
> > > https://github.com/NVIDIA/cbootimage-configs/blob/master/tegra124/nvidia/norrin/PM370_Hynix_2GB_H5TC4G63AFR_PBA_924MHz_01212014.bct.cfg
> > > by replacing all entries for each memory bank with the entries
> > > from here:
> > > https://github.com/coreboot/coreboot/blob/master/src/mainboard/google/nyan_big/bct/sdram-hynix-4GB-792.inc
> > > 
> > > I also added the SDRAM[0], SDRAM[1], SDRAM[2], SDRAM[3] each time
> > > in front of each line..
> > > This also booted for me, but also just leaves me with only 2Gb of
> > > my 4GB..
> > > 
> > > And since this problem also occurs, when chainloading u-boot,
> > > this must be a problem with u-boot, I think..
> > 
> > Not necessarily. U-Boot reads the memory size from EMC registers
> > and
> > those EMC registers are programmed based on the contents of the
> > BCT. So
> > if the BCT is wrong, then it's going to be wrong irrespective of
> > what
> > bootloader you use.
> > 
> > That said, I does indeed look as if U-Boot only supports 2 GiB of
> > memory
> > on 32-bit Tegra. arch/arm/mach-tegra/board.c contains the relevant
> > code.
> > If you look at dram_init() it uses query_sdram_size() to get the
> > size of
> > RAM. That will allow to convey up to 4 GiB of RAM, but then the
> > code in
> > board2.c (see dram_init_banksize()) only uses a single bank on 32-
> > bit
> > Tegra (CONFIG_PHYS_64BIT is only set for 64-bit Tegra).
> > 
> > According to the comments for dram_init_bank_size() we can't store
> > the
> > second bank because the .start field is 32 bits wide and therefore
> > can't
> > store the 0x100000000 start address for the bank.
> > 
> > It sounds to me like ARMV7_LPAE should perhaps select PHYS_64BIT to
> > make
> > sure all the appropriate variables can store addresses beyond the
> > 32-bit
> > boundary. It's one thing to not support > 2 GiB of memory in U-
> > Boot, but
> > it's an entirely different story if we can't even pass > GiB of
> > memory
> > to the kernel if they exist.
> > 
> > Stephen, do you remember if there was anything preventing us from
> > going
> > 64-bit for phys_addr_t and phys_size_t on LPAE enabled systems?
> 
> We probably just never tried since we never needed it and/or LPAE
> wasn't 
> available (in U-Boot SW) when it initially might have been useful. I 
> would not be surprised if some of the low-level Tegra code in U-Boot 
> conflates 64-bit support with SoC being >= T210, so things might not
> be 
> as simple as enabling CONFIG_PHYS_64BIT, but you never know...

Interesting discussion but I have to admit that I also only tested LPAE
once on the 4GB TK1 Chromebook running stock boot loader.

Unfortunately, the 4GB Apalis TK1 SKU got cancelled down the road so I
never got to actually really needing any of this in U-Boot. Sorry, not
being of much help here...

Cheers

Marcel