[U-Boot] [REGRESSION] tegra124: nyan-big: LPAE not working

[Stephen Warren]

On 2/11/19 3:48 AM, Thierry Reding wrote:
> On Mon, Feb 11, 2019 at 10:04:37AM +0000, Tristan Bastian wrote:
>>
>>
>>
>> Thierry Reding – Mon, 11. February 2019 10:38
>>> On Mon, Feb 11, 2019 at 09:20:33AM +0000, Tristan Bastian wrote:
>>>>
>>>> Thierry Reding – Mon, 11. February 2019 9:52
>>>>> On Sun, Feb 10, 2019 at 08:53:11PM +0100, Tristan Bastian wrote:
>>>>>> Thierry, do you have any news on this?
>>>>>>
>>>>>> I don't think, that google is going to push an updated version of
>>> coreboot
>>>>>> to each nyan-big device..
>>>>>> So reverting this patch at least for the nyan devices would be the best
>>> I
>>>>>> think..
>>>>>
>>>>> Yes, I agree. I had a brief chat with Rob Herring about this and he too
>>>>> agrees that we should revert it for now. I'll make it a manual revert
>>>>> and add a comment to the device tree node that will hopefully avoid any
>>>>> future janitorial "cleanups" of this sort.
>>>>
>>>> great news! :)
>>>>
>>>>>> BTW: I'm now running u-boot natively and it seems like u-boot always has
>>> a
>>>>>> problem with the memory..
>>>>>> If u-boot is used chainloaded to coreboot it is only getting 2GB of
>>> memory
>>>>>> and running u-boot natively also just gives me 2GB..
>>>>>> I've tested that with a kernel with "ARM: tegra: Fix unit_address_vs_reg
>>> DTC
>>>>>> warnings for /memory" reverted and also on a kernel, before this patch
>>> was
>>>>>> applied..
>>>>>
>>>>> It's possible that U-Boot doesn't support LPAE and therefore may not be
>>>>> able to use more than the 2 GiB of memory.
>>>>
>>>> So I at least enabled LPAE in u-boot with "CONFIG_ARMV7_LPAE=y" and
>>>> this was for some reason also needed to get some output on the
>>>> display..
>>>> I'm not sure why LPAE needs to activated in u-boot for display output
>>>> on the nyan-big..
>>>> Without LPAE enabled u-boot was still working, and booted linux, but
>>>> u-boot didn't display anything on screen, linux then did..
>>>
>>> Yeah, that's surprising. Perhaps without LPAE U-Boot thinks there's not
>>> enough memory for the framebuffer? There should be plenty, so maybe
>>> there is something else going on here.
>>>
>>>>> However, U-Boot should be
>>>>> able to tell the kernel exactly how much memory the system has and pass
>>>>> that on via device tree. That still does work, right?
>>>>
>>>> It seems like this is not working..
>>>> And this seems to be the case with both, u-boot chainloaded and
>>>> running u-boot natively..
>>>>
>>>> I've used these scripts for flashing:
>>>> github.com/NVIDIA/tegra-uboot-flasher-scripts
>>>> And used the norrin device since it seems like it is the nyan-big dev
>>>> board?
>>>> But going with the norrin device config should not be the issue here
>>>> since the problem also exists when chainloading u-boot, right?
>>>
>>> It could be a problem. The memory bank configuration is stored in what's
>>> called a BCT along with a bunch of other parameters that define what the
>>> memory controller needs to access the given memory chips. So if you've
>>> flashed a BCT that's for a board with only 2 GiB of memory you would end
>>> up with a system that can't address any more than that. It's somewhat
>>> surprising that memory accesses work at all with a BCT that's for
>>> different memory chips, but sometimes you can get lucky.
>>>
>>> You may want to try reflashing with the right BCT. The simplest would
>>> probably be to duplicate the cbootimage configuration for Norrin and
>>> substitute the relevant bits by what you have from the Chromebook flash
>>> utilities. Looks like the BCT in really the only thing you can replace
>>> there. Make sure to replace it with the one that matches your Chromebook
>>> and it should give you the right memory bank configuration.
>>
>> Is there a way to check if the correct bct for my 4GB model was used?
>> What I tried was to edit the existing bct config file of norrin from here:
>> https://github.com/NVIDIA/cbootimage-configs/blob/master/tegra124/nvidia/norrin/PM370_Hynix_2GB_H5TC4G63AFR_PBA_924MHz_01212014.bct.cfg
>> by replacing all entries for each memory bank with the entries from here:
>> https://github.com/coreboot/coreboot/blob/master/src/mainboard/google/nyan_big/bct/sdram-hynix-4GB-792.inc
>>
>> I also added the SDRAM[0], SDRAM[1], SDRAM[2], SDRAM[3] each time in front of each line..
>> This also booted for me, but also just leaves me with only 2Gb of my 4GB..
>>
>> And since this problem also occurs, when chainloading u-boot, this must be a problem with u-boot, I think..
> 
> Not necessarily. U-Boot reads the memory size from EMC registers and
> those EMC registers are programmed based on the contents of the BCT. So
> if the BCT is wrong, then it's going to be wrong irrespective of what
> bootloader you use.
> 
> That said, I does indeed look as if U-Boot only supports 2 GiB of memory
> on 32-bit Tegra. arch/arm/mach-tegra/board.c contains the relevant code.
> If you look at dram_init() it uses query_sdram_size() to get the size of
> RAM. That will allow to convey up to 4 GiB of RAM, but then the code in
> board2.c (see dram_init_banksize()) only uses a single bank on 32-bit
> Tegra (CONFIG_PHYS_64BIT is only set for 64-bit Tegra).
> 
> According to the comments for dram_init_bank_size() we can't store the
> second bank because the .start field is 32 bits wide and therefore can't
> store the 0x100000000 start address for the bank.
> 
> It sounds to me like ARMV7_LPAE should perhaps select PHYS_64BIT to make
> sure all the appropriate variables can store addresses beyond the 32-bit
> boundary. It's one thing to not support > 2 GiB of memory in U-Boot, but
> it's an entirely different story if we can't even pass > GiB of memory
> to the kernel if they exist.
> 
> Stephen, do you remember if there was anything preventing us from going
> 64-bit for phys_addr_t and phys_size_t on LPAE enabled systems?

We probably just never tried since we never needed it and/or LPAE wasn't 
available (in U-Boot SW) when it initially might have been useful. I 
would not be surprised if some of the low-level Tegra code in U-Boot 
conflates 64-bit support with SoC being >= T210, so things might not be 
as simple as enabling CONFIG_PHYS_64BIT, but you never know...