[U-Boot] [REGRESSION] tegra124: nyan-big: LPAE not working

Mon Mar 4 03:06:00 UTC 2019

On 2/11/2019 5:13 PM, Stephen Warren wrote:
> On 2/11/19 3:48 AM, Thierry Reding wrote:
>> On Mon, Feb 11, 2019 at 10:04:37AM +0000, Tristan Bastian wrote:
>>>
>>>
>>>
>>> Thierry Reding – Mon, 11. February 2019 10:38
>>>> On Mon, Feb 11, 2019 at 09:20:33AM +0000, Tristan Bastian wrote:
>>>>>
>>>>> Thierry Reding – Mon, 11. February 2019 9:52
>>>>>> On Sun, Feb 10, 2019 at 08:53:11PM +0100, Tristan Bastian wrote:
>>>>>>> Thierry, do you have any news on this?
>>>>>>>
>>>>>>> I don't think, that google is going to push an updated version of
>>>> coreboot
>>>>>>> to each nyan-big device..
>>>>>>> So reverting this patch at least for the nyan devices would be 
>>>>>>> the best
>>>> I
>>>>>>> think..
>>>>>>
>>>>>> Yes, I agree. I had a brief chat with Rob Herring about this and 
>>>>>> he too
>>>>>> agrees that we should revert it for now. I'll make it a manual revert
>>>>>> and add a comment to the device tree node that will hopefully 
>>>>>> avoid any
>>>>>> future janitorial "cleanups" of this sort.
>>>>>
>>>>> great news! :)
>>>>>
>>>>>>> BTW: I'm now running u-boot natively and it seems like u-boot 
>>>>>>> always has
>>>> a
>>>>>>> problem with the memory..
>>>>>>> If u-boot is used chainloaded to coreboot it is only getting 2GB of
>>>> memory
>>>>>>> and running u-boot natively also just gives me 2GB..
>>>>>>> I've tested that with a kernel with "ARM: tegra: Fix 
>>>>>>> unit_address_vs_reg
>>>> DTC
>>>>>>> warnings for /memory" reverted and also on a kernel, before this 
>>>>>>> patch
>>>> was
>>>>>>> applied..
>>>>>>
>>>>>> It's possible that U-Boot doesn't support LPAE and therefore may 
>>>>>> not be
>>>>>> able to use more than the 2 GiB of memory.
>>>>>
>>>>> So I at least enabled LPAE in u-boot with "CONFIG_ARMV7_LPAE=y" and
>>>>> this was for some reason also needed to get some output on the
>>>>> display..
>>>>> I'm not sure why LPAE needs to activated in u-boot for display output
>>>>> on the nyan-big..
>>>>> Without LPAE enabled u-boot was still working, and booted linux, but
>>>>> u-boot didn't display anything on screen, linux then did..
>>>>
>>>> Yeah, that's surprising. Perhaps without LPAE U-Boot thinks there's not
>>>> enough memory for the framebuffer? There should be plenty, so maybe
>>>> there is something else going on here.
>>>>
>>>>>> However, U-Boot should be
>>>>>> able to tell the kernel exactly how much memory the system has and 
>>>>>> pass
>>>>>> that on via device tree. That still does work, right?
>>>>>
>>>>> It seems like this is not working..
>>>>> And this seems to be the case with both, u-boot chainloaded and
>>>>> running u-boot natively..
>>>>>
>>>>> I've used these scripts for flashing:
>>>>> github.com/NVIDIA/tegra-uboot-flasher-scripts
>>>>> And used the norrin device since it seems like it is the nyan-big dev
>>>>> board?
>>>>> But going with the norrin device config should not be the issue here
>>>>> since the problem also exists when chainloading u-boot, right?
>>>>
>>>> It could be a problem. The memory bank configuration is stored in 
>>>> what's
>>>> called a BCT along with a bunch of other parameters that define what 
>>>> the
>>>> memory controller needs to access the given memory chips. So if you've
>>>> flashed a BCT that's for a board with only 2 GiB of memory you would 
>>>> end
>>>> up with a system that can't address any more than that. It's somewhat
>>>> surprising that memory accesses work at all with a BCT that's for
>>>> different memory chips, but sometimes you can get lucky.
>>>>
>>>> You may want to try reflashing with the right BCT. The simplest would
>>>> probably be to duplicate the cbootimage configuration for Norrin and
>>>> substitute the relevant bits by what you have from the Chromebook flash
>>>> utilities. Looks like the BCT in really the only thing you can replace
>>>> there. Make sure to replace it with the one that matches your 
>>>> Chromebook
>>>> and it should give you the right memory bank configuration.
>>>
>>> Is there a way to check if the correct bct for my 4GB model was used?
>>> What I tried was to edit the existing bct config file of norrin from 
>>> here:
>>> https://github.com/NVIDIA/cbootimage-configs/blob/master/tegra124/nvidia/norrin/PM370_Hynix_2GB_H5TC4G63AFR_PBA_924MHz_01212014.bct.cfg 
>>>
>>> by replacing all entries for each memory bank with the entries from 
>>> here:
>>> https://github.com/coreboot/coreboot/blob/master/src/mainboard/google/nyan_big/bct/sdram-hynix-4GB-792.inc 
>>>
>>>
>>> I also added the SDRAM[0], SDRAM[1], SDRAM[2], SDRAM[3] each time in 
>>> front of each line..
>>> This also booted for me, but also just leaves me with only 2Gb of my 
>>> 4GB..
>>>
>>> And since this problem also occurs, when chainloading u-boot, this 
>>> must be a problem with u-boot, I think..
>>
>> Not necessarily. U-Boot reads the memory size from EMC registers and
>> those EMC registers are programmed based on the contents of the BCT. So
>> if the BCT is wrong, then it's going to be wrong irrespective of what
>> bootloader you use.
>>
>> That said, I does indeed look as if U-Boot only supports 2 GiB of memory
>> on 32-bit Tegra. arch/arm/mach-tegra/board.c contains the relevant code.
>> If you look at dram_init() it uses query_sdram_size() to get the size of
>> RAM. That will allow to convey up to 4 GiB of RAM, but then the code in
>> board2.c (see dram_init_banksize()) only uses a single bank on 32-bit
>> Tegra (CONFIG_PHYS_64BIT is only set for 64-bit Tegra).
>>
>> According to the comments for dram_init_bank_size() we can't store the
>> second bank because the .start field is 32 bits wide and therefore can't
>> store the 0x100000000 start address for the bank.
>>
>> It sounds to me like ARMV7_LPAE should perhaps select PHYS_64BIT to make
>> sure all the appropriate variables can store addresses beyond the 32-bit
>> boundary. It's one thing to not support > 2 GiB of memory in U-Boot, but
>> it's an entirely different story if we can't even pass > GiB of memory
>> to the kernel if they exist.
>>
>> Stephen, do you remember if there was anything preventing us from going
>> 64-bit for phys_addr_t and phys_size_t on LPAE enabled systems?
> 
> We probably just never tried since we never needed it and/or LPAE wasn't 
> available (in U-Boot SW) when it initially might have been useful. I 
> would not be surprised if some of the low-level Tegra code in U-Boot 
> conflates 64-bit support with SoC being >= T210, so things might not be 
> as simple as enabling CONFIG_PHYS_64BIT, but you never know...

Good Evening,

I have finally built mainline u-boot for the rk3328-roc-cc board, and 
have actually run into this issue as well.
With the downstream 2017 u-boot, the board correctly reports 4GB to the 
kernel.
With mainline, no matter what I try, the kernel only ever sees 2GB.
I don't see any significant differences between the spin-up code from 
the downstream u-boot to the mainline u-boot.

I am starting to wonder if this is a u-boot regression, but since most 
embedded SBCs don't use 4GB of ram it does not crop up often.
Of course I may just be missing something, since this is my first foray 
into u-boot.