[U-Boot] PCIE_MEM_BUS for Freescale SoC

[York Sun]

On 11/23/2015 09:07 AM, Scott Wood wrote:
> On Mon, 2015-11-23 at 09:00 -0800, York Sun wrote:
>>
>> On 11/23/2015 08:24 AM, Scott Wood wrote:
>>> On Sat, 2015-11-21 at 15:29 -0800, York Sun wrote:
>>>>
>>>> On 11/21/2015 02:55 PM, York Sun wrote:
>>>>> Roy,
>>>>>
>>>>> Do you remember the reason why we use different virtual memory address
>>>>> from pci
>>>>> bus address with 36-bit? For example
>>>>>
>>>>> include/configs/P1022DS.h-496-#define CONFIG_SYS_PCIE1_MEM_VIRT
>>>>> 0xc0000000
>>>>> include/configs/P1022DS.h-497-#ifdef CONFIG_PHYS_64BIT
>>>>> include/configs/P1022DS.h:498:#define CONFIG_SYS_PCIE1_MEM_BUS 
>>>>>  0xe0000000
>>>>> include/configs/P1022DS.h-499-#define CONFIG_SYS_PCIE1_MEM_PHYS
>>>>> 0xc40000000ull
>>>>> include/configs/P1022DS.h-500-#else
>>>>> include/configs/P1022DS.h:501:#define CONFIG_SYS_PCIE1_MEM_BUS 
>>>>>  0xc0000000
>>>>> include/configs/P1022DS.h-502-#define CONFIG_SYS_PCIE1_MEM_PHYS
>>>>> 0xc0000000
>>>>> include/configs/P1022DS.h-503-#endif
>>>>>
>>>>> As far as I can tell, the following is the mapping
>>>>>
>>>>> TLB: MEM_VIRT=>MEM_PHYS
>>>>> PCI: MEM_BUS=>MEM_PHYS
>>>>> LAW: MEM_PHYS=>pcie interface
>>>>>
>>>>> Being different for MEM_VIRT and MEM_BUS cause confusion. When I run
>>>>> "pci
>>>>> header" command to show the BARs, I expect I can use "md" to access
>>>>> the
>>>>> BAR
>>>>> address. That's not the case if MEM_BUS is different from MEM_VIRT.
>>>>>
>>>>> I forget why we did this for 36-bit addressing. The MEM_VIRT is the
>>>>> same
>>>>> as
>>>>> MEM_BUS for 32-bit addressing. And why do we use the same MEM_BUS
>>>>> address
>>>>> for
>>>>> all PCIe hose? I know they are not conflicting, but is it necessary?
>>>>
>>>> (I hope Becky and Kumar still follow this mailing list)
>>>>
>>>> I dug out an old commit 4c78d4a6c01621721b732418e1c6da684a56bbb1 by
>>>> Becky
>>>> Bruce.
>>>> She believed overlapping the bus address for PCI controllers leaves more
>>>> space.
>>>> That's true. But we haven't use more than 512MB in u-boot. If we do need
>>>> more
>>>> space, we can easily move things around if we have PHYS_64BIT. If no
>>>> objection,
>>>> I'd like to change this back.
>>>
>>> I object.  It's not about how much RAM is used in U-Boot; it's about how
>>> much
>>> memory the OS needs to bounce-buffer for DMA.  The addresses set up by U
>>> -Boot
>>> should match what's in the device tree.  Yes, on 85xx Linux reprograms the
>>> ATMU based on the device tree rather than trusting U-Boot, but that
>>> doesn't
>>> mean every OS does.
>>
>> Isn't this backward? I mean the device tree should match u-boot,
> 
> Yes, and I don't want to deal with the churn of these device trees changing.
> 
>>  or u-boot should fix up the device tree, right?
> 
> In theory yes, but then you really are affecting Linux's usage.
> 
>> I checked several device tree files in kernel. We are not using more than
>> 512MB
>> for each PCI. What's the benefit of using the same address 0xe0000000?
> 
> Reducing the amount of RAM whose address matches a PCI bus address and thus
> can't be used for PCI DMA.
> 

Another thought, Linux scans and enumerates PCIe regardless U-boot. The BARs are
setup according to device tree. So setting done u-boot is overwritten.

And, reusing 0xe0000000 for all PCI(e) controllers only applies to PHYS_64BIT in
current code. Isn't address space more crowed for 32-bit?

Regarding the ATMU, if OS relies on u-boot to setup correct ATMU, it should have
correct device tree as well.

Those being said, I can leave the code as is. It is not broken badly, but
confusing if one tries to access BAR space manually.

York