[U-Boot] [Resend RFC PATCH 1/2] armv8: Fix dcache disable function

[Stephen Warren]

On 10/24/2016 04:59 AM, Mark Rutland wrote:
> On Fri, Oct 21, 2016 at 07:31:52PM +0000, york sun wrote:
>> On 10/20/2016 01:34 PM, Stephen Warren wrote:
>>> On 10/19/2016 11:06 PM, york sun wrote:
>>>> I understand the data in dirty cache is not lost when the dcache is
>>>> disabled. It is just not accessible. In my case, after flushing L1/L2 by
>>>> way/set, the data is probably in L3 cache. Without flushing L3, I have
>>>> stalled data (including the stack) in main memory.
>>>
>>> I assume "stale" not "stalled".
>>>
>>> Yes, I agree: If you have only flushed L1/L2 by set/way (or attempted to
>>> do everything, but the L3 cache doesn't implement by set/way
>>> operations), then L3 can indeed still contain dirty data, and hence main
>>> memory can be stale.
>>>
>>>> My previous test was trying to prove I can skip flushing L3 if I flush
>>>> the necessary VA.
>>>
>>> It depends whether your L3 cache is before/after the Level of
>>> Unification and/or the Level of Coherency for your HW, and whether you
>>> flush by VA to the PoU or PoC.
>>>
>>> Looking at your "[PATCH 2/2] armv8: Fix flush_dcache_all function", it
>>> uses __asm_flush_dcache_range() which cleans and invalidates to the
>>> point of coherency (it invokes the dc civac instruction).
>>>
>>>  > Now I when recall it carefully, I think I made a
>>>> mistake by flushing by VA _after_ flushing the cache by way/set. I might
>>>> have a positive result if I flushed the cache by VA first. I will repeat
>>>> this test when I get back to prove this theory.
>>>
>>> I'll assume you L3 cache is before the Point of Coherency. If so, then
>>> performing a clean/invalidate by VA to the PoC will affect all of L1,
>>> L2, and L3 caches. As such, you need only perform the c/i by VA, and you
>>> can entirely skip the c/i by set/way; I believe it would be redundant,
>>> assuming that the by VA operations cover all relevant VAs.
>>
>> I believe the PoC and PoU is before L3 for me.
>
> If you are using CCN, then the PoC is beyond the L3.
>
> Were the PoC before the L3, there would be no requirement to perform
> maintenance on the L3. The PoC is the point at which *all* accesses
> (cacheable or otherwise) see the same data.
>
> Per the ARM ARM (for ARMv8-A), maintenance to the PoC *must* affect
> system caches (including CCN).
>
>> I can clean/invalidate by VA, it may not cover all the cache lines. So
>> by set/way is still needed.
>
> The problem is figuring out which VA ranges require maintenance.
>
> Do we not have an idea of the set of memory banks present in the SoC?
> Like the memblock array in Linux?

There are two data structures in ARM U-Boot that describe memory layout:

1) A list of RAM memory regions. U-Boot uses these to know where to 
relocate itself to (it relocates itself to the top of RAM at boot), and 
to fill in the /memory node when booting an OS using DT (and other 
equivalent mechanisms when not using DT.)

2) For AArch64 specifically, there's a memory map table that defines all 
RAM and MMIO regions, and the translation table attributes required for 
those regions.

Judging by your comments later in the original message, it sounds like 
it'd be fine to read from these structures during any dcache clean 
routine provided the table has already been cleaned. That makes using 
the tables much more tractable:-)