[U-Boot] armv7 DMA and cache mangement functions

[Markus Niebel]

Am 27.08.2015 um 19:41 schrieb Mark Rutland:
> On Mon, Aug 24, 2015 at 08:54:17AM +0100, Markus Niebel wrote:
>> Hello,
>>
>> I'm not an expert in the low level details of this area. So please sorry if there are
>> wrong assumptions in this post post.
>>
>> Hardware: i.MX6 Solo (TQMa6 on custom Mainboard)
>> U-Boot: 2014.10
>> gcc: 4.8.3 
>>
>> We see an error using TFTP on i.MX6 that seems to triggered, if the code / data size goes
>> over a limit. Code changes have nothing to do with network stack, network drivers, 
>> memory mangement. TFTP will completely unusable: device sees frequently erroneous packages 
>> with different of wierd errors. If code stays below this size all works fine.
>>
>> Up to now we checked a lot of things. The following brought us to the assumption, that this
>> could be cache related:
>>
>> dynamically disable data cache before doing TFTP: 	TFTP works well again
>> running with disabled L2 cache (data cache enabled):	TFTP works well again
>>
>> Looking at the code in drivers/net/fec_mxc.c, function fec_recv we see a call to
>> invalidate_dcache_range before accessing the received ethernet data. When looking at
>> the code for invalidate_dcache_range in arch/arm/cpu/armv7/cache_v7.c an comparing
>> how the things done in linux and barebox we noticed that the order of L2 chache / data cache
>> invalidation is just swapped there. Applying this to the receive code for fec_mxc,
>> TFTP will work again.
>>
>> Question: is the order of cache invalidation important?
> 
> The order is important.
> 
> Consider the case where both the external and architected caches contain
> stale (but clean) cache lines for the region you care about.
> 
> If you invalidate the architected caches before the external L2, the
> architected caches may speculatively fetch (stale) data from the L2
> before the L2 is cleaned, and so in the end you may still see stale
> data in the architected caches.
> 
> If you invalidate the L2 first, the architected caches could
> speculatively fetch from the L2 (stale) or memory (new) while this is in
> progress, but they will then be invalidated, and from then on can only
> fetch the new data.
> 
> That assumes that both levels were clean to begin with. If they are not,
> then additional maintenance is required. It's also conceivable that
> caches could be implemented such that the above is insufficient, YMMV.
> 

Thank you for the clarification. Given your statement the code in question:

arch/arm/cpu/armv7/cache_v7.c

/*
 * Invalidates range in all levels of D-cache/unified cache used:
 * Affects the range [start, stop - 1]
 */
void invalidate_dcache_range(unsigned long start, unsigned long stop)
{
	v7_dcache_maint_range(start, stop, ARMV7_DCACHE_INVAL_RANGE);

	v7_outer_cache_inval_range(start, stop);
}

is wrong - and also the flush_dcache_all logic.

Given http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.faqs/ka14802.html
the chache invalidate handler needs to be fixed in a way, that also the platforms providing
atomic maintenance are taken in account.

Correct me if I'm wrong.

Regards 

Markus

> Thanks,
> Mark.
>