[U-Boot] i.MX51: FEC: Cache coherency problem?

[David Jander]

On Tue, 19 Jul 2011 07:36:32 -0700
"J. William Campbell" <jwilliamcampbell at comcast.net> wrote:

> On 7/19/2011 2:05 AM, Albert ARIBAUD wrote:
> > Le 19/07/2011 10:43, Aneesh V a écrit :
> >
> >>>> You would have to flush (before sending packets / starting external
> >>>> memory-to-device DMA) and invalidate (before reading received packets /
> >>>> after external device-to-memory DMA is done); using MMU and mapping
> >>>> cached/non-cached areas is IMO overkill, and will hurt CPU accesses to
> >>>> the xmit/receive buffers and descriptors.
> >>> So, you say actually what I did while exploring the problem would have
> >>> been a
> >>> correct way of solving this problem?
> >>>
> >>> Like this:
> >>>
> >>> 587 flush_cache(&fec->tbd_base[fec->tbd_index], 4);
> >> This is what is needed assuming the below is initiating a memory to
> >> peripheral DMA. Is your buffer only 4 bytes long?
> > Generally:
> >
> > - for sending data through a device that has its own, external, DMA
> > engine, you'll obviously need to flush the data buffer(s) but also any
> > DMA descriptors used by the engine, before you start the engine;
> >
> > - for rceiving, if you have to set up receive descriptors, you must
> > flush that before telling the device to enter receive mode (so that the
> > device reads the descriptors as you wrote them), and you should
> > invalidate the receive buffers at the latest when the device signals
> > that data has been received,
> Hi All,
> 
> >   or preferably long before (at the same time
> > you flushed the read descriptor, so that cache-related actions are
> > grouped in the same place in the code).
> I think this last statement is incorrect. You should invalidate the 
> cache for the receive buffers just before you intend to reference them. 
> If you do it right after receive mode is entered, subsequent access to 
> items NEAR the receive buffer may reload the cache with receive buffer 
> data before the dma is done, re-creating the problem you are trying to 
> avoid. Also, I don't know if ARM cache is write-back or write -thru, but 
> if it is write-back, the only way to avoid problems is to allocate the 
> receive buffers on cache line boundaries, so no "nearby" writes can 
> cause something in the DMA buffer to be corrupted. If all receive 
> buffers are allocated on cache-line boundaries (both start and end of 
> each buffer), you can invalidate the cache "early" under the assumption 
> that there will be no read accesses to the read buffers themselves until 
> after DMA is complete. IMHO it is better, even in this case., to 
> invalidate cache after dma is done but before referencing the read data.

Hey, thanks a lot for this bit of information! I really needed it! :-)
Indeed I hadn't thought about the situation of nearby memory accesses
allocating critical cache-lines...

Coming to think of it... would it really be that hard to enable the MMU in
u-boot and configure just two pools of flat, direct mapped linear memory to
malloc and dma_malloc from?
That would make fixing broken drivers trivial, and have other benefits (like
video memory coherence when caches are on)!
IMHO, changing the necessary malloc()'s for dma_malloc()'s is a lot less work
and easier to oversee than obscure cache-manipulation and alignment tricks all
over the place.... What do you think?

Best regards,

-- 
David Jander
Protonic Holland.