[U-Boot] i.MX51: FEC: Cache coherency problem?

[David Jander]

On Wed, 20 Jul 2011 10:56:07 +0200
Albert ARIBAUD <albert.u.boot at aribaud.net> wrote:

> Hi David,
> 
> Le 20/07/2011 08:29, David Jander a écrit :
> > On Tue, 19 Jul 2011 14:10:48 +0200
> > David Jander<david.jander at protonic.nl>  wrote:
> >
> >> On Tue, 19 Jul 2011 13:20:26 +0200
> >> Wolfgang Denk<wd at denx.de>  wrote:
> >>
> >>> Dear David Jander,
> >>>
> >>> In message<20110719131744.403a81e6 at archvile>  you wrote:
> >>>>
> >>>> Now I finally know what's wrong and am working on a proposed fix to make
> >>>> this one driver cache-aware.
> >>>
> >>> I would just like to point out that these efforts are highly
> >>> appreciated!
> >>
> >> Thanks.
> >> I'll try my best, but I am running out of time, and it is still not as it
> >> should. I still have trouble identifying the right place where receive
> >> buffer descriptors should be cache-invalidated. At one point, a magic
> >> flush_dcache_all() at a certain place in code made it work, but that
> >> can't be entirely correct, and if I replace it with only the necessary
> >> ranges, it doesn't work correctly anymore, so I guess this flush is also
> >> invalidating something I need elsewhere :-(
> >> I keep searching.... through the IMO fairly convoluted network code.
> >
> > Ok, that was not such a problem. I found few nice spots, where
> > cache-maintenance code should go IMO:
> >
> > In fec_rx_task_enable() cleaned RX BD's should be flushed.
> > In fec_tx_task_enable() dirty TX BD's should be flushed, as well as data
> > buffers.
> > In fec_send() Invalidate BD status fields before reading, also in the while
> > loop.
> > In fec_recv() Invalidate BD and data buffers just before every read.
> >
> > And of course align all buffers to cache-line size and pad around the end
> > of each buffer to cache-line size.
> >
> > Problem: It doesn't work (tftp transfer stops after a few packets)! :-(
> > I am giving up!
> >
> > Plan B: I'll now try to allocate all buffers in internal RAM (which is not
> > cached AFAIK).
> >
> > Any ideas?
> 
> Yes, one: I had issues with the Marvell Ethernet adapter, which has DMA 
> as well, not because of cache (it was not active at the time) but 
> because of instruction reordering done by the compiler when optimizing, 
> which resulted in out-of-order accesses to the descritpors and DMA 
> registered, so that the DMA start was written before the descriptors 
> were set up. The (proper and clean) solution was to introduce memory 
> barriers at strategic points of the driver to ensure that specific DMA 
> starts were done only after all writes to the descriptors (and possibly 
> buffers).

Hmmm. I know that issue, but AFAICS, the driver already uses readX() and
writeX() macros when accessing register and DMA memory. Those macros have
compiler barriers included.... and armv7 is still in-order execution ;-)
Thanks for the suggestion anyway :-)

Best regards,

-- 
David Jander
Protonic Holland.