[U-Boot] [Patch v3 2/4] ARMv8/FSL_LSCH3: Add FSL_LSCH3 SoC

[Mark Rutland]

On Thu, May 29, 2014 at 06:37:36PM +0100, Rob Herring wrote:
> On Thu, May 29, 2014 at 10:19 AM, York Sun <yorksun at freescale.com> wrote:
> > On 05/29/2014 06:19 AM, Rob Herring wrote:
> >> On Wed, May 28, 2014 at 6:46 PM, York Sun <yorksun at freescale.com> wrote:
> >
> > <snip>
> >
> >>> +static void set_pgtable_section(u64 *page_table, u64 index, u64 section,
> >>> +                               u8 memory_type)
> >>> +{
> >>> +       u64 value;
> >>> +
> >>> +       value = section | PMD_TYPE_SECT | PMD_SECT_AF;
> >>> +       value |= PMD_ATTRINDX(memory_type);
> >>> +       page_table[index] = value;
> >>> +}
> >>
> >> This function looks like it should be common.
> >>
> >
> > There is a common version in arch/arm/cpu/armv8/cache_v8.c. This version has
> > more flexibility. I am not sure which one will be used as common.
> 
> Then add the flexibility to the common one.
> 
> 
> >>> +       el = current_el();
> >>
> >> We really can't have u-boot running at random ELs in v8 for different
> >> platforms. It's a mess on v7. You should never be at EL3. u-boot could
> >> be defined to run at EL1, but then you need to be able to go back to
> >> EL2 to boot the kernel. So really u-boot should always run at EL2
> >> unless you are running in a VM, but that would be a different
> >> platform.
> >
> > I have to run u-boot at EL3. Otherwise I can't access Dickens. I vaguely
> > remember other reasons. I will have to dig it out if needed.
> 
> You may start in EL3, but then the early init code should drop to EL2.
> If you need EL3 later on for something PSCI does not address, then you
> are probably doing things wrong.
> 
> >>> + * flush_l3_cache
> >>> + * Dickens L3 cache can be flushed by transitioning from FAM to SFONLY power
> >>> + * state, by writing to HP-F P-state request register.
> >>
> >> Other SOCs will have Dickens. Are these registers FSL specific? If
> >> not, this should be common.
> >
> > I don't think they are FSL specific. But I haven't found a proper place to host
> > it. Can you share what other SoCs have Dickens? If they are not supported yet,
> > we can keep the code here until we are clear then move it out.
> 
> It is safe to say most if not all SOCs based on A53 and/or A57 will
> also be based on Dickens aka CCN-504.
> 
> >> Also, I believe the proper way to flush Dickens is using the
> >> architected cache flushing method where you walk the levels out to
> >> level 3.
> >
> > False. L3 cache gets flushed with instruction DCCIVAC. So if we flush the cache
> > by range, it works OK. But it doesn't work with DCISW or DCCISW. If we flush
> > cache by walking the levels, it doesn't work. We can only walk level 1 and level 2.
> 
> So the EL3 boot code should do any one-time invalidate all operations
> and u-boot in EL2 should only use range operations. There are
> limitations in the by way operations such as they are not SMP safe. If
> the by way operations are EL3 only, then that's probably a sign you
> are not doing things as intended. Or it could be an oversight and we
> need to figure out a common way to handle this across SOCs.

This is the only sane way of doing things.

The set/way operations can only be used for two things:

* One-off IMPLEMENTATION DEFINED cache initialisation (for power on).

* Emptying of a given CPU's architected cache levels (which is only
  necessary for power off).

In neither of these cases are the operations used to provide any sort of
visibility guarantee (i.e. they do not make uncached data visible to
CPUs, or cached data visible to other observers). In both of these cases
if you have PSCI this is hidden by the implementation.

It should be entirely possible to flush/clean the data/area you care
about by VA.

Given that this L3 cache and (IIRC the APM L3) can handle VA operations,
I would rather that U-boot and general purposes OSs assume that there is
such a cache present always, which is outside of our control and already
enabled. While this necessitates some cache flushing before turning the
MMU and caches on, it will otherwise be transparent.

This is what we exepect in Linux since c218bca74eea (arm64: Relax the
kernel cache requirements for boot). It makes a single image possible in
the presnce of system caches, and makes things consistent for KVM,
kexec, and chain-loading U-Boot.

Cheers,
Mark.