[U-Boot] [PATCH] ARM: vexpress: Remove virt and nonsec support for TC2

[Jon Medhurst (Tixy)]

On Wed, 2016-06-22 at 15:53 +0100, Andre Przywara wrote:
> Hi,
> 
> On 22/06/16 15:34, Jon Medhurst (Tixy) wrote:
> > When CPU's come out of reset they are in secure state supervisor mode,
> > so this is the state Linux kernel entry point is called in when it
> > brings up secondary CPU cores or the primary CPU restarts after power
> > management has sent it through an off/on transition.
> > 
> > As U-Boot starts the kernel in hypervisor mode and the kernel expects
> > and checks that CPUs start in the same state as initial boot this
> > results in a dead system. Specifically, it crashes early in boot when
> > the primary CPU runs the MCPM test [1] and even if power management
> > features are disabled it will still refuse to bring up any secondary
> > CPUs.
> > 
> > Fix this problem by removing U-Boot support for virt and nonsec
> > support on TC2.
> 
> So this disables any kind of virtualization support for TC2, which is a
> bit unfortunate.
> If I get this (and Sudeep's explanations) correctly, this new behaviour
> comes from the per-CPU-mailboxes setting in SCC 0x700, which is a
> setting in boot.txt on the uSD card.

Right, so I got the current U-Boot to boot Linux OK with all CPU's
coming up in hyp mode by:

- Clearing bits 12 and 13 in SCC 0x700 to disable per-cpu mailboxes and
  keep the secondary CPU powered up.

- Removing kernel configs
    CONFIG_ARCH_VEXPRESS_TC2_PM
    CONFIG_MCPM

I believe this works because the SCC change means that all CPU will be
running and waiting in the bootmonitor holding pen, from which they are
released when U-Boot calls smp_set_core_boot_addr. These secondary CPUs
then execute U-Boot code to switch to hypervisor mode, then enter a new
holding pen. (Running in RAM that can't get overwritten by Linux I hope!
)

Then when kernel boots and releases cores from holding pen they are in
hyp mode, like the main CPU. As we've also disabled power management in
the kernel, CPU's aren't ever powered down again and so stay in hyp
mode.

Actually, I just tried hotplugging a CPU which obviously the kernel
didn't like, so there is probably other kernel configs that need
tweaking to make things work.

> I wonder if we could make this virt/secure support a runtime decision
> then based on that register?
> So that a user can select whether she wants virtualisation or power
> management by changing the boot.txt setting and U-Boot transparently
> adapts to it, entering the kernel in either secure SVC or HYP.
>
> Does that make sense?
> I can look into a fix if this approach is fine.

Anything that gets U-Boot in it's default config working with vexpress
firmware and Linux kernel in their default config would be good
start :-)

I'm not sure that having an automatic selection in U-Boot is worth it,
given that anyone booting in hyp mode also needs to modify the firmware
and kernel configs to get things working. But I wouldn't object to such
an automatic selection either. After all, it's not like anyone really
uses or cares about this stuff, and if they are, they probably have
there own customised setup rather than using upstream defaults.

-- 
Tixy