Exception handling in HYP mode on ARMv7-A

[Jim Posen]

I've run into an issue with exception handling on the Raspberry Pi 3 in
32-bit
mode (config rpi_3_32b) and the ODroid XU4 (config odroid-xu3). When
testing
the "exception undefined" command, instead of printing out register info
and
resetting the CPU like it's supposed to, U-Boot just hangs.

I noticed that on both of these boards, U-Boot runs in hypervisor mode
(HYP)
because both the Pi's Cortex-A53 and the ODroid's Cortex-A7 support
virtualization extensions. This causes several issues with the ARMv7
exception
handling code, which assumes that exceptions are taken in EL1.

(All file names and line numbers referenced below are respect to the
v2021.10
release)

The first issue running in HYP mode is that the vector table base address
is
not set correctly. In arch/arm/cpu/armv7/start.S L78 and
arch/arm/lib/relocate.S L45, the VBAR register is loaded with the vector
table
base address. However, exceptions taken to EL2 use the address in the HVBAR
register, not VBAR.

The next issue is that the get_bad_stack macro (arch/arm/lib/vectors.S)
uses
"movs pc, lr" to return from the exception into supervisor mode, but this
is an
illegal instruction in HYP mode as it was replaced by "eret". Also, the lr
register does not hold the exception return address (which is used to
determine
the PC before the exception), since exceptions taken to EL2 store the
return
address in the new ELR_hyp register instead. I'm also not sure in reading
this
code why there's a switch to supervisor mode...

So to handle an exception to EL2 properly, an instruction like "msr
ELR_hyp,
lr" needs to be assembled, and this requires "-march=armv7ve" in the CC
flags,
not "-march=armv7-a".

I found it quite surprising that this didn't work correctly. I'd be happy
to
submit a patch for it, and could use some guidance. To change the CC flag
to
the correct architecture, it seems like introducing a CONFIG_CPU_V7VE may
be
the right approach. Then I'd say that for builds with this config, there
should
be two separate exception vector tables, one for EL1, which would be left
as it
currently is, and a new one for EL2, with this table loaded into HVBAR.

I'd appreciate your thoughts, thanks.