[U-Boot] Exception Level switching seems broken on RK3399

Thu May 24 11:07:51 UTC 2018

Vincente,

> On 19 May 2018, at 16:58, Vicente Bergas <vicencb at gmail.com> wrote:
> 
> Hello,
> I am writing this from a standalone Sapphire board [1],
> that is, without the Excavator base board.
> The CPU is the Rockchip RK3399, which implements ARMv8.0-A.
> 
> Currently the boot process is:
> 1.- Boot ROM
> 2.- SPL, provided as closed source binary blob [2]

SPL-support is available in mainline U-Boot.  We developed this for
the RK3399-Q7 and it has been successfully used on other RK3399
boards (e.g. I know that some Firefly-users are using this).

> 3.- ATF, closed source binary blob [3]
>     (not using the one from [2] because of stability issues)

Why use the closed-source blob, if the RK3399 is supported in the ATF
mainline and an ATF can be compiled from source?

> 4.- Mainline u-boot, master branch
> 5.- Mainline linux, master branch
> 
> I would like to use an opensource boot process.
> As a first approach I try to completely remove the ATF and
> replace the SPL with the one from u-boot.
> The modified boot process looks like:
> 1.- Boot ROM
> 2.- SPL, from mainline u-boot, master branch
> 3.- Mainline u-boot, master branch
> 4.- Mainline linux, master branch
> But it is not working.
> 
> The replaced SPL works fine and loads u-boot.
> U-boot also works fine, loads linux and jumps into it.

Yes, we’ve done some work to enable us to run U-Boot in EL3 on
the RK3399 (as we use it for programming the secure e-fuses on
the RK3399-Q7 in our factory programming setup).

> But then, linux never gets executed.
> 
> I have traced the issue to: arch/arm/include/asm/macro.h
> 202: msr  spsr_el3, \tmp
> 203: msr  elr_el3, \ep
> 204: eret // This is the last instruction executed
> 
> For testing, I have also set CONFIG_ARMV8_SWITCH_TO_EL1 and
> checked that switch_to_el1 from arch/arm/lib/bootm.c is not reached.
> 
> At this point I have a few questions:
> 1.- Is my first approach feasible? That is, is it possible to boot
>     this CPU without ATF?

It is feasible (i.e.: requires implementation work) but not recommended:
Linux will use PSCI to bring up the secondary CPUs.  We have run Linux
(limited to a single CPU) in EL3 on this CPU during our own board bringup,
but I would strongly discourage this as it will entail unnecessary effort.

> 2.- If so, what should I do to make it work? Probably it is just
>     a configuration issue, but I do not know what to check. [4]
> 3.- Else, why do I need ATF?

ATF is the secure monitor on ARMv8 and provides services such as PSCI
to start up secondary CPUs.  It will usually also be part of power-management
on most SoCs (after all: power configuration needs to be done in the secure
envelope).

Regards,
Philipp.

> 
> Regards,
>  Vicenç.
> 
> [1] http://opensource.rock-chips.com/wiki_Excavator_sapphire_board
> [2] https://github.com/rockchip-linux/rkbin/tree/29mirror
> [3] https://raw.githubusercontent.com/nightseas/armbian-build/master/packages/blobs/rockchip64/trust.img
> [4]
> CONFIG_ARM=y
> CONFIG_ARCH_ROCKCHIP=y
> CONFIG_SYS_TEXT_BASE=0x00200000
> CONFIG_SPL_LIBCOMMON_SUPPORT=y
> CONFIG_SPL_LIBGENERIC_SUPPORT=y
> CONFIG_SYS_MALLOC_F_LEN=0x4000
> CONFIG_ROCKCHIP_RK3399=y
> CONFIG_ROCKCHIP_SPL_RESERVE_IRAM=0x4000
> CONFIG_SPL_STACK_R_ADDR=0x80000
> # CONFIG_PSCI_RESET is not set
> CONFIG_CMD_HDMIDETECT=y
> CONFIG_DEFAULT_DEVICE_TREE="rk3399-evb"
> CONFIG_DEBUG_UART=y
> CONFIG_BOOTDELAY=1
> # CONFIG_DISPLAY_CPUINFO is not set
> # CONFIG_DISPLAY_BOARDINFO is not set
> CONFIG_SPL_STACK_R=y
> CONFIG_SPL_STACK_R_MALLOC_SIMPLE_LEN=0x4000
> # CONFIG_CMD_IMI is not set
> # CONFIG_CMD_XIMG is not set
> # CONFIG_CMD_SAVEENV is not set
> # CONFIG_CMD_FLASH is not set
> # CONFIG_CMD_LOADB is not set
> # CONFIG_CMD_LOADS is not set
> CONFIG_CMD_MMC=y
> CONFIG_CMD_USB=y
> # CONFIG_CMD_SETEXPR is not set
> # CONFIG_CMD_NET is not set
> # CONFIG_DOS_PARTITION is not set
> # CONFIG_ISO_PARTITION is not set
> CONFIG_SPL_OF_CONTROL=y
> CONFIG_REGMAP=y
> CONFIG_SPL_REGMAP=y
> CONFIG_SYSCON=y
> CONFIG_SPL_SYSCON=y
> CONFIG_CLK=y
> CONFIG_SPL_CLK=y
> CONFIG_ROCKCHIP_GPIO=y
> CONFIG_SYS_I2C_ROCKCHIP=y
> # CONFIG_MMC_WRITE is not set
> # CONFIG_MMC_HW_PARTITIONING is not set
> CONFIG_MMC_DW=y
> CONFIG_MMC_SDHCI=y
> CONFIG_MMC_SDHCI_ROCKCHIP=y
> CONFIG_DM_ETH=y
> # CONFIG_NETDEVICES is not set
> CONFIG_PINCTRL=y
> CONFIG_SPL_PINCTRL=y
> CONFIG_PINCTRL_ROCKCHIP_RK3399=y
> CONFIG_DM_PMIC=y
> CONFIG_PMIC_RK8XX=y
> CONFIG_REGULATOR_PWM=y
> CONFIG_DM_REGULATOR_FIXED=y
> CONFIG_REGULATOR_RK8XX=y
> CONFIG_PWM_ROCKCHIP=y
> CONFIG_RAM=y
> CONFIG_SPL_RAM=y
> CONFIG_BAUDRATE=1500000
> CONFIG_DEBUG_UART_BASE=0xFF1A0000
> CONFIG_DEBUG_UART_CLOCK=24000000
> CONFIG_DEBUG_UART_SHIFT=2
> CONFIG_SYSRESET=y
> CONFIG_USB=y
> CONFIG_USB_XHCI_HCD=y
> CONFIG_USB_XHCI_DWC3=y
> CONFIG_USB_EHCI_HCD=y
> CONFIG_USB_EHCI_GENERIC=y
> CONFIG_USB_STORAGE=y
> CONFIG_USB_KEYBOARD=y
> CONFIG_USB_HOST_ETHER=y
> CONFIG_USB_ETHER_ASIX=y
> CONFIG_USB_ETHER_ASIX88179=y
> CONFIG_USB_ETHER_MCS7830=y
> CONFIG_USB_ETHER_RTL8152=y
> CONFIG_USB_ETHER_SMSC95XX=y
> CONFIG_DM_VIDEO=y
> # CONFIG_BACKLIGHT_PWM is not set
> # CONFIG_VIDEO_BPP8 is not set
> # CONFIG_VIDEO_BPP16 is not set
> # CONFIG_VIDEO_BPP32 is not set
> # CONFIG_VIDEO_ANSI is not set
> # CONFIG_SYS_WHITE_ON_BLACK is not set
> CONFIG_DISPLAY=y
> CONFIG_VIDEO_ROCKCHIP=y
> CONFIG_DISPLAY_ROCKCHIP_HDMI=y
> # CONFIG_FAT_WRITE is not set
> CONFIG_USE_TINY_PRINTF=y
> CONFIG_ERRNO_STR=y
> # CONFIG_EFI_LOADER is not set