[U-Boot] Exception Level switching seems broken on RK3399

Thu May 24 16:48:44 UTC 2018

Hello Philipp,
your answer is much appreciated.

On Thu, May 24, 2018 at 1:07 PM, Dr. Philipp Tomsich
<philipp.tomsich at theobroma-systems.com> wrote:
> Vincente,
>
>> On 19 May 2018, at 16:58, Vicente Bergas <vicencb at gmail.com> wrote:
>>
>> Hello,
>> I am writing this from a standalone Sapphire board [1],
>> that is, without the Excavator base board.
>> The CPU is the Rockchip RK3399, which implements ARMv8.0-A.
>>
>> Currently the boot process is:
>> 1.- Boot ROM
>> 2.- SPL, provided as closed source binary blob [2]
>
> SPL-support is available in mainline U-Boot.  We developed this for
> the RK3399-Q7 and it has been successfully used on other RK3399
> boards (e.g. I know that some Firefly-users are using this).

Thank you!

>
>> 3.- ATF, closed source binary blob [3]
>>     (not using the one from [2] because of stability issues)
>
> Why use the closed-source blob, if the RK3399 is supported in the ATF
> mainline and an ATF can be compiled from source?

Currently I am using both binary blobs (SPL and ATF) because I could
not make it work another way. I'll give it another try.

>
>> 4.- Mainline u-boot, master branch
>> 5.- Mainline linux, master branch
>>
>> I would like to use an opensource boot process.
>> As a first approach I try to completely remove the ATF and
>> replace the SPL with the one from u-boot.
>> The modified boot process looks like:
>> 1.- Boot ROM
>> 2.- SPL, from mainline u-boot, master branch
>> 3.- Mainline u-boot, master branch
>> 4.- Mainline linux, master branch
>> But it is not working.
>>
>> The replaced SPL works fine and loads u-boot.
>> U-boot also works fine, loads linux and jumps into it.
>
> Yes, we’ve done some work to enable us to run U-Boot in EL3 on
> the RK3399 (as we use it for programming the secure e-fuses on
> the RK3399-Q7 in our factory programming setup).

I can indeed confirm that U-Boot runs fine in EL3.

>
>> But then, linux never gets executed.
>>
>> I have traced the issue to: arch/arm/include/asm/macro.h
>> 202: msr  spsr_el3, \tmp
>> 203: msr  elr_el3, \ep
>> 204: eret // This is the last instruction executed
>>
>> For testing, I have also set CONFIG_ARMV8_SWITCH_TO_EL1 and
>> checked that switch_to_el1 from arch/arm/lib/bootm.c is not reached.
>>
>> At this point I have a few questions:
>> 1.- Is my first approach feasible? That is, is it possible to boot
>>     this CPU without ATF?
>
> It is feasible (i.e.: requires implementation work) but not recommended:
> Linux will use PSCI to bring up the secondary CPUs.  We have run Linux
> (limited to a single CPU) in EL3 on this CPU during our own board bringup,
> but I would strongly discourage this as it will entail unnecessary effort.

There is a misunderstanding here. My intention was to run U-Boot in EL3,
then switch to EL2 or EL1 from within U-Boot and afterwards run Linux
in the lower EL.

>
>> 2.- If so, what should I do to make it work? Probably it is just
>>     a configuration issue, but I do not know what to check. [4]
>> 3.- Else, why do I need ATF?
>
> ATF is the secure monitor on ARMv8 and provides services such as PSCI
> to start up secondary CPUs.  It will usually also be part of power-management
> on most SoCs (after all: power configuration needs to be done in the secure
> envelope).

Do you mean that without ATF I can only run a single CPU core?

Regards,
  Vicenç.

>
> Regards,
> Philipp.
>
>>
>> Regards,
>>  Vicenç.
>>
>> [1] http://opensource.rock-chips.com/wiki_Excavator_sapphire_board
>> [2] https://github.com/rockchip-linux/rkbin/tree/29mirror
>> [3] https://raw.githubusercontent.com/nightseas/armbian-build/master/packages/blobs/rockchip64/trust.img
>> [4]
>> CONFIG_ARM=y
>> CONFIG_ARCH_ROCKCHIP=y
>> CONFIG_SYS_TEXT_BASE=0x00200000
>> CONFIG_SPL_LIBCOMMON_SUPPORT=y
>> CONFIG_SPL_LIBGENERIC_SUPPORT=y
>> CONFIG_SYS_MALLOC_F_LEN=0x4000
>> CONFIG_ROCKCHIP_RK3399=y
>> CONFIG_ROCKCHIP_SPL_RESERVE_IRAM=0x4000
>> CONFIG_SPL_STACK_R_ADDR=0x80000
>> # CONFIG_PSCI_RESET is not set
>> CONFIG_CMD_HDMIDETECT=y
>> CONFIG_DEFAULT_DEVICE_TREE="rk3399-evb"
>> CONFIG_DEBUG_UART=y
>> CONFIG_BOOTDELAY=1
>> # CONFIG_DISPLAY_CPUINFO is not set
>> # CONFIG_DISPLAY_BOARDINFO is not set
>> CONFIG_SPL_STACK_R=y
>> CONFIG_SPL_STACK_R_MALLOC_SIMPLE_LEN=0x4000
>> # CONFIG_CMD_IMI is not set
>> # CONFIG_CMD_XIMG is not set
>> # CONFIG_CMD_SAVEENV is not set
>> # CONFIG_CMD_FLASH is not set
>> # CONFIG_CMD_LOADB is not set
>> # CONFIG_CMD_LOADS is not set
>> CONFIG_CMD_MMC=y
>> CONFIG_CMD_USB=y
>> # CONFIG_CMD_SETEXPR is not set
>> # CONFIG_CMD_NET is not set
>> # CONFIG_DOS_PARTITION is not set
>> # CONFIG_ISO_PARTITION is not set
>> CONFIG_SPL_OF_CONTROL=y
>> CONFIG_REGMAP=y
>> CONFIG_SPL_REGMAP=y
>> CONFIG_SYSCON=y
>> CONFIG_SPL_SYSCON=y
>> CONFIG_CLK=y
>> CONFIG_SPL_CLK=y
>> CONFIG_ROCKCHIP_GPIO=y
>> CONFIG_SYS_I2C_ROCKCHIP=y
>> # CONFIG_MMC_WRITE is not set
>> # CONFIG_MMC_HW_PARTITIONING is not set
>> CONFIG_MMC_DW=y
>> CONFIG_MMC_SDHCI=y
>> CONFIG_MMC_SDHCI_ROCKCHIP=y
>> CONFIG_DM_ETH=y
>> # CONFIG_NETDEVICES is not set
>> CONFIG_PINCTRL=y
>> CONFIG_SPL_PINCTRL=y
>> CONFIG_PINCTRL_ROCKCHIP_RK3399=y
>> CONFIG_DM_PMIC=y
>> CONFIG_PMIC_RK8XX=y
>> CONFIG_REGULATOR_PWM=y
>> CONFIG_DM_REGULATOR_FIXED=y
>> CONFIG_REGULATOR_RK8XX=y
>> CONFIG_PWM_ROCKCHIP=y
>> CONFIG_RAM=y
>> CONFIG_SPL_RAM=y
>> CONFIG_BAUDRATE=1500000
>> CONFIG_DEBUG_UART_BASE=0xFF1A0000
>> CONFIG_DEBUG_UART_CLOCK=24000000
>> CONFIG_DEBUG_UART_SHIFT=2
>> CONFIG_SYSRESET=y
>> CONFIG_USB=y
>> CONFIG_USB_XHCI_HCD=y
>> CONFIG_USB_XHCI_DWC3=y
>> CONFIG_USB_EHCI_HCD=y
>> CONFIG_USB_EHCI_GENERIC=y
>> CONFIG_USB_STORAGE=y
>> CONFIG_USB_KEYBOARD=y
>> CONFIG_USB_HOST_ETHER=y
>> CONFIG_USB_ETHER_ASIX=y
>> CONFIG_USB_ETHER_ASIX88179=y
>> CONFIG_USB_ETHER_MCS7830=y
>> CONFIG_USB_ETHER_RTL8152=y
>> CONFIG_USB_ETHER_SMSC95XX=y
>> CONFIG_DM_VIDEO=y
>> # CONFIG_BACKLIGHT_PWM is not set
>> # CONFIG_VIDEO_BPP8 is not set
>> # CONFIG_VIDEO_BPP16 is not set
>> # CONFIG_VIDEO_BPP32 is not set
>> # CONFIG_VIDEO_ANSI is not set
>> # CONFIG_SYS_WHITE_ON_BLACK is not set
>> CONFIG_DISPLAY=y
>> CONFIG_VIDEO_ROCKCHIP=y
>> CONFIG_DISPLAY_ROCKCHIP_HDMI=y
>> # CONFIG_FAT_WRITE is not set
>> CONFIG_USE_TINY_PRINTF=y
>> CONFIG_ERRNO_STR=y
>> # CONFIG_EFI_LOADER is not set
>