IMX8M OP-TEE
Tim Harvey
tharvey at gateworks.com
Thu Oct 21 01:47:57 CEST 2021
On Wed, Oct 20, 2021 at 9:47 AM Igor Opaniuk <igor.opaniuk at foundries.io> wrote:
>
> Hi Tim,
>
> On Thu, Oct 14, 2021 at 10:03 PM Tim Harvey <tharvey at gateworks.com> wrote:
> >
> > On Mon, Oct 11, 2021 at 3:15 PM Tim Harvey <tharvey at gateworks.com> wrote:
> > >
> > > Greetings,
> > >
> > > Is anyone successfully booting U-Boot with OP-TEE support on the IMX8M?
> > >
> > > My understanding is that you need to add tee.bin to the images in the
> > > FIT image and include it in loadables following the ATF.
> > >
> > > While this was done with arch/arm/mach-imx/mkimage_fit_atf.sh before
> > > the switch to binman by simply having tee.bin in the U-Boot directory
> > > and passing in TEE_LOAD_ADDR (or accepting the default of 0xfe000000)
> > > once you switch to binman it needs to be added for your board.
> > >
> > > Additionally in order to use OP-TEE from U-Boot (ie for dek_blob
> > > command) you need to add a node with compatible=linaro,optee-tz as
> > > well.
> > >
> > > I've done the following to add OP-TEE for imx8mm_venice:
> > > diff --git a/configs/imx8mm_venice_defconfig b/configs/imx8mm_venice_defconfig
> > > index d85827b588..85d2b20810 100644
> > > -u-boot.dtsi
> > > index e0fa9ff4bf..7a71b974e1 100644
> > > --- a/arch/arm/dts/imx8mm-venice-u-boot.dtsi
> > > +++ b/arch/arm/dts/imx8mm-venice-u-boot.dtsi
> > > @@ -10,6 +10,13 @@
> > > multiple-images;
> > > };
> > >
> > > + firmware {
> > > + optee {
> > > + compatible = "linaro,optee-tz";
> > > + method = "smc";
> > > + };
> > > + };
> > > +
> > > wdt-reboot {
> > > compatible = "wdt-reboot";
> > > wdt = <&wdog1>;
> > > @@ -152,6 +159,16 @@
> > > };
> > > };
> > >
> > > + tee {
> > > + description = "TEE firmware";
> > > + type = "firmware";
> > > + arch = "arm64";
> > > + compression = "none";
> > > + data = "tee.bin";
> > > + load = <0xbe000000>;
> > > + entry = <0xbe000000>;
> > > + };
> > > +
> > > @fdt-SEQ {
> > > description = "NAME";
> > > type = "flat_dt";
> > > @@ -165,7 +182,7 @@
> > > @config-SEQ {
> > > description = "NAME";
> > > firmware = "uboot";
> > > - loadables = "atf";
> > > + loadables = "atf", "tee";
> > > fdt = "fdt-SEQ";
> > > };
> > > };
> > >
> > >
> > > However, when I attempt to boot I hang when the ATF is run.
> > >
> > > Where does the TEE_LOAD_ADDR come from specifically? I would think
> > > this needs to be defined when building tee and needs to match the load
> > > address used in the FIT image. It appears that perhaps this is supped
> > > to be DDR_BASE + DDR_SIZE - 32MIB but I'm not entirely sure.
> > >
> > > I'm currently using NXP's ATF (imx_5.4.3_2.0.0) and NXP's TEE
> > > (imx_5.4.70_2.3.0) and would also like to understand if NXP's branches
> > > are strictly required here and if so what the pros and cons of using
> > > them are.
> > >
> > > Anyone using IMX8MM OP-TEE that could point me in the right direction?
> > >
> >
> > Here's what I've learned so far via various responses, research and testing.
> >
> > There are multiple boot flows that can be used with OP-TEE:
> > 1. SPL -> ATF -> OP-TEE -> U-boot -> LInux
> This is the case for ARMv8: TF-A provides runtime service here as
> Secure Monitor (running on EL3), arbitrating context switches between
> Secure/Non-secure worlds. OP-TEE OS core is running on Secure EL1.
>
> > 2. SPL -> OP-TEE -> U-boot -> Linux
> This is the case for ARMv7, where there is only one Secure PL1,
> so OP-TEE also fulfills the function of SM (arbitrating context
> switches between NW/SW).
>
Igor,
Thanks for the explanation - this makes much more sense now!
> >
> > I'm not really clear what the pros and cons of each are - can anyone
> > shed some light on this?
> >
> > Method #1 is what I've found the most info on and have mostly gotten
> > working so I will go over that here. As IMX8MM runs on Cortex-A Arm
> > TrustZone is available and thus TF-A can be used.
> >
> > By the way, I'm a bit confused as to ATF vs TF-A. From what I can find:
> > - ATF (Arm Trusted Firmware) is a Trusted Firmware reference
> > implementation of secure world software for Armv7-A, Armv8-A, and
> > Armv8-M architectures.
> > - TF-A (Trusted Firmware-A) also appears to provide the same thing
> > Are these the same? Which one is it that NXP uses for imx-atf [1]
> It's the same product, which I guess was renamed to avoid confusion,
> as ARM introduced Trusted Firmware M for M-cores.
>
>
> >
> > When OP-TEE is used with TF-A the TF-A gets invoked with the TEE as
> > bl32 and main u-boot as bl33. Once it has done its startup TF-A jumps
> > into the bl32(OP-TEE) for the TEE startup, returns to TF-A and then
> > jumps to bl33(U-Boot).
> >
> > For this boot flow on IMX8MM you need to do the following:
> > - build TF-A (bl31.bin) with OP-TEE as a Secure Payload Dispatcher
> > (SPD) (add 'SPD=opteed' to env)
> > - build OP-TEE (tee.bin)
> > - add tee.bin to U-Boot FIT image along with TF-A binary in images node
> > * for boards still using arch/arm/mach-imx/mkimage_fit_atf.sh FIT
> > generation this is done automatically when tee.bin is U-Boot's build
> > directory but you also need to set TEE_LOAD_ADDR env to specify the
> > load addr of the TEE which depends on the board DRAM size
> > * for boards using binman the generation of the FIT image has moved
> > to using the 'binman' tool thus adding tee.bin requires you to modify
> > your board's u-boot.dtsi itb node to add tee.bin to itb/fit/images/
> > and add a reference to it in the loadables property of your boot
> > configuration following atf. This is shown in my patch above except
> > there I neglected to add the required 'tee_blob: blob-ext { filename =
> > "tee.bin"; }. Here you must set the TEE_LOAD_ADDR as the load address
> > manually.
> > - To enable OP-TEE support in U-Boot a /firmware/optee node needs to
> > be added to your board u-boot.dtsi with 'compatible =
> > "linaro,optee-tz"' and 'method = "smc"' nodes
> > * This node is used by the U-Boot dek_blob command
> Well, to be more precise it's just for probing optee (drivers/tee/optee)
> driver. It can be also used in AVB 2.0 implementation (in this case OP-TEE
> is used for RPMB eMMC access for storing
> rollback indexes or other persistent values to RPMB)
>
> > * If present then U-Boot will also copy it to your Linux dtb via
> > image_setup_libfdt->optee_copy_fdt_nodes
> > - To enable dek_blob in U-Boot enable CONFIG_CMD_DEKBLOB=y which will
> > enable CONFIG_IMX_OPTEE_DEK_ENCAP=y
> > - To enable optee driver in Linux enable CONFIG_TEE and CONFIG_OPTEE
> > and optionally CONFIG_HW_RANDOM_OPTEE
> > - Also, I believe you need to add memory reservations in the dt via a
> > /reserved-memory/optee_core and /reserved-memory/optee_shm which will
> > also be copied to the linux dtb as well if present and also depend on
> > the OP-TEE memory location/configuration
> >
> > If you are using NXP's imx-optee-os [2] for IMX8MM then you can use
> > the 'mx8mmevk' platform which simply configures the following which
> > can be overridden via env:
> > CFG_UART_BASE ?= UART2_BASE
> > CFG_DDR_SIZE ?= 0x80000000
> >
> > In core/arch/arm/plat-imx/conf.mk for imx8mm CFG_DDR_SIZE is used to
> > compute CFG_TZDRAM_START where optee gets loaded (TEE_LOAD_ADDR):
> > # put optee end of ddr for AARCH64
> > CFG_TZDRAM_START ?= ($(CFG_DRAM_BASE) - 0x02000000 + $(CFG_DDR_SIZE))
> > CFG_TZDRAM_SIZE ?= 0x01c00000
> > CFG_SHMEM_START ?= ($(CFG_TZDRAM_START) + $(CFG_TZDRAM_SIZE))
> > CFG_SHMEM_SIZE ?= 0x00400000
> >
> > thus:
> > CFG_DDR_SIZE = 0x40000000 (1GiB) results in
> > CFG_TZDRAM_START=0x7e000000 (TEE_LOAD_ADDR) and
> > CFG_SHMEM_START=0x7fc00000
> > CFG_DDR_SIZE = 0x80000000 (2GiB) results in
> > CFG_TZDRAM_START=0xbe000000 (TEE_LOAD_ADDR) and
> > CFG_SHMEM_START=0xbfc00000
> > CFG_DDR_SIZE = 0x100000000 (4GiB) results in
> > CFG_TZDRAM_START=0x13e000000 (TEE_LOAD_ADDR) and
> > CFG_SHMEM_START=0x13fc00000
> >
> > So for a 2GiB board your dt changes for a board already using binman
> > should look like this:
> > @@ -6,6 +6,29 @@
> > #include "imx8mm-u-boot.dtsi"
> >
> > / {
> > + firmware {
> > + optee {
> > + compatible = "linaro,optee-tz";
> > + method = "smc";
> > + };
> > + };
> > +
> > + reserved-memory {
> > + #address-cells = <2>;
> > + #size-cells = <2>;
> > + ranges;
> > +
> > + /* 32MiB */
> > + optee_core at be000000 {
> > + reg = <0x00 0xbe000000 0x00 0x2000000>;
> > + };
> > +
> > + /* 4K */
> > + optee_shm at bfc00000 {
> > + reg = <0x00 0xbfc00000 0x00 0x400000>;
> > + };
> > + };
> > +
> > binman: binman {
> > multiple-images;
> > };
> > @@ -152,6 +175,19 @@
> > };
> > };
> >
> > + tee {
> > + description = "TEE firmware";
> > + type = "firmware";
> > + arch = "arm64";
> > + compression = "none";
> > + load = <0xbe000000>;
> > + entry = <0xbe000000>;
> > +
> > + tee_blob: blob-ext {
> > + filename = "tee.bin";
> > + };
> > + };
> > +
> > @fdt-SEQ {
> > description = "NAME";
> > type = "flat_dt";
> > @@ -165,7 +201,7 @@
> > @config-SEQ {
> > description = "NAME";
> > firmware = "uboot";
> > - loadables = "atf";
> > + loadables = "atf", "tee";
> > fdt = "fdt-SEQ";
> > };
> > };
> >
> > I have done all of the above using imx-atf [1] and imx-optee-os [2] on
> > a 2GiB IMX8MM board and successfully boot to U-Boot where I verify
> > dek_blob works, and then successfully booting to Linux where I verify
> > that the Linux optee driver probes.
> >
> > But, what confuses me relates to optee memory usage and location
> > (TEE_LOAD_ADDR) based on testing on 1GiB and 4GiB IMX8MM boards:
> > - CFG_DDR_SIZE = 0x40000000 (1GiB); TEE_LOAD_ADDR=0x7e000000
> > - panic's at core/arch/arm/mm/core_mmu.c <check_pa_matches_va> on a
> > 1GiB DRAM board (why? makes no sense)
> > - panic's at core/arch/arm/mm/core_mmu.c <check_pa_matches_va> on a
> > 2GiB DRAM board (this makes sense as 0x7e000000 should be out of
> > bounds!)
> > - hangs in opteed_init on a 4GiB DRAM board (why?)
> > - CFG_DDR_SIZE = 0x80000000 (2GiB); TEE_LOAD_ADDR=0xbe000000
> > - boots on a 1GiB DRAM board... shouldn't that crash when accessing
> > 0xbe000000?
> > - boots on a 2GiB DRAM board
> > - boots on a 4GiB DRAM board
> > CFG_DDR_SIZE = 0x100000000 (4GiB); TEE_LOAD_ADDR=0x13e000000
> > - fails building: 'Error: u-boot.its:63.13-24 Value out of range for
> > 32-bit array element'
> > - also this may be an issue as currently IMX8MM boards have DRAM
> > limited to 3GiB due to various IP drivers not being 64bit ready
> >
> > Also, I find that the memory reservations produce an error when I TFTP
> > a file due to (tftp_init_load_addr -> lmb_init_and_reserve)
> > u-boot=> tftpboot $fdt_addr_r venice/imx8mm-venice-gw73xx-0x.dtb
> > Using ethernet at 30be0000 device
> > TFTP from server 192.168.1.146; our IP address is 192.168.1.1
> > Filename 'venice/imx8mm-venice-gw73xx-0x.dtb'.
> > ERROR: reserving fdt memory region failed (addr=bfc00000 size=400000 flags=0)
> >
> > Can anyone explain the memory configurations needed for IMX8MM OP-TEE?
>
> Well, I'd suggest just to take existing buildroot based setup with
> vanilla OP-TEE for
> Verdin iMX8MM ([1], [2], [3]), which I created ~2 years ago and just take a look
> what build flags are used there. I'm not sure if it's still works, as
> I don't have
> Verdin SoM at hand, however this definitely will save you a lot of
> time trying to build/bring
> up everything manually.
>
> Also there is a makefile for iMX8MQ EVK [4], which is currently maintained by
> Jens Wiklander from Linaro (never tried to use it, but looks like it's
> regularly updated, so
> it should just work out of the box).
>
> [1] https://github.com/OP-TEE/manifest/blob/master/verdin.xml
> [2] https://github.com/OP-TEE/build/blob/master/verdin.mk
> [3] https://optee.readthedocs.io/en/latest/building/index.html
> [4] https://github.com/OP-TEE/build/blob/master/imx.mk
I did manage to get OP-TEE booting properly on 1GiB IMX8MM boards.
An additional step above is to set BL32_BASE to what I calculate above
as TEE_LOAD_ADDR when building the ATF as by default it is set in
plat/imx/imx8m/imx8mm/platform.mk to 0xbe000000 so my issue was that
the ATF jump address was wrong.
Also above my /reserved-memory/optee_core node is defined wrong as a
size of 32MiB (0x2000000) overflows the optee_shm region thus the
optee_core size should be set to 0x1c00000 (this did not produce a
failure to boot, just a kernel warning/error).
I still do not have OP-TEE booting properly on a 4GiB IMX8MM board
however. Placing OP-TEE at the end of DRAM as imx8mm configuration
does puts it in a 64bit address space which if defined properly in the
its file (ie TEE_LOAD_ADDRESS="0x1 0x3fc00000" for pre-binman boards)
ends up failing to boot via eMMC due to the emmc driver
(drivers/mmc/fsl_esdhc_imx.c) not yet supporting 64bit addressing. I
have tried moving OP-TEE to lower memory such as the IMX8MN does by
specifying BL32_BASE=0x56000000 when building the ATF and
TEE_LOAD_ADDRESS=0x56000000 when building a pre-binman U-Boot) but
this ended up hanging in the ATF as well for still unknown reasons.
Best regards,
Tim
More information about the U-Boot
mailing list