imx8m optee load address?

Sat Jun 17 01:52:56 CEST 2023

On Thu, Jun 15, 2023 at 8:31 PM Peng Fan <peng.fan at oss.nxp.com> wrote:
>
>
>
> On 6/16/2023 9:56 AM, Tim Harvey wrote:
> > Greetings,
> >
> > I've seen several IMX8M boards include a firmware/optee node in the
> > U-Boot dt (git grep optee arch/arm/dts/imx8m*.dtsi) yet but none that
> > I see configure binman to actually add the binary in the u-boot.dtsi,
> > do anything to keep U-Boot from accessing the OPTEE memory, or
> > document how to configure and build OPTEE for imx8m. I would like to
> > add such support but I find it odd that OPTEE needs to be built
> > differently depending on the dram size.
> >
> > Prior to switching to binman (v2021.10)
> > arch/arm/mach-imx/mkimage_fit_atf.sh [1] would include tee.bin in the
> > FIT image if it was found in the current directory and it was expected
> > that you provide a proper TEE_LOAD_ADDR via the env (commit
> > a9eed6e1b8d8 ("imx: imx8m: introduce script to generate fit image").
> >
> > According to the IMX OPTEE documentation [2] the size and location of
> > OPTEE is hard coded (CFG_TZDRAM_START and CFG_TZDRAM_SIZE) but looking
> > at the imx-optee source [3] this is calculated based off of
> > CFG_DDR_SIZE (which you can provide via env at build time or just
> > provide CFG_TZDRAM_START and CFG_TZDRAM_SIZE via env directly).
>
> optee does not support PIE, it could only run at the address that built
> time defined.

Hi Peng,

I wasn't thinking about PIE but just building it at a location other
than the top of DRAM in order to come up with a generalized location
instead of having to customize u-boot.dtsi for different RAM sizes and
to workaround the 4GiB boundary issue.

If my understanding is correct optee's load address needs to match between:
1. atf: BL32_BASE at build time defines the address in secure memory
where BL2 loads the BL32 binary image (tee.bin) (must be aligned on a
page-size boundary)
2. optee: CFG_TZDRAM_START at build time (calculated based off of
CFG_DDR_SIZE) defines the address it is built to run at. Note you must
still define CFG_DDR_SIZE properly as this is passed to
imx_tzc_auto_configure()
3. uboot: tee.bin is contained in u-boot.itb FIT image and specifies
the address that U-Boot SPL loads tee.bin into

I'm really trying to understand the imx8m 4GiB limit issue as that
seems to be a huge limiation.

On an imx8mm board with 4GiB of DRAM configured with
atf:BL32_BASE=0x13e000000
optee:CFG_DDR_SIZE=0x100000000,CFG_TZDRAM_START=0x13e000000 and
u-boot.dtsi tee.bin load/entry=<0x1 0x3e000000> we get this:

U-Boot SPL 2023.04-00027-g578c653cbafa (Jun 16 2023 - 09:22:40 -0700)
GSCv3   : v58 0xf098 RST:VIN Thermal protection:disabled
RTC     : 1970-01-01   0:05:40 UTC
Model   : GW7201-01-EB
Serial  : 852455
MFGDate : 11-10-2020
PMIC    : MP5416 (IMX8MM)
DRAM    : LPDDR4 4 GiB 3000MT/s 1500MHz
WDT:   Started watchdog at 30280000 with servicing every 1000ms (60s timeout)
Trying to boot from eMMC
DTB     : imx8mm-venice-gw72xx-0x
Cannot use 64 bit addresses with SDMA
^^^ This is the imx mmc driver warning that it can't load data into
dram across 32bit boundary

Authenticate image from DDR location 0x401fcdc0...
NOTICE:  Do not release JR0 to NS as it can be used by HAB
ERROR:   mmap_add_region_check() failed. error -34
ASSERT: lib/xlat_tables_v2/xlat_tables_core.c:793
BACKTRACE: START: assert
0: EL3: 0x9289cc
1: EL3: 0x929efc
2: EL3: 0x92469c
3: EL3: 0x9248f4
4: EL3: 0x928850
5: EL3: 0x927594
6: EL3: 0x920128
7: EL3: 0x7eb19c
8: EL3: 0x7f3de0
BACKTRACE: END: assert

This is where the atf jumps to optee so I can understand why the above
doesn't work as U-Boot can't load tee.bin from the FIT to 0x13e000000
(not sure where it gets loaded... the 64bit address probably wraps).
I've also tried setting CFG_CORE_LARGE_PHYS_ADDR=y and
CFG_CORE_ARM64_PA_BITS=36 in optee as this is what the imx8mp-evk
config with 6GiB does and it ended with the same result but I do
notice that doing this changes the link address

The imx8mp-evk has 6GiB of dram on it. Does NXP not have a solution
for secure boot on boards with >3GiB of dram?  Is there some support
in their downstream U-Boot that I'm maybe missing? Personally I can't
even get NXP's lf_v2022.04 u-boot branch to build flash.bin for
imx8mp_evk so I can try it out.

Also, if I look at the latest NXP Yocto BSP I don't even see where
they configure BL32_BASE when building the atf which leaves it to
default to something SOC-specific that does not depend on dram size
and does not match the dram sizes for imx8mn_evk and imx8mp_evk
(https://github.com/nxp-imx/meta-imx/blob/kirkstone-5.15.71-2.2.0/meta-bsp/recipes-bsp/imx-atf/imx-atf_2.6.bb).

>
> And the TZASC needs following it to protect the region properly.
>
> >
> > This results in the OPTEE location of 0x7e000000 for 1GiB systems,
> > 0xbe000000 for 2GiB systems, and 0x13e000000 for 4GiB systems for
> > example. This dependence on dram size causes a couple of issues:
> >   - OPTEE location depends on build-time configuration and several
> > boards have run-time identification of their memory size
> >   - for 4GiB systems the address 0x13e000000 crosses the 32bit address
> > boundary and U-Boot hangs
>
> imx optee seems has this. But for overlapping with U-Boot, I doubt that.
>
> >
> > Is there any reason OPTEE can't be located wherever you want and not
> > depend on dram size?
>
> You could , just modify the imx *.mk file per my recall.

Right. So, for example on the same imx8mm board with 4GiB I can
configure uboot/atf/optee to load tee at top of 3GiB with
atf:BL32_BASE=0xfe000000
optee:CFG_DDR_SIZE=0xc0000000,CFG_TZDRAM_START=0xfe000000 and
u-boot.dtsi tee.bin load/entry=<0xfe000000>.

I finally found that this boots only if I also adjust the size
configured by board_phys_sdram_size() down to 3GiB otherwise we hang
somewhere before dram_init_banksize(). I am guessing this still has
something to do with missing a uboot carevout for the memory tee is
loaded in (and I'm not clear how to add a carve-out yet). I still
would like to understand this hang as while this is somewhat of a
work-around but it does cripple the board's memory size that gets
passed along to Linux.

>
> > I've tried to set it to 0x7e000000 (for a 1GiB
> > board) on a 4GiB board and U-Boot hangs near the DRAM banner which is
> > likely due to get_ram_size() stepping on OPTEE's memory range but then
>
> You may give a try to disable TZASC settings in OP-TEE first.
> And you also need to carve out the memory of OP-TEE from U-Boot MMU
> to avoid prefetch.

I had not thought about this. It looks like the TZASC provides
configurable protection over dram memory space. It looks like both the
atf and optee configure the TZASC. U-Boot SPL enable_tzc380() enables
TZASC. If I disable that it does not change the hang so perhaps the
atf/optee is re-enabling it?

>
> > I'm not clear why that doesn't occur when OPTEE is located at the top
> > of dram on a 1GiB and 2GiB system. How would I go about adding a
> > carevout for this?
>
> Are u using upstream optee or NXP downstream optee release?

I've been trying many combinations including upstream optee/atf as
well as NXP's latest branches (imx-optee-os-lf-6.1.1_1.0.0 and
imx-atx-lf_v2.6).

Best Regards,

Tim

>
> Regards,
> Peng.
>
> >
> > Best Regards,
> >
> > Tim
> > [1] https://elixir.bootlin.com/u-boot/v2021.10/source/arch/arm/mach-imx/mkimage_fit_atf.sh
> > [2] https://www.nxp.com/docs/en/user-guide/IMX_PORTING_GUIDE.pdf
> > [3] https://github.com/nxp-imx/imx-optee-os