[PATCH v4 5/5] test/py: Create a test for launching UEFI binaries from FIT images
Cristian Ciocaltea
cristian.ciocaltea at gmail.com
Sun Dec 29 19:39:20 CET 2019
On Sun, Dec 29, 2019 at 11:22:08AM +0100, Heinrich Schuchardt wrote:
> On 12/24/19 5:05 PM, Cristian Ciocaltea wrote:
> > This test verifies the implementation of the 'bootm' extension that
> > handles UEFI binaries inside FIT images (enabled via CONFIG_BOOTM_EFI).
> >
> > Signed-off-by: Cristian Ciocaltea <cristian.ciocaltea at gmail.com>
>
> Thanks a lot for devising this test.
>
> ---
>
> You are using variable env__efi_fit_tftp_file. To run the test on Gitlab
> and Travis CI a patch will be needed for:
>
> https://github.com/swarren/uboot-test-hooks.git
>
> I hope
>
> https://github.com/xypron/uboot-test-hooks/commit/20dcd721437dd5f7d7d3d235f7112246f43305d2
>
> will do the job.
>
> Once we have this applied we will have to adjust the config files for QEMU.
>
> ---
>
> I have been trying to run the test on qemu_arm64_defconfig using the
> following lines in u_boot_boardenv_qemu_arm64.py:
>
> env__efi_fit_tftp_file = {
> "fn": "helloworld.efi",
> "size": 4480,
> "crc32": "19f9c0ab",
> }
>
> I got an error:
>
> test/py/tests/test_efi_fit.py:417: in launch_efi
> addr = load_fit_from_host(fit) if is_sandbox else
> load_fit_from_tftp(fit)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> addr = f.get('addr', None)
> if not addr:
> > addr = u_boot_utils.find_ram_base(cons)
> E NameError: name 'u_boot_utils' is not defined
>
>
> When I provided addr:
>
> env__efi_fit_tftp_file = {
> "fn": "helloworld.efi",
> "size": 4480,
> "crc32": "19f9c0ab",
> "addr": 0x40400000,
> }
>
> I got the following error:
>
> => tftpboot 1073741824 helloworld.efi
> TFTP error: trying to overwrite reserved memory...
>
> I would have expected a command
>
> tftpboot 40400000 helloworld.efi
>
> to be issued.
>
> Same error with bootm:
>
> => bootm 1077936128#config-efi-nofdt
> "Synchronous Abort" handler, esr 0x96000010
> elr: 000000000001c36c lr : 00000000000140f4 (reloc)
>
> Please, fix the lines indicated below and verify that you can actually
> execute this test on the QEMU platform.
Thank you for the detailed report!
Unfortunately I have only tested on qemu_arm and somehow I missed the
check of having the address computed by the test suite. It used to work
before I changed the address data type to string - the reason was to
allow for more flexibility, e.g. providing values like '$kernel_addr_r'
instead of just precomputed numbers.
I'm going to extend the tests on qemu_arm64 as well.
>
> https://github.com/xypron/u-boot-build/tree/qemu-arm64/u-boot-test
>
> contains the files I use to run Python tests on qemu_arm64_defconfig.
>
> > ---
> > test/py/tests/test_efi_fit.py | 459 ++++++++++++++++++++++++++++++++++
> > 1 file changed, 459 insertions(+)
> > create mode 100644 test/py/tests/test_efi_fit.py
> >
> > diff --git a/test/py/tests/test_efi_fit.py b/test/py/tests/test_efi_fit.py
> > new file mode 100644
> > index 0000000000..e1f0e42694
> > --- /dev/null
> > +++ b/test/py/tests/test_efi_fit.py
> > @@ -0,0 +1,459 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2019, Cristian Ciocaltea <cristian.ciocaltea at gmail.com>
> > +#
> > +# Work based on:
> > +# - test_net.py
> > +# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
> > +# - test_fit.py
> > +# Copyright (c) 2013, Google Inc.
> > +#
> > +# Test launching UEFI binaries from FIT images.
> > +
> > +import os.path
> > +import pytest
> > +import u_boot_utils as util
>
> "as util" causes an error if you use u_boot_utils.* below. Below I
> indicate the places to change.
Fixed, thanks.
> > +
> > +"""
> > +Note: This test relies on boardenv_* containing configuration values to define
> > +which network environment is available for testing. Without this, the parts
> > +that rely on network will be automatically skipped.
> > +
> > +For example:
> > +
> > +# Boolean indicating whether the Ethernet device is attached to USB, and hence
> > +# USB enumeration needs to be performed prior to network tests.
> > +# This variable may be omitted if its value is False.
> > +env__net_uses_usb = False
> > +
> > +# Boolean indicating whether the Ethernet device is attached to PCI, and hence
> > +# PCI enumeration needs to be performed prior to network tests.
> > +# This variable may be omitted if its value is False.
> > +env__net_uses_pci = True
> > +
> > +# True if a DHCP server is attached to the network, and should be tested.
> > +# If DHCP testing is not possible or desired, this variable may be omitted or
> > +# set to False.
> > +env__net_dhcp_server = True
> > +
> > +# A list of environment variables that should be set in order to configure a
> > +# static IP. If solely relying on DHCP, this variable may be omitted or set to
> > +# an empty list.
> > +env__net_static_env_vars = [
> > + ('ipaddr', '10.0.0.100'),
> > + ('netmask', '255.255.255.0'),
> > + ('serverip', '10.0.0.1'),
> > +]
> > +
> > +# Details regarding a file that may be read from a TFTP server. This variable
> > +# may be omitted or set to None if TFTP testing is not possible or desired.
> > +# Additionally, when the 'size' is not available, the file will be generated
> > +# automatically in the TFTP root directory, as specified by the 'dn' field.
> > +env__efi_fit_tftp_file = {
> > + 'fn': 'test-efi-fit.img', # File path relative to TFTP root
> > + 'size': 3831, # File size
> > + 'crc32': '9fa3f79c', # Checksum using CRC-32 algorithm, optional
> > + 'addr': '$kernel_addr_r', # Loading address, optional
> > + 'dn': 'tftp/root/dir', # TFTP root directory path, optional
> > +}
> > +"""
> > +
> > +# Define the parametrized ITS data to be used for FIT images generation.
> > +its_data = '''
> > +/dts-v1/;
> > +
> > +/ {
> > + description = "EFI image with FDT blob";
> > + #address-cells = <1>;
> > +
> > + images {
> > + efi {
> > + description = "Test EFI";
> > + data = /incbin/("%(efi-bin)s");
> > + type = "%(kernel-type)s";
> > + arch = "%(sys-arch)s";
> > + os = "efi";
> > + compression = "%(efi-comp)s";
> > + load = <0x0>;
> > + entry = <0x0>;
> > + };
> > + fdt {
> > + description = "Test FDT";
> > + data = /incbin/("%(fdt-bin)s");
> > + type = "flat_dt";
> > + arch = "%(sys-arch)s";
> > + compression = "%(fdt-comp)s";
> > + };
> > + };
> > +
> > + configurations {
> > + default = "config-efi-fdt";
> > + config-efi-fdt {
> > + description = "EFI FIT w/ FDT";
> > + kernel = "efi";
> > + fdt = "fdt";
> > + };
> > + config-efi-nofdt {
> > + description = "EFI FIT w/o FDT";
> > + kernel = "efi";
> > + };
> > + };
> > +};
> > +'''
> > +
> > +# Define the parametrized FDT data to be used for DTB images generation.
> > +fdt_data = '''
> > +/dts-v1/;
> > +
> > +/ {
> > + #address-cells = <1>;
> > + #size-cells = <0>;
> > +
> > + model = "%(sys-arch)s %(fdt_type)s EFI FIT Boot Test";
> > + compatible = "%(sys-arch)s";
> > +
> > + reset at 0 {
> > + compatible = "%(sys-arch)s,reset";
> > + reg = <0>;
> > + };
> > +};
> > +'''
> > +
> > + at pytest.mark.buildconfigspec('bootm_efi')
> > + at pytest.mark.buildconfigspec('cmd_bootefi_hello_compile')
> > + at pytest.mark.buildconfigspec('fit')
> > + at pytest.mark.notbuildconfigspec('generate_acpi_table')
> > + at pytest.mark.requiredtool('dtc')
> > +def test_efi_fit_launch(u_boot_console):
> > + """Test handling of UEFI binaries inside FIT images.
> > +
> > + The tests are trying to launch U-Boot's helloworld.efi embedded into
> > + FIT images, in uncompressed or gzip compressed format.
> > +
> > + Additionally, a sample FDT blob is created and embedded into the above
> > + mentioned FIT images, in uncompressed or gzip compressed format.
> > +
> > + For more details, see launch_efi().
> > +
> > + The following test cases are currently defined and enabled:
> > + - Launch uncompressed FIT EFI & internal FDT
> > + - Launch uncompressed FIT EFI & FIT FDT
> > + - Launch compressed FIT EFI & internal FDT
> > + - Launch compressed FIT EFI & FIT FDT
> > + """
> > +
> > + def net_pre_commands():
> > + """Execute any commands required to enable network hardware.
> > +
> > + These commands are provided by the boardenv_* file; see the comment
> > + at the beginning of this file.
> > + """
> > +
> > + init_usb = cons.config.env.get('env__net_uses_usb', False)
> > + if init_usb:
> > + cons.run_command('usb start')
> > +
> > + init_pci = cons.config.env.get('env__net_uses_pci', False)
> > + if init_pci:
> > + cons.run_command('pci enum')
> > +
> > + def net_dhcp():
> > + """Execute the dhcp command.
> > +
> > + The boardenv_* file may be used to enable/disable DHCP; see the
> > + comment at the beginning of this file.
> > + """
> > +
> > + has_dhcp = cons.config.buildconfig.get('config_cmd_dhcp', 'n') == 'y'
> > + if not has_dhcp:
> > + cons.log.warning('CONFIG_CMD_DHCP != y: Skipping DHCP network setup')
> > + return False
> > +
> > + test_dhcp = cons.config.env.get('env__net_dhcp_server', False)
> > + if not test_dhcp:
> > + cons.log.info('No DHCP server available')
> > + return False
> > +
> > + cons.run_command('setenv autoload no')
> > + output = cons.run_command('dhcp')
> > + assert 'DHCP client bound to address ' in output
> > + return True
> > +
> > + def net_setup_static():
> > + """Set up a static IP configuration.
> > +
> > + The configuration is provided by the boardenv_* file; see the comment at
> > + the beginning of this file.
> > + """
> > +
> > + has_dhcp = cons.config.buildconfig.get('config_cmd_dhcp', 'n') == 'y'
> > + if not has_dhcp:
> > + cons.log.warning('CONFIG_NET != y: Skipping static network setup')
> > + return False
> > +
> > + env_vars = cons.config.env.get('env__net_static_env_vars', None)
> > + if not env_vars:
> > + cons.log.info('No static network configuration is defined')
> > + return False
> > +
> > + for (var, val) in env_vars:
> > + cons.run_command('setenv %s %s' % (var, val))
> > + return True
> > +
> > + def make_fpath(fname):
> > + """Compute the path of a given (temporary) file.
> > +
> > + Args:
> > + fname: The name of a file within U-Boot build dir.
> > + Return:
> > + The computed file path.
> > + """
> > +
> > + return os.path.join(cons.config.build_dir, fname)
> > +
> > + def make_efi(fname, comp):
> > + """Create an UEFI binary.
> > +
> > + This simply copies lib/efi_loader/helloworld.efi into U-Boot
> > + build dir and, optionally, compresses the file using gzip.
> > +
> > + Args:
> > + fname: The target file name within U-Boot build dir.
> > + comp: Flag to enable gzip compression.
> > + Return:
> > + The path of the created file.
> > + """
> > +
> > + bin_path = make_fpath(fname)
> > + util.run_and_log(cons,
> > + ['cp', make_fpath('lib/efi_loader/helloworld.efi'), bin_path])
> > + if comp:
> > + util.run_and_log(cons, ['gzip', '-f', bin_path])
> > + bin_path += '.gz'
> > + return bin_path
> > +
> > + def make_dtb(fdt_type, comp):
> > + """Create a sample DTB file.
> > +
> > + Creates a DTS file and compiles it to a DTB.
> > +
> > + Args:
> > + fdt_type: The type of the FDT, i.e. internal, user.
> > + comp: Flag to enable gzip compression.
> > + Return:
> > + The path of the created file.
> > + """
> > +
> > + # Generate resources referenced by FDT.
> > + fdt_params = {
> > + 'sys-arch': sys_arch,
> > + 'fdt_type' : fdt_type,
> > + }
> > +
> > + # Generate a test FDT file.
> > + dts = make_fpath('test-efi-fit-%s.dts' % fdt_type)
> > + with open(dts, 'w') as fd:
> > + fd.write(fdt_data % fdt_params)
> > +
> > + # Build the test FDT.
> > + dtb = make_fpath('test-efi-fit-%s.dtb' % fdt_type)
> > + util.run_and_log(cons, ['dtc', '-I', 'dts', '-O', 'dtb', '-o', dtb, dts])
> > + if comp:
> > + util.run_and_log(cons, ['gzip', '-f', dtb])
> > + dtb += '.gz'
> > + return dtb
> > +
> > + def make_fit(comp):
> > + """Create a sample FIT image.
> > +
> > + Runs 'mkimage' to create a FIT image within U-Boot build dir.
> > + Args:
> > + comp: Enable gzip compression for the EFI binary and FDT blob.
> > + Return:
> > + The path of the created file.
> > + """
> > +
> > + # Generate resources referenced by ITS.
> > + its_params = {
> > + 'sys-arch': sys_arch,
> > + 'efi-bin': os.path.basename(make_efi('test-efi-fit-helloworld.efi', comp)),
> > + 'kernel-type': 'kernel' if comp else 'kernel_noload',
> > + 'efi-comp': 'gzip' if comp else 'none',
> > + 'fdt-bin': os.path.basename(make_dtb('user', comp)),
> > + 'fdt-comp': 'gzip' if comp else 'none',
> > + }
> > +
> > + # Generate a test ITS file.
> > + its_path = make_fpath('test-efi-fit-helloworld.its')
> > + with open(its_path, 'w') as fd:
> > + fd.write(its_data % its_params)
> > +
> > + # Build the test ITS.
> > + fit_path = make_fpath('test-efi-fit-helloworld.fit')
> > + util.run_and_log(
> > + cons, [make_fpath('tools/mkimage'), '-f', its_path, fit_path])
> > + return fit_path
> > +
> > + def load_fit_from_host(f):
> > + """Load the FIT image using the 'host load' command and return its address.
> > +
> > + Args:
> > + f: Dictionary describing the FIT image to load, see env__efi_fit_test_file
> > + in the comment at the beginning of this file.
> > + Return:
> > + The address where the file has been loaded.
> > + """
> > +
> > + addr = f.get('addr', None)
> > + if not addr:
> > + addr = u_boot_utils.find_ram_base(cons)
>
> %s/u_boot_utils/util/
>
>
> > +
> > + output = cons.run_command(
> > + 'host load hostfs - %s %s/%s' % (addr, f['dn'], f['fn']))
> > + expected_text = ' bytes read'
> > + sz = f.get('size', None)
> > + if sz:
> > + expected_text = '%d' % sz + expected_text
> > + assert(expected_text in output)
> > +
> > + return addr
> > +
> > + def load_fit_from_tftp(f):
> > + """Load the FIT image using the tftpboot command and return its address.
> > +
> > + The file is downloaded from the TFTP server, its size and optionally its
> > + CRC32 are validated.
> > +
> > + Args:
> > + f: Dictionary describing the FIT image to load, see env__efi_fit_tftp_file
> > + in the comment at the beginning of this file.
> > + Return:
> > + The address where the file has been loaded.
> > + """
> > +
> > + addr = f.get('addr', None)
> > + if not addr:
> > + addr = u_boot_utils.find_ram_base(cons)
>
> %s/u_boot_utils/util/
>
>
> > +
> > + fn = f['fn']
> > + output = cons.run_command('tftpboot %s %s' % (addr, fn))
>
> You have to pass addr as hexadecimal number.
>
> output = cons.run_command('tftpboot %x %s' % (addr, fn))
As explained before, I eventually converted 'addr' to string
in order to allow values like '$kernel_addr_r' to be passed from the
environment (this is what I actually used during my local tests).
What I missed was the following:
addr = '%x' % util.find_ram_base(cons)
If this is not desired, I can revert to numbers only.
> > + expected_text = 'Bytes transferred = '
> > + sz = f.get('size', None)
> > + if sz:
> > + expected_text += '%d' % sz
> > + assert expected_text in output
> > +
> > + expected_crc = f.get('crc32', None)
> > + if not expected_crc:
> > + return addr
> > +
> > + if cons.config.buildconfig.get('config_cmd_crc32', 'n') != 'y':
> > + return addr
> > +
> > + output = cons.run_command('crc32 $fileaddr $filesize')
> > + assert expected_crc in output
> > +
> > + return addr
> > +
> > + def launch_efi(enable_fdt, enable_comp):
> > + """Launch U-Boot's helloworld.efi binary from a FIT image.
> > +
> > + An external image file can be downloaded from TFTP, when related
> > + details are provided by the boardenv_* file; see the comment at the
> > + beginning of this file.
> > +
> > + If the size of the TFTP file is not provided within env__efi_fit_tftp_file,
> > + the test image is generated automatically and placed in the TFTP root
> > + directory specified via the 'dn' field.
> > +
> > + When running the tests on Sandbox, the image file is loaded directly
> > + from the host filesystem.
> > +
> > + Once the load address is available on U-Boot console, the 'bootm'
> > + command is executed for either 'config-efi-fdt' or 'config-efi-nofdt'
> > + FIT configuration, depending on the value of the 'enable_fdt' function
> > + argument.
> > +
> > + Eventually the 'Hello, world' message is expected in the U-Boot console.
> > +
> > + Args:
> > + enable_fdt: Flag to enable using the FDT blob inside FIT image.
> > + enable_comp: Flag to enable GZIP compression on EFI and FDT
> > + generated content.
> > + """
> > +
> > + with cons.log.section('FDT=%s;COMP=%s' % (enable_fdt, enable_comp)):
> > + if is_sandbox:
> > + fit = {
> > + 'dn': cons.config.build_dir,
> > + 'addr': '${kernel_addr_r}',
> > + }
> > + else:
> > + # Init networking.
> > + net_pre_commands()
> > + net_set_up = net_dhcp()
> > + net_set_up = net_setup_static() or net_set_up
> > + if not net_set_up:
> > + pytest.skip('Network not initialized')
> > +
> > + fit = cons.config.env.get('env__efi_fit_tftp_file', None)
> > + if not fit:
> > + pytest.skip('No env__efi_fit_tftp_file binary specified in environment')
> > +
> > + sz = fit.get('size', None)
> > + if not sz:
> > + if not fit.get('dn', None):
> > + pytest.skip('Neither "size", nor "dn" info provided in env__efi_fit_tftp_file')
> > +
> > + # Create test FIT image.
> > + fit_path = make_fit(enable_comp)
> > + fit['fn'] = os.path.basename(fit_path)
> > + fit['size'] = os.path.getsize(fit_path)
> > +
> > + # Copy image to TFTP root directory.
> > + if fit['dn'] != cons.config.build_dir:
> > + util.run_and_log(cons, ['mv', '-f', fit_path, '%s/' % fit['dn']])
> > +
> > + # Load FIT image.
> > + addr = load_fit_from_host(fit) if is_sandbox else load_fit_from_tftp(fit)
> > +
> > + # Select boot configuration.
> > + fit_config = 'config-efi-fdt' if enable_fdt else 'config-efi-nofdt'
> > +
> > + # Try booting.
> > + cons.run_command(
> > + 'bootm %s#%s' % (addr, fit_config), wait_for_prompt=False)
>
> You have to pass the address as hexadecimal number.
>
> 'bootm %x#%s' % (addr, fit_config), wait_for_prompt=False)
Please see comments above.
> Best regards
>
> Heinrich
>
> > + if enable_fdt:
> > + cons.wait_for('Booting using the fdt blob')
> > + cons.wait_for('Hello, world')
> > + cons.wait_for('## Application terminated, r = 0')
> > + cons.restart_uboot();
> > +
> > + cons = u_boot_console
> > + # Array slice removes leading/trailing quotes.
> > + sys_arch = cons.config.buildconfig.get('config_sys_arch', '"sandbox"')[1:-1]
> > + is_sandbox = sys_arch == 'sandbox'
> > +
> > + try:
> > + if is_sandbox:
> > + # Use our own device tree file, will be restored afterwards.
> > + control_dtb = make_dtb('internal', False)
> > + old_dtb = cons.config.dtb
> > + cons.config.dtb = control_dtb
> > +
> > + # Run tests
> > + # - fdt OFF, gzip OFF
> > + launch_efi(False, False)
> > + # - fdt ON, gzip OFF
> > + launch_efi(True, False)
> > +
> > + if is_sandbox:
> > + # - fdt OFF, gzip ON
> > + launch_efi(False, True)
> > + # - fdt ON, gzip ON
> > + launch_efi(True, True)
> > +
> > + finally:
> > + if is_sandbox:
> > + # Go back to the original U-Boot with the correct dtb.
> > + cons.config.dtb = old_dtb
> > + cons.restart_uboot()
> >
>
More information about the U-Boot
mailing list