[U-Boot] [PATCH 14/14] test: Convert the vboot test to test/py

Teddy Reed teddy.reed at gmail.com
Sun Jul 3 23:38:48 CEST 2016


Hi Simon,

On Sun, Jul 3, 2016 at 8:40 AM, Simon Glass <sjg at chromium.org> wrote:
> Now that we have a suitable test framework we should move all tests into it.
> The vboot test is a suitable candidate. Rewrite it in Python and move the
> data files into an appropriate directory.
>
> Signed-off-by: Simon Glass <sjg at chromium.org>
> ---
>
>  test/README                                       |   1 -
>  test/py/tests/test_vboot.py                       | 185 ++++++++++++++++++++++
>  test/{ => py/tests}/vboot/sandbox-kernel.dts      |   0
>  test/{ => py/tests}/vboot/sandbox-u-boot.dts      |   0
>  test/{ => py/tests}/vboot/sign-configs-sha1.its   |   0
>  test/{ => py/tests}/vboot/sign-configs-sha256.its |   0
>  test/{ => py/tests}/vboot/sign-images-sha1.its    |   0
>  test/{ => py/tests}/vboot/sign-images-sha256.its  |   0
>  test/vboot/.gitignore                             |   3 -
>  test/vboot/vboot_test.sh                          | 151 ------------------
>  10 files changed, 185 insertions(+), 155 deletions(-)
>  create mode 100644 test/py/tests/test_vboot.py
>  rename test/{ => py/tests}/vboot/sandbox-kernel.dts (100%)
>  rename test/{ => py/tests}/vboot/sandbox-u-boot.dts (100%)
>  rename test/{ => py/tests}/vboot/sign-configs-sha1.its (100%)
>  rename test/{ => py/tests}/vboot/sign-configs-sha256.its (100%)
>  rename test/{ => py/tests}/vboot/sign-images-sha1.its (100%)
>  rename test/{ => py/tests}/vboot/sign-images-sha256.its (100%)
>  delete mode 100644 test/vboot/.gitignore
>  delete mode 100755 test/vboot/vboot_test.sh
>
> diff --git a/test/README b/test/README
> index 6cfee05..ee55972 100644
> --- a/test/README
> +++ b/test/README
> @@ -58,7 +58,6 @@ There are several ad-hoc tests which run outside the pytest environment:
>     test/image  - FIT and lagacy image tests (shell script and Python)
>     test/stdint - A test that stdint.h can be used in U-Boot (shell script)
>     trace       - Test for the tracing feature (shell script)
> -   vboot       - Test for verified boot (shell script)
>
>  The above should be converted to run as part of the pytest suite.
>
> diff --git a/test/py/tests/test_vboot.py b/test/py/tests/test_vboot.py
> new file mode 100644
> index 0000000..c779895
> --- /dev/null
> +++ b/test/py/tests/test_vboot.py
> @@ -0,0 +1,185 @@
> +# Copyright (c) 2013, Google Inc.
> +#
> +# SPDX-License-Identifier:     GPL-2.0+
> +#
> +# U-Boot Verified Boot Test
> +
> +"""
> +This tests verified boot in the following ways:
> +
> +For image verification:
> +- Create FIT (unsigned) with mkimage
> +- Check that verification shows that no keys are verified
> +- Sign image
> +- Check that verification shows that a key is now verified
> +
> +For configuration verification:
> +- Corrupt signature and check for failure
> +- Create FIT (with unsigned configuration) with mkimage
> +- Check that image veriication works

verification

> +- Sign the FIT and mark the key as 'required' for verification
> +- Check that image verification works
> +- Corrupt the signature
> +- Check that image verification no-longer works
> +
> +Tests run with both SHA1 and SHA256 hashing.
> +"""
> +
> +import pytest
> +import sys
> +import u_boot_utils as util
> +
> + at pytest.mark.buildconfigspec('fit_signature')
> +def test_vboot(u_boot_console):
> +    """Test verified boot signing with mkimage and verification with 'bootm'.
> +
> +    This works using sandbox only as it needs to update the device tree used
> +    by U-Boot to hold public keys from the signing process.
> +
> +    The SHA1 and SHA256 tests are combined into a single test since the
> +    key-generation process is quite slow and we want to avoid doing it twice.
> +    """
> +    def dtc(dts):
> +        """Run the device-tree compiler to compile a .dts file

In a few other places it reads: "device tree" without a dash. ;)

> +
> +        The output file will be the same as the input file but with a .dtb
> +        extension.
> +
> +        Args:
> +            dts: Device tree file to compile.
> +        """
> +        dtb = dts.replace('.dts', '.dtb')
> +        util.cmd(cons, 'dtc %s %s%s -O dtb '
> +                       '-o %s%s' % (dtc_args, datadir, dts, tmpdir, dtb))
> +
> +    def run_bootm(test_type, expect_string):
> +        """Run a 'bootm' command U-Boot.
> +
> +        This always starts a fresh U-Boot instance since the device tree may
> +        contain a new public key.
> +
> +        Args:
> +            test_type: A string identifying the test type
> +            expect_string: A string which is expected in the output

nit, punctuation.

> +        """
> +        cons.cleanup_spawn()
> +        cons.ensure_spawned()
> +        cons.log.action('%s: Test Verified Boot Run: %s' % (algo, test_type))
> +        output = cons.run_command_list(
> +            ['sb load hostfs - 100 %stest.fit' % tmpdir,
> +             'fdt addr 100',
> +             'bootm 100'])
> +        assert(expect_string in output)
> +
> +    def make_fit(its):
> +        """Make a new FIT from the .its source file
> +
> +        This runs 'mkimage -f' to create a new FIT.
> +
> +        Args:
> +            its: Filename containing .its source

nit, same

> +        """
> +        util.run_and_log(cons, [mkimage, '-D', dtc_args, '-f',
> +                                '%s%s' % (datadir, its), fit])
> +
> +    def sign_fit():
> +        """Sign the FIT
> +
> +        Signs the FIT and writes the signature into it. It also writes the
> +        public key into the dtb.
> +        """
> +        cons.log.action('%s: Sign images' % algo)
> +        util.run_and_log(cons, [mkimage, '-F', '-k', tmpdir, '-K', dtb,
> +                                '-r', fit])
> +
> +    def test_with_algo(sha):

nit, maybe sha_algo?

> +        """Test verified boot with the given hash algorithm

nit, same

> +
> +        This is the main part of the test code. The same procedure is followed
> +        for both hashing algorithms.
> +
> +        Args:
> +            sha: Either 'sha1' or 'sha256', to select the algorithm to use
> +        """
> +        global algo
> +
> +        algo = sha
> +
> +        # Compile our device tree files for kernel and U-Boot
> +        dtc('sandbox-kernel.dts')
> +        dtc('sandbox-u-boot.dts')
> +
> +        # Build the FIT, but don't sign anything yet
> +        cons.log.action('%s: Test FIT with signed images' % algo)
> +        make_fit('sign-images-%s.its' % algo)
> +        run_bootm('unsigned images', 'dev-')
> +
> +        # Sign images with our dev keys
> +        sign_fit()
> +        run_bootm('signed images', 'dev+')
> +
> +        # Create a fresh .dtb without the public keys
> +        dtc('sandbox-u-boot.dts')
> +
> +        cons.log.action('%s: Test FIT with signed configuration' % algo)
> +        make_fit('sign-configs-%s.its' % algo)
> +        run_bootm('unsigned config', '%s+ OK' % algo)
> +
> +        # Sign images with our dev keys
> +        sign_fit()
> +        run_bootm('signed config', 'dev+')
> +
> +        cons.log.action('%s: Check signed config on the host' % algo)
> +
> +        util.run_and_log(cons, [fit_check_sign, '-f', fit, '-k', tmpdir,
> +                                '-k', dtb])
> +
> +        # Increment the first byte of the signature, which should cause failure
> +        sig = util.cmd(cons, 'fdtget -t bx %s %s value' % (fit, sig_node))
> +        byte_list = sig.split()
> +        byte = int(byte_list[0], 16)
> +        byte_list = ['%x' % (byte + 1)] + byte_list[1:]
> +        sig = ' '.join(byte_list)
> +        util.cmd(cons, 'fdtput -t bx %s %s value %s' % (fit, sig_node, sig))
> +
> +        run_bootm('Signed config with bad hash', 'Bad Data Hash')
> +
> +        cons.log.action('%s: Check bad config on the host' % algo)
> +        util.run_and_log_expect_exception(cons, [fit_check_sign, '-f', fit,
> +                '-k', dtb], 1, 'Failed to verify required signature')
> +
> +    cons = u_boot_console
> +    tmpdir = cons.config.result_dir + '/'
> +    tmp = tmpdir + 'vboot.tmp'

Is there a need to clean these up afterward?

Python's tempfile might be helpful, you can supply the result_dir as
the directory prefix.

> +    datadir = 'test/py/tests/vboot/'
> +    fit = '%stest.fit' % tmpdir
> +    mkimage = cons.config.build_dir + '/tools/mkimage'
> +    fit_check_sign = cons.config.build_dir + '/tools/fit_check_sign'
> +    dtc_args = '-I dts -O dtb -i %s' % tmpdir
> +    dtb = '%ssandbox-u-boot.dtb' % tmpdir
> +    sig_node = '/configurations/conf at 1/signature at 1'

If these variables are used throughout the tests like globals, should
they be DATADIR, MKIMAGE, etc?

> +
> +    # Create an RSA key pair
> +    public_exponent = 65537
> +    util.cmd(cons, 'openssl genpkey -algorithm RSA -out %sdev.key '
> +                   '-pkeyopt rsa_keygen_bits:2048 '
> +                   '-pkeyopt rsa_keygen_pubexp:%d '
> +                   '2>/dev/null'  % (tmpdir, public_exponent))
> +
> +    # Create a certificate containing the public key
> +    util.cmd(cons, 'openssl req -batch -new -x509 -key %sdev.key -out '
> +                   '%sdev.crt' % (tmpdir, tmpdir))
> +
> +    # Create a number kernel image with zeroes
> +    with open('%stest-kernel.bin' % tmpdir, 'w') as fd:
> +        fd.write(5000 * chr(0))
> +

Any need to clean these up afterward or place them into
test-run-unique directories? Is the expectation that subsequent tests
will overwrite existing test data (also that vboot tests cannot run
concurrently)?

> +    try:
> +        # We need to use our own device tree file. Remember to restore it
> +        # afterwards.
> +        old_dtb = cons.config.dtb
> +        cons.config.dtb = dtb
> +        test_with_algo('sha1')
> +        test_with_algo('sha256')
> +    finally:
> +        cons.config.dtb = old_dtb
> diff --git a/test/vboot/sandbox-kernel.dts b/test/py/tests/vboot/sandbox-kernel.dts
> similarity index 100%
> rename from test/vboot/sandbox-kernel.dts
> rename to test/py/tests/vboot/sandbox-kernel.dts
> diff --git a/test/vboot/sandbox-u-boot.dts b/test/py/tests/vboot/sandbox-u-boot.dts
> similarity index 100%
> rename from test/vboot/sandbox-u-boot.dts
> rename to test/py/tests/vboot/sandbox-u-boot.dts
> diff --git a/test/vboot/sign-configs-sha1.its b/test/py/tests/vboot/sign-configs-sha1.its
> similarity index 100%
> rename from test/vboot/sign-configs-sha1.its
> rename to test/py/tests/vboot/sign-configs-sha1.its
> diff --git a/test/vboot/sign-configs-sha256.its b/test/py/tests/vboot/sign-configs-sha256.its
> similarity index 100%
> rename from test/vboot/sign-configs-sha256.its
> rename to test/py/tests/vboot/sign-configs-sha256.its
> diff --git a/test/vboot/sign-images-sha1.its b/test/py/tests/vboot/sign-images-sha1.its
> similarity index 100%
> rename from test/vboot/sign-images-sha1.its
> rename to test/py/tests/vboot/sign-images-sha1.its
> diff --git a/test/vboot/sign-images-sha256.its b/test/py/tests/vboot/sign-images-sha256.its
> similarity index 100%
> rename from test/vboot/sign-images-sha256.its
> rename to test/py/tests/vboot/sign-images-sha256.its
> diff --git a/test/vboot/.gitignore b/test/vboot/.gitignore
> deleted file mode 100644
> index 4631242..0000000
> --- a/test/vboot/.gitignore
> +++ /dev/null
> @@ -1,3 +0,0 @@
> -/*.dtb
> -/test.fit
> -/dev-keys
> diff --git a/test/vboot/vboot_test.sh b/test/vboot/vboot_test.sh
> deleted file mode 100755
> index 6d7abb8..0000000
> --- a/test/vboot/vboot_test.sh
> +++ /dev/null
> @@ -1,151 +0,0 @@
> -#!/bin/bash
> -#
> -# Copyright (c) 2013, Google Inc.
> -#
> -# Simple Verified Boot Test Script
> -#
> -# SPDX-License-Identifier:     GPL-2.0+
> -
> -set -e
> -
> -# Run U-Boot and report the result
> -# Args:
> -#      $1:     Test message
> -run_uboot() {
> -       echo -n "Test Verified Boot Run: $1: "
> -       ${uboot} -d sandbox-u-boot.dtb >${tmp} -c '
> -sb load hostfs - 100 test.fit;
> -fdt addr 100;
> -bootm 100;
> -reset'
> -       if ! grep -q "$2" ${tmp}; then
> -               echo
> -               echo "Verified boot key check failed, output follows:"
> -               cat ${tmp}
> -               false
> -       else
> -               echo "OK"
> -       fi
> -}
> -
> -echo "Simple Verified Boot Test"
> -echo "========================="
> -echo
> -echo "Please see doc/uImage.FIT/verified-boot.txt for more information"
> -echo
> -
> -err=0
> -tmp=/tmp/vboot_test.$$
> -
> -dir=$(dirname $0)
> -
> -if [ -z ${O} ]; then
> -       O=.
> -fi
> -O=$(readlink -f ${O})
> -
> -dtc="-I dts -O dtb -p 2000"
> -uboot="${O}/u-boot"
> -mkimage="${O}/tools/mkimage"
> -fit_check_sign="${O}/tools/fit_check_sign"
> -keys="${dir}/dev-keys"
> -echo ${mkimage} -D "${dtc}"
> -
> -echo "Build keys"
> -mkdir -p ${keys}
> -
> -PUBLIC_EXPONENT=${1}
> -
> -if [ -z "${PUBLIC_EXPONENT}" ]; then
> -       PUBLIC_EXPONENT=65537
> -fi
> -
> -# Create an RSA key pair
> -openssl genpkey -algorithm RSA -out ${keys}/dev.key \
> -    -pkeyopt rsa_keygen_bits:2048 \
> -    -pkeyopt rsa_keygen_pubexp:${PUBLIC_EXPONENT} 2>/dev/null
> -
> -# Create a certificate containing the public key
> -openssl req -batch -new -x509 -key ${keys}/dev.key -out ${keys}/dev.crt
> -
> -pushd ${dir} >/dev/null
> -
> -function do_test {
> -       echo do $sha test
> -       # Compile our device tree files for kernel and U-Boot
> -       dtc -p 0x1000 sandbox-kernel.dts -O dtb -o sandbox-kernel.dtb
> -       dtc -p 0x1000 sandbox-u-boot.dts -O dtb -o sandbox-u-boot.dtb
> -
> -       # Create a number kernel image with zeroes
> -       head -c 5000 /dev/zero >test-kernel.bin
> -
> -       # Build the FIT, but don't sign anything yet
> -       echo Build FIT with signed images
> -       ${mkimage} -D "${dtc}" -f sign-images-$sha.its test.fit >${tmp}
> -
> -       run_uboot "unsigned signatures:" "dev-"
> -
> -       # Sign images with our dev keys
> -       echo Sign images
> -       ${mkimage} -D "${dtc}" -F -k dev-keys -K sandbox-u-boot.dtb \
> -               -r test.fit >${tmp}
> -
> -       run_uboot "signed images" "dev+"
> -
> -
> -       # Create a fresh .dtb without the public keys
> -       dtc -p 0x1000 sandbox-u-boot.dts -O dtb -o sandbox-u-boot.dtb
> -
> -       echo Build FIT with signed configuration
> -       ${mkimage} -D "${dtc}" -f sign-configs-$sha.its test.fit >${tmp}
> -
> -       run_uboot "unsigned config" $sha"+ OK"
> -
> -       # Sign images with our dev keys
> -       echo Sign images
> -       ${mkimage} -D "${dtc}" -F -k dev-keys -K sandbox-u-boot.dtb \
> -               -r test.fit >${tmp}
> -
> -       run_uboot "signed config" "dev+"
> -
> -       echo check signed config on the host
> -       if ! ${fit_check_sign} -f test.fit -k sandbox-u-boot.dtb >${tmp}; then
> -               echo
> -               echo "Verified boot key check on host failed, output follows:"
> -               cat ${tmp}
> -               false
> -       else
> -               if ! grep -q "dev+" ${tmp}; then
> -                       echo
> -                       echo "Verified boot key check failed, output follows:"
> -                       cat ${tmp}
> -                       false
> -               else
> -                       echo "OK"
> -               fi
> -       fi
> -
> -       run_uboot "signed config" "dev+"
> -
> -       # Increment the first byte of the signature, which should cause failure
> -       sig=$(fdtget -t bx test.fit /configurations/conf at 1/signature at 1 value)
> -       newbyte=$(printf %x $((0x${sig:0:2} + 1)))
> -       sig="${newbyte} ${sig:2}"
> -       fdtput -t bx test.fit /configurations/conf at 1/signature at 1 value ${sig}
> -
> -       run_uboot "signed config with bad hash" "Bad Data Hash"
> -}
> -
> -sha=sha1
> -do_test
> -sha=sha256
> -do_test
> -
> -popd >/dev/null
> -
> -echo
> -if ${ok}; then
> -       echo "Test passed"
> -else
> -       echo "Test failed"
> -fi
> --
> 2.8.0.rc3.226.g39d4020
>
> _______________________________________________
> U-Boot mailing list
> U-Boot at lists.denx.de
> http://lists.denx.de/mailman/listinfo/u-boot

Thanks for this refactor! If the comments related to the sh to Python
are too nit-picky we can certainly change and expand the test harness
within additional patches later.

-- 
Teddy Reed V


More information about the U-Boot mailing list