[RFC] Start using guestfish for U-Boot fs tests

[Tom Rini]

On Fri, Jul 02, 2021 at 01:06:23PM -0600, Simon Glass wrote:
> Hi Tom,
> 
> On Fri, 2 Jul 2021 at 13:01, Tom Rini <trini at konsulko.com> wrote:
> >
> > Hey all,
> >
> > I started taking a look at moving to guestfish to see if this resolves
> > the latest problem I've run in to:
> > https://source.denx.de/u-boot/u-boot/-/jobs/284763#L307
> > which I think is due to guestmount not being done in time for the test.
> > So I started converting things to use guestfish directly:
> >
> > diff --git a/test/py/tests/test_fs/conftest.py b/test/py/tests/test_fs/conftest.py
> > index 7325486cdb1a..e8899cfdd118 100644
> > --- a/test/py/tests/test_fs/conftest.py
> > +++ b/test/py/tests/test_fs/conftest.py
> > @@ -265,10 +265,10 @@ def fs_obj_basic(request, u_boot_config):
> >      fs_ubtype = fstype_to_ubname(fs_type)
> >      check_ubconfig(u_boot_config, fs_ubtype)
> >
> > -    mount_dir = u_boot_config.persistent_data_dir + '/mnt'
> > +    data_dir = u_boot_config.persistent_data_dir + '/data'
> >
> > -    small_file = mount_dir + '/' + SMALL_FILE
> > -    big_file = mount_dir + '/' + BIG_FILE
> > +    small_file = data_dir + '/' + SMALL_FILE
> > +    big_file = data_dir + '/' + BIG_FILE
> >
> >      try:
> >
> > @@ -279,26 +279,14 @@ def fs_obj_basic(request, u_boot_config):
> >          return
> >
> >      try:
> > -        check_call('mkdir -p %s' % mount_dir, shell=True)
> > +        check_call('mkdir -p %s' % data_dir, shell=True)
> >      except CalledProcessError as err:
> >          pytest.skip('Preparing mount folder failed for filesystem: ' + fs_type + '. {}'.format(err))
> >          call('rm -f %s' % fs_img, shell=True)
> >          return
> >
> >      try:
> > -        # Mount the image so we can populate it.
> > -        mount_fs(fs_type, fs_img, mount_dir)
> > -    except CalledProcessError as err:
> > -        pytest.skip('Mounting to folder failed for filesystem: ' + fs_type + '. {}'.format(err))
> > -        call('rmdir %s' % mount_dir, shell=True)
> > -        call('rm -f %s' % fs_img, shell=True)
> > -        return
> > -
> > -    try:
> > -        # Create a subdirectory.
> > -        check_call('mkdir %s/SUBDIR' % mount_dir, shell=True)
> > -
> > -        # Create big file in this image.
> > +        # Create big file to copy in to the image.
> >          # Note that we work only on the start 1MB, couple MBs in the 2GB range
> >          # and the last 1 MB of the huge 2.5GB file.
> >          # So, just put random values only in those areas.
> > @@ -309,10 +297,14 @@ def fs_obj_basic(request, u_boot_config):
> >          check_call('dd if=/dev/urandom of=%s bs=1M count=1 seek=2499'
> >              % big_file, shell=True)
> >
> > -        # Create a small file in this image.
> > +        # Create a small file to copy in to the image.
> >          check_call('dd if=/dev/urandom of=%s bs=1M count=1'
> >             % small_file, shell=True)
> >
> > +        # Copy the files in to the image and add a subdirectory.
> > +        # Create a subdirectory.
> > +        check_call('guestfish add %s : run : mount /dev/sda / : mkdir /SUBDIR : copy-in %s %s /'
> > +            % (fs_img, big_file, small_file), shell=True)
> >          # Delete the small file copies which possibly are written as part of a
> >          # previous test.
> >          # check_call('rm -f "%s.w"' % MB1, shell=True)
> > @@ -357,13 +349,11 @@ def fs_obj_basic(request, u_boot_config):
> >
> >      except CalledProcessError as err:
> >          pytest.skip('Setup failed for filesystem: ' + fs_type + '. {}'.format(err))
> > -        umount_fs(mount_dir)
> >          return
> >      else:
> > -        umount_fs(mount_dir)
> >          yield [fs_ubtype, fs_img, md5val]
> >      finally:
> > -        call('rmdir %s' % mount_dir, shell=True)
> > +        call('rmdir %s' % data_dir, shell=True)
> >          call('rm -f %s' % fs_img, shell=True)
> >
> >  #
> >
> > The problem here is that a test run went from taking about 5 minutes to
> > taking about 17 minutes.  I can reduce this to closer to 15 minutes with
> > LIBGUESTFS_BACKEND=direct and using libguestfs-make-fixed-appliance to
> > make an appliance we reuse.  But that's still too long to be usable.
> > I'm hoping someone has some ideas here on how to improve things.
> 
> Well even 5 minutes is too long. Could we make the filesystems and
> files very small? It does not seem useful to make 2GB things.

Adding "not fs" brings me down to 2 minutes.  Doing "not fs and not efi"
(because I can see the efi secboot stuff taking visible time) brings me
to 1 minute.  Adding in "and not vboot" is 36 seconds.

That said, there's only a subset of the tests that we could turn in to
"use these static images" as we've also got tests to generate new signed
content and verify the signatures, and that's got to test both the sign
and verify paths.

> Also we could try harder to get the tests running in parallel. It is
> not that far away.

This, along with easier / clearer documentation on how to run tests
directly so that you can pass "not fs and not ..." might help too.  I
yanked this out of .gitlab-ci.yml today to make iterating on this under
docker easier, and have a slightly more robust one in my bin dir:

export TEST_PY_BD=sandbox;export UBOOT_TRAVIS_BUILD_DIR=/tmp/${TEST_PY_BD}
tools/buildman/buildman -o ${UBOOT_TRAVIS_BUILD_DIR} -w -E -W -e --board ${TEST_PY_BD}
virtualenv -p /usr/bin/python3 /tmp/venv
. /tmp/venv/bin/activate
pip install -r test/py/requirements.txt
export PATH=/opt/qemu/bin:/tmp/uboot-test-hooks/bin:${PATH}
export PYTHONPATH=/tmp/uboot-test-hooks/py/travis-ci
time ./test/py/test.py -ra --bd ${TEST_PY_BD} --build-dir "$UBOOT_TRAVIS_BUILD_DIR"

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20210702/9283ff37/attachment.sig>