[U-Boot] [RFC 1/3] test/py: convert fs-test.sh to pytest

[AKASHI Takahiro]

On Thu, Aug 30, 2018 at 12:01:32PM +0200, Heinrich Schuchardt wrote:
> On 08/30/2018 08:52 AM, AKASHI Takahiro wrote:
> > On Wed, Aug 29, 2018 at 11:36:51PM +0200, Heinrich Schuchardt wrote:
> >> On 08/23/2018 09:25 AM, AKASHI Takahiro wrote:
> >>> In this commit, the same set of test cases as in test/fs/fs-test.sh
> >>> is provided using pytest framework.
> >>> Actually, fs-test.sh provides three variants:"sb" (sb command), "nonfs"
> >>> (fatxx and etc.) and "fs" (hostfs), and this patch currently supports
> >>> only "nonfs" variant; So it is not a replacement of fs-test.sh for now.
> >>>
> >>> Simple usage:
> >>>   $ py.test test/py/tests/test_fs [<other options>]
> >>>
> >>> You may also specify filesystem types to be tested:
> >>>   $ py.test test/py/tests/test_fs --fs-type fat32 [<other options>]
> >>>
> >>> Signed-off-by: AKASHI Takahiro <takahiro.akashi at linaro.org>
> >>> ---
> >>>  test/py/tests/test_fs/conftest.py    | 175 +++++++++++++++++++
> >>>  test/py/tests/test_fs/fstest_defs.py |  10 ++
> >>>  test/py/tests/test_fs/test_basic.py  | 246 +++++++++++++++++++++++++++
> >>>  3 files changed, 431 insertions(+)
> >>>  create mode 100644 test/py/tests/test_fs/conftest.py
> >>>  create mode 100644 test/py/tests/test_fs/fstest_defs.py
> >>>  create mode 100644 test/py/tests/test_fs/test_basic.py
> >>>
> >>> diff --git a/test/py/tests/test_fs/conftest.py b/test/py/tests/test_fs/conftest.py
> >>> new file mode 100644
> >>> index 000000000000..fefeb4c9663f
> >>> --- /dev/null
> >>> +++ b/test/py/tests/test_fs/conftest.py
> >>> @@ -0,0 +1,175 @@
> >>> +# SPDX-License-Identifier:      GPL-2.0+
> >>> +# Copyright (c) 2018, Linaro Limited
> >>> +# Author: Takahiro Akashi <takahiro.akashi at linaro.org>
> >>> +
> >>> +import pytest
> >>> +import re
> >>> +from subprocess import call, check_call, check_output, CalledProcessError
> >>> +from fstest_defs import *
> >>> +
> >>> +supported_fs_basic = ['fat16', 'fat32', 'ext4']
> >>> +
> >>> +#
> >>> +# Filesystem test specific setup
> >>> +#
> >>> +def pytest_addoption(parser):
> >>> +    parser.addoption('--fs-type', action='append', default=None,
> >>> +        help='Targeting Filesystem Types')
> >>> +
> >>> +def pytest_configure(config):
> >>> +    global supported_fs_basic
> >>> +
> >>> +    def intersect(listA, listB):
> >>> +        return  [x for x in listA if x in listB]
> >>> +
> >>> +    supported_fs = config.getoption('fs_type')
> >>> +    if supported_fs:
> >>> +        print("*** FS TYPE modified: %s" % supported_fs)
> >>> +        supported_fs_basic =  intersect(supported_fs, supported_fs_basic)
> >>> +
> >>> +def pytest_generate_tests(metafunc):
> >>> +    if 'fs_obj_basic' in metafunc.fixturenames:
> >>> +        metafunc.parametrize('fs_obj_basic', supported_fs_basic,
> >>> +            indirect=True, scope='module')
> >>> +
> >>> +#
> >>> +# Helper functions
> >>> +#
> >>> +def fstype_to_ubname(fs_type):
> >>> +    if re.match('fat', fs_type):
> >>> +        return 'fat'
> >>> +    else:
> >>> +        return fs_type
> >>> +
> >>> +def check_ubconfig(config, fs_type):
> >>> +    if not config.buildconfig.get('config_cmd_%s' % fs_type, None):
> >>> +        pytest.skip('.config feature "CMD_%s" not enabled' % fs_type.upper())
> >>> +    if not config.buildconfig.get('config_%s_write' % fs_type, None):
> >>> +        pytest.skip('.config feature "%s_WRITE" not enabled'
> >>> +        % fs_type.upper())
> >>> +
> >>> +def mk_fs(config, fs_type, size, id):
> >>> +    fs_img = '%s.%s.img' % (id, fs_type)
> >>> +    fs_img = config.persistent_data_dir + '/' + fs_img
> >>> +
> >>> +    if fs_type == 'fat16':
> >>> +        mkfs_opt = '-F 16'
> >>> +    elif fs_type == 'fat32':
> >>> +        mkfs_opt = '-F 32'
> >>> +    else:
> >>> +        mkfs_opt = ''
> >>> +
> >>> +    if re.match('fat', fs_type):
> >>> +        fs_lnxtype = 'vfat'
> >>> +    else:
> >>> +        fs_lnxtype = fs_type
> >>> +
> >>> +    count = (size + 1023) / 1024
> >>> +
> >>> +    try:
> >>> +        check_call('rm -f %s' % fs_img, shell=True)
> >>> +        check_call('dd if=/dev/zero of=%s bs=1K count=%d'
> >>> +            % (fs_img, count), shell=True)
> >>> +        check_call('mkfs.%s %s %s'
> >>> +            % (fs_lnxtype, mkfs_opt, fs_img), shell=True)
> >>> +        return fs_img
> >>> +    except CalledProcessError:
> >>> +        call('rm -f %s' % fs_img, shell=True)
> >>> +        raise
> >>> +
> >>> +#
> >>> +# Fixture for basic fs test
> >>> +#     derived from test/fs/fs-test.sh
> >>> +#
> >>> +# NOTE: yield_fixture was deprecated since pytest-3.0
> >>> + at pytest.yield_fixture()
> >>> +def fs_obj_basic(request, u_boot_config):
> >>> +    fs_type = request.param
> >>> +    fs_img = ''
> >>> +
> >>> +    fs_ubtype = fstype_to_ubname(fs_type)
> >>> +    check_ubconfig(u_boot_config, fs_ubtype)
> >>> +
> >>> +    mount_dir = u_boot_config.persistent_data_dir + '/mnt'
> >>> +    small_file = mount_dir + '/' + SMALL_FILE
> >>> +    big_file = mount_dir + '/' + BIG_FILE
> >>> +    try:
> >>> +
> >>> +        # 3GiB volume
> >>> +        fs_img = mk_fs(u_boot_config, fs_type, 0xc0000000, '3GB')
> >>> +
> >>> +        # Mount the image so we can populate it.
> >>> +        check_call('mkdir -p %s' % mount_dir, shell=True)
> >>> +        check_call('sudo mount -o loop,rw %s %s'
> >>> +            % (fs_img, mount_dir), shell=True)
> >>
> >> Should I grant sudo to anybody who can commit to U-Boot?
> > 
> > I don't get your point.
> > I think using "sudo" solely in testing should be allowed.
> > 
> >> Just use exfat-fuse and fuse2fs.
> > 
> > It will not be a good idea to use those tools which are not
> > provided in every distribution.
> > I'd like to leave "sudo mount" statement as a backstop.
> 
> Fuse is a base functionality of Linux. On Travis we are using the Ubuntu
> distribution which contains said fuse file systems. Which distribuition
> does not have it?
> 
> Using sudo for me is a NO-NO. I will not run any test that uses sudo.

So this means that you have never tested file system using test-fs.sh.

Since my script is logically "general", it can, if we want, run against
other file systems as well. "sudo mount" is the only solution for those cases.

-Takahiro AKASHI


> Best regards
> 
> Heinrich