[U-Boot] [RFC 1/3] test/py: convert fs-test.sh to pytest

Thu Aug 30 10:56:41 UTC 2018

Hi Heinrich, Takahiro

On 08/30/2018 01:26 PM, AKASHI Takahiro wrote:
> On Thu, Aug 30, 2018 at 12:01:32PM +0200, Heinrich Schuchardt wrote:
>> On 08/30/2018 08:52 AM, AKASHI Takahiro wrote:
>>> On Wed, Aug 29, 2018 at 11:36:51PM +0200, Heinrich Schuchardt wrote:
>>>> On 08/23/2018 09:25 AM, AKASHI Takahiro wrote:
>>>>> In this commit, the same set of test cases as in test/fs/fs-test.sh
>>>>> is provided using pytest framework.
>>>>> Actually, fs-test.sh provides three variants:"sb" (sb command), "nonfs"
>>>>> (fatxx and etc.) and "fs" (hostfs), and this patch currently supports
>>>>> only "nonfs" variant; So it is not a replacement of fs-test.sh for now.
>>>>>
>>>>> Simple usage:
>>>>>    $ py.test test/py/tests/test_fs [<other options>]
>>>>>
>>>>> You may also specify filesystem types to be tested:
>>>>>    $ py.test test/py/tests/test_fs --fs-type fat32 [<other options>]
>>>>>
>>>>> Signed-off-by: AKASHI Takahiro <takahiro.akashi at linaro.org>
>>>>> ---
>>>>>   test/py/tests/test_fs/conftest.py    | 175 +++++++++++++++++++
>>>>>   test/py/tests/test_fs/fstest_defs.py |  10 ++
>>>>>   test/py/tests/test_fs/test_basic.py  | 246 +++++++++++++++++++++++++++
>>>>>   3 files changed, 431 insertions(+)
>>>>>   create mode 100644 test/py/tests/test_fs/conftest.py
>>>>>   create mode 100644 test/py/tests/test_fs/fstest_defs.py
>>>>>   create mode 100644 test/py/tests/test_fs/test_basic.py
>>>>>
>>>>> diff --git a/test/py/tests/test_fs/conftest.py b/test/py/tests/test_fs/conftest.py
>>>>> new file mode 100644
>>>>> index 000000000000..fefeb4c9663f
>>>>> --- /dev/null
>>>>> +++ b/test/py/tests/test_fs/conftest.py
>>>>> @@ -0,0 +1,175 @@
>>>>> +# SPDX-License-Identifier:      GPL-2.0+
>>>>> +# Copyright (c) 2018, Linaro Limited
>>>>> +# Author: Takahiro Akashi <takahiro.akashi at linaro.org>
>>>>> +
>>>>> +import pytest
>>>>> +import re
>>>>> +from subprocess import call, check_call, check_output, CalledProcessError
>>>>> +from fstest_defs import *
>>>>> +
>>>>> +supported_fs_basic = ['fat16', 'fat32', 'ext4']
>>>>> +
>>>>> +#
>>>>> +# Filesystem test specific setup
>>>>> +#
>>>>> +def pytest_addoption(parser):
>>>>> +    parser.addoption('--fs-type', action='append', default=None,
>>>>> +        help='Targeting Filesystem Types')
>>>>> +
>>>>> +def pytest_configure(config):
>>>>> +    global supported_fs_basic
>>>>> +
>>>>> +    def intersect(listA, listB):
>>>>> +        return  [x for x in listA if x in listB]
>>>>> +
>>>>> +    supported_fs = config.getoption('fs_type')
>>>>> +    if supported_fs:
>>>>> +        print("*** FS TYPE modified: %s" % supported_fs)
>>>>> +        supported_fs_basic =  intersect(supported_fs, supported_fs_basic)
>>>>> +
>>>>> +def pytest_generate_tests(metafunc):
>>>>> +    if 'fs_obj_basic' in metafunc.fixturenames:
>>>>> +        metafunc.parametrize('fs_obj_basic', supported_fs_basic,
>>>>> +            indirect=True, scope='module')
>>>>> +
>>>>> +#
>>>>> +# Helper functions
>>>>> +#
>>>>> +def fstype_to_ubname(fs_type):
>>>>> +    if re.match('fat', fs_type):
>>>>> +        return 'fat'
>>>>> +    else:
>>>>> +        return fs_type
>>>>> +
>>>>> +def check_ubconfig(config, fs_type):
>>>>> +    if not config.buildconfig.get('config_cmd_%s' % fs_type, None):
>>>>> +        pytest.skip('.config feature "CMD_%s" not enabled' % fs_type.upper())
>>>>> +    if not config.buildconfig.get('config_%s_write' % fs_type, None):
>>>>> +        pytest.skip('.config feature "%s_WRITE" not enabled'
>>>>> +        % fs_type.upper())
>>>>> +
>>>>> +def mk_fs(config, fs_type, size, id):
>>>>> +    fs_img = '%s.%s.img' % (id, fs_type)
>>>>> +    fs_img = config.persistent_data_dir + '/' + fs_img
>>>>> +
>>>>> +    if fs_type == 'fat16':
>>>>> +        mkfs_opt = '-F 16'
>>>>> +    elif fs_type == 'fat32':
>>>>> +        mkfs_opt = '-F 32'
>>>>> +    else:
>>>>> +        mkfs_opt = ''
>>>>> +
>>>>> +    if re.match('fat', fs_type):
>>>>> +        fs_lnxtype = 'vfat'
>>>>> +    else:
>>>>> +        fs_lnxtype = fs_type
>>>>> +
>>>>> +    count = (size + 1023) / 1024
>>>>> +
>>>>> +    try:
>>>>> +        check_call('rm -f %s' % fs_img, shell=True)
>>>>> +        check_call('dd if=/dev/zero of=%s bs=1K count=%d'
>>>>> +            % (fs_img, count), shell=True)
>>>>> +        check_call('mkfs.%s %s %s'
>>>>> +            % (fs_lnxtype, mkfs_opt, fs_img), shell=True)
>>>>> +        return fs_img
>>>>> +    except CalledProcessError:
>>>>> +        call('rm -f %s' % fs_img, shell=True)
>>>>> +        raise
>>>>> +
>>>>> +#
>>>>> +# Fixture for basic fs test
>>>>> +#     derived from test/fs/fs-test.sh
>>>>> +#
>>>>> +# NOTE: yield_fixture was deprecated since pytest-3.0
>>>>> + at pytest.yield_fixture()
>>>>> +def fs_obj_basic(request, u_boot_config):
>>>>> +    fs_type = request.param
>>>>> +    fs_img = ''
>>>>> +
>>>>> +    fs_ubtype = fstype_to_ubname(fs_type)
>>>>> +    check_ubconfig(u_boot_config, fs_ubtype)
>>>>> +
>>>>> +    mount_dir = u_boot_config.persistent_data_dir + '/mnt'
>>>>> +    small_file = mount_dir + '/' + SMALL_FILE
>>>>> +    big_file = mount_dir + '/' + BIG_FILE
>>>>> +    try:
>>>>> +
>>>>> +        # 3GiB volume
>>>>> +        fs_img = mk_fs(u_boot_config, fs_type, 0xc0000000, '3GB')
>>>>> +
>>>>> +        # Mount the image so we can populate it.
>>>>> +        check_call('mkdir -p %s' % mount_dir, shell=True)
>>>>> +        check_call('sudo mount -o loop,rw %s %s'
>>>>> +            % (fs_img, mount_dir), shell=True)
>>>>
>>>> Should I grant sudo to anybody who can commit to U-Boot?
>>>
>>> I don't get your point.
>>> I think using "sudo" solely in testing should be allowed.
>>>
>>>> Just use exfat-fuse and fuse2fs.
>>>
>>> It will not be a good idea to use those tools which are not
>>> provided in every distribution.
>>> I'd like to leave "sudo mount" statement as a backstop.
>>
>> Fuse is a base functionality of Linux. On Travis we are using the Ubuntu
>> distribution which contains said fuse file systems. Which distribuition
>> does not have it?
>>
>> Using sudo for me is a NO-NO. I will not run any test that uses sudo.
> 
> So this means that you have never tested file system using test-fs.sh.
> 
> Since my script is logically "general", it can, if we want, run against
> other file systems as well. "sudo mount" is the only solution for those cases.
> 
There are two general non-root implementations that I know of:

1) http://libguestfs.org/ which IIRC launches a small Linux VM in QEMU
    to do the filesystem accesses. I am not sure if the performance would be
    acceptable without KVM (which I assume we don't have in Travis).

2) https://github.com/lkl/linux which is a port of Linux to run in userspace
    as a library. It comes with tools like cptofs and lklfuse to access any
    filesystem Linux has a driver for. Sadly lkl isn't packaged in many distros.

- Tuomas