[U-Boot] [RFC PATCH] test: py: Disable sleep test for qemu targets

[Heinrich Schuchardt]

On 12/01/2017 03:46 PM, Michal Simek wrote:
> Qemu for arm32/arm64 has a problem with time setup.

Wouldn't it be preferable to fix the root cause?

> That's why sleep test is failing. Add boardidentity marker to remove
> specific boards from running that test.

Isn't this what 'boardspec' is used for?

test/py/pytest.ini:
boardspec: U-Boot: Describes the set of boards a test can/can't run on.

> 
> Signed-off-by: Michal Simek <michal.simek at xilinx.com>
> ---
> 
> If you want to add this checking in one patch and then put it to one
> test then it is fine for me.
> 
> ---
>   test/py/conftest.py         | 28 ++++++++++++++++++++++++++++
>   test/py/pytest.ini          |  1 +
>   test/py/tests/test_sleep.py |  1 +
>   3 files changed, 30 insertions(+)
> 
> diff --git a/test/py/conftest.py b/test/py/conftest.py
> index 6e66a48c15fd..1812ff340e6a 100644
> --- a/test/py/conftest.py
> +++ b/test/py/conftest.py
> @@ -436,6 +436,33 @@ def setup_boardspec(item):
>       if required_boards and ubconfig.board_type not in required_boards:
>           pytest.skip('board "%s" not supported' % ubconfig.board_type)
>   
> +def setup_boardidentity(item):
> +    """Process any 'boardidentity' marker for a test.
> +
> +    Such a marker lists the set of board identity that a test does/doesn't
> +    support. If tests are being executed on an unsupported board, the test is
> +    marked to be skipped.
> +
> +    Args:
> +        item: The pytest test item.
> +
> +    Returns:
> +        Nothing.
> +    """
> +    mark = item.get_marker('boardidentity')
> +    if not mark:
> +        return
> +    required_boards = []
> +    for board in mark.args:
> +        if board.startswith('!'):
> +            if ubconfig.board_identity == board[1:]:
> +                pytest.skip('board identity not supported')
> +                return
> +        else:
> +            required_boards.append(board)
> +    if required_boards and ubconfig.board_identity not in required_boards:
> +        pytest.skip('board identity not supported')
> +
>   def setup_buildconfigspec(item):
>       """Process any 'buildconfigspec' marker for a test.
>   
> @@ -503,6 +530,7 @@ def pytest_runtest_setup(item):
>   
>       start_test_section(item)
>       setup_boardspec(item)
> +    setup_boardidentity(item)
>       setup_buildconfigspec(item)
>       setup_requiredtool(item)
>   
> diff --git a/test/py/pytest.ini b/test/py/pytest.ini
> index 67e514f42058..9d64671814de 100644
> --- a/test/py/pytest.ini
> +++ b/test/py/pytest.ini
> @@ -8,4 +8,5 @@
>   [pytest]
>   markers =
>       boardspec: U-Boot: Describes the set of boards a test can/can't run on.
> +    boardidentity: U-Boot: Describes the board identity a test can/can't run on.
>       buildconfigspec: U-Boot: Describes Kconfig/config-header constraints.
> diff --git a/test/py/tests/test_sleep.py b/test/py/tests/test_sleep.py
> index 64e057132622..02a8a85b0e22 100644
> --- a/test/py/tests/test_sleep.py
> +++ b/test/py/tests/test_sleep.py
> @@ -5,6 +5,7 @@
>   import pytest
>   import time
>   
> + at pytest.mark.boardidentity("!qemu")

According to your commit message you don't want to exclude qemu-x86 here.

Best regards

Heinrich Schuchardt

>   def test_sleep(u_boot_console):
>       """Test the sleep command, and validate that it sleeps for approximately
>       the correct amount of time."""
>