[PATCH] test_sleep.py: make sleep time and margin configurable

[Stephen Warren]

On 6/16/20 9:11 PM, Simon Glass wrote:
> Hi Stephen,
> 
> On Mon, 8 Jun 2020 at 11:25, Stephen Warren <swarren at wwwdotorg.org> wrote:
>>
>> On 6/8/20 11:12 AM, Simon Glass wrote:
>>> Hi Stephen,
>>>
>>> On Mon, 8 Jun 2020 at 10:43, Stephen Warren <swarren at wwwdotorg.org> wrote:
>>>>
>>>> On 6/7/20 7:45 AM, Simon Glass wrote:
>>>>> On Thu, 4 Jun 2020 at 09:24, Heiko Schocher <hs at denx.de> wrote:
>>>>>>
>>>>>> make the sleep time and the margin configurable.
>>>>>>
>>>>>> Signed-off-by: Heiko Schocher <hs at denx.de>
>>>>>> ---
>>>>>>
>>>>>> travis build:
>>>>>> https://travis-ci.org/github/hsdenx/u-boot-test/builds/694545225
>>>>>>
>>>>>> This patch is needed as I start test/py now within tbot [1]. On
>>>>>> some configurations U-Boot is compiled on a build machine for
>>>>>> example in munich, while the board under test is in my lab in
>>>>>> hungary.
>>>>>>
>>>>>> So the 0.25 seconds default margin is often to low because
>>>>>> of latencies on the net.
>>>>>>
>>>>>> See as an example configuration (within tbot):
>>>>>>
>>>>>> https://github.com/EmbLux-Kft/tbot-tbot2go/blob/devel/boards/aristainetos.py#L29
>>>>>>
>>>>>> [1] http://tbot.tools/modules/tc.html#u-boot-test-py
>>>>>>
>>>>>>  test/py/tests/test_sleep.py | 14 +++++++++++---
>>>>>>  1 file changed, 11 insertions(+), 3 deletions(-)
>>>>>
>>>>> Reviewed-by: Simon Glass <sjg at chromium.org>
>>>>>
>>>>> Related, at some point we should change sandbox to fake the time
>>>>> movement since this test currently waits for three seconds even on
>>>>> sandbox.
>>>>
>>>> We definitely shouldn't do that; that's the exact kind of failure this
>>>> test is intended to detect.
>>>
>>> No, we're not looking for bugs in sandbox's time handling. We are just
>>> testing the plumbing associated with delaying (timer driver, etc.).
>>
>> The entire purpose of the test is to look for bugs in the backend
>> implementation of the time handling. I should know; I wrote the test!
> 
> OK and we have time_ut.c as well.
> 
> Perhaps we should disable this test on sandbox then? It really doesn't
> make sense to be testing timeouts on sandbox IMO and it costs us 3
> seconds on each test run.

I'd rather keep such policy out of the test framework. If you personally
don't want to run the test, you can always specify '-k not sleep' (if I
got the syntax right...) when running test.py. I think e.g. the Travis
test configuration might already do this, as an example.