[U-Boot] [PATCH 1/1] test/py/tests/test_sleep.py: test time approximately

[Heinrich Schuchardt]

On 10/05/2017 11:10 PM, Stephen Warren wrote:
> On 10/05/2017 01:52 PM, Heinrich Schuchardt wrote:
>> On qemu errors like
>> assert 2.999650001525879 >= 3
>> occur.
> 
> Can you work out why? I guess 1-999650001525879 is a really tiny amount,
> so perhaps it's OK. However, I'd like to keep the test strict if
> possible; maybe rather than:
> 
>> +    assert elapsed >= (sleep_time - 0.25)
> 
> can we use:
> 
>> +    assert elapsed >= (sleep_time - 0.01)
> 
> ?
> 
>> According to the comment in the code the test is meant to be
>> approximate. So we should accept some milliseconds less.
> 
> This test is deliberately very strict about the minimum time taken
> during sleep, and slightly sloppy about over-sleeping. That's because
> sleep should never wait too little time, but we allow a little extra
> time due to e.g. the test system being a bit busy and not noticing when
> the sleep wakes up.
>

You are making unreasonable assumptions about the accuracy of system clocks.

I am not aware of U-Boot making any guarantees that the clock is never
too fast.

If you want sleep to never consume less time than requested you would at
least always have to add 1 timer tick to the requested time in the sleep
function.

The test does not even use a timer function that is meant to measure
with the accuracy that you expect.

It uses Python function time.time() that is not guaranteed to have a
resolution better than one second and has no guarantee to be non-decreasing.

The right way to measure time for performance measurements is Python 3.3
function time.perf_counter().

QEMU provides no guarantees concerning time synchroniation. So shouldn't
we be happy with the little randomness that we see in some systems, e.g.

3.000195026397705
3.000035047531128
3.000230073928833
2.9996659755706787
3.0026118755340576
3.0025811195373535
3.0034120082855225
3.002816915512085
3.002542018890381
3.0020010471343994

Best regards

Heinrich