[U-Boot] [RFC PATCH] test: py: Disable sleep test for qemu targets

[Michal Simek]

On 5.12.2017 19:38, Tom Rini wrote:
> On Tue, Dec 05, 2017 at 11:20:57AM -0700, Stephen Warren wrote:
>> On 12/04/2017 04:21 PM, Tom Rini wrote:
>>> On Mon, Dec 04, 2017 at 10:14:06AM -0700, Stephen Warren wrote:
>>>> On 12/04/2017 08:30 AM, Tom Rini wrote:
>>>>> On Mon, Dec 04, 2017 at 03:21:04PM +0100, Michal Simek wrote:
>>>>>> On 4.12.2017 15:03, Tom Rini wrote:
>>>>>>> On Mon, Dec 04, 2017 at 02:55:45PM +0100, Michal Simek wrote:
>>>>>>>> On 1.12.2017 23:44, Tom Rini wrote:
>>>>>>>>> On Fri, Dec 01, 2017 at 10:07:54AM -0700, Stephen Warren wrote:
>>>>>>>>>> On 12/01/2017 08:19 AM, Michal Simek wrote:
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> On 1.12.2017 16:06, Heinrich Schuchardt wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 12/01/2017 03:46 PM, Michal Simek wrote:
>>>>>>>>>>>>> Qemu for arm32/arm64 has a problem with time setup.
>>>>>>>>>>>>
>>>>>>>>>>>> Wouldn't it be preferable to fix the root cause?
>>>>>>>>>>>
>>>>>>>>>>> Definitely that would be the best and IIRC I have tried to convince our
>>>>>>>>>>> qemu guy to do that but they have never done that.
>>>>>>>>>>
>>>>>>>>>> What is the exact failure condition? Is it simply that the test is still
>>>>>>>>>> slightly too strict about which delays it accepts, or is sleep outright
>>>>>>>>>> broken?
>>>>>>>>>>
>>>>>>>>>> You can use command-line option -k to avoid some tests. For example "-k not
>>>>>>>>>> sleep". That way, we don't have to hard-code the dependency into the test
>>>>>>>>>> source. Depending on the root cause (issue in U-Boot, or issue in just your
>>>>>>>>>> local version of qemu, or something that will never work) this might be
>>>>>>>>>> better?
>>>>>>>>>
>>>>>>>>> Even with the most recent relaxing of the sleep test requirements, I can
>>>>>>>>> still (depending on overall system load) have 'sleep' take too long, on
>>>>>>>>> QEMU.  I think it might have been half a second slow, but I don't have
>>>>>>>>> the log handy anymore.  Both locally and in travis we -k not sleep all
>>>>>>>>> of the qemu instances.
>>>>>>>>
>>>>>>>> ok. By locally do you mean just using -k not sleep?
>>>>>>>
>>>>>>> Yes, I have that in my CI scripts and similar.
>>>>>>
>>>>>> Wouldn't be easier to keep this in uboot-test-hooks repo with other
>>>>>> target setting?
>>>>>
>>>>> Or do as you did did and mark the tests as not allowed for qemu, yes.
>>>>>
>>>>>> What we are trying to do is that our testing group will run these tests
>>>>>> for me that's why it is just easier for me to change local
>>>>>> uboot-test-hooks repo instead of communicate with them what -k not XXX
>>>>>> parameters to add to certain scripts.
>>>>>>
>>>>>> It means in loop they will just run all tests on qemu, local targets and
>>>>>> in boardfarm. It is probably not big deal to tell them to add -k not
>>>>>> sleep for all qemu runs but I know that for some i2c testing qemu
>>>>>> doesn't emulate these devices that's why these tests fails. And the
>>>>>> amount of tests which we shouldn't run on qemu will probably grow.
>>>>>
>>>>> Well, I'm still open to possibly tweaking the allowed variance in the
>>>>> sleep test.  OTOH, if we just say "no QEMU" here, we can then go back to
>>>>> "sleep should be pretty darn accurate on HW" for the test too, and
>>>>> perhaps that's best.
>>>>
>>>> The fundamental problem of "over-sleeping" due to host system load/.. exists
>>>> with all boards. There's nothing specific to qemu here except that running
>>>> U-Boot on qemu on the host rather than on separate HW might more easily
>>>> trigger the "high load on the host" condition; I see the issue now and then
>>>> and manually retry that test, although that is a bit annoying.
>>>>
>>>> The original test was mostly intended to make sure that e U-Boot clock
>>>> didn't run at a significantly different rate to the host, since I had seen
>>>> that issue during development of some board support or as a regression
>>>> sometime. Perhaps the definition of "significantly different" should be more
>>>> like "1/2 rate or twice rate or more" rather than "off by a small fraction
>>>> of a second". That might avoid so many false positives.
>>>
>>> I've pushed this up to 10 seconds and 0.5s worth of overrun and on
>>> qemu-mips here I see a 13.2s sleep.  That's pretty close to 1/3rd fast
>>> and to me a wrong-clocking value, yes?
>>
>> For me the qemu-x86 build of mid-Nov commit of U-Boot running under the same
>> qemu version that U-Boot's Travis CI builds use, "sleep 10" takes about 10.5
>> seconds (including my reaction time), so ~13.2 does sound like it's probably
>> a bug. Or maybe qemu just isn't fast enough in its emulation to keep up with
>> real-time? I'd hope not for something simple like this, assuming you're
>> using a recent CPU, but maybe.
> 
> Yeah, I can do x86, ARM and PowerPC but it fails on MIPS.  And my build
> box isn't super new but an 8core/16thread E5-2670 should be good enough
> :)

I should spend some time to add also microblaze. :-)

Thanks,
Michal


-- 
Michal Simek, Ing. (M.Eng), OpenPGP -> KeyID: FE3D1F91
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel - Xilinx Microblaze
Maintainer of Linux kernel - Xilinx Zynq ARM and ZynqMP ARM64 SoCs
U-Boot custodian - Xilinx Microblaze/Zynq/ZynqMP SoCs


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: OpenPGP digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20171206/e7af911e/attachment.sig>