[U-Boot] [PATCH 00/23] efi_loader implement missing functions

[Rob Clark]

On Wed, Sep 6, 2017 at 12:18 AM, Heinrich Schuchardt <xypron.glpk at gmx.de> wrote:
> On 09/06/2017 01:48 AM, Rob Clark wrote:
>> On Tue, Sep 5, 2017 at 4:55 AM, Simon Glass <sjg at chromium.org> wrote:
>>> Hi,
>>>
>>> On 1 September 2017 at 22:45, Tom Rini <trini at konsulko.com> wrote:
>>>> On Wed, Aug 30, 2017 at 04:16:34AM +0800, Simon Glass wrote:
>>>>> Hi,
>>>>>
>>>>> On 29 August 2017 at 22:16, Rob Clark <robdclark at gmail.com> wrote:
>>>>>> On Tue, Aug 29, 2017 at 8:57 AM, Leif Lindholm <leif.lindholm at linaro.org> wrote:
>>>>>>> On Tue, Aug 29, 2017 at 02:26:48PM +0200, Alexander Graf wrote:
>>>>>>>>>>> I would add command
>>>>>>>>>>> bootefi selftest.efi
>>>>>>>>>>> to run the tests and provide the python wrapper code to add it to the
>>>>>>>>>>> test suite.
>>>>>>>>>>
>>>>>>>>>> I think that's a great idea, yes.
>>>>>>>>> I wonder how far we are from running UEFI tests (either the official
>>>>>>>>> ones, or I seem to remember hearing about some other test suite which
>>>>>>>>> didn't require UEFI shell)?
>>>>>>>>
>>>>>>>> Let's ask Leif, Ard and Dong.
>>>>>>>>
>>>>>>>> The official test suite definitely needs the UEFI Shell. Is the suite
>>>>>>>> publicly available by now?
>>>>>>>
>>>>>>> In binary form, you can access it already from the links on
>>>>>>> http://uefi.org/testtools
>>>>>>>
>>>>>>> Yes, 2.5 is latest release. No this is not a restriction ... the SCT
>>>>>>> releases have been lagging the specification releases a fair bit.
>>>>>>>
>>>>>>> The 2.5a package contains AARCH64, IA32 and X64 support (not ARM).
>>>>>>> Not that it couldn't contain ARM, it just hasn't been packaged.
>>>>>>>
>>>>>>>>> That seems more useful long term than re-inventing comprehensive UEFI
>>>>>>>>> test suite.  (Also, beyond just running shim/fallback/grub, I don't
>>>>>>>>> really have time to invent new tests for the stack of efi_loader
>>>>>>>>> patches I have.)
>>>>>>>>
>>>>>>>> Yes and no - it depends on the availability of the official suite :/.
>>>>>>>
>>>>>>> UEFI SCT is not yet open source/free software. Obviously, this is
>>>>>>> something Linaro has been lobbying for since day one of our
>>>>>>> involvement. There used to be little understanding for this, but that
>>>>>>> attitude has shifted substantially.
>>>>>>
>>>>>> So, if/until UEFI SCT is not an option, what about:
>>>>>>
>>>>>>   https://01.org/linux-uefi-validation
>>>>>>
>>>>>> (thx to pjones for pointing that out to me)
>>>>>
>>>>> Well in any case I'm not looking for a large functional test suite at
>>>>> this stage. It certainly could be useful, but not as a replacement for
>>>>> unit tests. The latter is for fast verification (so that everyone can
>>>>> run it as part of 'make tests') and easy identification of the
>>>>> location of bugs.
>>>>
>>>> I want to chime in here.  Unless we're talking hours of time, if "make
>>>> tests" takes 5 minutes to run, but that includes doing some sort of full
>>>> validation suite to the relevant EFI code, that seems like a win to me.
>>>> And if someone else is responsible for the contents of the tests and we
>>>> just have to confirm our expected results, that's an even bigger win.
>>>
>>> Yes it certainly is a win. But I'm already upset with how long the
>>> tests take to run so I would not be keen on anything that increases it
>>> much. How long does this test suite take to run normally?
>>>
>>
>> I'm not sure how long SCT would take to run until we get it working (I
>> have a growing stack of patches on my wip-enough-uefi-for-shell
>> branch, and have Shell.efi running but still debugging sct.efi)..
>> maybe someone who has run SCT on a real UEFI implementation knows..
>> But that said, there seems to be some potential to run a subset of the
>> tests.  Either way, it seems like most of the time that travis runs
>> take is building a million variants of u-boot.  I'm not sure the time
>> spent running tests is really the issue.
>>
>> BR,
>> -R
>>
> You could reduce the number of variants in .travis.yml by deleting
> entries from env:.

yeah, this is something I've done before to try to debug issues that
only show up in travis and that I could not reproduce locally

> With EFI the once that showed up my errors were:
>
> qemu-x86
> vexpress_ca15_tc2
> vexpress_ca9x4

imho, we need something aarch64 and using DM in the mix.. and so far
I've only seem the same issues on both vexpress platforms, so I'm not
sure how useful it is to test both (at least for a quick test run)

BR,
-R

> So maybe for a quick test (10 min) of your patches you can reduce to
> these and do a full test before submitting.
>
> Regards
>
> Heinrich