[U-Boot] [PATCH 00/23] efi_loader implement missing functions

[Heinrich Schuchardt]

On 09/06/2017 01:48 AM, Rob Clark wrote:
> On Tue, Sep 5, 2017 at 4:55 AM, Simon Glass <sjg at chromium.org> wrote:
>> Hi,
>>
>> On 1 September 2017 at 22:45, Tom Rini <trini at konsulko.com> wrote:
>>> On Wed, Aug 30, 2017 at 04:16:34AM +0800, Simon Glass wrote:
>>>> Hi,
>>>>
>>>> On 29 August 2017 at 22:16, Rob Clark <robdclark at gmail.com> wrote:
>>>>> On Tue, Aug 29, 2017 at 8:57 AM, Leif Lindholm <leif.lindholm at linaro.org> wrote:
>>>>>> On Tue, Aug 29, 2017 at 02:26:48PM +0200, Alexander Graf wrote:
>>>>>>>>>> I would add command
>>>>>>>>>> bootefi selftest.efi
>>>>>>>>>> to run the tests and provide the python wrapper code to add it to the
>>>>>>>>>> test suite.
>>>>>>>>>
>>>>>>>>> I think that's a great idea, yes.
>>>>>>>> I wonder how far we are from running UEFI tests (either the official
>>>>>>>> ones, or I seem to remember hearing about some other test suite which
>>>>>>>> didn't require UEFI shell)?
>>>>>>>
>>>>>>> Let's ask Leif, Ard and Dong.
>>>>>>>
>>>>>>> The official test suite definitely needs the UEFI Shell. Is the suite
>>>>>>> publicly available by now?
>>>>>>
>>>>>> In binary form, you can access it already from the links on
>>>>>> http://uefi.org/testtools
>>>>>>
>>>>>> Yes, 2.5 is latest release. No this is not a restriction ... the SCT
>>>>>> releases have been lagging the specification releases a fair bit.
>>>>>>
>>>>>> The 2.5a package contains AARCH64, IA32 and X64 support (not ARM).
>>>>>> Not that it couldn't contain ARM, it just hasn't been packaged.
>>>>>>
>>>>>>>> That seems more useful long term than re-inventing comprehensive UEFI
>>>>>>>> test suite.  (Also, beyond just running shim/fallback/grub, I don't
>>>>>>>> really have time to invent new tests for the stack of efi_loader
>>>>>>>> patches I have.)
>>>>>>>
>>>>>>> Yes and no - it depends on the availability of the official suite :/.
>>>>>>
>>>>>> UEFI SCT is not yet open source/free software. Obviously, this is
>>>>>> something Linaro has been lobbying for since day one of our
>>>>>> involvement. There used to be little understanding for this, but that
>>>>>> attitude has shifted substantially.
>>>>>
>>>>> So, if/until UEFI SCT is not an option, what about:
>>>>>
>>>>>   https://01.org/linux-uefi-validation
>>>>>
>>>>> (thx to pjones for pointing that out to me)
>>>>
>>>> Well in any case I'm not looking for a large functional test suite at
>>>> this stage. It certainly could be useful, but not as a replacement for
>>>> unit tests. The latter is for fast verification (so that everyone can
>>>> run it as part of 'make tests') and easy identification of the
>>>> location of bugs.
>>>
>>> I want to chime in here.  Unless we're talking hours of time, if "make
>>> tests" takes 5 minutes to run, but that includes doing some sort of full
>>> validation suite to the relevant EFI code, that seems like a win to me.
>>> And if someone else is responsible for the contents of the tests and we
>>> just have to confirm our expected results, that's an even bigger win.
>>
>> Yes it certainly is a win. But I'm already upset with how long the
>> tests take to run so I would not be keen on anything that increases it
>> much. How long does this test suite take to run normally?
>>
> 
> I'm not sure how long SCT would take to run until we get it working (I
> have a growing stack of patches on my wip-enough-uefi-for-shell
> branch, and have Shell.efi running but still debugging sct.efi)..
> maybe someone who has run SCT on a real UEFI implementation knows..
> But that said, there seems to be some potential to run a subset of the
> tests.  Either way, it seems like most of the time that travis runs
> take is building a million variants of u-boot.  I'm not sure the time
> spent running tests is really the issue.
> 
> BR,
> -R
> 
You could reduce the number of variants in .travis.yml by deleting
entries from env:.

With EFI the once that showed up my errors were:

qemu-x86
vexpress_ca15_tc2
vexpress_ca9x4

So maybe for a quick test (10 min) of your patches you can reduce to
these and do a full test before submitting.

Regards

Heinrich