[PATCH v3 6/6] test: add test for eficonfig secure boot key management

[Masahisa Kojima]

Hi Heinrich, Ilias, Simon,

On Mon, 17 Oct 2022 at 13:49, Masahisa Kojima
<masahisa.kojima at linaro.org> wrote:
>
> Hi Heinrich,
>
> On Sat, 15 Oct 2022 at 13:48, Heinrich Schuchardt <xypron.glpk at gmx.de> wrote:
> >
> > On 10/15/22 03:10, Simon Glass wrote:
> > > Hi Ilias,
> > >
> > > On Fri, 14 Oct 2022 at 09:59, Ilias Apalodimas
> > > <ilias.apalodimas at linaro.org> wrote:
> > >>
> > >> Hi Simon,
> > >>
> > >> On Fri, 14 Oct 2022 at 18:56, Simon Glass <sjg at chromium.org> wrote:
> > >>>
> > >>> Hi,
> > >>>
> > >>> On Fri, 14 Oct 2022 at 00:58, Masahisa Kojima
> > >>> <masahisa.kojima at linaro.org> wrote:
> > >>>>
> > >>>> Provide a unit test for the eficonfig secure boot key
> > >>>> management menu.
> > >>>>
> > >>>> Signed-off-by: Masahisa Kojima <masahisa.kojima at linaro.org>
> > >>>> ---
> > >>>> No change since v2
> > >>>>
> > >>>> newly created in v2
> > >>>>
> > >>>>   test/py/tests/test_eficonfig/conftest.py      |  84 +++-
> > >>>>   test/py/tests/test_eficonfig/defs.py          |  14 +
> > >>>>   .../test_eficonfig/test_eficonfig_sbkey.py    | 472 ++++++++++++++++++
> > >>>>   3 files changed, 568 insertions(+), 2 deletions(-)
> > >>>>   create mode 100644 test/py/tests/test_eficonfig/defs.py
> > >>>>   create mode 100644 test/py/tests/test_eficonfig/test_eficonfig_sbkey.py
> > >>>
> > >>> Please can this test be in C? Also, using down-arrow to select menus
> > >>> is brittle. Add a function to select the one you want, e.g. by name.
> > >>>
> > >>
> > >> Is there a very specific reason why we should do stuff like that in C?
> > >
> > > Yes, see here.
> > >
> > >>   Python is way easier to extend and test in our case.
> > >
> > > In what way? It seems a lot more complicated, plus the brittle nature
> > > of this test suggests it will be a hassle to maintain.
> > >
> > > https://u-boot.readthedocs.io/en/latest/develop/tests_writing.html#python-or-c
> > >
> > > There is a pending update here too:
> > >
> > > https://patchwork.ozlabs.org/project/uboot/patch/20221013122927.636867-15-sjg@chromium.org/
> >
> > The discussion touches different aspects:
> >
> > ** What does it take to make a GUI easily testable? **
> >
> > Relying on cursor movement for testing is fragile. This is why GUIs
> > often assign to each executable element the following:
> >
> > * Access key, e.g  <CTRL><S>
> >    A key combination that when entered will have the the same
> >    effect as selecting the GUI item
> > * Access string, '@SAVE'
> >    A string that when typed into the command field will have
> >    the same effect as selecting the GUI item
> >
> > Let's look at U-Boot's menu entry definition:
> >
> > struct bootmenu_entry {
> >      unsigned short int num;      // unique number 0 .. MAX_COUNT
> >      char key[3];                 // key identifier of number
> >      char *title;                 // title of entry
> >      char *command;               // hush command of entry
> >      enum boot_type type;         // boot type of entry
> >      u16 bootorder;               // order for each boot type
> >      struct bootmenu_data *menu;  // this bootmenu
> >      struct bootmenu_entry *next; // next menu entry (num+1)
> > }
> >
> > Our structure lacks an accessor element that can be used to select a
> > menu item without using cursor actions.
> >
> > Compound keystrokes like <CTRL><F6> are send as multiple bytes on the
> > console, e.g. 1b 5b 31 37 3b 35 7e.
> >
> > We may define a field shortcut of type char *. If the string is received
> > by the menu loop, let it activate the matching menu entry. Let cursor
> > actions (up, down, enter, space '+', '-') interrupt matching the
> > shortcut string.
> >
> > Instead we could also use a convention for the title:
> >
> > If a letter in the title is preceded by '&', this is the shortcut key.
> > This letter will be shown highlighted in the menu and the ampersand will
> > not be shown.
> >
> > This is probably easier to implement.
>
> I agree to the shortcut by '&', I will implement it and update the
> python test code.

The shortcut functionality is a kind of feature enhancement of eficonfig menus.
If there is no additional comment and feedback other than the shortcut feature,
could you consider to merge this series first?
I would like to get more user feedback.

I will send the follow up patches to support the shortcut feature
after this original
series is merged.

Thanks,
Masahisa Kojima

>
> Thanks,
> Masahisa Kojima
>
> >
> > Adding the shortcut facility will allow both for easier testing and
> > faster navigation.
> >
> > ** Choice of programming language **
> >
> > Several aspects control the choice of the programming language for tests:
> >
> > - Testing single library functions is only possible in C.
> > - Checking contents of internal structures is only possible in C.
> > - Testing the U-Boot's CLI is easily accessible in our Python framework.
> > - Preparation of complex test data is easier to do in Python.
> > - Mixed language tests should be avoided if not strictly necessary.
> >    It is much easier to maintain a single code source for a test.
> >
> > ** What to do for secure boot key management? **
> >
> > Secure boot key management requires complex preparation which fits well
> > into out Python testing framework.
> >
> > Once we provide shortcut keys to U-Boot menus the entries will be easily
> > accessible from Python.
> >
> > As we want to avoid complexity due to mixed language tests we should
> > stick to Python for testing key management at the user level.
> >
> > Additional tests in C for library functions managing keys and verifying
> > signatures would improve our code coverage.
> >
> > Best regards
> >
> > Heinrich