[PATCH v4 4/4] test: env: add test for env info sub-command

[Patrick DELAUNAY]

Hi Stephen,

> From: Stephen Warren <swarren at wwwdotorg.org>
> Sent: jeudi 18 juin 2020 00:32
> 
> On 6/16/20 2:01 AM, Patrick DELAUNAY wrote:
> > Hi Stephen,
> >
> >> From: Stephen Warren <swarren at wwwdotorg.org>
> >> Sent: mardi 16 juin 2020 00:09
> >>
> >> On 6/15/20 8:01 AM, Patrick Delaunay wrote:
> >>> Add a pytest for testing the env info sub-command:
> >>>
> >>> test_env_info: test command with several option
> >>>
> >>> test_env_info_quiet: test the result of the sub-command with quiet
> >>> option, '-q' as used for support in shell test; for example:
> >>>   if env info -p -d -q; then env save; fi
> >>
> >>> diff --git a/test/py/tests/test_env.py b/test/py/tests/test_env.py
> >>
> >>> + at pytest.mark.boardspec('sandbox')
> >>> + at pytest.mark.buildconfigspec('cmd_nvedit_info')
> >>> +def test_env_info(state_test_env):
> >>
> >> The body of these tests doesn't look like it tests something that's
> >> specific to sandbox, so I'm not sure why the test function is marked to only run
> on sandbox.
> >> Is it simply because other boards may store the environment
> >> differently and/or have valid saved environment in flash, so the
> >> responses to e.g. "env info" aren't the same everywhere? If so, I
> >> imagine that test_env_info_quiet() doesn't need to be sandbox-only, since
> there's no output in that case.
> >
> > The test is not really sandbox specific but I don't have easy way to
> > know on real board the ENV configuration (for the resut of command env info -p
> -d).
> >
> > In the test, I assume that  at least  CONFIG_ENV_IS_.... is activated
> > (for persistent storage) and if this target is selected in the weak function
> env_get_location.
> > And "env save" as be not be executed (default environment is used).
> >
> > And with quiet option, the test  the environment if is persistent
> > (result of "env -p -q" is 0) or using default ("env -d -q" result is 0).
> >
> > And in the next patch
> > http://patchwork.ozlabs.org/project/uboot/patch/20200616074048.7898-10
> > -patrick.delaunay at st.com/
> >
> > As the command "env erase" is not always supported according he environment
> target.
> >
> > I could test on real hardware but I need to check if I test all the possible result.
> 
> OK, I guess that makes sense for a start.

But I will propose a V5  to check command on real hardware
Just modify test_env_info to check the all possible strings.
 
> For testing on real HW, the typical approach would be to require that the board's
> test configuration define some env__xxx variables that define its capabilities.
> Then, the test can be made to depend on those values, or whether those variables
> are set at all.

For the next steps, I need to thinks about tests because ENV location is not only impacted
by CONFIG_ENV_IS_IN_.... or CONFIG_ENV_IS_NOWHERE 

These defined can be activated simultaneously and env location is detected at run
time: so it is difficult to predict the 'env info' result on real hardware.

On sandbox it is fixed because ENVL_NOWHERE is selected by default.

Patrick