[U-Boot] [U-Boot, v4, 2/2] test/py: add test for whitelist of variables while importing environment

[Tom Rini]

On Wed, Jun 27, 2018 at 09:52:50AM +0200, Quentin Schulz wrote:
> Hi Tom,
> 
> On Tue, Jun 26, 2018 at 10:44:59AM -0400, Tom Rini wrote:
> > On Tue, Jun 26, 2018 at 02:41:40PM +0200, Quentin Schulz wrote:
> > > Hi all,
> > > 
> > > On Wed, Jun 13, 2018 at 01:02:06PM -0600, Stephen Warren wrote:
> > > > On 06/13/2018 12:53 PM, Quentin Schulz wrote:
> > > > > Hi Tom,
> > > > > 
> > > > > On Wed, Jun 13, 2018 at 11:43:32AM -0400, Tom Rini wrote:
> > > > > > On Mon, Jun 04, 2018 at 11:47:30AM +0200, Quentin Schulz wrote:
> > > > > > 
> > > > > > > This tests that the importing of an environment with a specified
> > > > > > > whitelist works as intended.
> > > > > > > 
> > > > > > > If there are variables passed as parameter to the env import command,
> > > > > > >     those only should be imported in the current environment.
> > > > > > > 
> > > > > > > For each variable passed as parameter, if
> > > > > > >   - foo is bar in current env and bar2 in exported env, after importing
> > > > > > >   exported env, foo shall be bar2,
> > > > > > >   - foo does not exist in current env and foo is bar2 in exported env,
> > > > > > >   after importing exported env, foo shall be bar2,
> > > > > > >   - foo is bar in current env and does not exist in exported env (but is
> > > > > > >   passed as parameter), after importing exported env, foo shall be empty,
> > > > > > > 
> > > > > > > Any variable not passed as parameter should be left untouched.
> > > > > > > 
> > > > > > > Two other tests are made to test that size cannot be '-' if the checksum
> > > > > > > protection is enabled.
> > > > > > > 
> > > > > > > Signed-off-by: Quentin Schulz <quentin.schulz at bootlin.com>
> > > > > > > Reviewed-by: Simon Glass <sjg at chromium.org>
> > > > > > > Reviewed-by: Stephen Warren <swarren at nvidia.com>
> > > > > > 
> > > > > > This seems to not be working?
> > > > > > 
> > > > > > https://travis-ci.org/trini/u-boot/jobs/391504525
> > > > > > 
> > > > > 
> > > > > I just rebased on top of v2018.07-rc1, ran
> > > > > make mrproper
> > > > > ./test/py/test.py --bd sandbox --build
> > > > > 
> > > > > and the tests run fine ...
> > > > 
> > > > Most likely the failure is due to the test relying on some feature that
> > > > isn't enabled on the board being tested (emulated via qemu); you'll need to
> > > > add something like the following to indicate which feature the test relies
> > > > upon:
> > > > 
> > > > @pytest.mark.buildconfigspec('cmd_echo')
> > > > 
> > > 
> > > OK, I've added the dependency on cmd_importenv and cmd_exportenv, but
> > > that does not make it work any better.
> > > 
> > > I added my U-Boot repo to Travis and ran the tests. Here is the output
> > > of the job: https://travis-ci.org/QSchulz/u-boot/
> > > 
> > > Specifically, you have:
> > > https://travis-ci.org/QSchulz/u-boot/jobs/396742661
> > > https://travis-ci.org/QSchulz/u-boot/jobs/396742668
> > > https://travis-ci.org/QSchulz/u-boot/jobs/396742669
> > > https://travis-ci.org/QSchulz/u-boot/jobs/396742670
> > > https://travis-ci.org/QSchulz/u-boot/jobs/396742671
> > > 
> > > I've dumped the RAM after the `env export` and it looks pretty much
> > > empty compared to what I could see with sandbox tests.
> > > 
> > > Since all the other tests work, I'm not sure I actually introduced a
> > > regression or if it just never worked. I'll run tests without my patches
> > > that do a `env export` followed by a dump of the memory, a reset of the
> > > environement and a `env import` to see where we stand right now.
> > 
> > It's possible you've exposed a latent bug here.  What stood out to me at
> > the time was that it looked like we were using 0x0 for the address and
> > that's quite possibly an invalid location to be using on these
> > platforms.
> > 
> 
> Yes, looked weird to me as well but can't tell if that's a legit RAM
> address. Stephen says other tests interacting with RAM works on those
> configs so that should be working.
>> 
> So, I tested without my patches for whitelisting and it does not work
> for the same configs:
> 
> https://travis-ci.org/QSchulz/u-boot/jobs/396898590
> https://travis-ci.org/QSchulz/u-boot/jobs/396898597
> https://travis-ci.org/QSchulz/u-boot/jobs/396898598
> https://travis-ci.org/QSchulz/u-boot/jobs/396898599
> 
> I'm not introducing a regression with my patch. Do we know if env import
> and export is actually supported by those configs? How could we fix
> those? Let me know if I can help.

Maybe the tests need to depend on something else too then, that's not
enabled on these platforms?

> On another subject, I'm worried by these failing jobs:
> https://travis-ci.org/QSchulz/u-boot/jobs/396898591
> https://travis-ci.org/QSchulz/u-boot/jobs/396898592
> 
> Does this mean that we don't have a clean environnement for each and
> every test? (env import complains when I'm trying to import an
> environnement with ethXaddr in it, so I forcibly removed them from the
> environnement before exporting).

Right, tests are run serially and don't reset the board for each, iirc.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20180627/d78a188e/attachment.sig>