[U-Boot] [U-Boot, v4, 2/2] test/py: add test for whitelist of variables while importing environment

[Stephen Warren]

On 06/26/2018 06:41 AM, Quentin Schulz wrote:
> Hi all,
> 
> On Wed, Jun 13, 2018 at 01:02:06PM -0600, Stephen Warren wrote:
>> On 06/13/2018 12:53 PM, Quentin Schulz wrote:
>>> Hi Tom,
>>>
>>> On Wed, Jun 13, 2018 at 11:43:32AM -0400, Tom Rini wrote:
>>>> On Mon, Jun 04, 2018 at 11:47:30AM +0200, Quentin Schulz wrote:
>>>>
>>>>> This tests that the importing of an environment with a specified
>>>>> whitelist works as intended.
>>>>>
>>>>> If there are variables passed as parameter to the env import command,
>>>>>      those only should be imported in the current environment.
>>>>>
>>>>> For each variable passed as parameter, if
>>>>>    - foo is bar in current env and bar2 in exported env, after importing
>>>>>    exported env, foo shall be bar2,
>>>>>    - foo does not exist in current env and foo is bar2 in exported env,
>>>>>    after importing exported env, foo shall be bar2,
>>>>>    - foo is bar in current env and does not exist in exported env (but is
>>>>>    passed as parameter), after importing exported env, foo shall be empty,
>>>>>
>>>>> Any variable not passed as parameter should be left untouched.
>>>>>
>>>>> Two other tests are made to test that size cannot be '-' if the checksum
>>>>> protection is enabled.
>>>>>
>>>>> Signed-off-by: Quentin Schulz <quentin.schulz at bootlin.com>
>>>>> Reviewed-by: Simon Glass <sjg at chromium.org>
>>>>> Reviewed-by: Stephen Warren <swarren at nvidia.com>
>>>>
>>>> This seems to not be working?
>>>>
>>>> https://travis-ci.org/trini/u-boot/jobs/391504525
>>>>
>>>
>>> I just rebased on top of v2018.07-rc1, ran
>>> make mrproper
>>> ./test/py/test.py --bd sandbox --build
>>>
>>> and the tests run fine ...
>>
>> Most likely the failure is due to the test relying on some feature that
>> isn't enabled on the board being tested (emulated via qemu); you'll need to
>> add something like the following to indicate which feature the test relies
>> upon:
>>
>> @pytest.mark.buildconfigspec('cmd_echo')
>>
> 
> OK, I've added the dependency on cmd_importenv and cmd_exportenv, but
> that does not make it work any better.
> 
> I added my U-Boot repo to Travis and ran the tests. Here is the output
> of the job: https://travis-ci.org/QSchulz/u-boot/
> 
> Specifically, you have:
> https://travis-ci.org/QSchulz/u-boot/jobs/396742661
> https://travis-ci.org/QSchulz/u-boot/jobs/396742668
> https://travis-ci.org/QSchulz/u-boot/jobs/396742669
> https://travis-ci.org/QSchulz/u-boot/jobs/396742670
> https://travis-ci.org/QSchulz/u-boot/jobs/396742671
> 
> I've dumped the RAM after the `env export` and it looks pretty much
> empty compared to what I could see with sandbox tests.
> 
> Since all the other tests work, I'm not sure I actually introduced a
> regression or if it just never worked. I'll run tests without my patches
> that do a `env export` followed by a dump of the memory, a reset of the
> environement and a `env import` to see where we stand right now.
> 
> Before doing this test (which takes hours), my guess is that either `env
> export` is not working for the given configs, or there is something
> broken in the test framework (is the RAM address I get with
> u_boot_utils.find_ram_base() actually valid?).

I checked the qemu-x86 job from Tom's original failure:

https://travis-ci.org/trini/u-boot/jobs/391504523

At least two tests use find_ram_base(); test_net (uses a 4MB offset from 
the base) and test_md (uses the base as-is). Both these tests write data 
to the specified address, then check either the actual value, or a CRC32 
of a range of addresses, and both pass. This implies to me that 
find_ram_base() is returning the correct value, and the memory at that 
location works fine.

This doesn't rule out some problem in find_ram_base() on the other 
failing platforms of course; I didn't check those.