[U-Boot] [PATCH V2 1/7] test/py: Implement pytest infrastructure

[Michal Simek]

2015-12-16 19:09 GMT+01:00 Stephen Warren <swarren at wwwdotorg.org>:

> On 12/16/2015 10:43 AM, Michal Simek wrote:
>
>> Hi Stephen,
>>
>> 2015-12-16 17:27 GMT+01:00 Stephen Warren <swarren at wwwdotorg.org
>> <mailto:swarren at wwwdotorg.org>>:
>>
>>
>>     On 12/16/2015 08:11 AM, Michal Simek wrote:
>>
>>         On 9.12.2015 17:32, Stephen Warren wrote:
>>
>>             On 12/02/2015 03:18 PM, Stephen Warren wrote:
>>
>>                 This tool aims to test U-Boot by executing U-Boot shell
>>                 commands using
>>                 the
>>                 console interface. A single top-level script exists to
>>                 execute or attach
>>                 to the U-Boot console, run the entire script of tests
>>                 against it, and
>>                 summarize the results. Advantages of this approach are:
>>
>>                 - Testing is performed in the same way a user or script
>>                 would interact
>>                      with U-Boot; there can be no disconnect.
>>                 - There is no need to write or embed test-related code
>>                 into U-Boot
>>                 itself.
>>                      It is asserted that writing test-related code in
>>                 Python is simpler and
>>                      more flexible that writing it all in C.
>>                 - It is reasonably simple to interact with U-Boot in
>>                 this way.
>>
>>                 A few simple tests are provided as examples. Soon, we
>>                 should convert as
>>                 many as possible of the other tests in test/* and
>>                 test/cmd_ut.c too.
>>
>>                 In the future, I hope to publish (out-of-tree) the hook
>>                 scripts, relay
>>                 control utilities, and udev rules I will use for my own
>>                 HW setup.
>>
>>
>>             I finally got permission to publish these. Examples are at:
>>             https://github.com/swarren/uboot-test-hooks
>>
>>
>>         Interesting. What's the normal setup which you have for the board?
>>         I see from your description that you use numato usb relay - I
>>         expect one
>>         with more channels for reset.
>>         Then you are able to control boot mode. Is it also via the same
>>         relay?
>>         How do you power up the board?
>>
>>
>>     In my current setup I leave the board on all the time (or rather,
>>     manually turn on the power when I'm about to run the tests).
>>     Automating control of the power source is a step I'll take later.
>>
>>
>> ok.
>>
>>
>>     For Tegra, there are two important signals: reset and "force
>> recovery".
>>
>>
>>
>> Do you mean that these both signals are just connected out of chip?
>>
>
> Yes. Reset is typically driven into the PMIC, and the signal to request
> force recovery is driven into Tegra itself.
>
> Typically there are push-buttons on development boards to control those
> two signals. I've simply wired my relays across those buttons to simulate
> the button press.
>

ok I see.


>
>     Each of these has a separate relay, so the system currently uses 2
>>     relays per target board. The numato relay board I own has 8 relays,
>>     although there are a number of different models.
>>
>>
>> ok
>>
>>
>>     On Tegra, when reset is pulsed:
>>
>>     - If force-recovery is connected, the SoC enters USB recovery mode.
>>     In this state, SW can be downloaded over USB into RAM and executed.
>>
>>
>> Is this bootrom feature?
>>
>
> Yes.
>
> For xilinx boards there is all the time jtag available. It means
>> download can be done via jtag instead.
>>
>
> That sounds plausible. The only issue might be general system state; can
> you reset everything to POR defaults via JTAG before the download?



There is cpu reset, Soc reset on the board which can be used but I have IP
power switch. It means I can handle power which ensure correct state.


> If not, perhaps e.g. the eMMC controller was partially initialized by
> previous code, which might interfere with assumptions made by the new code
> that's downloaded?
>

I think my power switch solves this without any problem.


>
>     - If force-recovery is not connected, the SoC boots normally, from
>>     SW stored in flash (eMMC, SPI, ...)
>>
>>
>>     The example scripts always use recovery mode to download U-Boot into
>>     RAM rather than writing it to flash first and then resetting. This
>>     saves wear cycles on the flash, but does mean the download happens
>>     in the "reset" rather than "flash" script, which may make the
>>     examples a bit different than for some other SoCs.
>>
>>
>> Are you testing all boot modes? Because I expect these needs to be
>> tested too. Do you use SPL? If yes, are you going to test it in this way?
>>
>
> With those example scripts, cold boot isn't being tested. However, (a) I
> could define a new board ID (or pick up environment variables) to cause
> that to be tested sometimes (b) I don't recall having seen any differences
> between cold boot and recovery mode boot in the past; we get a lot of
> quicker/lower-wear test coverage this way without too much additional risk.
>

ok.


>
> SPL is in use. However, SPL on Tegra has a bit of a different job than it
> has on some other chips. The boot ROM always initializes SDRAM, and SPL
> actually runs on a different CPU and primarily has the job of booting the
> main CPU where the main U-Boot binary runs. For more information, see:
>
> ftp://download.nvidia.com/tegra-public-appnotes/index.html
>

Interesting.



>
>     Finally, the example scripts support two boards; my home/laptop dev
>>     setup that uses a Numato relay board to control the signals to the
>>     board I use there, and my work desktop dev setup that uses our
>>     "PM342" debug board to controll the signals. The latter works
>>     logically the same as the numato relay board, except it contains
>>     electronic switches driven by an FTDI chip.
>>
>> I expect this is FTDI chip on the target right?
>>
>
> It's actually a separate common debug board. Most/all of our development
> boards (and perhaps some production boards) have a standardized connector
> into which the common debug board plugs.
>


ok.
I think my setup is not that far from what you are using and I expect that
others SoCs will be very similar.
Do you have any other testcases which you are running and you haven't sent?

I will try to find out some time for testing this week. If not, then next
year.

Thanks,
Michal


-- 
Michal Simek, Ing. (M.Eng), OpenPGP -> KeyID: FE3D1F91
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel - Microblaze cpu - http://www.monstr.eu/fdt/
Maintainer of Linux kernel - Xilinx Zynq ARM architecture
Microblaze U-BOOT custodian and responsible for u-boot arm zynq platform