[U-Boot] U-Boot Regression Testing

[Simon Glass]

Hi Curt,

On Wed, Oct 9, 2013 at 10:22 AM, Curt Brune <curt at cumulusnetworks.com> wrote:
> Hello,
>
> I have some questions about how U-Boot regression testing works.  I am
> assuming some regression testing happens during the release period
> across some representative sample of boards and architectures.
>
> I know people check for compilation failures with MAKEALL, but I am
> wondering about runtime testing.  I also understand that testing
> platform specific boot code is rather difficult (or easy depending on
> your perspective) - it boots or it does not boot.
>
> To be concrete -- how are core U-Boot commands and features tested?
> For example how do you test that FIT image support is not broken or
> that the 'env' command and all its options work properly?
>
> Googling did not turn up much on how this is done.
>
> On the social side -- is that something the community helps out with
> or something DENX does, or a mix?
>
> Are you using a test framework of some kind, either home grown or open
> source?  These things tend to become home grown over time :)
>
> This kind of testing usually takes the form of 'chat' scripts
> communicating over serial consoles.  Perhaps you are using expect,
> pexpect, python nose?
>
> We have a project, of which U-Boot is a part, that is starting to span
> multiple boards and architectures.  We make a few modifications to
> U-Boot and I want to start automated regression testing as the number
> of boards increases.
>
> If an existing framework exists that folks are happy with I would love
> to hear about it.  Equally, I am interested to hear about what did
> *not* work for people.  Whatever method our project uses the scripts
> will be publicly available.

You can use sandbox, which is a build of U-Boot that runs under Linux.
There are some existing tests in test/ - you mention FIT support and
there is a basic test for that (could be expanded, in fact I may have
a patch somewhere).

Note that there is also buildman which aims to be the new MAKEALL eventually.

Regards,
Simon


>
> Cheers,
> Curt
> _______________________________________________
> U-Boot mailing list
> U-Boot at lists.denx.de
> http://lists.denx.de/mailman/listinfo/u-boot