[U-Boot] RFC: Testing U-Boot Part 1

[Wolfgang Denk]

Dear Simon Glass,

In message <CAPnjgZ3ZqFhZgtCAUSRngU3qB5ajxO9FthkmyuWurAWYkEvAHg at mail.gmail.com> you wrote:
> 
> Summary: I am quite keen on improving the test infrastructure in
> U-Boot. I would like to have a test suite that can run in a minute or
> two on a Linux PC and test all non-platform code.

I highly appreaciate such efforts!

> For speed, debugging and convenience, it would be nice to run U-Boot
> under a generic Linux environment on a workstation, and test all the
> generic non-platform code. The basic problem with this is that the
> non-platform code requires the platform code to operate. Even the x86
> platform code is designed for standalone operation on a particular x86
> board, and is not suitable for running under x86 Linux.

Some parts of what you are looking for can be done by running U-Boot
images in a QEMU based emulated envrionment.  So far we have only a
few targets in the tree that actually support this.  I wish we had
more of them.

I am aware that this is a different thing from what you propose here,
but I would like to state that we should not forget this option -
anybody who wants to work in that area will be more than welcome, too.


> To get around this I propose that we create a new ‘native’
> architecture. We write code in ‘arch/native’ which can run under
> Linux. Since all the non-platform code will happily run under this new
> ‘architecture’, we can then write tests which run quickly under x86
> Linux (or another Linux for that matter). This U-Boot 'architecture'
> should build natively on any 32/64-bit Linux machine since it just
> uses standard Linux system calls. Calls to Linux would be entirely
> within this arch/native subdirectory.

As Mike already pointed out, this is what BB has as "sandbox"
configuration.

> - Create a test SPI flash device, which is file-backed. Use this to
> write a test of the cmd_sf layer by issuing a series of commands and
> making sure that the correct SPI flash contents is obtained
>
> - Create a test MMC or IDE device, backed by a file. Use this to issue
> ext2 and fat commands to manipulate the filesystem. Then loopback
> mount the file and check from Linux that U-Boot did the right thing

In a later step it might even be possible to implement some "pass
through" mode to acces actual devices / drivers in some way.  Think of
using libusb to talk to USB devices.

> - Create a test Ethernet device with a mocked remote host attached.

I'm not sure what you mean by "a mocked remote host".  We should be
able to send and receive packets from a real network interface as
well.

> At this early stage I am looking for comments on the concept - how
> useful it is and how best to implement it.

Any step in this direction, how inclomplete it may be in the
beginning, is highly appreciated.  It may make sense to look over the
fence and study how BBis doing this.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Hokey religions and ancient weapons are  no  substitute  for  a  good
blaster at your side.                                      - Han Solo