[U-Boot] livetime of boards

[Wolfgang Denk]

Dear Tom,

In message <20131107205133.GT5925 at bill-the-cat> you wrote:
> 
> What we need to be careful of here is making sure whatever we grow is
> both useful and not overly complicated.  What I honestly wonder about is
> automated testing for commands (crc32 pops to mind only because I just
> fixed things) but otherwise having things broken down into a front end
> where people select what they did "Booted a ___ into ___ via ___",
> provide some output from a command (maybe add just a touch more info to
> 'version') and cover non-boot testing with copy/paste'able drop-downs.
> 
> I know automated testing is The Thing, but given N frameworks, everyone
> of them has issues because frankly, every SoC family has its own quirks
> about how boot and load and what is and is not even feasible, especially
> for the bootloader.

Agreed.  In the end, you will probably define a specific set of test
cases that can be run automatically on a board.  We will never have
100% coverage, nor any generic set of tests that fits all boards.

OK, a pretty large sub-set of common functionality can be tested in
the sandbox, which is a great achievement ...

I still firmly belive that the approach we've taken a long time ago,
i. e. to use a test framework (DUTS) in combination with a (today wiki
based) documentation framework (DULG) is something that can meet a
wide range of requirements.

Unfortunately, we haven't been able yet to find a test framework that
makes it easy for the average user to add new test cases.  I'm not
even talking here about the core of the framework, that has to deal
with things like attaching to the console of a board, control power
and/or reset state, detect if a JTAG debugger is attached and such
things.  In the current state, our code is expect based (and thus
implemented in tcl), and it's really a PITA to add new test cases to
it.

BUt it would require significant efforts to rwrite this in a more
modern context...

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
In C we had to code our own bugs, in C++ we can inherit them.