[tbot] [DENX] tbot: board hangs if no autoload

[Harald Seiler]

On Thu, 2018-11-15 at 11:57 +0100, Wolfgang Denk wrote:
> Dear Stefano,
> 
> In message <4e6fafee-0a11-dbdc-5ebd-15af7529f8d8 at denx.de> you wrote:
> > > U-Boot
> > > 
> > > - get sources
> > > - may apply patches to it
> > >   list of patches in a directory ?
> > >   get a list of patches from patchwork todo list?
> > > 
> > > - install u-boot on the board
> > > - check if really the new version of u-boot boots
> > 
> > I admit that I am not very interested how to build and to push software
> > on the board. There are a lot of different paths, and this is not what
> > the customers ask. They have their own buildserver (Yocto / buildroot /
> 
> However, this is exactly what you do when you prepare a pull-request
> for Tom in your function as U-Boot custodian...
> 
> Saving time there is useful, too, isn't it?
> 
> > 	- does network work ?
> > 	- does storage work ?
> > 	- do other u-boot peripherals work ?
> > 
> > Such cases - they are unaware of which board is running, and we can at
> 
> Thisis not really true, unless you define a number of parameters
> (address ranges for storing images etc. in RAM, for example)
> for all boards.  In the old DUTS setup we did this, but I'm not sure
> if it has been a useful approach.
> 
> > > - convert all DUTS testcases
> > >   http://git.denx.de/?p=duts.git
> > 
> > I do not thing this is a great idea. This "duts" is obsolete and I think
> > we have now a more generic and better concept with tbot. I think we
> 
> DUTS is indeed obsolete, but the test cases have always been useful,
> and they still are.  One issue with U-Boot is the lack of up-to-date
> documentation, and the major reason for that is the lack of a test
> suite to generate the needed log files.
> 
> @Harald: I forgot to ask this during your presentation: can tbot
> generate log files that contain just the commands sent to the target
> and the replies received?

Yes! I think I even demonstrated a use of the log, but I am not sure ...
TBot generates a logfile which contains a json stream (ugh) that should
have all info you could possibly want.  That of course includes all commands
and their output + returnvalue.

Heiko's tbot had the ability to use this log to generate documentation.  I had
a similar feature at one point, but it was pretty ugly and I haven't had time
yet to redo it in a better way.

> 
> I understand you play some tricks with fancy prompts etc., but can
> we get something like this?
> 
> > > Linux:
> > > 
> > > - get sources
> > > - may apply patches to it
> > > - install linux on the board
> > > - check if booted version is the expected one
> > > - create a register dump file
> > >   write register content into a register dump file
> > > - do register checks
> > 
> > See above. I think this is useful during a porting, but it is less
> > useful for a customer who wants to test functionalities. I would like to
> > have first a catalog of testcases with functionalities, like:
> 
> Stefano, I think you miss one key point:  tbot  is _not_ only for
> regression testing for a customer.  It is (and actually most of all)
> a tool that is supposed to free _you_ in your daily work from
> routine tasks.  Your brain is way to precious to be wasted on such
> non-creative routine jobs.  Let tbot do these, and put your brain to
> better use!
> 
> 
> Best regards,
> 
> Wolfgang Denk
> 

-- 
Harald

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-62  Fax: +49-8142-66989-80   Email: hws at denx.de