[tbot] Next steps

[Heiko Schocher]

Hello Harald,

Am 16.01.2019 um 11:01 schrieb Harald Seiler:
> Hello!
> 
> I want to ask for some input on the priorities of the following
> features.  Which of the following features/changes would you need
> the most?
> 
> ## Documentation Generation
>     ------------------------
> Old tbot had this feature and I definitely want it in new tbot as well.
> The basic idea is the following:  A generator that takes a test-run's logfile
> and a 'template' and uses these to generate a pdf documenting the process
> of reproducing the results.  The usecase is for example automatically generating
> documentation for board bringup.
> 
> ## Refactored Configuration
>     ------------------------
> Currently tbot takes two config files, one with `-l` and one with `-b`.
> The new system would only have one parameter `-c` which can be specified
> multiple times.  Each config file can then specify any of
> 
> 	* LAB
> 	* BOARD
> 	* UBOOT
> 	* LINUX
> 
> .  tbot will read the files in order and for each of the names takes
> the last one that was defined.
>     This would allow a more modularized config approach which (hopefully)
> makes sharing configs easier.
> 
> ## JTAG Debugger Integration
>     -------------------------
> New machine flavors for eg. BDI debuggers.
> 
> ## More Documentation
>     ------------------
> Right now, the docs are pretty lacking, especially for onboarding and
> getting started with tbot.  This has to change sooner rather than later
> but might not be the biggest prio right now ... You decide!
>     Another point to include here is "marketing":  I got feedback that
> while the docs provide a reference, there is way to little explanation
> what tbot is actually useful for.  "Why should I even use tbot?" The
> reason I haven't written much about this is, to be quite honest:  I am
> not yet sure.  Right now people are experimenting what tbot is good at,
> where it is still lacking, or where it might not the right choice
> at all.  I need some input here ... Tell me what you think!
> 
> ## Refactored Build-System
>     -----------------------
> As I previously mentioned, the current build testcases are not really
> in a state that is fun to use.  Before making them more feature rich
> there should be an overhaul of the design.
> 
> ## More builtin Testcases
>     ----------------------
> Right now, there are about 5 testcases included with tbot (not counting
> selftests).  While I don't want to stick every possible testcase into
> tbot core, there should definitely be a few more.  For example a testcase
> to build linux or to run U-Boot's test/py.
> 
> ## Examples
>     --------
> While the docs contain some code scattered about, there is no official
> working demo yet.  I think such a repo would help beginners a lot with
> understanding the basics of tbot.
> 
> ## DENX Internal: CI
>     -----------------
> New tbot should run in a CI for all our hardware at some point.  This needs
> to be set up so you can add your CI testcases. (Discussion about this should
> probably be moved to the internal ML, especially security considerations)
> 
> 
> If there is anything I missed, please mention that as well!

We should think about adding a feature for starting commands
and read the output of it later ...

For example you start on one machine "candump" which does not end, than
on another machine send a can message with cansend ... now you get output
from candump, see for example in my old, hacky tbot:

https://github.com/hsdenx/tbot/blob/master/src/tc/linux/tc_workfd_can.py#L100

and if you finished send a Ctrl-C to end candump ...

or linux top testcase example:

https://github.com/hsdenx/tbot/blob/master/src/tc/linux/tc_linux_top.py#L81

There I start on a machine "top" ... top outputs all n seconds the new
values ... read m loops, analyse the output and write interested values
into a txt files, which gnuplot understands and send Ctrl-C to end top
command ...

Now I can call gnuplot ... and have a nice image ...

bye,
Heiko
-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-52   Fax: +49-8142-66989-80   Email: hs at denx.de