[tbot] Next steps

[Claudius Heine]

Hi Harald,

Quoting Harald Seiler (2019-01-16 11:01:26)
> Hello!
> 
> I want to ask for some input on the priorities of the following
> features.  Which of the following features/changes would you need
> the most?

My personal priorities would be something like this: (highest prio first)
 - Refactored Configuration
 - possible JTAG Debugger integration next
   I put that here since it make sense to me to tackle that together with the
   refactored configuration
 - Refactored Build-System
   Building is a feature of tbot that other tools like lava lack (AFAIK)
 - Documentation Generation
   I currently have no customer expecting that, so for me personally not that important.
 - More builtin Testcases and Examples
   buildin testcases are examples and vice versa
 - More Documentation
   For me examples are worth more than documentation, since examples
   should show best practices and documentation I can get reading the
   code myself.

 - Denx CI
   I would say that will go in parallel to all of that, since we need to
   experience the strengths and weaknesses of tbot in real world
   circumstances in order to improve it.

regards,
Claudius

> ## Documentation Generation
>    ------------------------
> Old tbot had this feature and I definitely want it in new tbot as well.
> The basic idea is the following:  A generator that takes a test-run's logfile
> and a 'template' and uses these to generate a pdf documenting the process
> of reproducing the results.  The usecase is for example automatically generating
> documentation for board bringup.
> 
> ## Refactored Configuration
>    ------------------------
> Currently tbot takes two config files, one with `-l` and one with `-b`.
> The new system would only have one parameter `-c` which can be specified
> multiple times.  Each config file can then specify any of
> 
>         * LAB
>         * BOARD
>         * UBOOT
>         * LINUX
> 
> .  tbot will read the files in order and for each of the names takes
> the last one that was defined.
>    This would allow a more modularized config approach which (hopefully)
> makes sharing configs easier.
> 
> ## JTAG Debugger Integration
>    -------------------------
> New machine flavors for eg. BDI debuggers.
> 
> ## More Documentation
>    ------------------
> Right now, the docs are pretty lacking, especially for onboarding and
> getting started with tbot.  This has to change sooner rather than later
> but might not be the biggest prio right now ... You decide!
>    Another point to include here is "marketing":  I got feedback that
> while the docs provide a reference, there is way to little explanation
> what tbot is actually useful for.  "Why should I even use tbot?" The
> reason I haven't written much about this is, to be quite honest:  I am
> not yet sure.  Right now people are experimenting what tbot is good at,
> where it is still lacking, or where it might not the right choice
> at all.  I need some input here ... Tell me what you think!
> 
> ## Refactored Build-System
>    -----------------------
> As I previously mentioned, the current build testcases are not really
> in a state that is fun to use.  Before making them more feature rich
> there should be an overhaul of the design.
> 
> ## More builtin Testcases
>    ----------------------
> Right now, there are about 5 testcases included with tbot (not counting
> selftests).  While I don't want to stick every possible testcase into
> tbot core, there should definitely be a few more.  For example a testcase
> to build linux or to run U-Boot's test/py.
> 
> ## Examples
>    --------
> While the docs contain some code scattered about, there is no official
> working demo yet.  I think such a repo would help beginners a lot with
> understanding the basics of tbot.
> 
> ## DENX Internal: CI
>    -----------------
> New tbot should run in a CI for all our hardware at some point.  This needs
> to be set up so you can add your CI testcases. (Discussion about this should
> probably be moved to the internal ML, especially security considerations)
> 
> 
> If there is anything I missed, please mention that as well!
> 
> -- 
> Harald
> 
> DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: +49-8142-66989-62  Fax: +49-8142-66989-80   Email: hws at denx.de 
> 
> _______________________________________________
> tbot mailing list
> tbot at lists.denx.de
> https://lists.denx.de/listinfo/tbot

--
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch at denx.de

           PGP key: 6FF2 E59F 00C6 BC28 31D8 64C1 1173 CB19 9808 B153
                             Keyserver: hkp://pool.sks-keyservers.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: signature
URL: <http://lists.denx.de/pipermail/tbot/attachments/20190118/48eabfa4/attachment-0001.sig>