[tbot] [Discussion] Calling tbot from within the source directory

Thu Nov 29 12:46:47 UTC 2018

Hi Harald,

Quoting Harald Seiler (2018-11-29 12:33:42)
> On Thu, 2018-11-29 at 10:47 +0100, Stefano Babic wrote:
> > On 29/11/18 10:41, Claudius Heine wrote:
> > > Just a suggestion from me, calling tbot like this:
> > > 
> > >   tbot -c config.ini my-testcase
> > > 
> > > Where the ini file would contain something like this:
> > > 
> > >   [tbot]
> > >   verbose=2
> > >   cwd=/home/user/work/tbot/tbot-myproduct
> > >   lab=lab/my-lab.py
> > >   board=board/my-board.py
> > >   flags=flag1 flag2
> > >   testcase=my-default-testcase
> > > 
> > >   [project]
> > >   sources=work/source/my-project-source
> > >   outdir=work/source/my-project-source/out
> > >   nfsdir=/srv/nfs/my-project/user/
> > >   tftpdir=/srv/tftp/my-project/user/
> > > 
> > > What do you think?
> > 
> > See start of the thread, it was already proposed, but Harald prefers to
> > avoid such as configuration file.
> 
> I think, I should elaborate on why:
> 
> As I mentioned in my talk, the version of tbot you know is actually the second
> rewrite.  The first one had an extensive configuration system, supporting
> overlaying different config files, having configs depend on each other, and
> generating config values on the fly from python code.  The reason it got so
> complex was because I stumbled upon a lot of edge-cases that needed special
> handling.
> 
> It worked.  But it wasn't pretty.  And it was really hard to reason about.
> I was just short of adding something like `bitbake -e` which would allow
> you to trace where each value came from.

I agree that configuration can be done very complicated, but there is
always some middle ground. Just because we don't like 100% make
everything configurable and overwriteable and what ever, 10% or 50%
configurable might still be nice to have.

I like hybrid approaches. Sometimes its important to break with some
pattern in order to improve something. Its always hard to decide when to
stick and when to break with some design pattern. So maybe I am wrong in
this case but at some point blindly following any pattern will lead to a
bad system. I try to be pragmatic about such things.

> When I decided I need to restart the project again, I took some time to embrace
> the core principles of python [1] and to think about how they apply to tbot.
> 
> I decided to throw out the configuration file idea completely.  I decided that
> the most value is gained from tbot through bullet proof machine interaction and
> that I leave everything else up to the user.  Configuration is one such thing.
> For the lab and board there isn't really a way around it (although you could
> also just define your board classes in your testcase source file without issues ...
> it really is about convenience in this case).
>   But for anything more I feel like it shouldn't be tbot's job to handle it.
> 
> As I showed in the dcu-repo (only visible to Claudius), a user that needs to
> have config for each developer can add a config mechanism using pythons
> builtin `configparser`.  This has two advantages:
> 
>         1. tbot can stay small and focus on its core job: Machine Interaction
> 
>         2. A user that is not content with just a `configparser` based config
>            can easily decide to switch to something different.  You could even
>            think about pulling the config via network if that is what you want ...

I like what you did there and that prompted me to do it within the tbot
instead of having to do that in every testcase and copy paste it from
one to the next.

I am not talking about some king of over complicated configuration file
with lots of includes and what not. I am talking about some file that
can be loaded by tbot and contains just some key-value pairs.

It doesn't has to be an ini file, it can also be some python script that
is loaded by tbot to get some default values and key-value pairs to be
used in testcases.

> If tbot were to provide a config mechanism, I see two things happening:  For one,
> people who don't know about this or need something more complex will still use their
> own implementation (this violates "There should be one-- and preferably only one
> --obvious way to do it." from the Zen).  On the other hand, people would need more
> and more complex features added to this part of tbot which would take away time from
> actually improving the parts of tbot that count.
> 
> To quote the unix philosophy:
> 
>         Do one thing and do it well
> 
> There are already many existing python modules that handle configuration very well,
> so I think adding another implementation in tbot would be counterproductive.

I agree completly. Eighter use some simple config parser or just eval
some python file to get your configuration in case of tbot. My hope is that if
tbot provides some simple mechanism to get key-value pairs into testcases
then people will not try to implement their own in each and every
testcase making them incompatible to each other.

Why should they want complicated configuration mechanism if they already have
powerful composition in this version of tbot? I am not talking about
takeing that away, but just about providing some options to have some
project/user configuration that is valid for tbot based test
environments.

Also tbot command can be shorter even without alias or wrapper scripts, typing
less it IMO always better especially with tbot were that is the ultimate
goal :)

However if I understand Heiko correctly then all of this can be easily
done in a lab configuration. So we would have a "Lab" for every
project/user. Then it should be easy to reuse other "Lab"s to recreate
some sort of project/userconfiguration. I don't know how easy that is
currently, especially about how python finds other directories that
contain lab definitions in order to import them. I also down know how
much boilerplate each lab definition needs. If those are non-issues,
then I rest my case.

regards,
Claudius

--
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch at denx.de

           PGP key: 6FF2 E59F 00C6 BC28 31D8 64C1 1173 CB19 9808 B153
                             Keyserver: hkp://pool.sks-keyservers.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: signature
URL: <http://lists.denx.de/pipermail/tbot/attachments/20181129/74a47a4d/attachment.sig>