[tbot] [Discussion] Calling tbot from within the source directory

Stefano Babic sbabic at denx.de
Fri Nov 30 09:10:57 UTC 2018


Hi Claudius,

On 29/11/18 13:46, Claudius Heine wrote:
> Hi Harald,
> 
> Quoting Harald Seiler (2018-11-29 12:33:42)
>> On Thu, 2018-11-29 at 10:47 +0100, Stefano Babic wrote:
>>> On 29/11/18 10:41, Claudius Heine wrote:
>>>> Just a suggestion from me, calling tbot like this:
>>>>
>>>>   tbot -c config.ini my-testcase
>>>>
>>>> Where the ini file would contain something like this:
>>>>
>>>>   [tbot]
>>>>   verbose=2
>>>>   cwd=/home/user/work/tbot/tbot-myproduct
>>>>   lab=lab/my-lab.py
>>>>   board=board/my-board.py
>>>>   flags=flag1 flag2
>>>>   testcase=my-default-testcase
>>>>
>>>>   [project]
>>>>   sources=work/source/my-project-source
>>>>   outdir=work/source/my-project-source/out
>>>>   nfsdir=/srv/nfs/my-project/user/
>>>>   tftpdir=/srv/tftp/my-project/user/
>>>>
>>>> What do you think?
>>>
>>> See start of the thread, it was already proposed, but Harald prefers to
>>> avoid such as configuration file.
>>
>> I think, I should elaborate on why:
>>
>> As I mentioned in my talk, the version of tbot you know is actually the second
>> rewrite.  The first one had an extensive configuration system, supporting
>> overlaying different config files, having configs depend on each other, and
>> generating config values on the fly from python code.  The reason it got so
>> complex was because I stumbled upon a lot of edge-cases that needed special
>> handling.
>>
>> It worked.  But it wasn't pretty.  And it was really hard to reason about.
>> I was just short of adding something like `bitbake -e` which would allow
>> you to trace where each value came from.
> 
> I agree that configuration can be done very complicated, but there is
> always some middle ground. Just because we don't like 100% make
> everything configurable and overwriteable and what ever, 10% or 50%
> configurable might still be nice to have.

Just a new point: configurations like we thought (a .tbotrc, for
example) can often hide an issue an make a comparison between working
and not working systems difficult. As tbot-new is still young, I would
agree to try to go on without introducing too many variables: each
configuration file introduces different behaviors. We can also think
about if we can reach what you want without configuration files.

> 
> I like hybrid approaches. Sometimes its important to break with some
> pattern in order to improve something. Its always hard to decide when to
> stick and when to break with some design pattern. So maybe I am wrong in
> this case but at some point blindly following any pattern will lead to a
> bad system. I try to be pragmatic about such things.
> 
>> When I decided I need to restart the project again, I took some time to embrace
>> the core principles of python [1] and to think about how they apply to tbot.
>>
>> I decided to throw out the configuration file idea completely.  I decided that
>> the most value is gained from tbot through bullet proof machine interaction and
>> that I leave everything else up to the user.  Configuration is one such thing.
>> For the lab and board there isn't really a way around it (although you could
>> also just define your board classes in your testcase source file without issues ...
>> it really is about convenience in this case).
>>   But for anything more I feel like it shouldn't be tbot's job to handle it.
>>
>> As I showed in the dcu-repo (only visible to Claudius), a user that needs to
>> have config for each developer can add a config mechanism using pythons
>> builtin `configparser`.  This has two advantages:
>>
>>         1. tbot can stay small and focus on its core job: Machine Interaction
>>
>>         2. A user that is not content with just a `configparser` based config
>>            can easily decide to switch to something different.  You could even
>>            think about pulling the config via network if that is what you want ...
> 
> I like what you did there and that prompted me to do it within the tbot
> instead of having to do that in every testcase and copy paste it from
> one to the next.
> 
> I am not talking about some king of over complicated configuration file
> with lots of includes and what not. I am talking about some file that
> can be loaded by tbot and contains just some key-value pairs.
> 
> It doesn't has to be an ini file, it can also be some python script that
> is loaded by tbot to get some default values and key-value pairs to be
> used in testcases.
> 
>> If tbot were to provide a config mechanism, I see two things happening:  For one,
>> people who don't know about this or need something more complex will still use their
>> own implementation (this violates "There should be one-- and preferably only one
>> --obvious way to do it." from the Zen).  On the other hand, people would need more
>> and more complex features added to this part of tbot which would take away time from
>> actually improving the parts of tbot that count.
>>
>> To quote the unix philosophy:
>>
>>         Do one thing and do it well
>>
>> There are already many existing python modules that handle configuration very well,
>> so I think adding another implementation in tbot would be counterproductive.
> 
> I agree completly.

+1

> Eighter use some simple config parser or just eval
> some python file to get your configuration in case of tbot. My hope is that if
> tbot provides some simple mechanism to get key-value pairs into testcases
> then people will not try to implement their own in each and every
> testcase making them incompatible to each other.

But maybe we can have some "tbot libraries" or "3rd party" components
that are not part of tbot core, as many projects do.

> 
> Why should they want complicated configuration mechanism if they already have
> powerful composition in this version of tbot? I am not talking about
> takeing that away, but just about providing some options to have some
> project/user configuration that is valid for tbot based test
> environments.
> 
> Also tbot command can be shorter even without alias or wrapper scripts, typing
> less it IMO always better especially with tbot were that is the ultimate
> goal :)
> 
> 
> However if I understand Heiko correctly then all of this can be easily
> done in a lab configuration. So we would have a "Lab" for every
> project/user.

I think this is a great idea and we can try to explore better (or for
me, to better understand). IMHO lab is the "setup" fo the project, and I
guess your use case can be done via methods of the lab object. What about a
	lh = cx.enter_context(tbot.acquire_lab())
	src = lh.get_sources()

> Then it should be easy to reuse other "Lab"s to recreate
> some sort of project/userconfiguration. I don't know how easy that is

Me too

> currently, especially about how python finds other directories that
> contain lab definitions in order to import them.
> I also down know how
> much boilerplate each lab definition needs. If those are non-issues,
> then I rest my case.

Regards,
Stefano


-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
=====================================================================


More information about the tbot mailing list