[tbot] [Discussion] Calling tbot from within the source directory
sbabic at denx.de
Fri Nov 30 09:10:57 UTC 2018
On 29/11/18 13:46, Claudius Heine wrote:
> Hi Harald,
> Quoting Harald Seiler (2018-11-29 12:33:42)
>> On Thu, 2018-11-29 at 10:47 +0100, Stefano Babic wrote:
>>> On 29/11/18 10:41, Claudius Heine wrote:
>>>> Just a suggestion from me, calling tbot like this:
>>>> tbot -c config.ini my-testcase
>>>> Where the ini file would contain something like this:
>>>> flags=flag1 flag2
>>>> What do you think?
>>> See start of the thread, it was already proposed, but Harald prefers to
>>> avoid such as configuration file.
>> I think, I should elaborate on why:
>> As I mentioned in my talk, the version of tbot you know is actually the second
>> rewrite. The first one had an extensive configuration system, supporting
>> overlaying different config files, having configs depend on each other, and
>> generating config values on the fly from python code. The reason it got so
>> complex was because I stumbled upon a lot of edge-cases that needed special
>> It worked. But it wasn't pretty. And it was really hard to reason about.
>> I was just short of adding something like `bitbake -e` which would allow
>> you to trace where each value came from.
> I agree that configuration can be done very complicated, but there is
> always some middle ground. Just because we don't like 100% make
> everything configurable and overwriteable and what ever, 10% or 50%
> configurable might still be nice to have.
Just a new point: configurations like we thought (a .tbotrc, for
example) can often hide an issue an make a comparison between working
and not working systems difficult. As tbot-new is still young, I would
agree to try to go on without introducing too many variables: each
configuration file introduces different behaviors. We can also think
about if we can reach what you want without configuration files.
> I like hybrid approaches. Sometimes its important to break with some
> pattern in order to improve something. Its always hard to decide when to
> stick and when to break with some design pattern. So maybe I am wrong in
> this case but at some point blindly following any pattern will lead to a
> bad system. I try to be pragmatic about such things.
>> When I decided I need to restart the project again, I took some time to embrace
>> the core principles of python  and to think about how they apply to tbot.
>> I decided to throw out the configuration file idea completely. I decided that
>> the most value is gained from tbot through bullet proof machine interaction and
>> that I leave everything else up to the user. Configuration is one such thing.
>> For the lab and board there isn't really a way around it (although you could
>> also just define your board classes in your testcase source file without issues ...
>> it really is about convenience in this case).
>> But for anything more I feel like it shouldn't be tbot's job to handle it.
>> As I showed in the dcu-repo (only visible to Claudius), a user that needs to
>> have config for each developer can add a config mechanism using pythons
>> builtin `configparser`. This has two advantages:
>> 1. tbot can stay small and focus on its core job: Machine Interaction
>> 2. A user that is not content with just a `configparser` based config
>> can easily decide to switch to something different. You could even
>> think about pulling the config via network if that is what you want ...
> I like what you did there and that prompted me to do it within the tbot
> instead of having to do that in every testcase and copy paste it from
> one to the next.
> I am not talking about some king of over complicated configuration file
> with lots of includes and what not. I am talking about some file that
> can be loaded by tbot and contains just some key-value pairs.
> It doesn't has to be an ini file, it can also be some python script that
> is loaded by tbot to get some default values and key-value pairs to be
> used in testcases.
>> If tbot were to provide a config mechanism, I see two things happening: For one,
>> people who don't know about this or need something more complex will still use their
>> own implementation (this violates "There should be one-- and preferably only one
>> --obvious way to do it." from the Zen). On the other hand, people would need more
>> and more complex features added to this part of tbot which would take away time from
>> actually improving the parts of tbot that count.
>> To quote the unix philosophy:
>> Do one thing and do it well
>> There are already many existing python modules that handle configuration very well,
>> so I think adding another implementation in tbot would be counterproductive.
> I agree completly.
> Eighter use some simple config parser or just eval
> some python file to get your configuration in case of tbot. My hope is that if
> tbot provides some simple mechanism to get key-value pairs into testcases
> then people will not try to implement their own in each and every
> testcase making them incompatible to each other.
But maybe we can have some "tbot libraries" or "3rd party" components
that are not part of tbot core, as many projects do.
> Why should they want complicated configuration mechanism if they already have
> powerful composition in this version of tbot? I am not talking about
> takeing that away, but just about providing some options to have some
> project/user configuration that is valid for tbot based test
> Also tbot command can be shorter even without alias or wrapper scripts, typing
> less it IMO always better especially with tbot were that is the ultimate
> goal :)
> However if I understand Heiko correctly then all of this can be easily
> done in a lab configuration. So we would have a "Lab" for every
I think this is a great idea and we can try to explore better (or for
me, to better understand). IMHO lab is the "setup" fo the project, and I
guess your use case can be done via methods of the lab object. What about a
lh = cx.enter_context(tbot.acquire_lab())
src = lh.get_sources()
> Then it should be easy to reuse other "Lab"s to recreate
> some sort of project/userconfiguration. I don't know how easy that is
> currently, especially about how python finds other directories that
> contain lab definitions in order to import them.
> I also down know how
> much boilerplate each lab definition needs. If those are non-issues,
> then I rest my case.
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
More information about the tbot