[PATCH 0/3] RFC: test: Bring in the test hooks

[Heinrich Schuchardt]

On 5/2/25 17:06, Tom Rini wrote:
> On Fri, May 02, 2025 at 08:49:12AM -0600, Simon Glass wrote:
>> Hi Tom,
>>
>> On Fri, 2 May 2025 at 08:34, Tom Rini <trini at konsulko.com> wrote:
>>>
>>> On Thu, May 01, 2025 at 08:50:16PM -0600, Simon Glass wrote:
>>>
>>>> During a recent discussion with Heinrich we discussed why the hooks are
>>>> kept in a separate repo.
>>>>
>>>> The amount of code is small, a tenth of the size of the recently added
>>>> lwip, just by way of example. Testing is a critical part of U-Boot and
>>>> one of the things that distinguishes it from firmware projects that have
>>>> not kept up in this area. By having the tests somewhere else, we are
>>>> signalling that it is unusual, or difficult, or optional.
>>>>
>>>> The hooks mechanism also needs something of an update to take account of
>>>> real boards in 2025. That will be much easier to undertake if the code
>>>> that test/py talks to is in the same repo.
>>>>
>>>> This series brings the hook files in as first-class citizens of U-Boot.
>>>>
>>>> If we do go ahead with this, I will send a different series which has
>>>> separate commits (with correct author) in the u-boot-test-hooks repo.
>>>
>>> I think bringing more projects directly in to the repository is a bad
>>> idea. Your example of lwip isn't applicable because it's a read-only
>>> subtree that's maintained outside of the project (same as the dts
>>> subtree). But sure, lets "Say Yes". That said, we still need to:
>>> - Remove needless examples from the tree.
>>> - Not include personal labs directly in the tree.
>>>
>>> That last one is why I really think this is a bad idea. The point of
>>> having the hooks standalone is so that any given lab can easily add
>>> support for their lab and manage it, without worrying about disclosing
>>> internal layout. There's going to be hard coded default passwords there.

Secrets MUST NEVER reside in git repos.

I can't imagine that such irresponsible habits would be practiced by any 
accountable developer.

Please, have a look at

https://docs.github.com/en/actions/security-for-github-actions/security-guides/using-secrets-in-github-actions
https://tsi-ccdoc.readthedocs.io/en/master/ResOps/2019/gitlab/07_pass-build-secrets.html

to see how to do it properly.

The same is true for a private lab:
Use environment variables that are coming from outside the repository.

>>> There's going to be repository secrets there. That kind of information
>>> really should not be in a public repository. Integrating the hooks with
>>> mainline will make lab management harder, not easier. The point of the
>>> existing labs in u-boot-test-hooks is to provide samples.
>>>
>>> I think this is all why no, we should not go down this path.
>>
>> Is it worth discussing this, or is your mind made up? I have some
>> thoughts on the last one.
> 
> I think it's a terrible idea that I already said:
>> But sure, lets "Say Yes".
> 
> But please do spend time explaining your thoughts and perhaps others
> will also agree with you and I'll feel less bad taking this in?
> 

Currently when changing a test hook I have to go through these steps:

* fork u-boot-test-hooks
* push a commit to my fork
* update both .gitlab-ci.yaml and .azure-pipelines.yaml
* commit the change code.denx.de
* create a merge request for github.com/u-boot/u-boot

If the test hooks were in the same repo as main U-Boot, I could save two 
steps.

Best regards

Heinrich