[PATCH v1] CI: Add jsonschema python module

[Andrejs Cainikovs]

On 29/08/2023 17:37, Andrejs Cainikovs wrote:
> On 29/08/2023 17:04, Tom Rini wrote:
>> Interesting.  How exactly are you using these CI images? We have
>> tools/buildman/requirements.txt now to cover the newly-added modules in
>> CI, as we call pip on that.  But, it's not being used in the prepopulate
>> the pip cache stage in the Dockerfile.  And, I'm not immune to the idea
>> that we should instead be installing the distro package here.

Tom, I got the full picture right after I sent you my previous email.
This particular file, tools/buildman/requirements.txt in used in *CI*,
right.

Well, in this case my patch doesn't make much sense. Or it does.

I'd like to propose to install packages to runner image instead. I see
the reason why it might be convenient to have packages installed to
runner for particular jobs via .gitlab.yml - a bit less maintenance of
Docker image, which would need to be rebuilt every time there's new
dependency. However, if you think about pipeline execution times, on a
global scale, that's a waste. As an example, I see four occasions of:

pip install -r tools/buildman/requirements.txt

Twice in world build stage and twice in testsuites stage.

If we take another example, packages from py/requirements.txt are
installed 37 (if I counted correctly) times during entire testsuite
stage [2]. And I see you run them a lot [1].

Since you are maintainer of this, I will leave this up to you. But if
you're up for this proposal, I can do this change by myself. Of course,
this would imply rebuilding Docker image and pushing to registry from
your side, to keep stuff synchronized.

[1]: https://source.denx.de/u-boot/u-boot/-/pipelines/17556
[2]: https://source.denx.de/u-boot/u-boot/-/jobs/685270#L366

Regards,
Andrejs.

> 
> I see in Dockerfile test/py/requirements.txt and
> doc/sphinx/requirements.txt, but not tools/buildman/requirements.txt, so
> whatever is mentioned there is not part of the image on CI runners.
> 
> Following is the part of *our own* .gitlab.yml file,
> which uses a public Docker image
> (docker.io/trini/u-boot-gitlab-ci-runner:jammy-20230624-20Jul2023, as of
> now):
> 
> verdin-am62:
>   stage: build
>   script: |
>     ...
>     tools/buildman/buildman -o /tmp -P -E "verdin-am62" -M
>     ...
> 
> And this is how it fails in our CI without this change:
> 
> Building current source for 3 boards (3 threads, 14 jobs per thread)
>        arm:  +   verdin-am62_r5
> +binman: Unknown entry type 'ti-board-config' in node
> '/binman/board-cfg/ti-board-config' (expected etype/ti_board_config.py,
> error 'No module named 'jsonschema''
> +make[1]: *** [Makefile:1108: .binman_stamp] Error 1
> +make: *** [Makefile:177: sub-make] Error 2
> 

-- 
Best regards,
Andrejs Cainikovs