[RFC PATCH] Gitlab CI: Clean up more

[Tom Rini]

On Mon, Mar 17, 2025 at 09:12:14PM +0200, Ilias Apalodimas wrote:
> Hi Tom,
> 
> On Sat, 15 Mar 2025 at 16:17, Tom Rini <trini at konsulko.com> wrote:
> >
> > A problem we hit on our CI runners themselves, from time to time, is a
> > lack of disk space on the host. This is because Gitlab has no automation
> > itself around the removal of items it put in a "cache". While part of
> > the way to alleviate this is for the runners to run various docker
> > cleanup commands on their own, periodically, another thing to
> > potentially help would be to further clean-up in the job itself.
> >
> > This patch adds "git clean -dfx" in jobs that leave some files in the
> > source tree (these files may lead to a cache miss, more investigation
> > would be required) and deleting the board build directory. This part is
> > both a minimal size reclamation and minimal time increase (a few
> > seconds).
> >
> > The other part this adds is making the world build use "/tmp/world" as
> > the output directory not "/tmp" so that we can then delete "/tmp/world"
> > afterwards. This adds between 1m30 and 3m to the build.
> >
> > Signed-off-by: Tom Rini <trini at konsulko.com>
> > ---
> > Cc: Simon Glass <sjg at chromium.org>
> > Cc: Ilias Apalodimas <ilias.apalodimas at linaro.org>
> > ---
> >  .gitlab-ci.yml | 22 ++++++++++++++++++----
> >  1 file changed, 18 insertions(+), 4 deletions(-)
> >
> > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> > index 2dbe6325f334..ce7a391bbd00 100644
> > --- a/.gitlab-ci.yml
> > +++ b/.gitlab-ci.yml
> > @@ -63,8 +63,9 @@ stages:
> >          -r tools/buildman/requirements.txt -r tools/u_boot_pylib/requirements.txt
> >
> >    after_script:
> > +    - git clean -dfx
> >      - cp -v /tmp/${TEST_PY_BD}/*.{html,css,xml} .
> > -    - rm -rf /tmp/uboot-test-hooks /tmp/venv
> > +    - rm -rf /tmp/uboot-test-hooks /tmp/venv /tmp/${TEST_PY_BD}
> 
> This deletes less?

No? Note that we're adding the "git clean -dfx" and that must be before
we copy the files grabbed as artifacts in.

> >    script:
> >      # If we've been asked to use clang only do one configuration.
> >      - export UBOOT_TRAVIS_BUILD_DIR=/tmp/${TEST_PY_BD}
> > @@ -130,11 +131,13 @@ build all platforms in a single job:
> >          -r tools/buildman/requirements.txt
> >      - ret=0;
> >        git config --global --add safe.directory "${CI_PROJECT_DIR}";
> > -      ./tools/buildman/buildman -o /tmp -PEWM -x xtensa || ret=$?;
> > +      ./tools/buildman/buildman -o /tmp/world -PEWM -x xtensa || ret=$?;
> 
> Since you are making the tmp-> /tmp/world it might make more sense to
> define a variable for the output directory.

OK.

> >        if [[ $ret -ne 0 ]]; then
> > -        ./tools/buildman/buildman -o /tmp -seP;
> > +        ./tools/buildman/buildman -o /tmp/world -seP;
> >          exit $ret;
> >        fi;
> [...]
> 
> Other than that I think it looks sane. The added 1 - 3 min build time
> is small enough to ignore

I'm ambivalent about it, and since Simon is most concerned with overall
pipeline time I'd like his input here too.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20250317/e05c1f65/attachment.sig>