[PATCH v2 28/28] test: Add a test for booting Ubuntu 24.04

[Simon Glass]

Hi Tom,

On Tue, 18 Feb 2025 at 08:11, Tom Rini <trini at konsulko.com> wrote:
>
> On Tue, Feb 18, 2025 at 05:09:23AM -0700, Simon Glass wrote:
> > Hi Tom,
> >
> > On Mon, 17 Feb 2025 at 10:52, Tom Rini <trini at konsulko.com> wrote:
> > >
> > > On Sun, Feb 16, 2025 at 01:44:13PM -0700, Simon Glass wrote:
> > > > Now that U-Boot can boot this quickly, using kvm, add a test that the
> > > > installer starts up correctly.
> > > >
> > > > Use the qemu-x86_64 board in the SJG lab.
> > > >
> > > > Signed-off-by: Simon Glass <sjg at chromium.org>
> > > > ---
> > > >
> > > > Changes in v2:
> > > > - Add more patches to support booting with kvm
> > > > - Add new patch with a test for booting Ubuntu 24.04
> > > >
> > > >  .gitlab-ci.yml               |  5 ++++
> > > >  test/py/tests/test_distro.py | 53 ++++++++++++++++++++++++++++++++++++
> > > >  2 files changed, 58 insertions(+)
> > > >  create mode 100644 test/py/tests/test_distro.py
> > > >
> > > > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> > > > index 8c49d5b0a79..ec799e97c10 100644
> > > > --- a/.gitlab-ci.yml
> > > > +++ b/.gitlab-ci.yml
> > > > @@ -745,3 +745,8 @@ zybo:
> > > >    variables:
> > > >      ROLE: zybo
> > > >    <<: *lab_dfn
> > > > +
> > > > +qemu-x86_64:
> > > > +  variables:
> > > > +    ROLE: qemu-x86_64
> > > > +  <<: *lab_dfn
> > >
> > > I'm not sure why this is in your lab stanza, rather than the normal
> > > test.py QEMU stanza.
> >
> > Are you wanting to add the Ubuntu image into CI? It is quite large.
>
> If we're going to be able to run it on N platforms, yes, we need to
> think of a good way to cache the download. There's not a particular
> reason we can't run the stock Ubuntu RISC-V image on the two sifive
> targets and also qemu-riscv64, is there?

Yes, we can do that. It is pretty simple to set up in Labgrid and it
doesn't require all the runners to download a much larger image, etc.

>
> > > > diff --git a/test/py/tests/test_distro.py b/test/py/tests/test_distro.py
> > > > new file mode 100644
> > > > index 00000000000..51eec45cecc
> > > > --- /dev/null
> > > > +++ b/test/py/tests/test_distro.py
> > > > @@ -0,0 +1,53 @@
> > > > +# SPDX-License-Identifier: GPL-2.0+
> > > > +# Copyright 2025 Canonical Ltd.
> > > > +# Written by Simon Glass <simon.glass at canonical.com>
> > > > +
> > > > +import pytest
> > > > +
> > > > +DOWN = '\x1b\x5b\x42\x0d'
> > > > +
> > > > +# Enable early console so that the test can see if something goes wrong
> > > > +CONSOLE = 'earlycon=uart8250,io,0x3f8 console=uart8250,io,0x3f8'
> > > > +
> > > > + at pytest.mark.boardspec('qemu-x86_64')
> > > > + at pytest.mark.role('qemu-x86_64')
> > > > +def test_distro(ubman):
> > > > +    """Test that of-platdata can be generated and used in sandbox"""
> > > > +    with ubman.log.section('boot'):
> > > > +        ubman.run_command('boot', wait_for_prompt=False)
> > > > +
> > > > +    with ubman.log.section('Grub'):
> > > > +        # Wait for grub to come up and offset a menu
> > > > +        ubman.p.expect(['Try or Install Ubuntu'])
> > > > +
> > > > +        # Press 'e' to edit the command line
> > > > +        ubman.run_command('e', wait_for_prompt=False, send_nl=False)
> > > > +
> > > > +        # Wait until we see the editor appear
> > > > +        ubman.p.expect(['/casper/initrd'])
> > > > +
> > > > +        # Go down to the 'linux' line
> > > > +        ubman.send(DOWN * 3)
> > > > +
> > > > +        # Go to end of line
> > > > +        ubman.ctrl('E')
> > > > +
> > > > +        # Backspace to remove 'quiet splash'
> > > > +        ubman.send('\b' * len('quiet splash'))
> > > > +
> > > > +        # Send our noisy console
> > > > +        ubman.send(CONSOLE)
> > > > +
> > > > +        # Tell grub to boot
> > > > +        ubman.ctrl('X')
> > > > +        ubman.p.expect(['Booting a command list'])
> > > > +
> > > > +    with ubman.log.section('Linux'):
> > > > +        # Linux should start immediately
> > > > +        ubman.p.expect(['Linux version'])
> > > > +
> > > > +    with ubman.log.section('Ubuntu'):
> > > > +        # Shortly later, we should see this banner
> > > > +        ubman.p.expect(['Welcome to .*Ubuntu 24.04.1 LTS.*!'])
> > > > +
> > > > +    ubman.restart_uboot()
> > >
> > > And this seems very inflexible. Please see
> > > test/py/tests/test_net_boot.py for an example of how to have this be
> > > configurable and work on arbitrary platforms. What I assume is tricky is
> > > that the "role" part here is where you have a special disk image being
> > > passed. That too could be dealt with in u-boot-test-hooks in a few ways,
> > > and the images pre-fetched to the CI container. And if this was
> > > configurable similar to the example I noted above, it could check real
> > > hardware too.
> >
> > That wasn't the reaction I expected.
> >
> > Yes, it is inflexible, but it is a starting point. Isn't it better
> > than what we have today?
>
> Is your inflexible boot an OS test better than the flexible boot an OS
> test that we have today? No, it's not.

I didn't even know about it, or perhaps I forgot.

Perhaps this relates to getting the labgrid config published and
figuring out how to pass info from Labgrid to tests.

>
> > I would like to generalise this test to work on at least one real
> > board, preferably one that doesn't use grub.
>
> OK. The test we have today does that, if you check for the "Welcome to
> ..." string instead of the kernel has booted string. It also does
> netboot rather than run default bootcmd. But that's an easy enough test
> to write up. The only thing stopping me from doing that right now is I
> need to find a board in the lab where we installed an OS to eMMC and not
> SD card (some lab sd-mux issues).

OK. Labgrid has a 'features' thing which you can attach to targets, so
I should be able to use that to indicate that Ubuntu, Debian, Armbian,
etc. are available.

Regards,
Simon