Gitlab runner timeout on evb-ast2600 ?

Fri Apr 11 18:59:22 CEST 2025

On Fri, Apr 11, 2025 at 10:42:48AM -0600, Tom Rini wrote:
> On Fri, Apr 11, 2025 at 05:26:55PM +0200, Cédric Le Goater wrote:
> > On 4/11/25 17:01, Tom Rini wrote:
> > > On Fri, Apr 11, 2025 at 09:23:59AM +0200, Cédric Le Goater wrote:
> > > > Hi,
> > > > 
> > > > On 4/11/25 01:14, Tom Rini wrote:
> > > > > On Fri, Apr 11, 2025 at 08:07:15AM +0930, Joel Stanley wrote:
> > > > > > On Mon, 7 Apr 2025 at 23:25, Eugen Hristev <eugen.hristev at linaro.org> wrote:
> > > > > > > 
> > > > > > > Hi everyone,
> > > > > > > 
> > > > > > > I keep getting an error on running the CI loop on -next on my tree :
> > > > > > > 
> > > > > > > https://source.denx.de/u-boot/custodians/u-boot-at91/-/jobs/1090517
> > > > > > 
> > > > > > Thanks for the report.
> > > > > > 
> > > > > > > 
> > > > > > > ! _pytest.outcomes.Exit: Lab timeout: Marking connection bad - no other
> > > > > > > tests will run !
> > > > > > > 
> > > > > > > Any clue ? It's been like this for few days. It doesn't look like a
> > > > > > > faulty patch.
> > > > > > 
> > > > > > I can't see that link you sent me, it 403s.
> > > > > > 
> > > > > > Do you have any other traces or information to share?
> > > > > 
> > > > > It's been intermittent recently. I was able to put qemu + gdb together
> > > > > and see it's one of the early initcalls that fails and hang()s but
> > > > > didn't see why. You should be able to use commit 1f0281294d8b ("Merge
> > > > > patch series "Annotate switch/case fallthrough cases"") as a starting
> > > > > point of failure.
> > > > > 
> > > > 
> > > > Do you have logs ?
> > > > 
> > > > Things look ok from the QEMU side (latest 10.0.0-rc and 9.1) :
> > > >      uboot_build=/path/to/u-boot/
> > > >      flashsize=65536
> > > >      flash=./flash-ast2600-evb-uboot
> > > >      dd of=$flash bs=1K if=/dev/zero count=$flashsize
> > > >      dd of=$flash conv=notrunc bs=1K if=$uboot_build/spl/u-boot-spl.bin
> > > >      dd of=$flash conv=notrunc bs=1K if=$uboot_build/u-boot.img seek=64
> > > >      $ qemu-system-arm -M ast2600-evb -net nic,netdev=net0 -netdev user,id=net0 -nographic -snapshot -serial mon:stdio   -drive file=flash-ast2600-evb-uboot,format=raw,if=mtd
> > > >      qemu-system-arm: warning: nic ftgmac100.1 has no peer
> > > >      qemu-system-arm: warning: nic ftgmac100.2 has no peer
> > > >      qemu-system-arm: warning: nic ftgmac100.3 has no peer
> > > >      U-Boot SPL 2025.04-00921-gcb7555e93075 (Apr 11 2025 - 09:08:40 +0200)
> > > >      Trying to boot from RAM
> > > >      ## Checking hash(es) for config conf-1 ... OK
> > > >      ## Checking hash(es) for Image firmware-1 ... crc32+ OK
> > > >      ## Checking hash(es) for Image fdt-1 ... crc32+ OK
> > > >      U-Boot 2025.04-00921-gcb7555e93075 (Apr 11 2025 - 09:08:40 +0200)
> > > 
> > > Please note that it's inconsistent about what commit fails. I would
> > > expect 1f0281294d8b in U-Boot to fail but I don't know about
> > > cb7555e93075.
> > > 
> > > Also can you please check QEMU v8.2.0? That's what we're on, and I can
> > > try and test 9.1.3 or 10.0.0-rc and see if it's a QEMU problem instead.
> > > Thanks!
> > 
> > Same result :
> > 
> > qemu-system-arm -M ast2600-evb -net nic,netdev=net0 -netdev user,id=net0 -nographic -snapshot -serial mon:stdio   -drive file=flash-ast2600-evb-uboot,format=raw,if=mtd
> > qemu-system-arm: warning: nic ftgmac100.1 has no peer
> > qemu-system-arm: warning: nic ftgmac100.2 has no peer
> > qemu-system-arm: warning: nic ftgmac100.3 has no peer
> > 
> > U-Boot SPL 2025.04-00881-g1f0281294d8b (Apr 11 2025 - 17:23:25 +0200)
> > Trying to boot from RAM
> > ## Checking hash(es) for config conf-1 ... OK
> > ## Checking hash(es) for Image firmware-1 ... crc32+ OK
> > ## Checking hash(es) for Image fdt-1 ... crc32+ OK
> > 
> > 
> > U-Boot 2025.04-00881-g1f0281294d8b (Apr 11 2025 - 17:23:25 +0200)
> > 
> > Model: Aspeed BMC
> > DRAM:  already initialized, 1008 MiB (capacity:1024 MiB, VGA:64 MiB), ECC off
> > Core:  266 devices, 24 uclasses, devicetree: separate
> > WDT:   Started watchdog at 1e785000 with servicing every 1000ms (60s timeout)
> > MMC:   sdhci at 1e750100: 0
> > Loading Environment from SPIFlash... SF: Detected mx25u51245g with page size 256 Bytes, erase size 64 KiB, total 64 MiB
> > *** Warning - bad CRC, using default environment
> > 
> > In:    serial at 1e784000
> > Out:   serial at 1e784000
> > Err:   serial at 1e784000
> > Net:   eth0: ftgmac at 1e660000, eth1: ftgmac at 1e680000, eth2: ftgmac at 1e670000, eth3: ftgmac at 1e690000
> > Hit any key to stop autoboot:  0
> > libfdt fdt_check_header(): FDT_ERR_BADMAGIC
> > Error loading kernel FIT image
> > => QEMU 8.2.10 monitor - type 'help' for more information
> > (qemu) q
> > 
> > 
> > What about the compiler ?
> 
> It's:
> https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/13.2.0/x86_64-gcc-13.2.0-nolibc-aarch64-linux.tar.xz

And part of what makes this a pain to debug is that trivial changes
break/fail it here. For example, ff0b7d741dec fails for me but if I make
a tiny source change so the version string is ff0b7d741dec-dirty it
passes. So you may need something longer for your ${uboot_build}
directory, mine is /home/trini/work/u-boot/u-boot.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20250411/28e76928/attachment.sig>