U-Boot v2023.01 broken on riscv64 qemu virt platform: "fdt fixup event failed"

Thu Jan 12 00:42:49 CET 2023

Hi Heinrich,

On Wed, 11 Jan 2023 at 16:40, Heinrich Schuchardt <xypron.glpk at gmx.de> wrote:
>
> On 1/12/23 00:29, Simon Glass wrote:
> > () Hi Heinrich,
> >
> > On Wed, 11 Jan 2023 at 16:22, Heinrich Schuchardt <xypron.glpk at gmx.de> wrote:
> >>
> >> On 1/11/23 20:08, Karsten Merker wrote:
> >>> Hello,
> >>>
> >>> it looks like U-Boot v2023.01 is currently broken for the riscv64
> >>> architecture on the qemu "virt" platform; the boot process of a
> >>> riscv64 VM fails during FDT fixup:
> >>>
> >>> -----8<----------8<----------8<----------8<----------8<----------8<-----
> >>>
> >>> OpenSBI v1.1
> >>>      ____                    _____ ____ _____
> >>>     / __ \                  / ____|  _ \_   _|
> >>>    | |  | |_ __   ___ _ __ | (___ | |_) || |
> >>>    | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
> >>>    | |__| | |_) |  __/ | | |____) | |_) || |_
> >>>     \____/| .__/ \___|_| |_|_____/|____/_____|
> >>>           | |
> >>>           |_|
> >>>
> >>> Platform Name             : riscv-virtio,qemu
> >>> Platform Features         : medeleg
> >>> Platform HART Count       : 4
> >>> Platform IPI Device       : aclint-mswi
> >>> Platform Timer Device     : aclint-mtimer @ 10000000Hz
> >>> Platform Console Device   : uart8250
> >>> Platform HSM Device       : ---
> >>> Platform Reboot Device    : sifive_test
> >>> Platform Shutdown Device  : sifive_test
> >>> Firmware Base             : 0x80000000
> >>> Firmware Size             : 312 KB
> >>> Runtime SBI Version       : 1.0
> >>>
> >>> Domain0 Name              : root
> >>> Domain0 Boot HART         : 2
> >>> Domain0 HARTs             : 0*,1*,2*,3*
> >>> Domain0 Region00          : 0x0000000002000000-0x000000000200ffff (I)
> >>> Domain0 Region01          : 0x0000000080000000-0x000000008007ffff ()
> >>> Domain0 Region02          : 0x0000000000000000-0xffffffffffffffff (R,W,X)
> >>> Domain0 Next Address      : 0x0000000080200000
> >>> Domain0 Next Arg1         : 0x0000000082200000
> >>> Domain0 Next Mode         : S-mode
> >>> Domain0 SysReset          : yes
> >>>
> >>> Boot HART ID              : 2
> >>> Boot HART Domain          : root
> >>> Boot HART Priv Version    : v1.12
> >>> Boot HART Base ISA        : rv64imafdch
> >>> Boot HART ISA Extensions  : time,sstc
> >>> Boot HART PMP Count       : 16
> >>> Boot HART PMP Granularity : 4
> >>> Boot HART PMP Address Bits: 54
> >>> Boot HART MHPM Count      : 16
> >>> Boot HART MIDELEG         : 0x0000000000001666
> >>> Boot HART MEDELEG         : 0x0000000000f0b509
> >>>
> >>>
> >>> U-Boot 2023.01+dfsg-1 (Jan 10 2023 - 03:18:09 +0000)
> >>>
> >>> CPU:   rv64imafdch_zicsr_zifencei_zihintpause_zba_zbb_zbc_zbs_sstc
> >>> Model: riscv-virtio,qemu
> >>> DRAM:  8 GiB
> >>> Core:  31 devices, 15 uclasses, devicetree: board
> >>> Flash: 32 MiB
> >>> Loading Environment from nowhere... OK
> >>> In:    serial at 10000000
> >>> Out:   serial at 10000000
> >>> Err:   serial at 10000000
> >>> Net:   eth0: virtio-net#2
> >>> Working FDT set to ff7344b0
> >>> Hit any key to stop autoboot:  0
> >>>
> >>> Device 0: QEMU VirtIO Block Device
> >>>               Type: Hard Disk
> >>>               Capacity: 102400.0 MB = 100.0 GB (209715200 x 512)
> >>> ... is now current device
> >>> Scanning virtio 0:1...
> >>> Found /boot/extlinux/extlinux.conf
> >>> Retrieving file: /boot/extlinux/extlinux.conf
> >>> U-Boot menu
> >>> 1:    Debian GNU/Linux bookworm/sid 6.1.0-1-riscv64
> >>> 2:    Debian GNU/Linux bookworm/sid 6.1.0-1-riscv64 (rescue target)
> >>> 3:    Debian GNU/Linux bookworm/sid 6.0.0-6-riscv64
> >>> 4:    Debian GNU/Linux bookworm/sid 6.0.0-6-riscv64 (rescue target)
> >>> 5:    Debian GNU/Linux bookworm/sid 6.0.0-5-riscv64
> >>> 6:    Debian GNU/Linux bookworm/sid 6.0.0-5-riscv64 (rescue target)
> >>> Enter choice: 1:      Debian GNU/Linux bookworm/sid 6.1.0-1-riscv64
> >>> Retrieving file: /boot/initrd.img-6.1.0-1-riscv64
> >>> Retrieving file: /boot/vmlinux-6.1.0-1-riscv64
> >>> append: root=/dev/vda1 rw noquiet
> >>> Moving Image from 0x84000000 to 0x80200000, end=815e5000
> >>> ## Flattened Device Tree blob at ff7344b0
> >>>      Booting using the fdt blob at 0xff7344b0
> >>> Working FDT set to ff7344b0
> >>>      Using Device Tree in place at 00000000ff7344b0, end 00000000ff738dea
> >>> Working FDT set to ff7344b0
> >>> ERROR: fdt fixup event failed: -22
> >>>    - must RESET the board to recover.
> >>>
> >>> FDT creation failed! hanging...### ERROR ### Please RESET the board ###
> >>>
> >>> -----8<----------8<----------8<----------8<----------8<----------8<-----
> >>>
> >>> Software versions used:
> >>> - OpenSBI 1.1 (Debian package: opensbi 1.1-2)
> >>> - QEMU 7.2.0 (Debian package: qemu-system-misc 1:7.2+dfsg-1+b2)
> >>> - U-Boot v2023.01 (Debian package: u-boot-qemu 2023.01+dfsg-1)
> >>>
> >>> The issue is independent from the kernel that gets booted.  With
> >>> U-Boot v2022.10 everything works without problems.  I have used
> >>> git bisect (with qemu-riscv64_smode_defconfig) to narrow down the
> >>> specific change that triggers the issue and that has resulted in
> >>> the following commit:
> >>>
> >>> -----8<----------8<----------8<----------8<----------8<----------8<-----
> >>>
> >>> commit a56f663f07073713042bb0fd08053aeb667e717b (HEAD)
> >>> Author: Simon Glass <sjg at chromium.org>
> >>> Date:   Thu Oct 20 18:23:14 2022 -0600
> >>>
> >>>       vbe: Add info about the VBE device to the fwupd node
> >>>
> >>>       At present we put the driver in the /chosen node in U-Boot. This is a bit
> >>>       strange, since U-Boot doesn't normally use that node itself. It is better
> >>>       to put it under the bootstd node.
> >>>
> >>>       To make this work we need to copy create the node under /chosen when
> >>>       fixing up the device tree. Copy over all the properties so that fwupd
> >>>       knows what to do.
> >>>
> >>>       Update the sandbox device tree accordingly.
> >>>
> >>>       Signed-off-by: Simon Glass <sjg at chromium.org>
> >>>
> >>> -----8<----------8<----------8<----------8<----------8<----------8<-----
> >>>
> >>> If you should happen to run Debian/unstable, the easiest way to
> >>> reproduce the problem is as follows:
> >>>
> >>> $ sudo apt install qemu-system-misc opensbi u-boot-qemu
> >>>
> >>> You can then either manually set up a riscv64 VM image as described at
> >>> https://wiki.debian.org/RISC-V#Setting_up_a_riscv64_virtual_machine
> >>> or you can use a prebuilt VM image from the Debian Quick Image
> >>> Baker as described at https://people.debian.org/~gio/dqib/:
> >>>
> >>> $ wget -O riscv64-virt.zip https://gitlab.com/api/v4/projects/giomasce%2Fdqib/jobs/artifacts/master/download?job=convert_riscv64-virt
> >>> $ unzip -x riscv64-virt.zip
> >>> $ cd artifacts
> >>>
> >>> and boot the VM image with
> >>>
> >>> $ qemu-system-riscv64 -smp 4 -nographic -machine virt -m 8G -bios /usr/lib/riscv64-linux-gnu/opensbi/generic/fw_jump.elf -kernel /usr/lib/u-boot/qemu-riscv64_smode/u-boot.bin -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-device,rng=rng0 -append "console=ttyS0 rw root=/dev/vda1 scsi_mod.use_blk_mq=0" -device virtio-blk-device,drive=hd0 -drive file=image.qcow2,format=qcow2,id=hd0 -device virtio-net-device,netdev=usernet -netdev user,id=usernet,hostfwd=tcp::22222-:22
> >>>
> >>> Regards,
> >>> Karsten
> >>
> >> Thanks Karsten for reporting the issue.
> >>
> >> Bisection points to Simon's patch:
> >>
> >> a56f663f0707371
> >> vbe: Add info about the VBE device to the fwupd node
> >>
> >> In bootmeth_vbe_simple_ft_fixup() probing device vbe_simple fails
> >> because all fwupd related properties are missing in the QEMU device-tree.
> >>
> >> The following change is enough to make QEMU boot again:
> >>
> >> CONFIG_BOOTMETH_VBE=n
> >>
> >> @Simon:
> >>
> >> CONFIG_BOOTMETH_VBE should default to 'no' as only the sandbox provides
> >> the required data in the devicetree. Or bootmeth_vbe_simple_ft_fixup()
> >> should ignore a failure to probe vbe_simple.
> >
> > It is supposed to ignore that failure. Could it be that there is a bug
> > in vbe_find_next_device() that it should set *devp to NULL when it
> > doesn't find anything?
>
> The vbe_simple device exists on qemu-riscv64_smode_defconfig but cannot
> be probed.

OK, so setting *devp = NULL before the last 'return 0' might help?

>
> >
> > How come this failure does not show in CI?
>
> We currently lack tests to actually start a binary via the Linux legacy
> entry point.

Ah OK, so the fixup never runs.

Regards,
Simon