U-Boot v2023.01 broken on riscv64 qemu virt platform: "fdt fixup event failed"

Thu Jan 12 01:01:55 CET 2023

On 1/12/23 00:59, Tom Rini wrote:
> On Thu, Jan 12, 2023 at 12:40:05AM +0100, Heinrich Schuchardt wrote:
>> On 1/12/23 00:29, Simon Glass wrote:
>>> () Hi Heinrich,
>>>
>>> On Wed, 11 Jan 2023 at 16:22, Heinrich Schuchardt <xypron.glpk at gmx.de> wrote:
>>>>
>>>> On 1/11/23 20:08, Karsten Merker wrote:
>>>>> Hello,
>>>>>
>>>>> it looks like U-Boot v2023.01 is currently broken for the riscv64
>>>>> architecture on the qemu "virt" platform; the boot process of a
>>>>> riscv64 VM fails during FDT fixup:
>>>>>
>>>>> -----8<----------8<----------8<----------8<----------8<----------8<-----
>>>>>
>>>>> OpenSBI v1.1
>>>>>       ____                    _____ ____ _____
>>>>>      / __ \                  / ____|  _ \_   _|
>>>>>     | |  | |_ __   ___ _ __ | (___ | |_) || |
>>>>>     | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
>>>>>     | |__| | |_) |  __/ | | |____) | |_) || |_
>>>>>      \____/| .__/ \___|_| |_|_____/|____/_____|
>>>>>            | |
>>>>>            |_|
>>>>>
>>>>> Platform Name             : riscv-virtio,qemu
>>>>> Platform Features         : medeleg
>>>>> Platform HART Count       : 4
>>>>> Platform IPI Device       : aclint-mswi
>>>>> Platform Timer Device     : aclint-mtimer @ 10000000Hz
>>>>> Platform Console Device   : uart8250
>>>>> Platform HSM Device       : ---
>>>>> Platform Reboot Device    : sifive_test
>>>>> Platform Shutdown Device  : sifive_test
>>>>> Firmware Base             : 0x80000000
>>>>> Firmware Size             : 312 KB
>>>>> Runtime SBI Version       : 1.0
>>>>>
>>>>> Domain0 Name              : root
>>>>> Domain0 Boot HART         : 2
>>>>> Domain0 HARTs             : 0*,1*,2*,3*
>>>>> Domain0 Region00          : 0x0000000002000000-0x000000000200ffff (I)
>>>>> Domain0 Region01          : 0x0000000080000000-0x000000008007ffff ()
>>>>> Domain0 Region02          : 0x0000000000000000-0xffffffffffffffff (R,W,X)
>>>>> Domain0 Next Address      : 0x0000000080200000
>>>>> Domain0 Next Arg1         : 0x0000000082200000
>>>>> Domain0 Next Mode         : S-mode
>>>>> Domain0 SysReset          : yes
>>>>>
>>>>> Boot HART ID              : 2
>>>>> Boot HART Domain          : root
>>>>> Boot HART Priv Version    : v1.12
>>>>> Boot HART Base ISA        : rv64imafdch
>>>>> Boot HART ISA Extensions  : time,sstc
>>>>> Boot HART PMP Count       : 16
>>>>> Boot HART PMP Granularity : 4
>>>>> Boot HART PMP Address Bits: 54
>>>>> Boot HART MHPM Count      : 16
>>>>> Boot HART MIDELEG         : 0x0000000000001666
>>>>> Boot HART MEDELEG         : 0x0000000000f0b509
>>>>>
>>>>>
>>>>> U-Boot 2023.01+dfsg-1 (Jan 10 2023 - 03:18:09 +0000)
>>>>>
>>>>> CPU:   rv64imafdch_zicsr_zifencei_zihintpause_zba_zbb_zbc_zbs_sstc
>>>>> Model: riscv-virtio,qemu
>>>>> DRAM:  8 GiB
>>>>> Core:  31 devices, 15 uclasses, devicetree: board
>>>>> Flash: 32 MiB
>>>>> Loading Environment from nowhere... OK
>>>>> In:    serial at 10000000
>>>>> Out:   serial at 10000000
>>>>> Err:   serial at 10000000
>>>>> Net:   eth0: virtio-net#2
>>>>> Working FDT set to ff7344b0
>>>>> Hit any key to stop autoboot:  0
>>>>>
>>>>> Device 0: QEMU VirtIO Block Device
>>>>>                Type: Hard Disk
>>>>>                Capacity: 102400.0 MB = 100.0 GB (209715200 x 512)
>>>>> ... is now current device
>>>>> Scanning virtio 0:1...
>>>>> Found /boot/extlinux/extlinux.conf
>>>>> Retrieving file: /boot/extlinux/extlinux.conf
>>>>> U-Boot menu
>>>>> 1:    Debian GNU/Linux bookworm/sid 6.1.0-1-riscv64
>>>>> 2:    Debian GNU/Linux bookworm/sid 6.1.0-1-riscv64 (rescue target)
>>>>> 3:    Debian GNU/Linux bookworm/sid 6.0.0-6-riscv64
>>>>> 4:    Debian GNU/Linux bookworm/sid 6.0.0-6-riscv64 (rescue target)
>>>>> 5:    Debian GNU/Linux bookworm/sid 6.0.0-5-riscv64
>>>>> 6:    Debian GNU/Linux bookworm/sid 6.0.0-5-riscv64 (rescue target)
>>>>> Enter choice: 1:      Debian GNU/Linux bookworm/sid 6.1.0-1-riscv64
>>>>> Retrieving file: /boot/initrd.img-6.1.0-1-riscv64
>>>>> Retrieving file: /boot/vmlinux-6.1.0-1-riscv64
>>>>> append: root=/dev/vda1 rw noquiet
>>>>> Moving Image from 0x84000000 to 0x80200000, end=815e5000
>>>>> ## Flattened Device Tree blob at ff7344b0
>>>>>       Booting using the fdt blob at 0xff7344b0
>>>>> Working FDT set to ff7344b0
>>>>>       Using Device Tree in place at 00000000ff7344b0, end 00000000ff738dea
>>>>> Working FDT set to ff7344b0
>>>>> ERROR: fdt fixup event failed: -22
>>>>>     - must RESET the board to recover.
>>>>>
>>>>> FDT creation failed! hanging...### ERROR ### Please RESET the board ###
>>>>>
>>>>> -----8<----------8<----------8<----------8<----------8<----------8<-----
>>>>>
>>>>> Software versions used:
>>>>> - OpenSBI 1.1 (Debian package: opensbi 1.1-2)
>>>>> - QEMU 7.2.0 (Debian package: qemu-system-misc 1:7.2+dfsg-1+b2)
>>>>> - U-Boot v2023.01 (Debian package: u-boot-qemu 2023.01+dfsg-1)
>>>>>
>>>>> The issue is independent from the kernel that gets booted.  With
>>>>> U-Boot v2022.10 everything works without problems.  I have used
>>>>> git bisect (with qemu-riscv64_smode_defconfig) to narrow down the
>>>>> specific change that triggers the issue and that has resulted in
>>>>> the following commit:
>>>>>
>>>>> -----8<----------8<----------8<----------8<----------8<----------8<-----
>>>>>
>>>>> commit a56f663f07073713042bb0fd08053aeb667e717b (HEAD)
>>>>> Author: Simon Glass <sjg at chromium.org>
>>>>> Date:   Thu Oct 20 18:23:14 2022 -0600
>>>>>
>>>>>        vbe: Add info about the VBE device to the fwupd node
>>>>>
>>>>>        At present we put the driver in the /chosen node in U-Boot. This is a bit
>>>>>        strange, since U-Boot doesn't normally use that node itself. It is better
>>>>>        to put it under the bootstd node.
>>>>>
>>>>>        To make this work we need to copy create the node under /chosen when
>>>>>        fixing up the device tree. Copy over all the properties so that fwupd
>>>>>        knows what to do.
>>>>>
>>>>>        Update the sandbox device tree accordingly.
>>>>>
>>>>>        Signed-off-by: Simon Glass <sjg at chromium.org>
>>>>>
>>>>> -----8<----------8<----------8<----------8<----------8<----------8<-----
>>>>>
>>>>> If you should happen to run Debian/unstable, the easiest way to
>>>>> reproduce the problem is as follows:
>>>>>
>>>>> $ sudo apt install qemu-system-misc opensbi u-boot-qemu
>>>>>
>>>>> You can then either manually set up a riscv64 VM image as described at
>>>>> https://wiki.debian.org/RISC-V#Setting_up_a_riscv64_virtual_machine
>>>>> or you can use a prebuilt VM image from the Debian Quick Image
>>>>> Baker as described at https://people.debian.org/~gio/dqib/:
>>>>>
>>>>> $ wget -O riscv64-virt.zip https://gitlab.com/api/v4/projects/giomasce%2Fdqib/jobs/artifacts/master/download?job=convert_riscv64-virt
>>>>> $ unzip -x riscv64-virt.zip
>>>>> $ cd artifacts
>>>>>
>>>>> and boot the VM image with
>>>>>
>>>>> $ qemu-system-riscv64 -smp 4 -nographic -machine virt -m 8G -bios /usr/lib/riscv64-linux-gnu/opensbi/generic/fw_jump.elf -kernel /usr/lib/u-boot/qemu-riscv64_smode/u-boot.bin -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-device,rng=rng0 -append "console=ttyS0 rw root=/dev/vda1 scsi_mod.use_blk_mq=0" -device virtio-blk-device,drive=hd0 -drive file=image.qcow2,format=qcow2,id=hd0 -device virtio-net-device,netdev=usernet -netdev user,id=usernet,hostfwd=tcp::22222-:22
>>>>>
>>>>> Regards,
>>>>> Karsten
>>>>
>>>> Thanks Karsten for reporting the issue.
>>>>
>>>> Bisection points to Simon's patch:
>>>>
>>>> a56f663f0707371
>>>> vbe: Add info about the VBE device to the fwupd node
>>>>
>>>> In bootmeth_vbe_simple_ft_fixup() probing device vbe_simple fails
>>>> because all fwupd related properties are missing in the QEMU device-tree.
>>>>
>>>> The following change is enough to make QEMU boot again:
>>>>
>>>> CONFIG_BOOTMETH_VBE=n
>>>>
>>>> @Simon:
>>>>
>>>> CONFIG_BOOTMETH_VBE should default to 'no' as only the sandbox provides
>>>> the required data in the devicetree. Or bootmeth_vbe_simple_ft_fixup()
>>>> should ignore a failure to probe vbe_simple.
>>>
>>> It is supposed to ignore that failure. Could it be that there is a bug
>>> in vbe_find_next_device() that it should set *devp to NULL when it
>>> doesn't find anything?
>>
>> The vbe_simple device exists on qemu-riscv64_smode_defconfig but cannot
>> be probed.
>>
>>>
>>> How come this failure does not show in CI?
>>
>> We currently lack tests to actually start a binary via the Linux legacy
>> entry point.
>
> It would be good if we had more tests like test/nokia_rx51_test.sh that
> booted up a full image. I feel like for platforms that are EBBR
> compliant it should be less of a problem to do so than it is for
> nokia_rx51 where part of the challenge is that we need the custom QEMU
> for it at all.
>

The EBBR/EFI part is covered by starting GRUB.

The missing part is booting non-EFI images.

Best regards

Heinrich