NVMe boot issues on RockPro64

[Punit Agrawal]

Hi,

I get the following errors when booting Linux from an ADATA XPG SX8200
NVMe on a RockPro64.

[    3.705205] rockchip-pcie f8000000.pcie: unexpected IRQ, INT0
[    3.705226] rockchip-pcie f8000000.pcie: unexpected IRQ, INT0
[    3.705247] rockchip-pcie f8000000.pcie: unexpected IRQ, INT0
[    3.705331] rockchip-pcie f8000000.pcie: unexpected IRQ, INT0
[    3.705352] rockchip-pcie f8000000.pcie: unexpected IRQ, INT0
[    3.705373] rockchip-pcie f8000000.pcie: unexpected IRQ, INT0

At which point boot hangs. Has anybody come across these errors when
using NVMe?

Using an alternate device (sd card) to load the kernel / initrd doesn't
cause the issue and the drive works fine when used as a root device in
Linux subsequently.

On digging further, I found that uboot exits with the NVME interrupt
line (PCI legacy interrupt) active when making any access to the
device. Even just running "nvme scan" leads to the active interrupt
line.

After sprinkling some prints in the uboot NVMe driver, it seems that the
interrupt goes active right at the beginning of setting up the IO queues
(nvme_setup_io_queues). This is also the first time the admin queue is
used; when issuing the command to setup the number of queues
(NVME_FEAT_NUM_QUEUES). For some reason, updating the CQ head doorbell
doesn't clear the interrupt.

The active interrupt doesn't bother uboot as it ignores the device
interrupt but causes an issue latter when linux boots.

Has anybody faced similar issues with NVMe and uboot? Any idea on how to
stop the interrupt line from triggering? Or de-activating it on exit?

Let me know if there's anything I can provide to help debug the
problem. Also, happy to try any patches or suggestions.

Thanks,
Punit