[RFC] dev_phys_to_bus() and PCI

[Mark Kettenis]

I'm working on support for Apple's M1 systems in U-Boot.  The idea is
that this can be used as a "payload" for the m1n1 bootloader that is
being developed by Hector Martin for Asahi Linux in order to provide
an UEFI implementation that can boot a standard arm64 OS.

My current code, which can be found on the "apple-m1-m1n1" branch at:

  https://github.com/kettenis/u-boot/tree/apple-m1-m1n1

can already do this.  I'm booting OpenBSD/arm64 this way and I've also
booted into grub using a standard arm64 Debian installer.

One of the big challenges I'm facing is that the integrated PCIe
devices on the system sit behind an IOMMU that (as far as we can tell)
doesn't support any kind of bypass mode.  I don't think we want full
IOMMU support in U-Boot, so the approach I'm currently taking is that
I set up the IOMMU to map a chunk of memory at the "top of RAM" where
U-Boot resides after relocation.  But in order to use this mapping I
need to do DMA address translation.

Fortunately Nicolas Saenz Julienne recently introduced
dev_phys_to_bus() and dev_bus_to_phys() interfaces to do this.  Those
interfaces make use of a dma-ranges property in the device tree which
doesn't work so well for PCI devices though.  However, the PCI code in
U-Boot already has a way to describe DMA address translation through
its regions support.  Hooking this up to dev_phys_to_bus() and
dev_bus_to_phys() is fairly easy as illustrated by the diff below.

Would this be viable approach?  This could also help adding support
for PCIe devices on the Raspberry Pi CM4.


commit 4f0e989c7a765291a38b7d10da562f23c5f31239
Author: Mark Kettenis <kettenis at openbsd.org>
Date:   Fri Mar 12 20:23:11 2021 +0100

    dm: Add PCI support to dev_phys_to_bus()/dev_bus_to_phys()
    
    For PCI devices, call dm_pci_phys_to_bus()/dm_pci_bus_to_phys()
    to do the address translation.  This uses the regions set up
    by the PCI host controller driver to do the translation as a
    single translation offset may not be sufficient in this case.
    
    Signed-off-by: Mark Kettenis <kettenis at openbsd.org>

diff --git a/include/phys2bus.h b/include/phys2bus.h
index 866b8b51a8..13d23ef4bb 100644
--- a/include/phys2bus.h
+++ b/include/phys2bus.h
@@ -23,14 +23,21 @@ static inline unsigned long bus_to_phys(unsigned long bus)
 
 #if CONFIG_IS_ENABLED(DM)
 #include <dm/device.h>
+#include <pci.h>
 
 static inline dma_addr_t dev_phys_to_bus(struct udevice *dev, phys_addr_t phys)
 {
+	if (device_is_on_pci_bus(dev))
+		return dm_pci_phys_to_bus(dev, phys, PCI_REGION_SYS_MEMORY);
+
 	return phys - dev_get_dma_offset(dev);
 }
 
 static inline phys_addr_t dev_bus_to_phys(struct udevice *dev, dma_addr_t bus)
 {
+	if (device_is_on_pci_bus(dev))
+		return dm_pci_bus_to_phys(dev, bus, PCI_REGION_SYS_MEMORY);
+
 	return bus + dev_get_dma_offset(dev);
 }
 #else