Testing pci_mvebu.c with Kirkwood SoCs
Tony Dinh
mibodhi at gmail.com
Mon Nov 8 01:58:27 CET 2021
Hi Pali,
Looks like it is working! But in a weird way. Please see my responses
in between, and also in the test log below.
On Sun, Nov 7, 2021 at 3:45 PM Pali Rohár <pali at kernel.org> wrote:
>
> Hello!
>
> On Sunday 07 November 2021 12:56:37 Tony Dinh wrote:
> > Hi Pali,
> >
> > I've applied your patch, and ran some tests. Looks like we got to the
> > bind, but it was not probing (or the probe died silently). Later, I
> > tried pci enum and got some errors during probing.
>
> It looks like that arch/arm/mach-kirkwood/cpu.c already setup PCIe
> windows. Therefore when pci_mvebu.c tries it too, it fails on error
> "conflicts with another window" which you see in the log.
>
> Could you try to comment calling of "mvebu_mbus_add_window_by_id"
> function in pci_mvebu.c? As PCIe window setup should be done exactly
> once.
>
> Also try to call 'pci 0' and 'pci 1' after the 'pci enum'.
>
> > Here is the log (I added some debug printf to drivers/pci/pci_mvebu.c
> > and drivers/core/device.c to see the progress).
> >
> > - U-Boot boot log:
> >
> > U-Boot 2022.01-tld-1-00054-g52207514ba-dirty (Nov 06 2021 - 18:11:41 -0700)
> > Pogoplug V4
> >
> > SoC: Kirkwood 88F6281_A1
> > DRAM: 128 MiB
> > mvebu_pcie_bind: begin
> > mvebu_pcie_bind: got an available node
> > Bound device to pcie at 82000000
> > mvebu_pcie_bind: bound
> > mvebu_pcie_bind: exit 0
> > Bound device pcie at 82000000 to mbus at f1000000
> > Bound device mbus at f1000000 to root_driver
> > Bound device ehci at 50000 to ocp at f1000000
> > Bound device ethernet-controller at 72000 to ocp at f1000000
> > Bound device sata at 80000 to ocp at f1000000
> > Bound device mvsdio at 90000.blk to mvsdio at 90000
> > Bound device mvsdio at 90000 to ocp at f1000000
> > Bound device ocp at f1000000 to root_driver
> > NAND: 128 MiB
> > MMC: mvsdio at 90000: 0
> > Loading Environment from NAND... *** Warning - bad CRC, using default
> > environment
> >
> > In: serial
> > Out: serial
> > Err: serial
> > Net:
> > Warning: ethernet-controller at 72000 (eth0) using random MAC address -
> > 66:43:1f:50:f0:e7
> > eth0: ethernet-controller at 72000
> > Hit any key to stop autoboot: 0
> >
> > Pogo_V4> pci
> > No such bus
> > Pogo_V4> pci 0
> > No such bus
> > Pogo_V4> pci 1
> > No such bus
> >
> > Pogo_V4> pci enum
> > mvebu_pcie_probe:
> > Cannot add window '4:e8', conflicts with another window
> > PCIe unable to add mbus window for mem at 90000000+10000000
> > Cannot add window '4:e0', conflicts with another window
> > PCIe unable to add mbus window for IO at c0000000+00010000
> > mvebu_pcie_probe: exit 0
> > Bound device pci_0:0.0 to pcie0.0
> > Bound device xhci_pci to pci_0:0.0
> >
> > Pogo_V4> pci regions
> > # Bus start Phys start Size Flags
> > 0 0x0000000090000000 0x0000000090000000 0x0000000010000000 mem
> > 1 0x0000000000000000 0x0000000000000000 0x0000000008000000 mem sysmem
> > 2 0x00000000c0000000 0x00000000c0000000 0x0000000000010000 io io
> >
> > - Linux dmesg (grep for pci and xhci)
> >
> > [ 13.163011][ T1] mvebu-pcie mbus at f1000000:pcie at 82000000: host
> > bridge /mbus at f1000000/pcie at 82000000 ranges:
> > [ 13.163147][ T1] mvebu-pcie mbus at f1000000:pcie at 82000000:
> > MEM 0x00f1040000..0x00f1041fff -> 0x0000040000
> > [ 13.163217][ T1] mvebu-pcie mbus at f1000000:pcie at 82000000:
> > MEM 0xffffffffffffffff..0x00fffffffe -> 0x0100000000
> > [ 13.163268][ T1] mvebu-pcie mbus at f1000000:pcie at 82000000:
> > IO 0xffffffffffffffff..0x00fffffffe -> 0x0100000000
> > [ 13.163754][ T1] mvebu-pcie mbus at f1000000:pcie at 82000000: PCI
> > host bridge to bus 0000:00
> > [ 13.163791][ T1] pci_bus 0000:00: root bus resource [bus 00-ff]
> > [ 13.163829][ T1] pci_bus 0000:00: root bus resource [mem
> > 0xf1040000-0xf1041fff] (bus address [0x00040000-0x00041fff])
> > [ 13.163862][ T1] pci_bus 0000:00: root bus resource [mem
> > 0xe0000000-0xefffffff]
> > [ 13.163892][ T1] pci_bus 0000:00: root bus resource [io 0x1000-0xeffff]
> > [ 13.164075][ T1] pci 0000:00:01.0: [11ab:6281] type 01 class 0x060400
> > [ 13.164133][ T1] pci 0000:00:01.0: reg 0x38: [mem
> > 0x00000000-0x000007ff pref]
> > [ 13.165893][ T1] PCI: bus0: Fast back to back transfers disabled
> > [ 13.165949][ T1] pci 0000:00:01.0: bridge configuration invalid
> > ([bus 00-00]), reconfiguring
> > [ 13.166276][ T1] pci 0000:01:00.0: [1b73:1009] type 00 class 0x0c0330
> > [ 13.166334][ T1] pci 0000:01:00.0: reg 0x10: [mem
> > 0x00000000-0x0000ffff 64bit]
> > [ 13.166381][ T1] pci 0000:01:00.0: reg 0x18: [mem
> > 0x00000000-0x00000fff 64bit]
> > [ 13.166424][ T1] pci 0000:01:00.0: reg 0x20: [mem
> > 0x00000000-0x00000fff 64bit]
> > [ 13.166561][ T1] pci 0000:01:00.0: supports D1
> > [ 13.166588][ T1] pci 0000:01:00.0: PME# supported from D0 D1 D3hot
> > [ 13.166685][ T1] pci 0000:01:00.0: 2.000 Gb/s available PCIe
> > bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:01.0 (capable
> > of 4.000 Gb/s with 5.0 GT/s PCIe x1 link)
> > [ 13.197158][ T1] PCI: bus1: Fast back to back transfers disabled
> > [ 13.197208][ T1] pci_bus 0000:01: busn_res: [bus 01-ff] end is
> > updated to 01
> > [ 13.197538][ T1] pci 0000:00:01.0: BAR 14: assigned [mem
> > 0xe0000000-0xe00fffff]
> > [ 13.197581][ T1] pci 0000:00:01.0: BAR 6: assigned [mem
> > 0xe0100000-0xe01007ff pref]
> > [ 13.197624][ T1] pci 0000:01:00.0: BAR 0: assigned [mem
> > 0xe0000000-0xe000ffff 64bit]
> > [ 13.197670][ T1] pci 0000:01:00.0: BAR 2: assigned [mem
> > 0xe0010000-0xe0010fff 64bit]
> > [ 13.197715][ T1] pci 0000:01:00.0: BAR 4: assigned [mem
> > 0xe0011000-0xe0011fff 64bit]
> > [ 13.197759][ T1] pci 0000:00:01.0: PCI bridge to [bus 01]
> > [ 13.197792][ T1] pci 0000:00:01.0: bridge window [mem
> > 0xe0000000-0xe00fffff]
> > [ 13.197985][ T1] pcieport 0000:00:01.0: enabling device (0140 -> 0142)
> > [ 13.198135][ T1] pci 0000:01:00.0: enabling device (0140 -> 0142)
> >
> > [ 15.619756][ T1] xhci_hcd 0000:01:00.0: xHCI Host Controller
> > [ 15.625694][ T1] xhci_hcd 0000:01:00.0: new USB bus registered,
> > assigned bus number 2
> > [ 15.635643][ T1] xhci_hcd 0000:01:00.0: hcc params 0x200073a1
> > hci version 0x100 quirks 0x0000000000080010
> > [ 15.646477][ T1] xhci_hcd 0000:01:00.0: xHCI Host Controller
> > [ 15.652432][ T1] xhci_hcd 0000:01:00.0: new USB bus registered,
> > assigned bus number 3
> > [ 15.660560][ T1] xhci_hcd 0000:01:00.0: Host supports USB 3.0 SuperSpeed
> >
> > Note that this board PCI and USB 3.0 have been working in the Linux
> > kernel for quite many years.
> >
> > - These are relevant configs used in the build (not sure if I need to
> > use CONFIG_DM_PCI_COMPAT?)
> >
> > CONFIG_DM=y
> > CONFIG_CMD_PCI=y
> > CONFIG_PCI=y
> > CONFIG_PCI_MVEBU=y
> > CONFIG_PCI_ENDPOINT=y
> > CONFIG_DM_PCI_COMPAT=y
> > CONFIG_USB_XHCI_HCD=y
> > CONFIG_USB_XHCI_PCI=y
> > CONFIG_USB_XHCI_MVEBU=y
>
> Following options should be enough:
>
> CONFIG_CMD_PCI=y
> CONFIG_PCI=y
> # CONFIG_DM_PCI_COMPAT is not set
> CONFIG_PCI_PNP=y
> CONFIG_PCI_MVEBU=y
Since the PCI bus in this box is used for USB 3.0, I've included
various XHCI configs to see if I can probe this USB 3.0 flash drive
also (that's the ultimate test).
>
> Option CONFIG_PCI_PNP=y is required to see endpoint cards. Without it
> U-Boot would see only PCIe Root Ports. But for first basic tests it
> should be enough.
>
> > BTW, the topic is no longer kwboot, should we move this to another new
> > thread? i.e. Testing PCI MVEBU with Kirkwood SoCs.
>
> Changed :-)
Thanks :)
Here is the log of a successful spin up for the PCI bus and USB 3.0
drive. Please see my commentary ### in between the logs.
U-Boot 2022.01-rc1-00054-g52207514ba-dirty (Nov 07 2021 - 16:03:46 -0800)
Pogoplug V4
SoC: Kirkwood 88F6281_A1
DRAM: 128 MiB
device_probe:
device_probe: exit 0
### The device_probe above seems to have nothing happening.
mvebu_pcie_bind: begin
mvebu_pcie_bind: got an available node
Bound device to pcie at 82000000
mvebu_pcie_bind: bound
mvebu_pcie_bind: exit 0
Bound device pcie at 82000000 to mbus at f1000000
Bound device mbus at f1000000 to root_driver
Bound device ehci at 50000 to ocp at f1000000
Bound device ethernet-controller at 72000 to ocp at f1000000
Bound device sata at 80000 to ocp at f1000000
Bound device mvsdio at 90000.blk to mvsdio at 90000
Bound device mvsdio at 90000 to ocp at f1000000
Bound device ocp at f1000000 to root_driver
NAND: 128 MiB
MMC: device_probe:
device_probe:
device_probe:
device_probe: exit 0
device_probe: exit 0
device_probe:
device_probe:
mvsdio at 90000: 0
Loading Environment from NAND... *** Warning - bad CRC, using default
environment
In: serial
Out: serial
Err: serial
Net: device_probe:
device_probe:
Warning: ethernet-controller at 72000 (eth0) using random MAC address -
86:b8:84:ee:a9:a5
device_probe: exit 0
eth0: ethernet-controller at 72000
Hit any key to stop autoboot: 0
Pogo_V4> usb reset
resetting USB...
Bus ehci at 50000: device_probe:
device_probe:
USB EHCI 1.00
device_probe: exit 0
scanning bus ehci at 50000 for devices... Bound device usb_hub to ehci at 50000
device_probe:
device_probe:
Bound device usb_hub to usb_hub
device_probe:
device_probe:
device_probe: exit 0
Bound device usb_mass_storage to usb_hub
device_probe:
device_probe:
Bound device usb_mass_storage.lun0 to usb_mass_storage
device_probe: exit 0
device_probe: exit 0
3 USB Device(s) found
scanning usb for storage devices... 1 Storage Device(s) found
### Try to bring up both USB drives (2.0 and 3.0). But only the USB
2.0 was up as shown above.
Pogo_V4> pci enum
device_probe:
device_probe:
device_probe:
device_probe:
device_probe: exit 0
device_probe: exit 0
mvebu_pcie_probe:
Cannot add window '4:e8', conflicts with another window
PCIe unable to add mbus window for mem at 90000000+10000000
Cannot add window '4:e0', conflicts with another window
PCIe unable to add mbus window for IO at c0000000+00010000
mvebu_pcie_probe: exit 0
Bound device pci_0:0.0 to pcie0.0
device_probe:
device_probe:
Bound device xhci_pci to pci_0:0.0
device_probe: exit 0
device_probe: exit 0
device_probe:
### So I did the pci enum above to get the XHCI bound to PCIe.
Pogo_V4> usb reset
resetting USB...
Bus ehci at 50000: device_probe:
device_probe:
USB EHCI 1.00
device_probe: exit 0
Bus xhci_pci: device_probe:
device_probe:
Register 400081f NbrPorts 4
Starting the controller
USB XHCI 1.00
device_probe: exit 0
scanning bus ehci at 50000 for devices... Bound device usb_hub to ehci at 50000
device_probe:
device_probe:
Bound device usb_hub to usb_hub
device_probe:
device_probe:
device_probe: exit 0
Bound device usb_mass_storage to usb_hub
device_probe:
device_probe:
Bound device usb_mass_storage.lun0 to usb_mass_storage
device_probe: exit 0
device_probe: exit 0
3 USB Device(s) found
scanning bus xhci_pci for devices... Bound device usb_hub to xhci_pci
device_probe:
device_probe:
Bound device usb_mass_storage to usb_hub
device_probe:
device_probe:
Bound device usb_mass_storage.lun0 to usb_mass_storage
device_probe: exit 0
device_probe: exit 0
2 USB Device(s) found
scanning usb for storage devices... 2 Storage Device(s) found
### Now both drives are up!
Pogo_V4> pci 0
device_probe:
Scanning PCI devices on bus 0
BusDevFun VendorId DeviceId Device Class Sub-Class
_____________________________________________________________
00.00.00 0x11ab 0x6281 Bridge device 0x04
Pogo_V4> pci 1
device_probe:
Scanning PCI devices on bus 1
BusDevFun VendorId DeviceId Device Class Sub-Class
_____________________________________________________________
01.00.00 0x1b73 0x1009 Serial bus controller 0x03
### And the pci command has meaningful output at this point as shown above.
Pogo_V4> usb tree
USB device tree:
1 Hub (480 Mb/s, 0mA)
| u-boot EHCI Host Controller
|
+-2 Hub (480 Mb/s, 100mA)
| GenesysLogic USB2.0 Hub
|
+-3 Mass Storage (480 Mb/s, 224mA)
SanDisk Ultra Fit 4C531001560827107320
1 Hub (5 Gb/s, 0mA)
| U-Boot XHCI Host Controller
|
+-2 Mass Storage (5 Gb/s, 224mA)
SanDisk Cruzer Glide 3.0 4C530000130418116112
Pogo_V4> usb info
1: Hub, USB Revision 2.0
- u-boot EHCI Host Controller
- Class: Hub
- PacketSize: 64 Configurations: 1
- Vendor: 0x0000 Product 0x0000 Version 1.0
Configuration: 1
- Interfaces: 1 Self Powered 0mA
Interface: 0
- Alternate Setting 0, Endpoints: 1
- Class Hub
- Endpoint 1 In Interrupt MaxPacket 8 Interval 255ms
2: Hub, USB Revision 2.0
- GenesysLogic USB2.0 Hub
- Class: Hub
- PacketSize: 64 Configurations: 1
- Vendor: 0x05e3 Product 0x0610 Version 144.48
Configuration: 1
- Interfaces: 1 Self Powered Remote Wakeup 100mA
Interface: 0
- Alternate Setting 0, Endpoints: 1
- Class Hub
- Endpoint 1 In Interrupt MaxPacket 1 Interval 12ms
- Endpoint 1 In Interrupt MaxPacket 1 Interval 12ms
3: Mass Storage, USB Revision 2.10
- SanDisk Ultra Fit 4C531001560827107320
- Class: (from Interface) Mass Storage
- PacketSize: 64 Configurations: 1
- Vendor: 0x0781 Product 0x5583 Version 1.0
Configuration: 1
- Interfaces: 1 Bus Powered 224mA
Interface: 0
- Alternate Setting 0, Endpoints: 2
- Class Mass Storage, Transp. SCSI, Bulk only
- Endpoint 1 In Bulk MaxPacket 512
- Endpoint 2 Out Bulk MaxPacket 512
1: Hub, USB Revision 3.0
- U-Boot XHCI Host Controller
- Class: Hub
- PacketSize: 512 Configurations: 1
- Vendor: 0x0000 Product 0x0000 Version 1.0
Configuration: 1
- Interfaces: 1 Self Powered 0mA
Interface: 0
- Alternate Setting 0, Endpoints: 1
- Class Hub
- Endpoint 1 In Interrupt MaxPacket 8 Interval 255ms
2: Mass Storage, USB Revision 3.0
- SanDisk Cruzer Glide 3.0 4C530000130418116112
- Class: (from Interface) Mass Storage
- PacketSize: 512 Configurations: 1
- Vendor: 0x0781 Product 0x5597 Version 1.0
Configuration: 1
- Interfaces: 1 Bus Powered 224mA
Interface: 0
- Alternate Setting 0, Endpoints: 2
- Class Mass Storage, Transp. SCSI, Bulk only
- Endpoint 1 In Bulk MaxPacket 1024
- Endpoint 2 Out Bulk MaxPacket 1024
### The XHCI Host Controller and the SanDisk Cruzer Glide are now
active as shown above.
### So list the files on this SanDisk Cruzer Glide to make sure we can read it.
Pogo_V4> ext2ls usb 1:1 /
device_probe:
device_probe:
device_probe: exit 0
<DIR> 4096 .
<DIR> 4096 ..
522666 uboot.2021.07-tld-1.pogo_v4.bin
524288 uboot.2021.07-tld-1.pogo_v4.mtd0.kwb
Great works Pali!
I think some more investigation is needed. Why did we need to do "pci
enum", and then "usb reset", in that order, to get the PCI bus and the
XHCI controller probed? there must be something missing in the process
somewhere between the Device uclass, the PCI uclass, and the pci_mvebu
uclass?
Thanks,
Tony
More information about the U-Boot
mailing list