[U-Boot-Users] RE: PCI bus/bridge mis-configuration

[Brendan Beahan]

SOLVED !!!

The problem lay in ppcboot.  An logical error when scanning the ROM_ADDRESS register led ppcboot to think that there was a very large expansion ROM when in fact there was none.  The resulting address map was unworkable, so linux disabled regions that looked problematic. 

I don't know if the erroneous code was part of my vendors port, or the original ppcboot source.

I hope this helps someone some day.

Brendan Beahan.

-----Original Message-----
From: Brendan Beahan 
Sent: Friday, 7 February 2003 10:24 AM
To: 'u-boot-users at lists.sourceforge.net'
Subject: PCI bus/bridge mis-configuration


Hello all,

I've got a problem with PCI bus/bridge/device configuration on an embedded system.  I don't know if the problem is created by the boot loader (ppcboot) or the linux kernel (or even the HW ), but I thought someone on this forum might be able to shed some light on the issue.

I've used ppcboot's pciinfo command to visually verify that each device seems to be configured correctly.  But when linux "boots" some resource regions simply don't appear to be allocated, and error messages are generated.  One PMC module on the PCI bus becomes unavailable.

Can anyone shed some light on what might be going on here, or have any suggestions on what I might to track the cause of the problem down ?

I have included more detailed info below, namely pciinfo output, kernel error messages, lspci output.

Thank you ...

Firstly, the pci bus "tree" diagram, as reported by "lspci -t" is this:

-+-[01]---0a.0-[02-03]----0f.0-[03]----00.0
\-[00]-+-08.0
+-08.1
+-08.2
\-08.4

The significant devices are :
01:0a.0 is a PCI-PCI bridge on the single board computer
02:0f.0 is a PCI-PCI bridge on the cPCI PMC carrier card
03:00.0 is a PLX device on a PMC module

The essence of the problem is this: The SBC uses boots using ppcboot, which seems to setup the PCI bus OK, (as seen using the pciinfo command). However the linux kernel fails to set-up the memory resources correctly at the two PCI-PCI bridges, which in turn prevents proper initialisation of the PMC devices. These are the error messages:

PCI: Cannot allocate resource region 0 of device 03:01.0
PCI: Cannot allocate resource region 2 of device 03:01.0
PCI: Cannot allocate resource region 3 of device 03:01.0
PCI: Failed to allocate resource 0(0-1ff) for 03:01.0
PCI: Failed to allocate resource 2(0-3fffff) for 03:01.0
PCI: Failed to allocate resource 3(0-3fffff) for 03:01.0

I include a more detailed transcript below, including
pciinfo output
PCI messages during kernel initialisation
lspci and lspci -b output

My SBC has a local PMC site, so I can mount my PMS device on bus 0 and remove the carrier card. This works fine, so I suspect configuration of the bridges is the problem.

A prefetchable region (c0200000-c03fffff) is set up inside a non-prefetchable region (00000000-cfdfffff) which I though might be suspect, but it works OK when I "mount" the PMC module locally.

Detailed transcript ...

GMS PPCBoot 1.1.6a (Jan 23 2003 - 12:56:37)

CPU: MPC7410 v17.4 @ 466.666 MHz
Board: Unknown
.
.
.
=> pciinfo 1
Scanning PCI devices on bus 1
.
.
.
Found PCI device 10, function 0:
vendor ID = 0x8086
device ID = 0xb154
command register = 0x0147
status register = 0x02b0
revision ID = 0x00
class code = 0x06 (Bridge device )
sub class code = 0x04
programming interface = 0x00
cache line = 0x08
latency time = 0x10
header type = 0x01
BIST = 0x00
base address 0 = 0x00000000
base address 1 = 0x00000000
primary bus number = 0x01
secondary bus number = 0x02
subordinate bus number = 0x03
secondary latency timer = 0x10
IO base = 0xf1
IO limit = 0xf1
secondary status = 0x2280
memory base = 0x0000
memory limit = 0xcfd0
prefetch memory base = 0xc021
prefetch memory limit = 0xc031
prefetch memory base upper = 0x00000000
prefetch memory limit upper = 0x00000000
IO base upper 16 bits = 0x0010
IO limit upper 16 bits = 0x0010
expansion ROM base address = 0x00000000
interrupt line = 0x00
interrupt pin = 0x00
bridge control = 0x000b
=> pciinfo 2
Scanning PCI devices on bus 2

Found PCI device 15, function 0:
vendor ID = 0x1011
device ID = 0x0024
command register = 0x0147
status register = 0x0290
revision ID = 0x03
class code = 0x06 (Bridge device )
sub class code = 0x04
programming interface = 0x00
cache line = 0x08
latency time = 0x10
header type = 0x01
BIST = 0x00
base address 0 = 0x00000000
base address 1 = 0x00000000
primary bus number = 0x02
secondary bus number = 0x03
subordinate bus number = 0x03
secondary latency timer = 0x10
IO base = 0xf1
IO limit = 0xf1
secondary status = 0x2280
memory base = 0x0000
memory limit = 0xcfd0
prefetch memory base = 0xc021
prefetch memory limit = 0xc031
prefetch memory base upper = 0x00000000
prefetch memory limit upper = 0x00000000
IO base upper 16 bits = 0x0010
IO limit upper 16 bits = 0x0010
expansion ROM base address = 0x00000000
interrupt line = 0x00
interrupt pin = 0x00
bridge control = 0x000b
=> pciinfo 3
Scanning PCI devices on bus 3

Found PCI device 1, function 0:
vendor ID = 0x10b5
device ID = 0x9656
command register = 0x0147
status register = 0x02b0
revision ID = 0x02
class code = 0x06 (Bridge device )
sub class code = 0x80
programming interface = 0x00
cache line = 0x08
latency time = 0x20
header type = 0x00
BIST = 0x00
base address 0 = 0xcfdffe00
base address 1 = 0x0010ff01
base address 2 = 0xcf800000
base address 3 = 0xcf400000
base address 4 = 0x00000000
base address 5 = 0x00000000
cardBus CIS pointer = 0x00000000
sub system vendor ID = 0x4144
sub system ID = 0x0042
expansion ROM base address = 0x00000000
interrupt line = 0x68
interrupt pin = 0x01
min Grant = 0x00
max Latency = 0x00

=> tftboopboot
.
## Booting image at 01000000 ...
Image Name: Linux-2.4.20-rc1
Created: 2003-01-30 0:48:06 UTC
Image Type: PowerPC Linux Kernel Image (gzip compressed)
.
.
.
PCI: Probing PCI hardware
PCI: bridge rsrc 0..fffff (100), parent c0226400
PCI: bridge rsrc 80400000..bfffffff (200), parent c022641c
PCI: bridge rsrc 100000..1fffff (100), parent c0226400
PCI: bridge rsrc c0200000..f7ffffff (200), parent c022641c
PCI: bridge rsrc 10f000..10ffff (101), parent c0386138
PCI: bridge rsrc c0200000..c03fffff (1201), parent c0386154
PCI: bridge rsrc 10f000..10ffff (101), parent c06a5124
PCI: bridge rsrc c0200000..c03fffff (1201), parent c06a515c
PCI:00:08.1: Resource 4: 00000400-0000040f (f=101)
PCI:00:08.2: Resource 4: 0000ffe0-0000ffff (f=101)
PCI:03:01.0: Resource 0: cfdffe00-cfdfffff (f=200)
PCI: Cannot allocate resource region 0 of device 03:01.0
PCI:03:01.0: Resource 1: 0010ff00-0010ffff (f=101)
PCI:03:01.0: Resource 2: cf800000-cfbfffff (f=200)
PCI: Cannot allocate resource region 2 of device 03:01.0
PCI:03:01.0: Resource 3: cf400000-cf7fffff (f=200)
PCI: Cannot allocate resource region 3 of device 03:01.0
PCI: Failed to allocate resource 0(0-1ff) for 03:01.0
PCI: Failed to allocate resource 2(0-3fffff) for 03:01.0
PCI: Failed to allocate resource 3(0-3fffff) for 03:01.0
. 
.
.
Give root password for maintenance
(or type Control-D for normal startup): 
debian1:~# lspci -bvv
.
.
.
01:0a.0 PCI bridge: Intel Corp.: Unknown device b154 (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B-
Status: Cap+ 66Mhz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 16, cache line size 08
Bus: primary=01, secondary=02, subordinate=03, sec-latency=16
I/O behind bridge: 0010f000-0010ffff
Memory behind bridge: 00000000-cfdfffff
Prefetchable memory behind bridge: 00000000c0200000-00000000c0300000
BridgeCtl: Parity+ SERR+ NoISA- VGA+ MAbort- >Reset- FastB2B-
Capabilities: [dc] Power Management version 1
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-
Bridge: PM- B3+

02:0f.0 PCI bridge: Digital Equipment Corporation DECchip 21152 (rev 03) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 16, cache line size 08
Bus: primary=02, secondary=03, subordinate=03, sec-latency=16
I/O behind bridge: 0010f000-0010ffff
Memory behind bridge: 00000000-cfdfffff
Prefetchable memory behind bridge: 00000000c0200000-00000000c0300000
BridgeCtl: Parity+ SERR+ NoISA- VGA+ MAbort- >Reset- FastB2B-
Capabilities: [dc] Power Management version 1
Flags: PMEClk- DSI- D1- D2- AuxCurrent=220mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-
Bridge: PM- B3+

03:01.0 Bridge: PLX Technology, Inc.: Unknown device 9656 (rev 02)
Subsystem: Unknown device 4144:0042
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B-
Status: Cap+ 66Mhz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 32, cache line size 08
Interrupt: pin A routed to IRQ 104
Region 0: Memory at cfdffe00 (32-bit, non-prefetchable)
Region 1: I/O ports at 10ff00
Region 2: Memory at cf800000 (32-bit, non-prefetchable)
Region 3: Memory at cf400000 (32-bit, non-prefetchable)
Capabilities: [40] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [48] #00 [0000]

debian1:~# lspci -vv
01:0a.0 PCI bridge: Intel Corp.: Unknown device b154 (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B-
Status: Cap+ 66Mhz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 16, cache line size 08
Bus: primary=01, secondary=02, subordinate=03, sec-latency=16
I/O behind bridge: 0010f000-0010ffff
Memory behind bridge: 00000000-cfdfffff
Prefetchable memory behind bridge: 00000000c0200000-00000000c0300000
BridgeCtl: Parity+ SERR+ NoISA- VGA+ MAbort- >Reset- FastB2B-
Capabilities: [dc] Power Management version 1
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-
Bridge: PM- B3+

02:0f.0 PCI bridge: Digital Equipment Corporation DECchip 21152 (rev 03) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 16, cache line size 08
Bus: primary=02, secondary=03, subordinate=03, sec-latency=16
I/O behind bridge: 0010f000-0010ffff
Memory behind bridge: 00000000-cfdfffff
Prefetchable memory behind bridge: 00000000c0200000-00000000c0300000
BridgeCtl: Parity+ SERR+ NoISA- VGA+ MAbort- >Reset- FastB2B-
Capabilities: [dc] Power Management version 1
Flags: PMEClk- DSI- D1- D2- AuxCurrent=220mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-
Bridge: PM- B3+

03:01.0 Bridge: PLX Technology, Inc.: Unknown device 9656 (rev 02)
Subsystem: Unknown device 4144:0042
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B-
Status: Cap+ 66Mhz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 32, cache line size 08
Interrupt: pin A routed to IRQ 104
Region 0: Memory at <ignored> (32-bit, non-prefetchable) [size=512]
Region 1: I/O ports at 10ff00 [size=256]
Region 2: Memory at <ignored> (32-bit, non-prefetchable) [size=4M]
Region 3: Memory at <ignored> (32-bit, non-prefetchable) [size=4M]
Capabilities: [40] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [48] #00 [0000]