[U-Boot] BayTrail PCIe x4 slot (Soft-Strap?)

[Stefan Roese]

Hi Bin,

On 08.07.2016 12:34, Bin Meng wrote:
> On Fri, Jul 8, 2016 at 6:13 PM, Stefan Roese <sr at denx.de> wrote:
>> Hi Bin,
>>
>> On 08.07.2016 11:22, Bin Meng wrote:
>>> On Fri, Jul 8, 2016 at 4:44 PM, Stefan Roese <sr at denx.de> wrote:
>>>> Hi Bin,
>>>>
>>>> On 08.07.2016 04:27, Bin Meng wrote:
>>>>> On Thu, Jul 7, 2016 at 11:52 PM, Stefan Roese <sr at denx.de> wrote:
>>>>>> Hi!
>>>>>>
>>>>>> I do have BayTrail / FSP related question. I'm currently trying
>>>>>> to use a DFI QSeven SoM which has one x4 PCIe slot instead
>>>>>> of the usual 4 x1 slots. So all 4 PCIe lanes are used by the
>>>>>> first PCIe controller. With the current U-Boot, all 4 PCIe
>>>>>> controllers are enabled by the FSP :
>>>>>>
>>>>>> 00:1c.0 PCI bridge: Intel Corporation Atom Processor E3800 Series PCI Express Root Port 1 (rev 11)
>>>>>> 00:1c.1 PCI bridge: Intel Corporation Atom Processor E3800 Series PCI Express Root Port 2 (rev 11)
>>>>>> 00:1c.2 PCI bridge: Intel Corporation Atom Processor E3800 Series PCI Express Root Port 3 (rev 11)
>>>>>> 00:1c.3 PCI bridge: Intel Corporation Atom Processor E3800 Series PCI Express Root Port 4 (rev 11)
>>>>>>
>>>>>> In this configuration, the x4 PCIe card that is installed in the
>>>>>> PCIe slot is not detected. The system always generated hotplug
>>>>>> events for all for ports, but the link is not established.
>>>>>>
>>>>>> The original DFI BIOS only enables the first PCIe controller. The
>>>>>> controllers 1...3 are not visible via lspci. Here the 4x link
>>>>>> is established and the 4x PCIe card is detected correctly.
>>>>>>
>>>>>> My question now is, how can I enable this 4x link on the first
>>>>>> PCIe controller via U-Boot / FSP? I have found no option on how
>>>>>> to configure the PCIe controllers in the FSP dts properties.
>>>>>> So that only the first controller is enabled and visible via
>>>>>> lspci etc. The BayTrail datasheet mentions this to configure the
>>>>>> PCIe setup in chapter "23.2.1 Root Port Configurations":
>>>>>>
>>>>>> "
>>>>>> Root port configurations are set by SoftStraps stored in SPI flash,
>>>>>> and the default option is “(4) x1”. Links for each root port will
>>>>>> train automatically to the maximum possible for each port.
>>>>>> "
>>>>>>
>>>>>
>>>>> Correct. It's determined by the soft strap value in the SPI flash, to
>>>>> be specific, in the descriptor.bin.
>>>>
>>>> Thanks.
>>>>
>>>> I'm still wondering, if there isn't any documentation about these
>>>> soft-strap values that are stored in the SPI flash for this Atom SoC.
>>>> Does someone have a link to such a document? Or any further infos?
>>>> Can the values be simply changed in the binary image, or is there
>>>> some sort of CRC on the image that will fail after such a change?
>>>>
>>>
>>> Just checked the datasheet, other than the description you quoted in
>>> preivous thread, no detailed info regarding to this soft-strap values.
>>> Looks there is no public document that has such info.
>>>
>>>>>> Where is this SoftStraps in the SPI flash located? I've found this
>>>>>> page mentioning that its a offset 0x100:
>>>>>>
>>>>>> https://embedded.communities.intel.com/thread/8539
>>>>>>
>>>>>> But I fail to find any documentation for all those Soft-Strap
>>>>>> Registers / Values in the SPI flash. Does anyone have some further
>>>>>> infos / documentation on this? How to enable 4x PCIe lanes
>>>>>> for one PCIe controller on BayTrail / Atom?
>>>>>>
>>>>>
>>>>> Please try this:
>>>>>
>>>>> If you have the original DFI BIOS, extract the descriptor.bin using
>>>>> U-Boot's ifdtool (ifdtool -x dfi_bios), and use this descriptor.bin to
>>>>> generate u-boot.rom. Or if you don't have the original DFI BIOS, dump
>>>>> it using Dediprog SF100, and do the same.
>>>>
>>>> Thanks.
>>>>
>>>> I do have the original BIOS. And with its descriptor.bin I get the
>>>> following compilation error:
>>>>
>>>> Region Intel ME is 3141632(0x2ff000) bytes. File is 5238784(0x4ff000)  bytes. Not injecting.
>>>> Makefile:1070: recipe for target 'u-boot.rom' failed
>>>> make: *** [u-boot.rom] Error 1
>>>
>>> This indicates the original BIOS's descriptor.bin defines a smaller
>>> me.bin than what you are using.
>>>
>>>>
>>>> When I now also use the extracted me.bin to replace the one that
>>>> I'm currently using, u-boot.rom can be built just fine. But
>>>> unfortunately it does not boot to the prompt. I can see some
>>>> early debug output but that's it. I also tried to use the debug
>>>> FSP version but with this compilation fails with this error:
>>>>
>>>
>>> By early debug output, do you mean: U-Boot's debug UART output? At
>>> which point does it hang?
>>>
>>>> Output file is too small. (8388608 < 8421376)
>>>> Makefile:1070: recipe for target 'u-boot.rom' failed
>>>> make: *** [u-boot.rom] Error 1
>>>>
>>>
>>> This indicates the debug version FSP has larger size so that
>>> u-boot.rom does not fit.
>>>
>>>> Has someone been able to use the FSP debug version? Bin, do you
>>>> know how this can be integrated without this image size
>>>> overflow? And which UART will be used by this FSP debug?
>>>>
>>>
>>> If you are using debug version FSP, please adjust the
>>> CONFIG_FSP_ADDR=0xfffb0000.
>>
>> Ah, okay. This info was missing. Is this documented somewhere? If
>> not, could you please add this to the x86 README? Or I'll send a
>> patch for this next week.
> 
> See arch/x86/Kconfig:
> 
> 226 config FSP_ADDR
> 227         hex "Firmware Support Package binary location"
> 228         depends on HAVE_FSP
> 229         default 0xfffc0000
> 230         help
> 231           FSP is not Position Independent Code (PIC) and the whole
> FSP has to
> 232           be rebased if it is placed at a location which is
> different from the
> 233           perferred base address specified during the FSP build. Use Intel's
> 234           Binary Configuration Tool (BCT) to do the rebase.
> 235
> 236           The default base address of 0xfffc0000 indicates that
> the binary must
> 237           be located at offset 0xc0000 from the beginning of a 1MB
> flash device
> 
> But BayTrail-specific version base address of the debug version is
> undocumented. Feel free to send a patch on this.

Sure. I'll send a patch for this beginning of next week.
 
>>
>>> The debug FSP uses the SoC integrated
>>> legacy UART for the debug output.
>>
>> Please find below the output of the debug FSP:
>>
>> FSP Got Notification. Notification Value : 0x00000020
>> FSP Post PCI Enumeration ...
>> Install PPI: 30CFE3E7-3DE1-4586-BE20-DEABA1B3B793
>> Notify: PPI Guid: 30CFE3E7-3DE1-4586-BE20-DEABA1B3B793, Peim notify entry point: 7B797CEA
>> PchReadyToBootEvent() Start
>> PciERWORegInit() Start
>> PciERWORegInit() End
>> PchReadyToBootEvent() End
>> Setting baudrate for SIO HS UART...
>> FSP Got Notification. Notification Value : 0x00000040
>> FSP Ready To Boot ...
>> Install PPI: 7CE88FB3-4BD7-4679-87A8-A8D8DEE50D2B
>> Notify: PPI Guid: 7CE88FB3-4BD7-4679-87A8-A8D8DEE50D2B, Peim notify entry point: 7B79802F
>> PchInitExitBootCallback() Start
>> LockPciDevCap() BusNumber = 0x0
>> LockPciDevCap() Bus = 0x0,Dev = 0x0,Func = 0x0
>> LockPciDevCap() PciEAddressBase = 0xE0000000
>> LockPciDevCap() StatusReg = 0x0
>> LockPciDevCap() Bus = 0x0,Dev = 0x2,Func = 0x0
>> LockPciDevCap() PciEAddressBase = 0xE0010000
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() CapId = 0xD0
>> LockPciDevCap() While CAPBILITY = 0x9001
>> LockPciDevCap() While CapId = 0x90
>> LockPciDevCap() While CAPBILITY = 0xB005
>> LockPciDevCap() While CapId = 0xB0
>> LockPciDevCap() While CAPBILITY = 0x9
>> LockPciDevCap() While CapId = 0x0
>> LockPciDevCap() Bus = 0x0,Dev = 0x11,Func = 0x0
>> LockPciDevCap() PciEAddressBase = 0xE0088000
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() CapId = 0x80
>> LockPciDevCap() While CAPBILITY = 0x1
>> LockPciDevCap() While CapId = 0x0
>> LockPciDevCap() Bus = 0x0,Dev = 0x12,Func = 0x0
>> LockPciDevCap() PciEAddressBase = 0xE0090000
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() CapId = 0x80
>> LockPciDevCap() While CAPBILITY = 0x1
>> LockPciDevCap() While CapId = 0x0
>> LockPciDevCap() Bus = 0x0,Dev = 0x13,Func = 0x0
>> LockPciDevCap() PciEAddressBase = 0xE0098000
>> LockPciDevCap() StatusReg = 0x2B0
>> LockPciDevCap() StatusReg = 0x2B0
>> LockPciDevCap() CapId = 0x80
>> LockPciDevCap() While CAPBILITY = 0x7005
>> LockPciDevCap() While CapId = 0x70
>> LockPciDevCap() While CAPBILITY = 0xA801
>> LockPciDevCap() While CapId = 0xA8
>> LockPciDevCap() While CAPBILITY = 0x12
>> LockPciDevCap() While CapId = 0x0
>> LockPciDevCap() Bus = 0x0,Dev = 0x15,Func = 0x0
>> LockPciDevCap() PciEAddressBase = 0xE00A8000
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() CapId = 0x80
>> LockPciDevCap() While CAPBILITY = 0x1
>> LockPciDevCap() While CapId = 0x0
>> LockPciDevCap() Bus = 0x0,Dev = 0x16,Func = 0x0
>> LockPciDevCap() PciEAddressBase = 0xE00B0000
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() CapId = 0x80
>> LockPciDevCap() While CAPBILITY = 0x1
>> LockPciDevCap() While CapId = 0x0
>> LockPciDevCap() Bus = 0x0,Dev = 0x17,Func = 0x0
>> LockPciDevCap() PciEAddressBase = 0xE00B8000
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() CapId = 0x80
>> LockPciDevCap() While CAPBILITY = 0x1
>> LockPciDevCap() While CapId = 0x0
>> LockPciDevCap() Bus = 0x0,Dev = 0x1C,Func = 0x0
>> LockPciDevCap() PciEAddressBase = 0xE00E0000
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() CapId = 0x40
>> LockPciDevCap() While CAPBILITY = 0x8010
>> LockPciDevCap() While CapId = 0x80
>> LockPciDevCap() While CAPBILITY = 0x9005
>> LockPciDevCap() While CapId = 0x90
>> LockPciDevCap() While CAPBILITY = 0xA00D
>> LockPciDevCap() While CapId = 0xA0
>> LockPciDevCap() While CAPBILITY = 0x1
>> LockPciDevCap() While CapId = 0x0
>> LockPciDevCap() Bus = 0x0,Dev = 0x1D,Func = 0x0
>> LockPciDevCap() PciEAddressBase = 0xE00E8000
>> LockPciDevCap() StatusReg = 0x290
>> LockPciDevCap() StatusReg = 0x290
>> LockPciDevCap() CapId = 0x50
>> LockPciDevCap() While CAPBILITY = 0x5801
>> LockPciDevCap() While CapId = 0x58
>> LockPciDevCap() While CAPBILITY = 0x980A
>> LockPciDevCap() While CapId = 0x98
>> LockPciDevCap() While CAPBILITY = 0x13
>> LockPciDevCap() While CapId = 0x0
>> LockPciDevCap() Bus = 0x0,Dev = 0x1E,Func = 0x0
>> LockPciDevCap() PciEAddressBase = 0xE00F0000
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() CapId = 0x80
>> LockPciDevCap() While CAPBILITY = 0x1
>> LockPciDevCap() While CapId = 0x0
>> LockPciDevCap() Bus = 0x0,Dev = 0x1E,Func = 0x1
>> LockPciDevCap() PciEAddressBase = 0xE00F1000
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() CapId = 0x80
>> LockPciDevCap() While CAPBILITY = 0x1
>> LockPciDevCap() While CapId = 0x0
>> LockPciDevCap() Bus = 0x0,Dev = 0x1E,Func = 0x2
>> LockPciDevCap() PciEAddressBase = 0xE00F2000
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() CapId = 0x80
>> LockPciDevCap() While CAPBILITY = 0x1
>> LockPciDevCap() While CapId = 0x0
>> LockPciDevCap() Bus = 0x0,Dev = 0x1E,Func = 0x3
>> LockPciDevCap() PciEAddressBase = 0xE00F3000
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() CapId = 0x80
>> LockPciDevCap() While CAPBILITY = 0x1
>> LockPciDevCap() While CapId = 0x0
>> LockPciDevCap() Bus = 0x0,Dev = 0x1E,Func = 0x4
>> LockPciDevCap() PciEAddressBase = 0xE00F4000
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() CapId = 0x80
>> LockPciDevCap() While CAPBILITY = 0x1
>> LockPciDevCap() While CapId = 0x0
>> LockPciDevCap() Bus = 0x0,Dev = 0x1E,Func = 0x5
>> LockPciDevCap() PciEAddressBase = 0xE00F5000
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() StatusReg = 0x10
>> LockPciDevCap() CapId = 0x80
>> LockPciDevCap() While CAPBILITY = 0x1
>> LockPciDevCap() While CapId = 0x0
>> LockPciDevCap() Bus = 0x0,Dev = 0x1F,Func = 0x0
>> LockPciDevCap() PciEAddressBase = 0xE00F8000
>> LockPciDevCap() StatusReg = 0x210
>> LockPciDevCap() StatusReg = 0x210
>> LockPciDevCap() CapId = 0xE0
>> LockPciDevCap() While CAPBILITY = 0x9
>> LockPciDevCap() While CapId = 0x0
>> LockPciDevCap() Bus = 0x0,Dev = 0x1F,Func = 0x3
>> LockPciDevCap() PciEAddressBase = 0xE00FB000
>> LockPciDevCap() StatusReg = 0x290
>> LockPciDevCap() StatusReg = 0x290
>> LockPciDevCap() CapId = 0x50
>> LockPciDevCap() While CAPBILITY = 0x1
>> LockPciDevCap() While CapId = 0x0
>> ConfigureMiscAtBoot() Start
>> ConfigureMiscAtBoot() End
>> ConfigureLpeAtBoot() Start
>> ConfigureLpeAtBoot() End
>> ConfigureOtgAtBoot() Start
>> ConfigureOtgAtBoot() End
>> ConfigureLpssAtBoot() Start
>> ConfigureLpssAtBoot() End
>> ConfigureSccAtBoot() Start
>> Using eMMC 4.5
>> Overwrite the nslew/pslew -> 0
>> ConfigureSccAtBoot() End
>> ConfigureXhciAtBoot() Start
>> ConfigureXhciAtBoot() End
>> PchInitExitBootCallback() End
>> Notify: PPI Guid: 7CE88FB3-4BD7-4679-87A8-A8D8DEE50D2B, Peim notify entry point: 7B7C0619
>> Notify: PPI Guid: 7CE88FB3-4BD7-4679-87A8-A8D8DEE50D2B, Peim notify entry point: 7B781876
>> MP ReadyToBootEvent()
>> Detected 4 CPU threads
>> Install PPI: 605EA650-C65C-42E1-BA80-91A52AB618C6
>> ============= PEIM FSP is Completed =============
>>
>>
>> Thats all. Nothing after this. Can you see some problems with
>> this debug FSP output? Helping us to get this U-Boot version with
>> this updated flash descriptor binary running?
>>
> 
> Looks FSP has completed its execution at least for the
> fsp_notify(post_pci_phase). so U-Boot hangs somewhere after the
> fsp_notify(). Can you add some debug output in U-Boot to figure out
> where it hangs?

That's what I'm doing right now. And I'm getting a bit further here.
I need to stop now, as the weekend is approaching in big steps! I'll
keep you updated on this.

BTW: I noticed that just changing one Soft Strap value in the flash
descriptor (via some hex editor) leads to a complete non-bootable board.
No early debug output at all in this case. I'm guessing that some CRC
is generated over this flash descriptor image and if this does not
match, booting will not continue. If someone has some insight knowledge
on this flash descriptor image / CRC generation, please share this with
me. All input on this is really appreciated.

Thanks,
Stefan