[U-Boot] Cavium/Marvell Octeon Support

[Aaron Williams]

Hi all,

I have been tasked with porting our Octeon U-Boot to the latest U-Boot and merging it upstream. This will involve a very significant amount of code that generally will not be compatible with other MIPS processors due to our needs and requirements. For example, the start.S will need to be completely different than what is present. For example, our existing start.S is 3577 lines of code in order to deal with things like RAS, exceptions, virtual memory and more. We need to use virtual memory since U-Boot can be loaded at any 4MB boundary in memory, not just 0xbfc00000. A number of drivers will need to be updated in order to properly map pointers to physical addresses. This is needed anyway, since I see numerous drivers that assume that a pointer is a DMA address. For MIPS this is never the case (I'm looking at XHCI).

The new Octeon U-Boot will be native 64-bit instead of how the earlier one was 32-bit using the N32 ABI (so 64-bit addresses could be accessed). We had to jump through some hoops to make a 32-bit U-Boot fully support 64-bit hardware.

I think we can shrink the code by removing support for starting "simple executive" tasks. Simple executive tasks are bare metal applications that can run on dedicated cores beside Linux (or without Linux). I will also not be porting any support for anything older than Octeon3.

We also make heavy use of our SDK in order to perform hardware initialization and networking. In our old U-Boot, we have almost 900K lines of code. I can cut out much of this but much will remain.

We also have added extensive infrastructure for handling SFP and QSFP cables as well as very extensive phy support for phys from Aquantia/Marvell, Vitesse/Microsemi, Inphi/Cortina and an Avago gearbox. Our customer wants us to port all of this to the new U-Boot and upstream it. I'm worried about the sheer amount of code since it is absolutely massive. Some of these phy drivers are extremely complex and need to tie into the SFP management. We also need to use a background polling thread while at the command prompt. A fair bit of our phy code is not in the normal phy drivers because it did not fit the model. Some of these phy drivers need to interact with the SFP support code in order to handle hot plug events in order to reconfigure themselves based on the cable type. The existing SFP code handles everything from SFP to SFP28 as well as QSFP and 100G QSFP (never tested).

In the old U-Boot the PHY support had to be significantly enhanced due to requirements for hot-plugging and how some of the PHYs are configured. It gets quite complicated with phys like the Inphi where one phy can handle either four ports (XFI/SGMII) or a single 4-lane port (XLAUI). It gets even worse since in some boards we use reclocking chips and there is one chip that handles the receive path of a QSFP and another that handles the transmit path. Further complicating things, with a QSFP it can be treated either as XLAUI or as four XFI ports, so you can have four ports spread across two chips, with each port using different slices of each chip. In the case of the Inphi/Cortina chip, a single device can handle one or four ports based on the configuration and it is configured by "slice" which is basically an offset into the MDIO register space. We had to jump through hoops in order to have this stuff work in a sane way in the device tree. We added entries for SFP and QSFP slots in the device tree which point to the MACs, GPIOs and I2C bus because pointing them to the phys just got too insane. This will need to be ported to the new U-Boot. It should not break the existing support since most of it was implemented outside of the core PHY handling code. In the port, it would be far better if this could be integrated in. The SFP management code is architecture agnostic as is all of the PHY support. The callbacks for the SFP support are used by the MAC which then notifies the PHY since the MAC often needs to reconfigure itself. It can handle some crazy configurations.

While I see some phy drivers that we also support, i.e. Cortina, our drivers tend to have a lot more functionality. For example, all of our phy drivers that support firmware support commands for upgrading the firmware as well as things like cable testing and other features.

Our bootloader needs to be able to be booted from a variety of sources, including SPI, eMMC, NOR flash and booting over the PCI bus from a host system. This is one reason we use virtual memory. The other reason is that it eliminates the need to perform relocation. Our start.S code handles all of these different cases as well as exception handling.

I will also say up front that the memory initialization code is a mess and quite large (it was written by a hardware engineer who never heard of functions).

One thing is that this will break mips unless it is refactored like ARM is, for example, separating armv7 and armv8. This way we could have arch/mips/cpu/octeon. I did this with the old bootloader to separate our stuff. I'm open to suggestions as for the naming. I don't see how we can share much of the code with the other MIPS CPUs.

All in all, I think the final port will add between 500K-1M lines of code for the Octeon CPU. It is much more extensive than what is required for OcteonTX since in the latter case most of the hardware initialization is done by earlier stage bootloaders and the ATF handles things like SFP port management and many of the networking operations.

I'm not sure how well I'll be able to upstream all of this code at this point since I was just handed this task. We already have at least 1M lines of code added to the old U-Boot which is based off of 2013.08 with a lot of backports.