[U-Boot] [EXT] Re: Cavium/Marvell Octeon Support

[Aaron Williams]

On Wednesday, October 30, 2019 3:05:25 PM PDT Tom Rini wrote:
> External Email
> 
> ----------------------------------------------------------------------
> 
> On Wed, Oct 23, 2019 at 03:50:00AM +0000, Aaron Williams wrote:
> > Hi all,
> > 
> > I have been tasked with porting our Octeon U-Boot to the latest U-Boot
> > and merging it upstream.
> 
> [snip]
> 
> I want to jump back up back to the top of this thread.  And first I want
> to say that I am glad that there is official desire to upstream support.
> This is good.  My concern is that the plan seems to be, at a very high
> level, "get everything we have for every feature upstream".  But as been
> said elsewhere this would roughly double the total LOC for the project,
> and it's not like we're a new project with a small handful of things :)
> It's impossible for the community to review that much code in any
> meaningful way over anything less than a period of several years.  I
> know you've said that to support various customer use cases you need all
> sorts of other things, and while I'm certain that's true, I believe the
> plan needs to be to step back and pick the smallest possible testable
> unit, and upstream it.  And add to it, small pieces at a time.  Thanks!

It might be easier if I were a maintainer for our SOC to limit the needed 
review of a fair bit of the code. I have already found I can cut out a large 
chunk of our code by removing support for our older models. Much of the code 
has been very well tested, for example our serdes initialization and DRAM 
initialization code. That's not to say I can't do some cleanup.

The changes to U-Boot itself should be relatively small as long as I can keep 
much of our code under arch/mips/arch-octeon and arch/mips/cpu/octeon much 
like how our ARM code is. For anything that is applicable to other 
architectures I will place it in the appropriate locations. 

Our ARM code is quite a bit simpler than MIPS because on ARM most of the heavy 
lifting is done by our "BDK" bootloader as well as ATF. On ARM, U-Boot doesn't 
need to deal with SFPs, serdes initialization, DRAM initialization, hot plug 
or a myriad of other issues.

I noticed the same with X86.  If it weren't for these other layers the X86 
code would also be quite large.

I have already identified quite a few very large files that will be removed, 
such as the error handling code for or Octeon2 and earlier CPUs.

Additionally I have identified a number of register definition files I can get 
rid of, including one that's 2.3MB in size! These files tend to be huge 
because they contain definitions for every single chip and revision as well as 
big and little endian definitions. On top of that, there are a huge number of 
comments. Each field contains all of the text that is in our hardware 
reference manuals. There is no dearth of comments. I should be able to cut the 
size of the remaining files to 1/4th their current size or even smaller.

Some files will still remain quite large, however, such as our serdes 
initialization and DRAM initialization code (which I plan to re-architect 
because the original author didn't believe in functions due to stack 
limitations. (it is well commented though). If you ever want to learn all the 
gory details of DDR4 link training and finding trends and so-forth it's all in 
there. The current memory initialization code is over 1MB in size. I plan to 
cut this down and break it up in a clean manner. The initialization code has 
grown in complexity and size over the years as various instabilities have been 
identified and fixed. The DRAM initialization code for our OcteonTX2 CPU is 
almost as large, though this code has been cleaned up and re-written. There 
really is no way to avoid this. The OcteonTX and OcteonTX2 DDR initialization 
code is similar to that for Octeon. In the case of U-Boot on our ARM SoCs, 
though, initialization is done before U-Boot is loaded.

I'll move the init code to drivers/ram/marvell/octeon. It will be about twice 
as large as the AXP driver (which only handles DDR3). The serdes init code I 
figure could go under drivers/soc/marvell/octeon.

I noticed that there are several directories under drivers for memory. There's 
drivers/ram, drivers/memory and drivers/ddr. These should be consolidated. I 
think some code might be able to be common, such as the SPD decoding code. 
It's even possible that some algorithms might be able to be made common such 
as deskew training and read/write leveling.

In terms of Octeon specific features, there really aren't too many of those 
but most of the ones we have are essential in the bootloader. There's no 
avoiding the Serdes and low-level network initialization. The serdes init code 
works across all networking interface types (SGMII, 1000Base-X, XAUI, RXAUI, 
XFI, XLAUI, 25G (XLAUI), SATA, PCIe, SRIO plus all the variants (i.e. KR). It 
also configures all the clocks and equalization. It's not like a simple 
gigabit NIC nor is it offloaded to some other layer. Some of this code will 
come later, for example support for NUMA with CN78XX (96 cores, 256GiB of 
RAM).

Currently we are using 39MB under arch/mips. I think I can easily cut this 
down to 15MB or smaller, especially by moving some code here to the 
appropriate driver directories (i.e. DRAM,  pcie, watchdog, etc.)

It will still be a large SoC, though.

-Aaron