[U-Boot] U-boot Porting

[Aaron Williams]

Hi Andreas,

I would love to see OCTEON support in the mainline as well, though I am 
not sure how I should go about this since it is a substantial amount of 
code. Fortunately most of the changes are not to the common code, and 
many of the common code changes are feature enhancements portable to 
other platforms as well.

One big problem I have is I have no way of testing any of my stuff on 
non-Cavium boards or CPUs so testing my changes and making sure I didn't 
break anything is rather difficult. I am also not too familiar with git 
or how to actually submit the patches with it though I can work with our 
Linux kernel developer on this.

When I redid U-Boot using the up-to-date code I made a large effort to 
keep our stuff separate from the rest of the U-Boot code so much of our 
stuff is fairly well isolated. I placed most of our code under 
arch/mips/cpu/octeon and arch/mips/include/asm/arch-octeon (though some 
of the files in arch/mips/include/asm also had to be changed). Most of 
the rest of the code is in board/octeon/xxx though there's a fair amount 
in board/octeon/common.

Most major changes to the common code were done by marking the common 
code functions as "weak" and replacing them elsewhere.

A few of the files in U-Boot are also shared by some other utilities, 
such as our code for loading and executing simple executive applications 
and initializing DDR memory.

One constantly moving target in U-Boot is the simple executive code. We 
support writing se applications which are basically ELF executables that 
run directly on the OCTEON processor without an operating system 
underneath. These can be run in parallel with other applications on 
different cores along with Linux. The main thing involved here is the 
memory management. We use a "named block" memory scheme which is used by 
the Linux kernel and simple executive applications to find and share 
various data structures such as the flat device tree (BE and LE 
versions), temporary load address, linux reserved address, PCI console 
memory block and some JTAG information. We need this because we can have 
multiple applications running on different cores.

U-Boot itself always runs on core 0 but needs to be aware of the other 
CPU cores (currently up to 32).

We have little in common with the other MIPS processors since we have to 
support 64-bit mode. To do this we compile it with our own toolchain 
using the N32 ABI. I believe most of our changes are now in the mainline 
GCC as well now.

We also do not use arch/mips/lib/board.c since our version is so 
different. I will be the first to admit that the code needs to be 
cleaned up since board_init_f is huge.

We have done some interesting things with our U-Boot image. Our U-Boot 
binary can be executed at any memory address as long as it's 4MB aligned 
(or 64K aligned when running from flash). We have a single binary image 
that can be booted directly from RAM (when booting over eJTAG or 
PCI/PCIe), out of L2 cache and out of flash. By doing this we can also 
store two copies of U-Boot in flash (the same binary image) to support a 
failsafe and normal bootloader.

2702: common board code
28,867: support for 33 different boards
13,568: configuration .h files for all of our boards
2,215: board_octeon.c (replacement for board.c)
24,004: .c files in arch/mips/cpu/octeon/
10,147: .c files in arch/mips/cpu/octeon/commands/
155,994: .c files from our simple executive which are linked to U-Boot. 
This includes files for various errata, networking support, clock, 
memory management used by our simple executive environment as well as 
code to deal with many of the modules in all of our various chips.
10,166: .c files for OCTEON-specific commands, some of which could be 
merged into the core code.
279,576: .h files for our simple executive. Some of these files are very 
large because they include generated header files for all of the 
register definitions for all of our chips (including different revisions).

The most common change we have to make to any PCI drivers is to support 
the appropriate address mapping. In our case the virtual address needs 
to be translated to a PCI DMA address, which may be different than a 
physical address. Fortunately the mechanisms are available in the PCI 
driver to support this. The drivers also need to be extended to be able 
to pass 64-bit addresses to the devices.

For devices not on the PCI bus (i.e. USB EHCI) we have to use a 
different mechanism in order to always pass the physical address.

All accesses to registers have to go through wrappers (which are usually 
already in place) since all of our non-PCI device registers are always 
mapped into various 64-bit address spaces.

Here's a list of changes from a quick glance:

Added and changed drivers:
- TI tmp42x temperature sensor
- NX SAA56004X temperature sensor
(I think I wrote one of the other temp sensor drivers as well)
- Power driver for ispPAC chip
- OCTEON MDIO driver
- OCTEON USB driver for OCTEON I and OCTEON
- USB clock init code for OCTEON II USB EHCI interface (slight 
modifications to EHCI driver for OCTEON II to handle virtual memory 
mapping and 64-bit addressing)
- Intel E1000 driver - minor change to support 64-bit addressing and 
virtual to PCI DMA addressing.
- OCTEON GPIO driver
- OCTEON MMC/SD/SDHC/SDXC driver
- OCTEON SPI driver
- Fixed AHCI SATA driver so it compiles and works
- Modified NAND driver in order to support large NAND chips, also added 
support for multi-bit ECC
- Fixed CFI NOR flash driver to properly handle 8-bit mode.
- Minor changes to PCI driver

Drivers not under drivers/ but under arch/mips/cpu/octeon:
- Watchdog
- compact flash (this also requires a rather hacked up version of the 
cmd_ide.c file in order to support multiple CF devices properly)
- ethernet (including management, SGMII, XAUI, SPI and some other 
interface types)
- PCI driver
- PCI console driver (for OCTEON NIC/PCI target boards)
- serial
- I2C
- MIPS micro assembler (required for watchdog support)

lib changes:
- Made md5, sha1, sha256 and crc32 functions weak since we supply OCTEON 
replacement functions that take advantage of encryption/hash/CRC 
instructions to reduce code size
- Updated lzma code to latest
- Added bch.c for handling multi-bit ECC.
- Enabled and/or added a few more string functions
- Made a few other functions "weak"

Common/other changes:
- Minor changes to various commands so that if an address of 0 is 
supplied by the user it will use the default load address (typically 
0x20000000 for us)
- Some environment changes in order to support the environment being in 
RAM when the bootloader is started.
- RAM environment support
- Support for user-defined functions to be called whenever setenv is 
called to hook environment variable changes.
- MTD and flash filesystem support for NAND > 4GiB to match Linux kernel.
- Support for prompt to be created dynamically and also defined in an 
environment variable (including in hush shell)
- Minor change to DHCP code to allow for custom options
- Support for DHCP option to set TFTP server IP
- Added commands for bunzip and unlzma
- Minor change to mmc command to enable some additional functionality
- Minor change to dlmalloc.c init code so to not zero out memory in our 
simulator case.
- Minor changes to usb code to handle some buggy hardware (i.e. SanDisk 
Cruzer similar to Linux).
- Support for go command, elf and some other commands to call a function 
so we can perform clean-up before handing execution off (i.e. shutting 
down networking support, freeing resources).

Octeon specific commands:
- bootloaderupdate
- bootloadervalidate
- bootoct (boots simple executive applications)
- bootoctlinux (boots the OCTEON Linux kernel)
- base64 - sets 64-bit base address for md64, etc.
- cmp64/cp64/loop64/md64/mm64 (64-bit versions of cp/md/mm), also added 
.d for 64-bit accesses
- tftp (alias for tftpboot since so many customers rely on tftp)
- read64/read64b/read64l/read64s - read a 64-bit memory address (used to 
read registers)
- write64/write64b/write64l/write64s - writes a 64-bit memory address 
(used to write registers)
- octnand - OCTEON-specific NAND read/write routines used for writing 
bootloaders, etc. to NAND
- qlm - print/modify our QLM interface JTAG settings (a QLM is an I/O 
interface that can be SGMII/XAUI/ILK/PCIe and various other high-speed 
interface types).
- octreginfo - prints out all of the CP0 registers and TLB entries.
- namedprint - prints out all named memory blocks
- freeprint - prints out all free memory blocks
- namedalloc - allocate a named memory block
- octwd - configure OCTEON watchdog support
- tlv_eeprom - display or modify the contents of a board-specific I2C 
EEPROM used by Cavium boards which contains board identification and 
configuration information.
- eraseenv - erases the environment flash block
- bootstage3 - searches for a stage 3 bootloader, used when booting from 
MMC/SD.
- octbootbus - Prints all of the timing parameters on our boot bus or 
allows the timing parameters to be changed for various devices


-Aaron

On 07/06/2012 01:19 AM, Andreas Bießmann wrote:
> Dear Aaron Williams,
>
> On 06.07.2012 01:52, Aaron Williams wrote:
>> Hi Zahid,
>>
>> I am in charge of U-Boot for OCTEON MIPS. I have not pushed the changes
>> back upstream since the amount of code is enormous (over 430K lines of
>> code!). Granted, a huge percentage of that is from generated register
>> files but it is still a huge amount of code. Just the DRAM
>> initialization code is 9600 lines of code for DDR2 and DDR3 support.
>>
>> We also do things that no other architecture does with U-Boot such as we
>> always run in TLB mapped memory. The code is available under GPL but you
>> need to contact support at cavium.com. With the TLB mapping it no longer
>> matters where U-Boot is loaded in memory. The same U-Boot binary image
>> can start executing at any 64K flash boundary in the first 4MB of flash
>> (so we support the same image as a failsafe and standard bootloader),
>> any 4MB boundary in RAM (when booted over PCI or eJTAG) and out of L2
>> cache (our top of trunk copies itself from flash to the L2 cache very
>> early on to speed up memory initialization). The TLB mapping also allows
>> U-Boot to be copied to the very top of memory since KSEG0 is rather
>> limited to only 256MB. This is essential since we can support many GB of
>> RAM which otherwise requires 64-bit addressing.
>>
>> I need to push some of my changes back upstream since I have added some
>> drivers for some temperature sensors, power monitors, fixed the AHCI
>> driver and added a few features over the standard U-Boot (such as
>> dynamically generated prompt support).
> I would really like to see the whole stuff in mainline in favor of out
> of tree patches maintained only inside Cavium.
> Mainline U-Boot may also benefit from your generic changes and you will
> get reviews for free.
>
> Best regards
>
> Andreas Bießmann
>


-- 
Aaron Williams
Software Engineer
Cavium, Inc.
(408) 943-7198  (510) 789-8988 (cell)