[U-Boot] boot-up time optimization. Where to start?

[Wolfgang Denk]

Dear Simon Schwarz,

In message <BANLkTi=HfOsg73_GCC=4aVoi_XUNbcf9nQ at mail.gmail.com> you wrote:
>
> I just started to work on my bachelor thesis. It is about "Linux boot-up
> time optimization". The past days I spend analyzing what consumes the most
> time in the boot process.

A lot of efforts have alreay been spent in this area.  A ton of
specialized solutions exist, some of them even documetned.  I highly
recommend to study the available information first.  A good starting
point might be this page: http://www.elinux.org/Boot_Time

>  I found that u-boot takes pretty much as long as the whole Linux kernel
> (the one we are using).

This depends on a lot of things: architecture, boot device, memory
architecture, etc. etc.

> I started digging into the source and I think I have a big picture of what
> is going on. I already learned from the mailing list that it is a good idea
> to start a discussion early if you plan to change something and want it
> upstream. At this point of my thesis I'am free to choose where I start -
> only string attached is that if it is platform specific it has to be TI
> OMAP3.
> 
> So here is my question: Where do you see the most potential to optimize
> u-boot?
> 
> I already have two bullets on my list (just some ideas - maybe totally
> unrealistic *g*):
> - Use Hardware specific copy commands
> - build the checksum while moving the kernel to RAM

Eventually you will find that these are not _that_ important.

A few thoughts, mostly unsorted:

- Booting fast means running fast, so make sure your CPU can run as
  fast as possible: chose an optimized clock configuration; chose
  optimized memory timings; make sure all caches are enabled.

- Embedded processors usually have tiny caches, so performance is
  often limited by the memory bandwidth: a device with a 32 bit wide
  memory bus will be faster than one with a 16 bit bus.

- Memory technology is imprtant: DDR3 is much faster than plain old
  SDRAM.

- Chose the storage device wisely: using NOR flash as boot device
  allows direct execution of the code from the NOR (often using a 32
  bit wide bus); if you use NAND flash as storage you will have a
  relatively slow download and a way more complicated protocol for
  booting.

- Benchmark your system before you make assumptions.  For example, it
  is not possible to decide in general if a compressed or an
  uncompressed Linux kernel image is a better choice - if you have a
  fast bus and a slow CPU, then the CPU performance for uncompressing
  the code may be the bottleneck, and an uncompressed kernel may be
  way faster; if you have a fast CPU and a relatively slow storage
  device it may be much more efficient to load the smaller, compressed
  kernel image, and the CPU will still be mostly idle uncompressing
  it.

- Keep in mind that you are always trading efforts and costs, boot
  time and reliability/security against each other. You always get a
  maximum of two items, never all three (good, cheap, fast). If you
  optimize for speed, you may have to accept not only higher
  development efforts, but for example also higher production coses
  (like when needing bigger boot devices for uncompressed images or
  file systems), but also have to give up security/reliabilty by for
  example switching off the checksum protection of the images.

- Optimize your configuration: U-Boot in it's general configuration is
  more of a bring-up and debug tool, combined with a powerful machine
  for implementing fancy features like software updates and such, but
  not optimized at all for fast booting.

  Removing unused code (both in U-Boot and Linux) makes the memory
  footprint smaller and thus loading faster.

- Apply the old Antoine de Saint-Exupery aphorism: "Perfection is
  reached, not when there is no longer anything to add, but when
  there is no longer anything to take away."  For example when booting
  form NAND flash, we have a number of steps: some ROM code loads a
  small block of code from NAND into some (usually limited) memory;
  this code (in U-Boot terms the nand_spl code) will load the rest of
  U-Boot and start it; U-Boot will then initialize more of the system
  and relocate itself to the upper end o the RAM; then U-Boot will
  load the Linux kernel and start it.

  This allows for a powerful and very flexible system, but how much
  of this power and flexibility do you really need on a system where
  your primary goal is to boot fast, i. e. to make U-Boot goes out of
  the way as fast as possible?

  If you want to boot fast, why do you go through all these steps,
  then?  Why does not your nand_spl code load the Linux kernel
  directly and start it, instead of wasting time with all these other
  steps and phases?  [Note that this is the approach taken by all
  these systems that report sub-second boot times.]

- Don't stop the optimization with the loading of the Linux kernel.
  The choice of the root file system type, the strategy for driver
  initialization (immediate with statically linked drivers versus lazy
  with dynamically loaded modues) and for starting software services
  is at least as well as important.

  Keep in mind that on most systems it's trivial (with standard
  U-Boot) to have Linux starting the first user space code within 5 or
  6 seconds after port-on.  But on many systems the appliucation
  startup will take much longer than that.

- And on each step of you path benchmark again, and focus on the hot
  spots.  It makes little sense to spend a week of effort on reducing
  the execustion time of function foo() to 5% of it's original value,
  when this cunction contributes only 1% to the total boot time.

Hope this helps.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
But the only way of discovering the limits  of  the  possible  is  to
venture a little way past them into the impossible.
                         - _Profiles of the Future_ (1962; rev. 1973)
                  ``Hazards of Prophecy: The Failure of Imagination''