[U-Boot] Announcing easy Linux installation on Allwinner devices for non-geek users

[Luc Verhaegen]

On Thu, Jan 22, 2015 at 02:49:03PM +0200, Siarhei Siamashka wrote:
> Hello,
> 
> This is a somewhat long e-mail.
> 
> 1. The problem
> 2. A possible solution
> 3. Relocatable SD card
> 4. NAND
> 5. Support more devices!
> 6. Reliability tests
> 7. FEL mode installer application
> 8. A31+ support
> 9. Summary
> 
> =====================================================================
> == 1. The problem
> =====================================================================
> 
> First of all, let's see what is the difference between various
> Allwinner boards/devices and the Raspberry Pi (which is considered
> by many as a role model of being user friendly and easy to use).
> 
> Raspberry Pi offers only a few hardware variants with minor differences.
> And Raspberry Pi users are familiar with the concept of downloading SD
> card images with Raspbian distro and writing them to the SD card (there
> are tools to easily do this even in Windows). This works as a way to
> quickly get the system up and running on their devices. And this is
> the skill that they already have and expect to be applicable to other
> devices.
> 
> On the other hand, http://linux-sunxi.org/Category:Devices lists 124
> pages dedicated to different Allwinner based devices at the moment.
> Most of the Allwinner based devices have a HDMI connector and are able
> to run a "desktop" GNU/Linux system with a big desktop monitor just
> like the Raspberry Pi. Many Allwinner devices have USB host ports.
> And the ones which don't, are at least equipped with USB OTG. While
> definitely not prefect, USB OTG is still suitable for a "desktop"
> system when used with something like
>     http://linux-sunxi.org/USB_OTG_Charging_Hub
> 
> And here we have a problem. The linux-sunxi wiki contains a lot of
> useful information about the hardware and reasonably detailed
> instructions about compiling the u-boot and the kernel for each of
> the devices. But that's not what the non-geek users want. The non-geek
> users expect to find SD card images for their devices, just like
> this is handled for the Raspberry Pi.
> 
> What happens if non-geek users do not find the right SD card images
> for their devices? Do they follow the instructions to compile the
> u-boot and the kernel themselves? Of course not. The users just pick
> some random SD card image, which "seems" to be likely compatible.
> That's why they are trying Cubieboard SD card images on random
> tablets or other devboards. This rarely ends up well. Want an
> example? Here it is:
>     https://www.mail-archive.com/linux-sunxi@googlegroups.com/msg08343.html
> 
> Also have a look at
>     http://olimex.wordpress.com/2014/10/27/a20-olinuxino-lime2-review-and-updates/
> for a nice warning message "EDIT: Something important! Debian and
> Android images for LIME and LIME2 are different because of the memory
> and Gigabit ethernet, so if you have LIME download the images for
> LIME if you have LIME2 download the images for LIME2!!!". And check
> the discussion in comments about Cubian, Bananian and other board
> specific distributions.
> 
> Surprisingly, compiling u-boot just happens to be rather difficult
> for non-geeks. Geeks are of course perfectly fine with the nice and
> detailed instructions, which are readily available:
>     http://lists.denx.de/pipermail/u-boot/2014-December/199351.html
> 
> =====================================================================
> == 2. A possible solution
> =====================================================================
> 
> While working on DRAM controller bugfixes for Allwinner A10/A13/A20,
> it was discovered that, among other things, DRAM bus width and density
> autodetection can be implemented:
>     http://lists.denx.de/pipermail/u-boot/2014-July/183981.html
> 
> This alone provides an obvious opportunity to use the same u-boot
> binary on multiple devices even if they use different DRAM
> size/configuration. And the natural evolution of this is the support
> of Allwinner A10/A13/A20 SoC in the same u-boot binary by just doing
> runtime SoC type detection:
>     http://lists.denx.de/pipermail/u-boot/2014-August/185218.html
> 
> This was a cool demo, but having only the u-boot prompt on the serial
> console and the ability to boot from the SD card is not something
> that can immediately help non-geek users.
> 
> Now there is an important thing to consider. The same u-boot
> binary can boot on different Allwinner devices and initialize
> DRAM successfully. But what about the other peripherals? Some of
> the peripherals strictly need configuration parameters, which can't
> be automatically detected at runtime in any way (LCD displays for
> example). So what is the safe subset of hardware, which is
> universally usable in generic u-boot binaries? Basically, we can
> rely on the assumption that everything that is used by BROM, can be
> safely autodetected and used by the universal u-boot binaries too.
> 
> For example, the UART serial console is not really used by BROM, and
> this was a kind of hack in the previous demo. At that time it looked
> like the serial console could be configured correctly according to
> some heuristic rules. However later it turned out that the heuristics
> does not really work on some A13 devices and they may have mutually
> incompatible UART configurations:
>     https://www.mail-archive.com/linux-sunxi@googlegroups.com/msg07057.html
> This particular problem just confirms the general rule about
> relying only on what is also used by BROM. Oh, and some functionality
> is also provided by dedicated SoC pins, which are strictly used only
> for a single purpose and can't be re-configured for doing anything
> else. This can be safely used too.
> 
> Anyway, the really missing part was the user friendly output and user
> friendly input for generic u-boot binaries. HDMI is widely used in
> Allwinner hardware and it supports autoconfiguration. USB host ports
> use dedicated pins and only enabling/disabling power can be device
> specific. The missing USB power can be solved by using a powered USB
> hub, which is not very convenient, but still a workable solution.
> 
> After the initial discussion and planning on the IRC, Luc Verhaegen
> stepped in to implement the video driver for u-boot, together
> with simplefb support for the linux kernel:
>     http://lists.denx.de/pipermail/u-boot/2014-August/185193.html
> 
> And Roman Byshko stepped in to implement the USB EHCI support:
>     http://lists.denx.de/pipermail/u-boot/2014-July/183349.html
> 
> Many thanks to them for spearheading the development of these
> important features! And of course, Hans de Goede did a great
> job bugfixing and further improving this code, probably investing
> even more efforts than the initial contributors. Not to mention
> the participation in lengthy simplefb political battles, which
> looked really scary and discouraging, but ended up well.
> 
> With all these features in place, now we can do something like this:
>     https://github.com/ssvb/sunxi-bootsetup/releases/tag/20141215-sunxi-bootsetup-prototype
> 
> It is a demo of a universal SD card image, which should be bootable
> on any Allwinner A10/A10s/A20 device. With an installer of u-boot
> v2015.01-rc3 as the initial demo simple payload. By using a
> HDMI monitor for output and a USB keyboard or FEL button for input,
> it offers a menu based user interface. The menu allows to select
> the exact type of the user's Allwinner device and install the
> right bootloader for it.
> 
> In principle, even now it is usable as a base for the SD card distro
> image. Maybe with http://www.berryterminal.com/doku.php/berryboot
> chained as the next installation stage.
> 
> There is of course a huge room for improvements, which are yet to be
> done. Some of the potential improvements are listed below.
> 
> =====================================================================
> == 3. Relocatable SD card
> =====================================================================
> 
> Maybe not every user really needs a full-fledged bootloader, so a basic
> device independent bootloader with just SD card support and HDMI can
> be probably used not only for the initial setup of the hardware type,
> but kept indefinitely for booting the linux kernel too. Naturally, in
> order to get good performance, the DRAM parameters need to be patched
> into the SPL, replacing the generic failsafe ones.
> 
> The DRAM parameters can (and should) be tied to the unique chip ID:
>     http://linux-sunxi.org/SID_Register_Guide
> So that if anyone tries to move the SD card to a different Allwinner
> device, this situation can be detected and the menu with the device
> type selection can be shown again on the HDMI monitor instead of
> failing in a non-obvious way. Storing SID values and DRAM settings
> for more than one device is also an option, so repeatedly swapping
> between two devices will trigger the device type selection menu
> activation only once for each device.
> 
> =====================================================================
> == 4. NAND
> =====================================================================
> 
> NAND is the hardware, which is supported by BROM. Which means that it
> should be usable in a generic way by the universal device independent
> u-boot binary. NAND is a perfect place to store the device specific
> information. So that the user can avoid the annoying device type
> selection choices.
> 
> =====================================================================
> == 5. Support more devices!
> =====================================================================
> 
> The number of supported Allwinner devices in u-boot v2015.01 is
> really small. A few more devices are being added for v2015.04
> While the progress is steady, I'm not convinced that the support
> for all the 100+ Allwinner devices can be added in a reasonable
> time frame.
> 
> The owners of some these devices are non-geeks and will not be able
> to submit patches to u-boot and the linux kernel on their own, even
> if provided with detailed instructions. This process just does not
> scale. Moreover, it is not very nice to force the users to master
> a useless skill, such as FEX knowledge.
> 
> So I see the automatic conversion as the only reasonable solution.
> 
> Yes, something like this already supposedly exits:
>     https://github.com/mripard/sunxi-babelfish
> 
> I don't know how much progress has the 'sunxi-babelfish' project
> made so far (and to be honest, even did not try it). But in my
> opinion it has some fatal deficiencies in the design, based on
> just reading its README:
>   1. Implemented in the C language, while scripting languages are
>      orders of magnitude more suitable for such task and allow much
>      faster development.
>   2. This approach does not look very testable/debuggable.
>   3. It apparently does not help mainlining. The output of the
>      conversion does not look like it is intended to be used as
>      a template for the DTS file submission to the mainline kernel.
> 
> So the right solution in my opinion is a set of scripts for the
> sunxi-boards repository. The scripts need to parse the FEX files,
> which are conveniently already collected in sunxi-boards. They
> need to support different FEX dialects as input (this is really
> important!) and 3 types of output:
>   1. The defconfigs for u-boot
>   2. The DTS files for the linux kernel
>   3. The FEX file in a dialect, which is compatible with the sunxi-3.4
>      kernel
> 
> The type of the FEX dialect can be derived from the soc-specific
> directory names in the sunxi-boards repository. 
> 
> Just to get an idea of how this could possibly work, one may look at
>     http://linux-sunxi.org/index.php?title=LCD&oldid=12858
> These are the results of automatic conversion of FEX into the LCD
> parameters for u-boot. The manual work needs to be reduced as much
> as possible.
> 
> =====================================================================
> == 6. Reliability tests
> =====================================================================
> 
> The problem is explained in
>     http://linux-sunxi.org/Hardware_Reliability_Tests
> 
> These tests just need to be added as part of the installation process.
> Maybe also do speed grading of the CPU and DRAM to store this
> information for use by cpufreq and DRAM init.
> 
> The undervolting/overclocking functionality can be also provided.
> Because some users are trying overclocking regardless of any warnings,
> it's better to provide at least some safety parachute to ensure that
> they don't hurt themselves.
> 
> =====================================================================
> == 7. FEL mode installer application
> =====================================================================
> 
> Instead of booting from the SD card (which does not require the
> desktop PC), the installation can be done in FEL mode by running a
> special application on the desktop PC. Other than this, a lot of
> functionality can be shared between these two installation methods.
> 
> This feature is critical for the devices without HDMI, such A13 and A23.
> 
> =====================================================================
> == 8. A31+ support
> =====================================================================
> 
> The current demo only supports A10, A10s and A20. Naturally, A31
> support is also very much desired.
> 
> =====================================================================
> == 9. Summary
> =====================================================================
> 
> Here is the demo, please try it and provide feedback:
>     https://github.com/ssvb/sunxi-bootsetup/releases/tag/20141215-sunxi-bootsetup-prototype
> 
> This demo may be already fully usable to provide device independent SD
> card images with Linux distributions for Allwinner devices (maybe with
> some minor tweaks). If you are maintaining one of these Cubian, Bananian
> or the other pre-made SD card images, it is possible to extend them to
> support more devices. Feel free to ask if you need some help.
> 
> This is not a finished work, more improvements are coming. And this is
> posted to the mainline u-boot mailing list, because I want to see some
> essential features implemented in the mainline u-boot. Forking is
> definitely not the goal.
> 
> The key assumption about the device independent u-boot binaries:
> 
>     Anything that is supported by BROM, can be autoconfigured and
>     used by the generic device independent u-boot binaries too.
> 
> The key assumption about FEX files conversion:
> 
>     The FEX files originate from Android devices, do work on Android
>     devices and allow to use all the hardware properly, so all the
>     necessary information is available for conversion. But we need to
>     be aware of mutually incompatible FEX dialects used by different
>     SoC generations and handle them gracefully.
> 
> 
> Thanks for reading and reaching the end of this e-mail.
> 
> Special thanks to Luc Verhaegen, Roman Byshko and Hans de Goede for
> their work on u-boot.
> 
> -- 
> Best regards,
> Siarhei Siamashka

While the technical advancement is great, and really does deserve a 
massive round of applause, i do not buy into the logic for making things 
easier for absolutely everyone.

Yes, we have 123 devices on our wiki, and we have tons of easy, 
well-structured guides to follow on how to do get things going. We are 
_very_ unique in that respect, and people only need a limited degree of 
geekiness to successfully make it through those. Given, it is more than 
what the average rpi user manages, but the rpi is a single device with 
device specific images. Let rpi users use the rpi, let the more 
dedicated and intellegent ones move over to more capable hw.

Yes, we have 123 devices on our wiki, but 75 of those are listed as 
NDH_TODO. Then there is a ton of devices for which some community 
members have added fex files and uboot support, but which are not 
documented in any way, several of them cannot even be identified by 
anyone but the original committer.

Telling people that a universal image is now available is only going to:
a) fool people (well, not really fool, but you too know how the minds of 
  a large portion of our users work) into thinking that they get full 
  device support. This makes it much more likely that they throw in the 
  towel after having booted, or that they come into our irc channel and 
  baselessly complain.
b) remove peoples willingness to work through our new device howto:
 1) we all miss out on valuable data and information (including you)
 2) multiply the overhead of getting from the basic device support to 
    more fully utilizing the device by the number of users, as every 
    user would do that from scratch.
c) increase the incidents of people trashing their NAND and losing their 
 fex files, with usually no original vendor images or fex files as a 
 backup available.

I have long held that we should lower the barrier as much as possible, 
but that peoples willingness to follow clear and concise guides and 
compile u-boot and a kernel (which is not rocket science) is a necessary 
hurdle. Those portions of our userbase that lack a measurable attention 
span and/or the willingness to read through a few pages on our wiki are 
simply damaging, and should be excluded. Similarly, people who do not 
work through the new device howto and document their devices, they have 
no right to expect any help from our community. Some level of geekiness 
is required. But which non-geek wants to put a linux on their android 
device anyway? Those few that want that do so because they either are, 
or because they want to be. Give them that small hurdle to climb over.

Realizing the danger of excluding non development boards with the push 
to mainline, and thus devicetree, i pushed Maxime to create babelfish. I 
see something similar happening here, although less directly so. By 
reducing things to a single image install, the device support is also
actively reduced. By offering ready made support for the hw feature set 
of your average cheap development board, we are once again making sure 
that tablets have less of a chance of getting support.

I also see a serious increase of android users who just trashed their   
NAND who come to use thinking we are clearvoyant timetravelling 
magicians who with a single incantation will make their android whole 
again, but stable and faster this time round.

So please, stop calling this a universal route for non-geeks. Call it a 
universal rescue system, which in itself is a magnificent achievement. A 
universal rescue system tells people that this is just the start, or a 
safety net, and it could increase peoples willingness to risk working 
through our NDH. An easy universal installation for non-geeks is only 
going to lead to a lot of disappointment on many sides.

Luc Verhaegen.