[U-Boot] Announcing easy Linux installation on Allwinner devices for non-geek users

[Hans de Goede]

Hi Siarhei,

See the bottom for my reply to all this.

On 22-01-15 13:49, Siarhei Siamashka wrote:
> Hello,
>
> This is a somewhat long e-mail.
>
> 1. The problem
> 2. A possible solution
> 3. Relocatable SD card
> 4. NAND
> 5. Support more devices!
> 6. Reliability tests
> 7. FEL mode installer application
> 8. A31+ support
> 9. Summary
>
> =====================================================================
> == 1. The problem
> =====================================================================
>
> First of all, let's see what is the difference between various
> Allwinner boards/devices and the Raspberry Pi (which is considered
> by many as a role model of being user friendly and easy to use).
>
> Raspberry Pi offers only a few hardware variants with minor differences.
> And Raspberry Pi users are familiar with the concept of downloading SD
> card images with Raspbian distro and writing them to the SD card (there
> are tools to easily do this even in Windows). This works as a way to
> quickly get the system up and running on their devices. And this is
> the skill that they already have and expect to be applicable to other
> devices.
>
> On the other hand, http://linux-sunxi.org/Category:Devices lists 124
> pages dedicated to different Allwinner based devices at the moment.
> Most of the Allwinner based devices have a HDMI connector and are able
> to run a "desktop" GNU/Linux system with a big desktop monitor just
> like the Raspberry Pi. Many Allwinner devices have USB host ports.
> And the ones which don't, are at least equipped with USB OTG. While
> definitely not prefect, USB OTG is still suitable for a "desktop"
> system when used with something like
>      http://linux-sunxi.org/USB_OTG_Charging_Hub
>
> And here we have a problem. The linux-sunxi wiki contains a lot of
> useful information about the hardware and reasonably detailed
> instructions about compiling the u-boot and the kernel for each of
> the devices. But that's not what the non-geek users want. The non-geek
> users expect to find SD card images for their devices, just like
> this is handled for the Raspberry Pi.
>
> What happens if non-geek users do not find the right SD card images
> for their devices? Do they follow the instructions to compile the
> u-boot and the kernel themselves? Of course not. The users just pick
> some random SD card image, which "seems" to be likely compatible.
> That's why they are trying Cubieboard SD card images on random
> tablets or other devboards. This rarely ends up well. Want an
> example? Here it is:
>      https://www.mail-archive.com/linux-sunxi@googlegroups.com/msg08343.html
>
> Also have a look at
>      http://olimex.wordpress.com/2014/10/27/a20-olinuxino-lime2-review-and-updates/
> for a nice warning message "EDIT: Something important! Debian and
> Android images for LIME and LIME2 are different because of the memory
> and Gigabit ethernet, so if you have LIME download the images for
> LIME if you have LIME2 download the images for LIME2!!!". And check
> the discussion in comments about Cubian, Bananian and other board
> specific distributions.
>
> Surprisingly, compiling u-boot just happens to be rather difficult
> for non-geeks. Geeks are of course perfectly fine with the nice and
> detailed instructions, which are readily available:
>      http://lists.denx.de/pipermail/u-boot/2014-December/199351.html
>
> =====================================================================
> == 2. A possible solution
> =====================================================================
>
> While working on DRAM controller bugfixes for Allwinner A10/A13/A20,
> it was discovered that, among other things, DRAM bus width and density
> autodetection can be implemented:
>      http://lists.denx.de/pipermail/u-boot/2014-July/183981.html
>
> This alone provides an obvious opportunity to use the same u-boot
> binary on multiple devices even if they use different DRAM
> size/configuration. And the natural evolution of this is the support
> of Allwinner A10/A13/A20 SoC in the same u-boot binary by just doing
> runtime SoC type detection:
>      http://lists.denx.de/pipermail/u-boot/2014-August/185218.html
>
> This was a cool demo, but having only the u-boot prompt on the serial
> console and the ability to boot from the SD card is not something
> that can immediately help non-geek users.
>
> Now there is an important thing to consider. The same u-boot
> binary can boot on different Allwinner devices and initialize
> DRAM successfully. But what about the other peripherals? Some of
> the peripherals strictly need configuration parameters, which can't
> be automatically detected at runtime in any way (LCD displays for
> example). So what is the safe subset of hardware, which is
> universally usable in generic u-boot binaries? Basically, we can
> rely on the assumption that everything that is used by BROM, can be
> safely autodetected and used by the universal u-boot binaries too.
>
> For example, the UART serial console is not really used by BROM, and
> this was a kind of hack in the previous demo. At that time it looked
> like the serial console could be configured correctly according to
> some heuristic rules. However later it turned out that the heuristics
> does not really work on some A13 devices and they may have mutually
> incompatible UART configurations:
>      https://www.mail-archive.com/linux-sunxi@googlegroups.com/msg07057.html
> This particular problem just confirms the general rule about
> relying only on what is also used by BROM. Oh, and some functionality
> is also provided by dedicated SoC pins, which are strictly used only
> for a single purpose and can't be re-configured for doing anything
> else. This can be safely used too.
>
> Anyway, the really missing part was the user friendly output and user
> friendly input for generic u-boot binaries. HDMI is widely used in
> Allwinner hardware and it supports autoconfiguration. USB host ports
> use dedicated pins and only enabling/disabling power can be device
> specific. The missing USB power can be solved by using a powered USB
> hub, which is not very convenient, but still a workable solution.
>
> After the initial discussion and planning on the IRC, Luc Verhaegen
> stepped in to implement the video driver for u-boot, together
> with simplefb support for the linux kernel:
>      http://lists.denx.de/pipermail/u-boot/2014-August/185193.html
>
> And Roman Byshko stepped in to implement the USB EHCI support:
>      http://lists.denx.de/pipermail/u-boot/2014-July/183349.html
>
> Many thanks to them for spearheading the development of these
> important features! And of course, Hans de Goede did a great
> job bugfixing and further improving this code, probably investing
> even more efforts than the initial contributors. Not to mention
> the participation in lengthy simplefb political battles, which
> looked really scary and discouraging, but ended up well.
>
> With all these features in place, now we can do something like this:
>      https://github.com/ssvb/sunxi-bootsetup/releases/tag/20141215-sunxi-bootsetup-prototype
>
> It is a demo of a universal SD card image, which should be bootable
> on any Allwinner A10/A10s/A20 device. With an installer of u-boot
> v2015.01-rc3 as the initial demo simple payload. By using a
> HDMI monitor for output and a USB keyboard or FEL button for input,
> it offers a menu based user interface. The menu allows to select
> the exact type of the user's Allwinner device and install the
> right bootloader for it.
>
> In principle, even now it is usable as a base for the SD card distro
> image. Maybe with http://www.berryterminal.com/doku.php/berryboot
> chained as the next installation stage.
>
> There is of course a huge room for improvements, which are yet to be
> done. Some of the potential improvements are listed below.
>
> =====================================================================
> == 3. Relocatable SD card
> =====================================================================
>
> Maybe not every user really needs a full-fledged bootloader, so a basic
> device independent bootloader with just SD card support and HDMI can
> be probably used not only for the initial setup of the hardware type,
> but kept indefinitely for booting the linux kernel too. Naturally, in
> order to get good performance, the DRAM parameters need to be patched
> into the SPL, replacing the generic failsafe ones.
>
> The DRAM parameters can (and should) be tied to the unique chip ID:
>      http://linux-sunxi.org/SID_Register_Guide
> So that if anyone tries to move the SD card to a different Allwinner
> device, this situation can be detected and the menu with the device
> type selection can be shown again on the HDMI monitor instead of
> failing in a non-obvious way. Storing SID values and DRAM settings
> for more than one device is also an option, so repeatedly swapping
> between two devices will trigger the device type selection menu
> activation only once for each device.
>
> =====================================================================
> == 4. NAND
> =====================================================================
>
> NAND is the hardware, which is supported by BROM. Which means that it
> should be usable in a generic way by the universal device independent
> u-boot binary. NAND is a perfect place to store the device specific
> information. So that the user can avoid the annoying device type
> selection choices.
>
> =====================================================================
> == 5. Support more devices!
> =====================================================================
>
> The number of supported Allwinner devices in u-boot v2015.01 is
> really small. A few more devices are being added for v2015.04
> While the progress is steady, I'm not convinced that the support
> for all the 100+ Allwinner devices can be added in a reasonable
> time frame.
>
> The owners of some these devices are non-geeks and will not be able
> to submit patches to u-boot and the linux kernel on their own, even
> if provided with detailed instructions. This process just does not
> scale. Moreover, it is not very nice to force the users to master
> a useless skill, such as FEX knowledge.
>
> So I see the automatic conversion as the only reasonable solution.
>
> Yes, something like this already supposedly exits:
>      https://github.com/mripard/sunxi-babelfish
>
> I don't know how much progress has the 'sunxi-babelfish' project
> made so far (and to be honest, even did not try it). But in my
> opinion it has some fatal deficiencies in the design, based on
> just reading its README:
>    1. Implemented in the C language, while scripting languages are
>       orders of magnitude more suitable for such task and allow much
>       faster development.
>    2. This approach does not look very testable/debuggable.
>    3. It apparently does not help mainlining. The output of the
>       conversion does not look like it is intended to be used as
>       a template for the DTS file submission to the mainline kernel.
>
> So the right solution in my opinion is a set of scripts for the
> sunxi-boards repository. The scripts need to parse the FEX files,
> which are conveniently already collected in sunxi-boards. They
> need to support different FEX dialects as input (this is really
> important!) and 3 types of output:
>    1. The defconfigs for u-boot
>    2. The DTS files for the linux kernel
>    3. The FEX file in a dialect, which is compatible with the sunxi-3.4
>       kernel
>
> The type of the FEX dialect can be derived from the soc-specific
> directory names in the sunxi-boards repository.
>
> Just to get an idea of how this could possibly work, one may look at
>      http://linux-sunxi.org/index.php?title=LCD&oldid=12858
> These are the results of automatic conversion of FEX into the LCD
> parameters for u-boot. The manual work needs to be reduced as much
> as possible.
>
> =====================================================================
> == 6. Reliability tests
> =====================================================================
>
> The problem is explained in
>      http://linux-sunxi.org/Hardware_Reliability_Tests
>
> These tests just need to be added as part of the installation process.
> Maybe also do speed grading of the CPU and DRAM to store this
> information for use by cpufreq and DRAM init.
>
> The undervolting/overclocking functionality can be also provided.
> Because some users are trying overclocking regardless of any warnings,
> it's better to provide at least some safety parachute to ensure that
> they don't hurt themselves.
>
> =====================================================================
> == 7. FEL mode installer application
> =====================================================================
>
> Instead of booting from the SD card (which does not require the
> desktop PC), the installation can be done in FEL mode by running a
> special application on the desktop PC. Other than this, a lot of
> functionality can be shared between these two installation methods.
>
> This feature is critical for the devices without HDMI, such A13 and A23.
>
> =====================================================================
> == 8. A31+ support
> =====================================================================
>
> The current demo only supports A10, A10s and A20. Naturally, A31
> support is also very much desired.
>
> =====================================================================
> == 9. Summary
> =====================================================================
>
> Here is the demo, please try it and provide feedback:
>      https://github.com/ssvb/sunxi-bootsetup/releases/tag/20141215-sunxi-bootsetup-prototype
>
> This demo may be already fully usable to provide device independent SD
> card images with Linux distributions for Allwinner devices (maybe with
> some minor tweaks). If you are maintaining one of these Cubian, Bananian
> or the other pre-made SD card images, it is possible to extend them to
> support more devices. Feel free to ask if you need some help.
>
> This is not a finished work, more improvements are coming. And this is
> posted to the mainline u-boot mailing list, because I want to see some
> essential features implemented in the mainline u-boot. Forking is
> definitely not the goal.
>
> The key assumption about the device independent u-boot binaries:
>
>      Anything that is supported by BROM, can be autoconfigured and
>      used by the generic device independent u-boot binaries too.
>
> The key assumption about FEX files conversion:
>
>      The FEX files originate from Android devices, do work on Android
>      devices and allow to use all the hardware properly, so all the
>      necessary information is available for conversion. But we need to
>      be aware of mutually incompatible FEX dialects used by different
>      SoC generations and handle them gracefully.
>
>
> Thanks for reading and reaching the end of this e-mail.
>
> Special thanks to Luc Verhaegen, Roman Byshko and Hans de Goede for
> their work on u-boot.

You're welcome, and thank you too for all your work on sunxi support,
especially the sun4i/sun5i/sun7i dram work.

So about the above mail, I've a number of things to say:

1) I think presenting the user with a list of devices to choose is a good idea,
but I also think your solution is over complicated.

The user will likely use a PC to download and write the image to the sdcard,
why not simply provide a simple app on the PC side to select the board and
write the correct u-boot binary ? I had a little bash script doing just that
for the Fedora images, granted this requires the user to have Linux, so
maybe we need someone to get to write a win32 util for this, so that we
can offer both options ?

To me this seems much more simple, and it will e.g. also work for a23 / a33
based tablets, which do not have hdmi and are really attractive for little
hobby projects due to their low price (35-40 usd for a complete tablet).

2) I would love to see a good fex file parser both the generate u-boot defconfig
files and kernel dts files

3) As for the whole store info in nand based on sid idea, with the recent
readonly nand patches posted to the list, which AFAIK do not need any nand
parameters, we could do one better and read the dram timings from the nand
for the SPL, and the in real u-boot read and parse script.bin from the nanda
partition. This is a bit of a wild idea I admit, but it could work, 2 problems
with it are:

a) It assume a standard Allwinner android nand contents, so not good for devices
where people want to actually write a normal linux distro to the nand / bricked
devices

b) Does not work on devices without nand (e.g. some olinuxino-lime models)

c) Does not really help for the kernel, we could generate a dtb on the fly
on u-boot based on the fex file contents, but that is going to be very tricky,
esp with the dtb files evolving as we start supporting more and more hw features
in the upstream kernel.

Regards,

Hans


p.s.

Any chance you could park this very cool project for a bit, and generate
some patches to start using standard clock speed based jedec timings for
dram on sun4i / sun5i / sun7i ?  I would like to get those patches into
my sunxi-wip branch soon-ish so that we can give them a good shake down
before queueing them up for merging into u-boot v2015.07 .