[ELDK] U-Boot configuration kernel + rootfs squashfs image in NAND Flash, copy rootfs to ram, mount rootfs from ram

Christoph Petzold cpetzoldatwork at gmx.de
Wed Jan 16 09:05:27 CET 2013

On 15.01.2013 20:31, Wolfgang Denk wrote:
> Dear Christoph,
> In message <50F575DB.2000403 at gmx.de> you wrote:
>>> Second, you write some 200+ lines of description what you did and what
>>> you think, and then ask for "help or suggestions", but I cannot see
>>> any clear statement of what your specific questions are,
>> I stated my problem in the Subject and at the beginning of my mail.
> Sorry, but I disagree.  You desribe a task, i.e. what you want to do,
> but you don't really state which exact parts are not working as
> expected or which exact questions you want to haave answered.

I agree to disagree. Obviously i do have a slightly different opinion of 
how "task description", "specific questions" and "clear satements" are 
defined. I thank you for sharing your point of view and i hope to be 
able to learn from it.

>> - kernel from NAND and root file system with squashfs from ram"
> But why the "from RAM" ?

The Argument is Speed. I have just stopped times with a simple stopwatch 
to show the Speed of sdram compared to the speed of nand-read.

MX28 U-Boot >  nand read ${rootfsaddr} rootfs ${rootfssize}

<<<<<<<<<<<<< start stopwatch >>>>>>>>>

NAND read: device 0 offset 0x540000, size 0x2031040
Skipping bad block 0x00e20000
Skipping bad block 0x01ec0000
  33755200 bytes read: OK

<<<<<<<<<<<<< stop stopwatch  >>>>>>>> ~9 seconds rounded down
<<<<<<<<<<<<< reset and restart stopwatch >>>>>>

MX28 U-Boot > nboot linux

Loading from nand0, offset 0x140000
    Image Name:   Linux-3.3.0-karo
    Image Type:   ARM Linux Kernel Image (uncompressed)
    Data Size:    2633456 Bytes = 2.5 MiB
    Load Address: 40008000
    Entry Point:  40008000

<<<<<<<<<<<<< stop stopwatch  >>>>>>>> ~1 second rounded up
<<<<<<<<<<<<< reset and restart stopwatch >>>>>>

MX28 U-Boot > bootm ${loadaddr} ${rootfsaddr}
## Booting kernel from Legacy Image at 40100000 ...
    Image Name:   Linux-3.3.0-karo
    Image Type:   ARM Linux Kernel Image (uncompressed)
    Data Size:    2633456 Bytes = 2.5 MiB
    Load Address: 40008000
    Entry Point:  40008000
    Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 40800000 ...
    Image Name:   ramdisk_sqashfs
    Image Type:   ARM Linux RAMDisk Image (uncompressed)
    Data Size:    33755136 Bytes = 32.2 MiB
    Load Address: 40800000
    Entry Point:  40800000
    Verifying Checksum ... OK
    Loading Kernel Image ... OK

Starting kernel ...

<<<<<<<<<<<<< stop stopwatch  >>>>>>>> less then 1 second

As you pointed out here:

>> ## Loading init Ramdisk from Legacy Image at 40800000 ...
>>      Image Name:   ramdisk_sqashfs
>>      Image Type:   ARM Linux RAMDisk Image (uncompressed)
>>      Data Size:    33755136 Bytes = 32.2 MiB
>>      Load Address: 40800000
>>      Entry Point:  40800000
>>      Verifying Checksum ... OK
>>      Loading Kernel Image ... OK
> So you load the U-Boot image with your rootfs at 40800000; it starts
> with the 64 byte image header, followed (at 40800040) by the raw
> rootfs data. Then you perform a memcpy() of the whole image from
> 40800040 to 40800000.  This appears a serious waste of time to me.

i configured u-boot to copy the whole rootfs image from ram to ram, 
which is appears to be a serious waste of time. Thank you for pointing 
that out. I did not intent this behavior, off course. The manpage of
memcpy states that src and dest must not overlap. I do not know how 
memcpy copys the bytes in memory. If it copys the first byte of src to 
the first byte of dest and then the second byte of src to the second 
byte of dest and so on, and if u-boot really doeas memcpy from 40800040 
to 40800000 it should work, but generally i would expect garbage in 
memory when src and dest addresses do overlap.

This example shows how much time is spent copying ~32MB in ram. It is 
less then a second. I consider this time significantly smaller then 9 
seconds, which are needed to copy 32MB from nand to ram. I would like to 
use this experimentation data to decide which file system boot 
configuration to chose.

>> Boot and start of our main application with standard nand boot
>> configuration (Kernel + rootfsimage(Jffs2)) take around one minute.
> Did you analyze where the time gets spent?

No i did not analyze where the time gets spent. I just wanted to 
experiment with different partition- file system- and boot 
configurations. I wanted to have reproducible and comparable data.

> OK, JFFS2 is pretty slow,
> especially when well used and when mounting larger partitions.
> Did you try squashfs (or UBIFS!!) from NAND instead?

I did try squashfs and the kernel booted and found the rootfs image but 
i was not able to see the init process finished because somewhere in it 
the system stops doing anything but this is another story...

I did not try UBIFS yet. As far as i believe to know the UBI file system 
actually needs an UBI-Volume. According to 
http://www.linux-mtd.infradead.org/doc/ubifs.html the developers of UBI 
and UBIFS wanted to solve two separate problems. It seems reasonable to 
do flash specific tasks like wear-leveling seperated from the filesystem 
some how. But the whole process of creating an UBI volume, creating an 
ubifs image, putting the ubifs image into a volume, telling the kernel 
to use the ubifs rotfs image seems to be quite complex. I can do a lot 
of mistakes along this way. This is why i have not tried UBIFS yet.

>>> [1] http://catb.org/esr/faqs/smart-questions.html
>> Exactly which section should i read?
> All of them ?

I have skimmed all of them and have found out nothing new. The points 
mentioned in this document appear to be obvious to me. I would not dare 
to write to a mailing list before reading tons of manuals, archieves of 
mailing lists and How-Tos.

>> I would like to find out if my
>> "unusual" configuration differs from the standard configuration with
>> respect to the boot time and application start time.
> What are your expectations?  When using it from NAND, only the parts
> that are really referenced will be loaded (and uncompressed) to RAM,
> and you have the full RAM available for buffer cache. Things you need
> only rarely or never will eventually never be loaded at all. When
> using it as some form of ramdisk, you always have to load the whole
> image to RAM, before you can copy (and uncompress) to another RAM
> area the parts you need.
> My expectation would be that this approach has to be slower, first due
> to the full loading into RAM, and second to the loss of RAM space
> which means everything will permanently run slower.

See the experimentation data with the stop watch above.

> Why don't you check how the kenrel is actually configured?

I tried to find a kiernel entry point in the kernel configuration menu 
and i did cat .config | grep -i entry
and i did not get any result. In arch/arm/boot i did not find where it 
and i  also used google to find out. From now on i will stop mentioning, 
that i have used google to find out about a problem or a question 
because it seems obvious to me to do this.

> Sorry, we can't te;;. We don;t see the boot args that were actually
> received by the Linux kernel, nor can we check which attempts the
> kernel might have made to access the rootfs - from the snippets of the
> boot log we cannot even guess what might have gone wrong.
>> Which kernel parameters do i have to specify to have the kernel find and
>> mount the squashfs root file system image in ram?
> Hm... there are a number of documents out there describing that.  Did
> you try reading some? See for example http://bit.ly/13A127u

This makes me a little sad :( Please do not tell me to use google. I 
would not ask people to fiddle with my problems, when i had not tried 
hard to solve them with the help of google, archieves and manuals...before.

>> I believe i did some thing wrong but i do not know what.
> I believe you are right, but this is not enough information ...

So i should have given more data? If it is not enough information, which 
information do you need, the Kernel command line or a whole log file of 
the serial terminal session?

Starting kernel ...

Booting Linux on physical CPU 0
Linux version 3.3.0-karo (entwickler at entwickler-vm) (gcc version 4.5.2 
20101204 (prerelease) (GCC) ) #13 PREEMPT Tue Jan 15 15:02:57 CET 2013
CPU: ARM926EJ-S [41069265] revision 5 (ARMv5TEJ), cr=00053177
CPU: VIVT data cache, VIVT instruction cache
Machine: Ka-Ro electronics TX28 module
Memory policy: ECC disabled, Data cache writeback
On node 0 totalpages: 32256
free_area_init_node: node 0, pgdat c0521d9c, node_mem_map c0560000
   Normal zone: 256 pages used for memmap
   Normal zone: 0 pages reserved
   Normal zone: 32000 pages, LIFO batch:7
pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
pcpu-alloc: [0] 0
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 32000
Kernel command line: consoleblank=0 console=ttyAMA0,115200 
tx28_base=stkv3 tx28_otg_mode=device rw debug panic=1 mxsfb.mode=VGA 
root=/dev/ram rw ramdisk_size=0x2031040

> Best regards,
> Wolfgang Denk

More information about the eldk mailing list