kernel doesn't start on Odroid U2 unless setting initrd_high
Tom Rini
trini at konsulko.com
Thu Jan 12 01:10:15 CET 2023
On Wed, Jan 11, 2023 at 10:08:59PM +0100, Joost van Zwieten wrote:
>
>
> On Wed, Jan 11, 2023 at 15:38, Tom Rini <trini at konsulko.com> wrote:
> > On Wed, Jan 11, 2023 at 09:28:42PM +0100, Joost van Zwieten wrote:
> > >
> > >
> > > On Tue, Jan 10, 2023 at 18:29, Tom Rini <trini at konsulko.com> wrote:
> > > > On Wed, Jan 11, 2023 at 12:10:42AM +0100, Joost van Zwieten wrote:
> > > > >
> > > > >
> > > > > On Tue, Jan 10, 2023 at 18:08, Tom Rini <trini at konsulko.com>
> > > wrote:
> > > > > > On Wed, Jan 11, 2023 at 12:01:46AM +0100, Joost van Zwieten
> > > wrote:
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Jan 10, 2023 at 13:41, Tom Rini
> > > <trini at konsulko.com>
> > > > > wrote:
> > > > > > > > On Tue, Jan 10, 2023 at 09:13:32AM +0100, Joost van
> > > Zwieten
> > > > > wrote:
> > > > > > > >
> > > > > > > > > Dear maintainers,
> > > > > > > > >
> > > > > > > > > As of commit 4963f63fe61f15329d77472a762b1d8bf754d24b
> > > > > U-Boot
> > > > > > > fails
> > > > > > > > > to start
> > > > > > > > > a kernel (with `bootz`) on my Odroid U2 unless I
> > > force
> > > > > > > > > `initrd_high`, e.g.
> > > > > > > > > to `0x50000000`. With commit 4963f63f and
> > > `initrd_high`
> > > > > > > *unset* I
> > > > > > > > > get the
> > > > > > > > > following output:
> > > > > > > > >
> > > > > > > > > U-Boot 2020.10-rc2-00314-g4963f63fe6 (Jan 09 2023
> > > -
> > > > > 23:59:31
> > > > > > > > > +0100)
> > > > > > > > >
> > > > > > > > > CPU: Exynos4412 @ 1 GHz
> > > > > > > > > Model: Odroid based on Exynos4412
> > > > > > > > > Type: u3
> > > > > > > > > DRAM: 2 GiB
> > > > > > > > > LDO20 at VDDQ_EMMC_1.8V: set 1800000 uV; enabling
> > > > > > > > > LDO22 at VDDQ_EMMC_2.8V: set 2800000 uV; enabling
> > > > > > > > > LDO21 at TFLASH_2.8V: set 2800000 uV; enabling
> > > > > > > > > MMC: SAMSUNG SDHCI: 2, EXYNOS DWMMC: 0
> > > > > > > > > Loading Environment from MMC... *** Warning - bad
> > > CRC,
> > > > > using
> > > > > > > > > default
> > > > > > > > > environment
> > > > > > > > >
> > > > > > > > > In: serial
> > > > > > > > > Out: serial
> > > > > > > > > Err: serial
> > > > > > > > > Boot device: MMC(2)
> > > > > > > > > Net: No ethernet found.
> > > > > > > > > Hit any key to stop autoboot: 0
> > > > > > > > > Odroid # env set fk_kvers 5.10.0-20-armmp
> > > > > > > > > Odroid # load mmc ${mmcbootdev}:${mmcbootpart}
> > > > > ${fdt_addr_r}
> > > > > > > > > /boot/dtbs/${fk_kvers}/exynos4412-odroidu3.dtb
> > > > > > > > > 53440 bytes read in 50 ms (1 MiB/s)
> > > > > > > > > Odroid # load mmc ${mmcbootdev}:${mmcbootpart}
> > > > > > > ${kernel_addr_r}
> > > > > > > > > /boot/vmlinuz-${fk_kvers}
> > > > > > > > > 4973056 bytes read in 182 ms (26.1 MiB/s)
> > > > > > > > > Odroid # load mmc ${mmcbootdev}:${mmcbootpart}
> > > > > > > ${ramdisk_addr_r}
> > > > > > > > > /boot/initrd.img-${fk_kvers}
> > > > > > > > > 22231585 bytes read in 777 ms (27.3 MiB/s)
> > > > > > > > > Odroid # env set bootargs console=ttySAC1,115200n8
> > > > > > > > > Odroid # bootz ${kernel_addr_r}
> > > > > > > ${ramdisk_addr_r}:${filesize}
> > > > > > > > > ${fdt_addr_r}
> > > > > > > > > Kernel image @ 0x41000000 [ 0x000000 - 0x4be200 ]
> > > > > > > > > ## Flattened Device Tree blob at 40800000
> > > > > > > > > Booting using the fdt blob at 0x40800000
> > > > > > > > > Loading Ramdisk to b9947000, end bae7aa21 ...
> > > OK
> > > > > > > > > Loading Device Tree to b9936000, end b99460bf
> > > ... OK
> > > > > > > > >
> > > > > > > > > Starting kernel ...
> > > > > > > > >
> > > > > > > > > And that's all I ever see. Normally the initrd loads
> > > a
> > > > > module
> > > > > > > that
> > > > > > > > > causes an
> > > > > > > > > LED on the Odroid to blink, and this is not happening
> > > > > either,
> > > > > > > so
> > > > > > > > > I'm pretty
> > > > > > > > > confident the kernel doesn't start or at least
> > > crashes
> > > > > before
> > > > > > > > > producing
> > > > > > > > > output. If I set `initrd_high` to `0x50000000` (or
> > > > > something
> > > > > > > in the
> > > > > > > > > neighborhood) the kernel starts just fine:
> > > > > > > > >
> > > > > > > > > U-Boot 2020.10-rc2-00314-g4963f63fe6 (Jan 09 2023
> > > -
> > > > > 23:59:31
> > > > > > > > > +0100)
> > > > > > > > >
> > > > > > > > > CPU: Exynos4412 @ 1 GHz
> > > > > > > > > Model: Odroid based on Exynos4412
> > > > > > > > > Type: u3
> > > > > > > > > DRAM: 2 GiB
> > > > > > > > > LDO20 at VDDQ_EMMC_1.8V: set 1800000 uV; enabling
> > > > > > > > > LDO22 at VDDQ_EMMC_2.8V: set 2800000 uV; enabling
> > > > > > > > > LDO21 at TFLASH_2.8V: set 2800000 uV; enabling
> > > > > > > > > MMC: SAMSUNG SDHCI: 2, EXYNOS DWMMC: 0
> > > > > > > > > Loading Environment from MMC... *** Warning - bad
> > > CRC,
> > > > > using
> > > > > > > > > default
> > > > > > > > > environment
> > > > > > > > >
> > > > > > > > > In: serial
> > > > > > > > > Out: serial
> > > > > > > > > Err: serial
> > > > > > > > > Boot device: MMC(2)
> > > > > > > > > Net: No ethernet found.
> > > > > > > > > Hit any key to stop autoboot: 0
> > > > > > > > > Odroid # env set fk_kvers 5.10.0-20-armmp
> > > > > > > > > Odroid # load mmc ${mmcbootdev}:${mmcbootpart}
> > > > > ${fdt_addr_r}
> > > > > > > > > /boot/dtbs/${fk_kvers}/exynos4412-odroidu3.dtb
> > > > > > > > > 53440 bytes read in 49 ms (1 MiB/s)
> > > > > > > > > Odroid # load mmc ${mmcbootdev}:${mmcbootpart}
> > > > > > > ${kernel_addr_r}
> > > > > > > > > /boot/vmlinuz-${fk_kvers}
> > > > > > > > > 4973056 bytes read in 181 ms (26.2 MiB/s)
> > > > > > > > > Odroid # load mmc ${mmcbootdev}:${mmcbootpart}
> > > > > > > ${ramdisk_addr_r}
> > > > > > > > > /boot/initrd.img-${fk_kvers}
> > > > > > > > > 22231585 bytes read in 777 ms (27.3 MiB/s)
> > > > > > > > > Odroid # env set bootargs console=ttySAC1,115200n8
> > > > > > > > > Odroid # env set initrd_high 0x50000000
> > > > > > > > > Odroid # bootz ${kernel_addr_r}
> > > > > > > ${ramdisk_addr_r}:${filesize}
> > > > > > > > > ${fdt_addr_r}
> > > > > > > > > Kernel image @ 0x41000000 [ 0x000000 - 0x4be200 ]
> > > > > > > > > ## Flattened Device Tree blob at 40800000
> > > > > > > > > Booting using the fdt blob at 0x40800000
> > > > > > > > > Loading Ramdisk to 4eacc000, end 4ffffa21 ...
> > > OK
> > > > > > > > > Loading Device Tree to bae6a000, end bae7a0bf
> > > ... OK
> > > > > > > > >
> > > > > > > > > Starting kernel ...
> > > > > > > > >
> > > > > > > > > [ 0.000000] Booting Linux on physical CPU 0xa00
> > > > > > > > > <truncated>
> > > > > > > > >
> > > > > > > > > The difference between those two runs is the location
> > > > > where
> > > > > > > U-Boot
> > > > > > > > > loads the
> > > > > > > > > initrd. The parent commit of 4963f63f boots fine
> > > without
> > > > > > > setting
> > > > > > > > > `initrd_high`:
> > > > > > > > >
> > > > > > > > > U-Boot 2020.10-rc2-00313-gdfaf6a5797 (Jan 10 2023
> > > -
> > > > > 00:13:19
> > > > > > > > > +0100)
> > > > > > > > >
> > > > > > > > > CPU: Exynos4412 @ 1 GHz
> > > > > > > > > Model: Odroid based on Exynos4412
> > > > > > > > > Type: u3
> > > > > > > > > DRAM: 2 GiB
> > > > > > > > > LDO20 at VDDQ_EMMC_1.8V: set 1800000 uV; enabling
> > > > > > > > > LDO22 at VDDQ_EMMC_2.8V: set 2800000 uV; enabling
> > > > > > > > > LDO21 at TFLASH_2.8V: set 2800000 uV; enabling
> > > > > > > > > MMC: SAMSUNG SDHCI: 2, EXYNOS DWMMC: 0
> > > > > > > > > Loading Environment from MMC... *** Warning - bad
> > > CRC,
> > > > > using
> > > > > > > > > default
> > > > > > > > > environment
> > > > > > > > >
> > > > > > > > > In: serial
> > > > > > > > > Out: serial
> > > > > > > > > Err: serial
> > > > > > > > > Boot device: MMC(2)
> > > > > > > > > Net: No ethernet found.
> > > > > > > > > Hit any key to stop autoboot: 0
> > > > > > > > > Odroid # env set fk_kvers 5.10.0-20-armmp
> > > > > > > > > Odroid # load mmc ${mmcbootdev}:${mmcbootpart}
> > > > > ${fdt_addr_r}
> > > > > > > > > /boot/dtbs/${fk_kvers}/exynos4412-odroidu3.dtb
> > > > > > > > > 53440 bytes read in 49 ms (1 MiB/s)
> > > > > > > > > Odroid # load mmc ${mmcbootdev}:${mmcbootpart}
> > > > > > > ${kernel_addr_r}
> > > > > > > > > /boot/vmlinuz-${fk_kvers}
> > > > > > > > > 4973056 bytes read in 181 ms (26.2 MiB/s)
> > > > > > > > > Odroid # load mmc ${mmcbootdev}:${mmcbootpart}
> > > > > > > ${ramdisk_addr_r}
> > > > > > > > > /boot/initrd.img-${fk_kvers}
> > > > > > > > > 22231585 bytes read in 776 ms (27.3 MiB/s)
> > > > > > > > > Odroid # env set bootargs console=ttySAC1,115200n8
> > > > > > > > > Odroid # bootz ${kernel_addr_r}
> > > > > > > ${ramdisk_addr_r}:${filesize}
> > > > > > > > > ${fdt_addr_r}
> > > > > > > > > Kernel image @ 0x41000000 [ 0x000000 - 0x4be200 ]
> > > > > > > > > ## Flattened Device Tree blob at 40800000
> > > > > > > > > Booting using the fdt blob at 0x40800000
> > > > > > > > > Loading Ramdisk to 4eacc000, end 4ffffa21 ...
> > > OK
> > > > > > > > > Loading Device Tree to 4eabb000, end 4eacb0bf
> > > ... OK
> > > > > > > > >
> > > > > > > > > Starting kernel ...
> > > > > > > > >
> > > > > > > > > [ 0.000000] Booting Linux on physical CPU 0xa00
> > > > > > > > > <truncated>
> > > > > > > > >
> > > > > > > > > I've tested two different kernels from Debian
> > > (buster and
> > > > > > > bullseye)
> > > > > > > > > and both
> > > > > > > > > have the same problem. I'm building U-Boot using the
> > > > > default
> > > > > > > config
> > > > > > > > > for
> > > > > > > > > board `odroid` and boot the Odroid from an SD card
> > > with
> > > > > > > firmware
> > > > > > > > > (`bl1`,
> > > > > > > > > `bl2` and `tzsw`) from Hardkernel's clone of the
> > > u-boot
> > > > > > > repository
> > > > > > > > > [1].
> > > > > > > > > There are no peripherals connected apart from the SD
> > > card
> > > > > and
> > > > > > > the
> > > > > > > > > serial
> > > > > > > > > console. Please let me know if you need more
> > > information.
> > > > > > > >
> > > > > > > > Interesting. So what does "bdi" show, both in the older
> > > good
> > > > > and
> > > > > > > > then current tree? Specifically the information about
> > > where
> > > > > DRAM
> > > > > > > is, and
> > > > > > > > how large it is? Thanks.
> > > > > > > >
> > > > > > > > --
> > > > > > > > Tom
> > > > > > >
> > > > > > > Both 4963f63f (not working) and its parent (working)
> > > produce
> > > > > the
> > > > > > > same
> > > > > > > output:
> > > > > > >
> > > > > > > boot_params = 0x40000100
> > > > > > > DRAM bank = 0x00000000
> > > > > > > -> start = 0x40000000
> > > > > > > -> size = 0x10000000
> > > > > > > DRAM bank = 0x00000001
> > > > > > > -> start = 0x50000000
> > > > > > > -> size = 0x10000000
> > > > > > > DRAM bank = 0x00000002
> > > > > > > -> start = 0x60000000
> > > > > > > -> size = 0x10000000
> > > > > > > DRAM bank = 0x00000003
> > > > > > > -> start = 0x70000000
> > > > > > > -> size = 0x10000000
> > > > > > > DRAM bank = 0x00000004
> > > > > > > -> start = 0x80000000
> > > > > > > -> size = 0x10000000
> > > > > > > DRAM bank = 0x00000005
> > > > > > > -> start = 0x90000000
> > > > > > > -> size = 0x10000000
> > > > > > > DRAM bank = 0x00000006
> > > > > > > -> start = 0xa0000000
> > > > > > > -> size = 0x10000000
> > > > > > > DRAM bank = 0x00000007
> > > > > > > -> start = 0xb0000000
> > > > > > > -> size = 0x0ff00000
> > > > > > > memstart = 0x40000000
> > > > > > > memsize = 0x7ff00000
> > > > > > > flashstart = 0x00000000
> > > > > > > flashsize = 0x00000000
> > > > > > > flashoffset = 0x00000000
> > > > > > > baudrate = 115200 bps
> > > > > > > relocaddr = 0xbfe83000
> > > > > > > reloc off = 0x7c083000
> > > > > > > Build = 32-bit
> > > > > > > current eth = unknown
> > > > > > > ethaddr = (not set)
> > > > > > > IP addr = <NULL>
> > > > > > > fdt_blob = 0xbae7bf00
> > > > > > > new_fdt = 0xbae7bf00
> > > > > > > fdt_size = 0x00002fa0
> > > > > > > lmb_dump_all:
> > > > > > > memory.cnt = 0x1
> > > > > > > memory.size = 0x0
> > > > > > > memory.reg[0x0].base = 0x40000000
> > > > > > > .size = 0x7ff00000
> > > > > > >
> > > > > > > reserved.cnt = 0x1
> > > > > > > reserved.size = 0x0
> > > > > > > reserved.reg[0x0].base = 0xbae7acd0
> > > > > > > .size = 0x5085330
> > > > > > > arch_number = 0x000010c1
> > > > > > > TLB addr = 0xbfef0000
> > > > > > > irq_sp = 0xbae7bef0
> > > > > > > sp start = 0xbae7bee0
> > > > > > > Board Type = 0
> > > > > > > Early malloc usage: f0 / 400
> > > > > > >
> > > > > > > I'm not sure if this is relevant, but with the failing
> > > commit
> > > > > I can
> > > > > > > get the
> > > > > > > kernel to start with `initrd_high` set to `0x70000000` or
> > > > > lower,
> > > > > > > but not
> > > > > > > higher.
> > > > > >
> > > > > > OK, so 8 whole banks, of 256MB each. Does this platform have
> > > 2GB
> > > > > of
> > > > > > memory for real?
> > > > > >
> > > > > > --
> > > > > > Tom
> > > > >
> > > > > Yes, it does. See https://www.hardkernel.com/shop/odroid-u3/ .
> > > >
> > > > Thanks. I guess my current thought is that in the absence of
> > > adding
> > > > earlycon to the kernel args, to perhaps get more info out of the
> > > kernel
> > > > as to when / where it's dying, that bootm_size should be set to
> > > > 0x10000000 (256MB) to replicate the old behavior, on this
> > > platform.
> > > > Jaehoon, do you have any other ideas?
> > > >
> > > > --
> > > > Tom
> > >
> > > I was not aware of the existence of `earlycon`. Running the first
> > > failing
> > > commit 4963f63f with `earlycon` and with `initrd_high` *unset*
> > > reveals a
> > > kernel panic:
> > >
> > > Kernel image @ 0x41000000 [ 0x000000 - 0x4be200 ]
> > > ## Flattened Device Tree blob at 40800000
> > > Booting using the fdt blob at 0x40800000
> > > Loading Ramdisk to b9947000, end bae7aa21 ... OK
> > > Loading Device Tree to b9936000, end b99460bf ... OK
> > >
> > > Starting kernel ...
> > >
> > > [ 0.000000] Booting Linux on physical CPU 0xa00
> > > [ 0.000000] Linux version 5.10.0-20-armmp
> > > (debian-kernel at lists.debian.org) (gcc-10 (Debian 10.2.1-6) 10.2.1
> > > 20210110,
> > > GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian 5.10.158-2
> > > (2022-12-13)
> > > [ 0.000000] CPU: ARMv7 Processor [413fc090] revision 0
> > > (ARMv7),
> > > cr=10c5387d
> > > [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT
> > > aliasing
> > > instruction cache
> > > [ 0.000000] OF: fdt: Machine model: Hardkernel ODROID-U3
> > > board based
> > > on Exynos4412
> > > [ 0.000000] earlycon: exynos4210 at MMIO 0x13810000 (options
> > > '')
> > > [ 0.000000] printk: bootconsole [exynos4210] enabled
> > > [ 0.000000] Memory policy: Data cache writealloc
> > > [ 0.000000] efi: UEFI not found.
> > > [ 0.000000] Reserved memory: created DMA memory pool at
> > > 0xbf700000,
> > > size 8 MiB
> > > [ 0.000000] OF: reserved mem: initialized node
> > > region_mfc_right,
> > > compatible id shared-dma-pool
> > > [ 0.000000] Reserved memory: created DMA memory pool at
> > > 0xbd300000,
> > > size 36 MiB
> > > [ 0.000000] OF: reserved mem: initialized node
> > > region_mfc_left,
> > > compatible id shared-dma-pool
> > > [ 0.000000] cma: Reserved 16 MiB at 0xbc000000
> > > [ 0.000000] Samsung CPU ID: 0xe4412220
> > > [ 0.000000] Zone ranges:
> > > [ 0.000000] DMA [mem
> > > 0x0000000040000000-0x000000006fffffff]
> > > [ 0.000000] Normal empty
> > > [ 0.000000] HighMem [mem
> > > 0x0000000070000000-0x00000000bd2fffff]
> >
> > So here we go. Initrd cannot be in highmem. I cannot find this
> > documented, quickly, under https://www.kernel.org/doc/html/latest/ so I
> > guess this is more just background knowledge than formal knowledge. This
> > also matches up with your noting that initrd_high=0x70000000 works.
> >
> > --
> > Tom
>
> I found this [1]. Section 5 mentions placing the initrd in lowmem, if I
> understand everything correctly.
Ah, right, there it is, I just skimmed too quickly.
> Setting `bootm_size` to ` 0x10000000` works for me. I can write a patch that
> adds `bootm_size=0x100000000` to `CFG_EXTRA_ENV_SETTINGS` in
> `include/configs/odroid.h` if you want, but I don't understand the fix, high
> mem and low mem well enough to write a decent commit message. I can
> certainly test (a backport of, see bug reports [2], [3]) a patch.
>
> Thanks for identifying the problem and proposing a fix!
So, bootm_size is the variable which tells U-Boot that when relocating
initrd/fdt around, it must be within memory start + bootm_size, as a way
of enforcing the low-memory requirement (or similar constraints, aarch64
says first 1GB of memory). So technically, could set that to 0x30000000
instead of 0x10000000 which is just what the implicit limit was before
the change in question. A patch as you suggest would be right, so yes,
please do and thanks again for testing / digging on the platform itself!
--
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20230111/b411cd60/attachment.sig>
More information about the U-Boot
mailing list