[U-Boot] Please pull u-boot-dm

Thu Jul 12 21:26:35 UTC 2018

On 07/12/2018 12:17 PM, Stephen Warren wrote:
> On 07/12/2018 09:52 AM, Stephen Warren wrote:
>> On 07/11/2018 06:12 PM, Simon Glass wrote:
>>> Hi Stephen,
>>>
>>> On 11 July 2018 at 16:01, Stephen Warren <swarren at wwwdotorg.org> wrote:
>>>> On 07/10/2018 02:24 PM, Simon Glass wrote:
>>>>>
>>>>> Hi Stephen,
>>>>>
>>>>> On 10 July 2018 at 13:53, Stephen Warren <swarren at wwwdotorg.org> 
>>>>> wrote:
>>>>>>
>>>>>> On 07/10/2018 12:47 PM, Tom Rini wrote:
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jul 09, 2018 at 01:53:43PM -0600, Simon Glass wrote:
>>>>>>>
>>>>>>>> Hi Tom.
>>>>>>>>
>>>>>>>> Here are some test-coverage and DM core enhancements. Also it 
>>>>>>>> adds a
>>>>>>>> way to access the binman definition from U-Boot.
>>>>>>>>
>>>>>>>>
>>>>>>>> The following changes since commit
>>>>>>>> 8c5d4fd0ec222701598a27b26ab7265d4cee45a3:
>>>>>>>>
>>>>>>>>      Prepare v2018.07 (2018-07-09 10:24:14 -0400)
>>>>>>>>
>>>>>>>> are available in the Git repository at:
>>>>>>>>
>>>>>>>>      git://git.denx.de/u-boot-dm.git
>>>>>>>>
>>>>>>>> for you to fetch changes up to
>>>>>>>> 16b8d6b76992690c65c58dc8b0591496cc5e46ef:
>>>>>>>>
>>>>>>>>      binman: Support updating the device tree with calc'd info
>>>>>>>> (2018-07-09 09:11:00 -0600)
>>>>>>
>>>>>>
>>>>>>
>>>>>> This pull has caused intermittent/random build errors on my Jenkins
>>>>>> system.
>>>>>> The log shows:
>>>>>>
>>>>>>>     LD      spl/u-boot-spl
>>>>>>>     OBJCOPY spl/u-boot-spl-nodtb.bin
>>>>>>>     COPY    spl/u-boot-spl.bin
>>>>>>>     BINMAN  u-boot-tegra.bin
>>>>>>>     BINMAN  u-boot-nodtb-tegra.bin
>>>>>>>     BINMAN  u-boot-dtb-tegra.bin
>>>>>>> binman: pylibfdt error -9: FDT_ERR_BADMAGIC
>>>>>>>
>>>>>>>
>>>>>>> /var/lib/jenkins/workspace/u-boot-denx_uboot-master-build/src/u-boot/Makefile:1244: 
>>>>>>>
>>>>>>> recipe for target 'u-boot-tegra.bin' failed
>>>>>>> make[1]: *** [u-boot-tegra.bin] Error 1
>>>>>>> make[1]: *** Waiting for unfinished jobs....
>>>>>>> make[1]: Leaving directory
>>>>>>>
>>>>>>> '/var/lib/jenkins/workspace/u-boot-denx_uboot-master-build/build/u-boot/beaver' 
>>>>>>>
>>>>>>> Makefile:148: recipe for target 'sub-make' failed
>>>>>>> make: *** [sub-make] Error 2
>>>>>>> make: Leaving directory
>>>>>>> '/var/lib/jenkins/workspace/u-boot-denx_uboot-master-build/src/u-boot' 
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> This doesn't happen every time; my Jenkins system builds 25 
>>>>>> Tegra/sandbox
>>>>>> boards, yet a varying set of boards fail each time I trigger the 
>>>>>> build:
>>>>>> Just
>>>>>> beaver the first time, then just colibri_t20 and ventana, then just
>>>>>> medcom-wide. Note that the system performs incremental builds, if 
>>>>>> that
>>>>>> matters.
>>>>>
>>>>>
>>>>> That might be the fdt_resize() problem which David Gibson has just
>>>>> sorted out upstream. If you can run binman -D (to get a stack trace)
>>>>> that might help. I should be able to do a patch if that is the
>>>>> problem.
>>>>
>>>>
>>>> Is this the backtrace you're looking for?
>>>>
>>>>> [swarren at swarren-lx1 u-boot]$
>>>>> CROSS_COMPILE=/home/swarren/shared/gcc-linaro-7.2.1-2017.11-i686_arm-linux-gnueabihf/bin/arm-linux-gnueabihf- 
>>>>>
>>>>> sh -c "make O=build-beaver -s beaver_defconfig && make 
>>>>> O=build-beaver -s
>>>>> -j8"
>>>>> arch/arm/dts/tegra30-apalis.dtb: Warning 
>>>>> (avoid_unnecessary_addr_size):
>>>>> /i2c at 7000d000/tps65911 at 2d/regulators: unnecessary 
>>>>> #address-cells/#size-cells
>>>>> without "ranges" or child "reg" property
>>>>> arch/arm/dts/tegra30-beaver.dtb: Warning 
>>>>> (avoid_unnecessary_addr_size):
>>>>> /i2c at 7000d000/tps65911 at 2d/regulators: unnecessary 
>>>>> #address-cells/#size-cells
>>>>> without "ranges" or child "reg" property
>>>>> binman: pylibfdt error -9: FDT_ERR_BADMAGIC
>>>>>
>>>>> Traceback (most recent call last):
>>>>>    File "../tools/binman/binman", line 120, in RunBinman
>>>>>      ret_code = control.Binman(options, args)
>>>>>    File
>>>>> "/home/swarren/shared/git_wa/tegra-uboot-flasher/u-boot/tools/binman/control.py", 
>>>>>
>>>>> line 128, in Binman
>>>>>      dtb = fdt.FdtScan(fname)
>>>>>    File
>>>>> "/home/swarren/shared/git_wa/tegra-uboot-flasher/u-boot/tools/binman/../dtoc/fdt.py", 
>>>>>
>>>>> line 459, in FdtScan
>>>>>      dtb = Fdt(fname)
>>>>>    File
>>>>> "/home/swarren/shared/git_wa/tegra-uboot-flasher/u-boot/tools/binman/../dtoc/fdt.py", 
>>>>>
>>>>> line 315, in __init__
>>>>>      self._fdt_obj = libfdt.Fdt(fd.read())
>>>>>    File "scripts/dtc/pylibfdt/libfdt.py", line 207, in __init__
>>>>>      check_err(fdt_check_header(self._fdt));
>>>>>    File "scripts/dtc/pylibfdt/libfdt.py", line 160, in check_err
>>>>>      raise FdtException(val)
>>>>> FdtException: pylibfdt error -9: FDT_ERR_BADMAGIC
>>>>> /home/swarren/shared/git_wa/tegra-uboot-flasher/u-boot/Makefile:1244:
>>>>> recipe for target 'u-boot-tegra.bin' failed
>>>>> make[1]: *** [u-boot-tegra.bin] Error 1
>>>>> make[1]: *** Waiting for unfinished jobs....
>>>
>>> Thanks for that. But actually that is not something I have seen or can
>>> explain. Here it is reading the DT at the start and somehow failing in
>>> the check. That DT is created by the U-Boot build system. Is it
>>> possible that your system has its own libfdt installed?
>>>
>>> As it happened, the pylibfdt changes have been applied upstream today,
>>> so I'll prepare a patch to sync U-Boot up with that. Hopefully that
>>> will resolve any issues, but I am not sure.
>>
>> I have the standard Ubuntu 16.04 packages installed, but no 
>> non-standard local builds:
>>
>> device-tree-compiler                     1.4.0+dfsg-2
>> libfdt1:amd64                            1.4.0+dfsg-2
> 
> I suspect this is a missing dependency in the makefiles, or perhaps some 
> process isn't waiting for its child to exit. The code that's finding the 
> bad FDT magic appears to be reading from a file that hasn't yet been 
> written yet:
> 
>> [pid 26251] execve("../tools/binman/binman", 
>> ["../tools/binman/binman", "-d", "u-boot.dtb", "-O", ".", "-I", ".", 
>> "-I", "../board/nvidia/beaver", "spl/u-boot-spl"], [/* 158 vars */] 
>> <unfinished ...>
> 
>> [pid 26255] execve("../tools/binman/binman", 
>> ["../tools/binman/binman", "-d", "u-boot.dtb", "-O", ".", "-I", ".", 
>> "-I", "../board/nvidia/beaver", "spl/u-boot-spl"], [/* 158 vars */] 
>> <unfinished ...>
> 
>> [pid 26255] open("./u-boot-out.dtb", O_WRONLY|O_CREAT|O_TRUNC, 0666 
>> <unfinished ...>
>> [pid 26251] <... close resumed> )       = 0
>> [pid 26251] close(3)                    = 0
>> [pid 26251] open("./u-boot-out.dtb", O_RDONLY) = 3
>> [pid 26251] fstat(3, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
>> [pid 26251] fstat(3, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
>> [pid 26251] lseek(3, 0, SEEK_CUR)       = 0
>> [pid 26251] lseek(3, 0, SEEK_CUR)       = 0
>> [pid 26251] fstat(3, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
>> [pid 26251] read(3, "", 4096)           = 0
>> [pid 26251] close(3)                    = 0
>> [pid 26251] fstat(1, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
>> [pid 26251] write(1, "binman: pylibfdt error -9: FDT_E"..., 44binman: 
>> pylibfdt error -9: FDT_ERR_BADMAGIC
> 
>> [pid 26255] write(3, 
>> "\320\r\376\355\0\0We\0\0\0008\0\0Q\340\0\0\0(\0\0\0\21\0\0\0\20\0\0\0\0"..., 
>> 20480 <unfinished ...>

Specifically, this Makefile rule:

u-boot-dtb-tegra.bin u-boot-tegra.bin u-boot-nodtb-tegra.bin: \
		spl/u-boot-spl u-boot.bin FORCE
	$(call if_changed,binman)

runs that same binman invocation for all 3 target binaries separately 
and in parallel, and tools/binman/control.py:Binman() uses the same 
hard-coded filename for an output (or temporary?) file in each 
invocation. I'm not sure if we can simply replace:

             fname = tools.GetOutputFilename('u-boot-out.dtb')

with a call to tempfile.mkstemp() or similar?