[U-Boot] [PATCH] Allow for parallel builds and saved output

[Simon Glass]

Hi Kumar,

On Tue, Nov 8, 2011 at 5:48 AM, Kumar Gala <galak at kernel.crashing.org> wrote:
>
> On Nov 3, 2011, at 11:22 AM, Simon Glass wrote:
>
>> Hi Andy,
>>
>> On Thu, Nov 3, 2011 at 12:28 AM, Andy Fleming <afleming at freescale.com> wrote:
>>> The MAKEALL script cleverly runs make with the appropriate options
>>> to use all of the cores on the system, but your average U-Boot build
>>> can't make much use of more than a few cores.  If you happen to have
>>> a many-core server, your builds will leave most of the system idle.
>>>
>>> In order to make full use of such a system, we need to build multiple
>>> targets in parallel, and this requires directing make output into
>>> multiple directories. We add a BUILD_NBUILDS variable, which allows
>>> users to specify how many builds to run in parallel. I've found that
>>> 16 is too many on my system (fork starts to fail). When BUILD_NBUILDS
>>> is set greater than 1, we redefine BUILD_DIR for each build to be
>>> ${BUILD_DIR}/${target}. Also, we make "./build" the default BUILD_DIR
>>> when BUILD_NBUILDS is greater than 1.
>>>
>>> Once each build finishes, we run make clean on its directory, to reduce
>>> the footprint, and also remove all .depend files.
>>>
>>> As a result, we are left with a build directory with all of the built
>>> targets still there for use, which means anyone who wanted to use
>>> MAKEALL as part of a test harness can now do so.
>>>
>>> Signed-off-by: Andy Fleming <afleming at freescale.com>
>>> ---
>>> Inspired by all the MAKEALL improvements, I decided to clean up my old
>>> one for parallel builds. I think this version addresses the concerns
>>> raised last time...
>>
>
> I see an issue if I have a build in working tree already.  I think we might need a distclean before starting
>
> I also it seems this does things in clumps of BUILD_NBUILDS.  What I mean is, if BUILD_NBUILDS is 4, you do 4 and wait for all 4 to finish before starting the 5th job once the 1st is complete.  Anything we can do about that to always try and have 4 builds going until the end?
>
> - k

What is the issue you see? Given that it uses an empty output dir for
each build, I can't see what could be wrong.

I am seeing a strange issue which is probably my fault. In a whole
MAKEALL maybe one build (always mx31 so far) fails with errors like
this:

$ more logs/mx31pdk.ERR
mv: cannot stat `/c/cosarm/buildall/u-boot/build/mx31pdk/include/autoconf.mk.tmp
': No such file or directory
/c/cosarm/buildall/u-boot/include/timestamp.h:27: fatal error: generated/timesta
mp_autogenerated.h: No such file or directory
compilation terminated.
make[1]: *** No rule to make target `/c/cosarm/buildall/u-boot/build/mx31pdk/arc
h/arm/cpu/arm1136/.depend', needed by `_depend'.  Stop.
In file included from /c/cosarm/buildall/u-boot/include/version.h:27:0,
                 from mkimage.c:26:
/c/cosarm/buildall/u-boot/include/timestamp.h:27:47: fatal error: generated/time
stamp_autogenerated.h: No such file or directory
compilation terminated.


/c/cosarm/buildall/u-boot/include/timestamp.h:27: fatal error: generated/timesta
mp_autogenerated.h: No such file or directory
compilation terminated.
make[1]: *** No rule to make target `/c/cosarm/buildall/u-boot/build/imx31_phyco
re/arch/arm/cpu/arm1136/.depend', needed by `_depend'.  Stop.
In file included from /c/cosarm/buildall/u-boot/include/version.h:27:0,
                 from mkimage.c:26:
/c/cosarm/buildall/u-boot/include/timestamp.h:27:47: fatal error: generated/time
stamp_autogenerated.h: No such file or directory
compilation terminated.
make[1]: *** [/c/cosarm/buildall/u-boot/build/imx31_phycore/tools/mkimage.o] Err
or 1
make: *** [tools] Error 2


I really can't see that Andy's change can affect this, but perhaps it
provokes an existing bug by changing the timing. I see the issue when
building 16 U-Boots at a time.

I haven't looked into it in detail - maybe there is a dependency
missing in the Makefile. But I would have thought that this line would
ensure that these files exist before anything else is done:

depend dep:	$(TIMESTAMP_FILE) $(VERSION_FILE) \
...

Regards,
Simon