[U-Boot] [PATCH] Allow for parallel builds and saved output

[Kumar Gala]

On Nov 20, 2011, at 12:11 AM, Simon Glass wrote:

> Hi Kumar,
> 
> On Tue, Nov 8, 2011 at 5:48 AM, Kumar Gala <galak at kernel.crashing.org> wrote:
>> 
>> On Nov 3, 2011, at 11:22 AM, Simon Glass wrote:
>> 
>>> Hi Andy,
>>> 
>>> On Thu, Nov 3, 2011 at 12:28 AM, Andy Fleming <afleming at freescale.com> wrote:
>>>> The MAKEALL script cleverly runs make with the appropriate options
>>>> to use all of the cores on the system, but your average U-Boot build
>>>> can't make much use of more than a few cores.  If you happen to have
>>>> a many-core server, your builds will leave most of the system idle.
>>>> 
>>>> In order to make full use of such a system, we need to build multiple
>>>> targets in parallel, and this requires directing make output into
>>>> multiple directories. We add a BUILD_NBUILDS variable, which allows
>>>> users to specify how many builds to run in parallel. I've found that
>>>> 16 is too many on my system (fork starts to fail). When BUILD_NBUILDS
>>>> is set greater than 1, we redefine BUILD_DIR for each build to be
>>>> ${BUILD_DIR}/${target}. Also, we make "./build" the default BUILD_DIR
>>>> when BUILD_NBUILDS is greater than 1.
>>>> 
>>>> Once each build finishes, we run make clean on its directory, to reduce
>>>> the footprint, and also remove all .depend files.
>>>> 
>>>> As a result, we are left with a build directory with all of the built
>>>> targets still there for use, which means anyone who wanted to use
>>>> MAKEALL as part of a test harness can now do so.
>>>> 
>>>> Signed-off-by: Andy Fleming <afleming at freescale.com>
>>>> ---
>>>> Inspired by all the MAKEALL improvements, I decided to clean up my old
>>>> one for parallel builds. I think this version addresses the concerns
>>>> raised last time...
>>> 
>> 
>> I see an issue if I have a build in working tree already.  I think we might need a distclean before starting
>> 
>> I also it seems this does things in clumps of BUILD_NBUILDS.  What I mean is, if BUILD_NBUILDS is 4, you do 4 and wait for all 4 to finish before starting the 5th job once the 1st is complete.  Anything we can do about that to always try and have 4 builds going until the end?
>> 
>> - k
> 
> What is the issue you see? Given that it uses an empty output dir for
> each build, I can't see what could be wrong.

I think if I have an existing build in my source tree it screws up between that and the O= build.

> I am seeing a strange issue which is probably my fault. In a whole
> MAKEALL maybe one build (always mx31 so far) fails with errors like
> this:
> 
> $ more logs/mx31pdk.ERR
> mv: cannot stat `/c/cosarm/buildall/u-boot/build/mx31pdk/include/autoconf.mk.tmp
> ': No such file or directory
> /c/cosarm/buildall/u-boot/include/timestamp.h:27: fatal error: generated/timesta
> mp_autogenerated.h: No such file or directory
> compilation terminated.
> make[1]: *** No rule to make target `/c/cosarm/buildall/u-boot/build/mx31pdk/arc
> h/arm/cpu/arm1136/.depend', needed by `_depend'.  Stop.
> In file included from /c/cosarm/buildall/u-boot/include/version.h:27:0,
>                 from mkimage.c:26:
> /c/cosarm/buildall/u-boot/include/timestamp.h:27:47: fatal error: generated/time
> stamp_autogenerated.h: No such file or directory
> compilation terminated.
> 
> 
> /c/cosarm/buildall/u-boot/include/timestamp.h:27: fatal error: generated/timesta
> mp_autogenerated.h: No such file or directory
> compilation terminated.
> make[1]: *** No rule to make target `/c/cosarm/buildall/u-boot/build/imx31_phyco
> re/arch/arm/cpu/arm1136/.depend', needed by `_depend'.  Stop.
> In file included from /c/cosarm/buildall/u-boot/include/version.h:27:0,
>                 from mkimage.c:26:
> /c/cosarm/buildall/u-boot/include/timestamp.h:27:47: fatal error: generated/time
> stamp_autogenerated.h: No such file or directory
> compilation terminated.
> make[1]: *** [/c/cosarm/buildall/u-boot/build/imx31_phycore/tools/mkimage.o] Err
> or 1
> make: *** [tools] Error 2
> 
> 
> I really can't see that Andy's change can affect this, but perhaps it
> provokes an existing bug by changing the timing. I see the issue when
> building 16 U-Boots at a time.
> 
> I haven't looked into it in detail - maybe there is a dependency
> missing in the Makefile. But I would have thought that this line would
> ensure that these files exist before anything else is done:
> 
> depend dep:	$(TIMESTAMP_FILE) $(VERSION_FILE) \
> ...
> 
> Regards,
> Simon