[U-Boot] MAKEALL

[Simon Glass]

Hi,

On 12 February 2014 03:42, Masahiro Yamada <yamada.m at jp.panasonic.com> wrote:
> Hi Albert,
>
>
>> > It might also be worth looking at tools/buildman, which automatically
>> > allocates one build thread per CPU.
>>
>> Jumping in late, but my question is incidental and not urgent anyway.
>>
>> Would using buildman make the multiple build / multiple CPU code in
>> MAKEALL useless? I'm wondering whether we could apply the Unix
>> philosophy here (1), let buildman alone deal with handling parallel
>> builds, and remove code from MAKEALL.
>

This is a good summary thank you, but I have a few comments as well.

> I think parallel build feature is missing from buildman.
> So, buildman cannot check if -j option is working correctly.

It is not used by default, but there is a -j option. If you use -j,
first make sure you reduce the number of threads to compensate. So for
example you can use:

   buildman -j 8 -T 1

to use -j8 with only a single thread on a 8-core machine. I typically
find that building with -j1 and increasing the thread count is much
faster. Another tip if you are not using your machine for anything
else is to use -T with a number 20-30% larger than the number of CPUs
you have.

>
> Note:
> Please do not be confused by the difference of
> what "parallel" means.
>
> [1] MAKEALL runs single "make" thread by default.
> (You can change this with BUILD_NBUILDS variable)
> One "make" thread runs multiple jobs by giving -j option.
> (You can change this with BUILD_NCPUS variable)
>
> [2] buildman runs multiple "make" threads.
> Each make thread runs one job (that is, always  -j 1 ).
>

Indeed, this is the default behaviour, but it can be changed with options.

>
>
> Besides parallel build , I notice some differences between MAKEALL and
> buildman.
>
>  - MAKEALL runs "make mrproper" everty time before "make",
>     but buildman doesn't.
>     This means objects are remaining, that were generated
>     by the previous commit.
>     So, even if some build rules in makefiles get broken at an
>     intermediate commit,  buildman possibly cannot detect the error.

That is correct. To get the MAKEALL behaviour you need to give the -f option.

>
>  - MAKEALL can select in-tree-build or out-of-tree build
>    by BUILD_DIR option, but buildman always does out-of-tree build.

That's right, in fact buildman does not support in-tree build at all
unfortunately. However, given that every thread has its own git tree,
it would actually be fairly easy to add this feature.

>
> (Simon, please correct me if I am wrong.)
>
> If you touch only C sources, buildman is enough (and faster).
>
> But if you change make targets, it is highly recommended
> to check with MAKEALL.
> At least, I need both. (for Kbuild and Kconfig work)
>
> I guess we cannot replace MAKEALL with buildman for now.

The intent was that buildman supported everything that MAKEALL does
and more. I believe that is true except for the in-tree build you
mention above. However, I think there might also be some small
difference between the way the -f option is implemented in buildman,
and the way MAKEALL works. It might be worth splitting the -f option
into two:

- one to force rebuild of a board
- another to force a reconfigure on every build

Then we could be sure. In summary, if we do that and support in-tree
builds then I think buildman has all the features of MAKEALL.

BTW a few other options that buildman has, which people may not have noticed:

-S   - shows increase/decrease of image sizes across commits
-B   - same but for individual functions
-k    - keep the build outputs
-d    - show detailed board information
-S   - allows building every second commit, or only first and last, etc.

Regards,
Simon