[U-Boot] [PATCH v2 0/3] Add some missing buildman features and deprecate MAKEALL

[Tom Rini]

On Mon, Aug 04, 2014 at 02:41:45PM -0600, Simon Glass wrote:
> Hi Tom,
> 
> On 4 August 2014 07:23, Tom Rini <trini at ti.com> wrote:
> > On Mon, Jul 28, 2014 at 11:53:27PM -0600, Simon Glass wrote:
> >
> >> Buildman has been around for a little over a year and is used by a fair
> >> number of U-Boot developers. However quite a few people still use MAKEALL.
> >>
> >> Buildman was intended to replace MAKEALL, so perhaps now is a good time to
> >> start that process.
> >
> > OK, lets start.  First, I had to make up a special directory with
> > symlinks to the various toolchains, in order to get them to work with
> > buildman, due to the various layouts:
> >
> > $ ls -l ~/work/u-boot/toolchain-links/
> > total 4
> > lrwxrwxrwx 1 trini trini 38 Aug  1 17:09 arc_gnu_4.8-R2_prebuilt_uclibc_be -> /opt/arc_gnu_4.8-R2_prebuilt_uclibc_be
> > lrwxrwxrwx 1 trini trini 38 Aug  1 17:09 arc_gnu_4.8-R2_prebuilt_uclibc_le -> /opt/arc_gnu_4.8-R2_prebuilt_uclibc_le
> > lrwxrwxrwx 1 trini trini 24 Aug  1 17:10 bfin-elf -> /opt/uClinux-45/bfin-elf
> > lrwxrwxrwx 1 trini trini 32 Aug  1 17:09 m68k-linux -> /opt/gcc-4.9.0-nolibc/m68k-linux
> > lrwxrwxrwx 1 trini trini 34 Aug  1 17:09 microblazeel-xilinx-linux-gnu -> /opt/microblazeel-xilinx-linux-gnu
> > lrwxrwxrwx 1 trini trini 32 Aug  1 17:09 microblaze-xilinx-linux-gnu -> /opt/microblaze-xilinx-linux-gnu
> > lrwxrwxrwx 1 trini trini 66 Aug  1 17:15 mips32-linux -> /opt/eldk-5.2.1/mips/sysroots/i686-eldk-linux/usr/bin/mips32-linux
> > lrwxrwxrwx 1 trini trini 36 Aug  1 17:09 nds32le-linux-glibc-v1f -> /opt/nds32le/nds32le-linux-glibc-v1f
> > lrwxrwxrwx 1 trini trini 20 Aug  1 17:09 sparc-elf-3.4.4 -> /opt/sparc-elf-3.4.4
> >
> > I tried a few things but I couldn't get a toolchain section setup that
> > found all of those.  Second:
> 
> OK, can you give me a trace of the output of 'buildman
> --list-tool-chains' - I should be able to figure out what is wrong.

Sure but I think the problem was I couldn't get it to recurse all the
way down, just a few levels (which isn't a bad thing exactly, it'd be
going pretty deep otherwise).

> > (testing the MMC pull request)
> > $ ./tools/buildman/buildman -b master
> > 'arc|blackfin|m68k|microblaze|nds32|sparc|mips'
> > No section: 'make-flags'
> > Building 11 commits for 121 boards (6 threads, 1 job per thread)
> > Cloning repo for thread 0
> > Cloning repo for thread 1
> > Cloning repo for thread 2
> > Cloning repo for thread 3
> > Cloning repo for thread 4
> > Cloning repo for thread 5
> >  1155   88   88 /1331   axs101
> >
> > So it's nice that it spit out a thread per commit, but I really only
> > care right now if master builds right.  And I've gone blind and can't
> > find the end result logs of what failed to build (88 failures doesn't
> > quite match the warning/errors I get for my wrapper around
> > toolchains+MAKEALL).
> 
> You can use the -s flag to see the broken builds, and -se to see
> errors also. Is that what you mean?

That's closer, yeah.  Oh, and -c 1 seems to do be what I want, except I
get:
$ ./tools/buildman/buildman -se 'blackfin' -c 1
No section: 'make-flags'
Traceback (most recent call last):
  File "./tools/buildman/buildman", line 135, in <module>
    control.DoBuildman(options, args)
  File "/home/trini/work/u-boot/u-boot/tools/buildman/control.py", line 138, in DoBuildman
    1)
  File "/home/trini/work/u-boot/u-boot/tools/buildman/../patman/patchstream.py", line 389, in GetMetaDataForList
    stdout = command.RunPipe(pipe, capture=True).stdout
  File "/home/trini/work/u-boot/u-boot/tools/buildman/../patman/command.py", line 50, in RunPipe
    user_pipestr =  '|'.join([' '.join(pipe) for pipe in pipe_list])
TypeError: sequence item 8: expected string, NoneType found

> The output is in ../<branch>/<commit>/<board> but if you want to keep
> the binaries, you need -k also.

Ah, OK.

> > I also didn't quite see how to plugin an arm64 toolchain.
> 
> I haven't tried - can you please point me to one and I'll give it a go.

There should be something newer than
https://launchpad.net/linaro-toolchain-binaries but I'm failing to find
it quickly.

> > Next up, is there some way to have multiple toolchains for an arch
> > available?  On some compute resources I have at work I throw
> > arm/eldk4.2, arm/eldk5.2, arm/linaro-2013.03 (since we've pinned to that
> > for some projects) and some various combos for powerpc as well.  I
> > didn't see how to do that with buildman aside from different ~/.buildman
> > files per invocation.
> 
> Yes I remember you talking about this before. One option would be to
> add toolchain as a new dimension in the matrix. It might be painful
> though. So to be clear, you want to define multiple toolchains for an
> arch, and have buildman always build with all of them?

Well, a --config-file= would be good enough so I could have
buildman-eldk54.cfg, buildman-eldk42.cfg, etc, etc.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140804/d612919b/attachment.pgp>