[PATCHv4] Azure/GitLab: Move to gcc-11.1.0 and LLVM-11

Fri Jul 2 16:59:39 CEST 2021

On Fri, Jul 2, 2021 at 10:42 PM Tom Rini <trini at konsulko.com> wrote:
>
> - Move to gcc-11.1.0 builds from kernel.org for supported platforms and
>   LLVM-11 for those tests.
> - As Heinrich has noted, the RISC-V platform specification has a profile
>   OS-A for running rich operating systems like Linux and BSD. This profile
>   requires 64bit and UEFI conforming to the EBBR. Only the 'embedded'
>   profile may use 32bit.  Given this, drop grub for 32bit RISC-V as it no
>   longer compiles with gcc-11.1 and upstream is unlikely to fix it:
>   https://www.mail-archive.com/grub-devel@gnu.org/msg30736.html
> - Update to grub-2.06 release to address other issues of building with
>   gcc-11.1.
> - Update to newer Xtensa (gcc-9.2.0) and ARC (gcc-10.2) toolchains
>
> Cc: Heinrich Schuchardt <xypron.glpk at gmx.de>
> Cc: Bin Meng <bmeng.cn at gmail.com>
> Cc: Simon Glass <sjg at chromium.org>
> Cc: Rick Chen <rick at andestech.com>
> Signed-off-by: Tom Rini <trini at konsulko.com>
> ---
> Changes in v4:
> - Given Heinrich's explanation, continue dropping grub for riscv32 and
>   explain why we can better in the commit message.
> - The x86 build problem noted in v3 has been fixed, thanks Bin!
>
> Changes in v3:
> - Post the right patch this time.
> - Switch to gcc-11.1.0 builds rather than pre-release
> - grub-2.06 is out, use that rather than -rc1
> - Update Xtensa and ARC toolchains.
> - Note that right now this causes x86 to fail to link, which I did not
>   see with gcc-11.0 tests.
>
> Changes in v2:
> - None, I re-sent v1 by accident
> ---
>  .azure-pipelines.yml    |  3 +-
>  .gitlab-ci.yml          |  3 +-
>  tools/docker/Dockerfile | 85 +++++++++++++++++------------------------
>  3 files changed, 36 insertions(+), 55 deletions(-)
>
> diff --git a/.azure-pipelines.yml b/.azure-pipelines.yml
> index 5ac4d648ec9e..3d4cd76c6061 100644
> --- a/.azure-pipelines.yml
> +++ b/.azure-pipelines.yml
> @@ -2,7 +2,7 @@ variables:
>    windows_vm: vs2017-win2016
>    ubuntu_vm: ubuntu-18.04
>    macos_vm: macOS-10.15
> -  ci_runner_image: trini/u-boot-gitlab-ci-runner:focal-20210416-10Jun2021
> +  ci_runner_image: trini/u-boot-gitlab-ci-runner:focal-20210416-14Jun2021
>    # Add '-u 0' options for Azure pipelines, otherwise we get "permission
>    # denied" error when it tries to "useradd -m -u 1001 vsts_azpcontainer",
>    # since our $(ci_runner_image) user is not root.
> @@ -300,7 +300,6 @@ jobs:
>            cp ~/grub_x86.efi ${UBOOT_TRAVIS_BUILD_DIR}/
>            cp ~/grub_x64.efi ${UBOOT_TRAVIS_BUILD_DIR}/
>            cp /opt/grub/grubriscv64.efi ${UBOOT_TRAVIS_BUILD_DIR}/grub_riscv64.efi
> -          cp /opt/grub/grubriscv32.efi ${UBOOT_TRAVIS_BUILD_DIR}/grub_riscv32.efi
>            cp /opt/grub/grubaa64.efi ${UBOOT_TRAVIS_BUILD_DIR}/grub_arm64.efi
>            cp /opt/grub/grubarm.efi ${UBOOT_TRAVIS_BUILD_DIR}/grub_arm.efi
>            virtualenv -p /usr/bin/python3 /tmp/venv
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 41a2c18fa180..b7c049825102 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -2,7 +2,7 @@
>
>  # Grab our configured image.  The source for this is found at:
>  # https://source.denx.de/u-boot/gitlab-ci-runner
> -image: trini/u-boot-gitlab-ci-runner:focal-20210416-10Jun2021
> +image: trini/u-boot-gitlab-ci-runner:focal-20210416-14Jun2021
>
>  # We run some tests in different order, to catch some failures quicker.
>  stages:
> @@ -38,7 +38,6 @@ stages:
>      - cp ~/grub_x86.efi $UBOOT_TRAVIS_BUILD_DIR/
>      - cp ~/grub_x64.efi $UBOOT_TRAVIS_BUILD_DIR/
>      - cp /opt/grub/grubriscv64.efi $UBOOT_TRAVIS_BUILD_DIR/grub_riscv64.efi
> -    - cp /opt/grub/grubriscv32.efi $UBOOT_TRAVIS_BUILD_DIR/grub_riscv32.efi
>      - cp /opt/grub/grubaa64.efi $UBOOT_TRAVIS_BUILD_DIR/grub_arm64.efi
>      - cp /opt/grub/grubarm.efi $UBOOT_TRAVIS_BUILD_DIR/grub_arm.efi
>      - virtualenv -p /usr/bin/python3 /tmp/venv
> diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile
> index 95951f4bef89..d0e2642da585 100644
> --- a/tools/docker/Dockerfile
> +++ b/tools/docker/Dockerfile
> @@ -12,24 +12,24 @@ ENV DEBIAN_FRONTEND=noninteractive
>  # Add LLVM repository
>  RUN apt-get update && apt-get install -y gnupg2 wget xz-utils && rm -rf /var/lib/apt/lists/*
>  RUN wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add -
> -RUN echo deb http://apt.llvm.org/focal/ llvm-toolchain-focal-10 main | tee /etc/apt/sources.list.d/llvm.list
> +RUN echo deb http://apt.llvm.org/focal/ llvm-toolchain-focal-11 main | tee /etc/apt/sources.list.d/llvm.list
>
> -# Manually install the kernel.org "Crosstool" based toolchains for gcc-7.3
> -RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/9.2.0/x86_64-gcc-9.2.0-nolibc-aarch64-linux.tar.xz | tar -C /opt -xJ
> -RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/9.2.0/x86_64-gcc-9.2.0-nolibc-arm-linux-gnueabi.tar.xz | tar -C /opt -xJ
> -RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/9.2.0/x86_64-gcc-9.2.0-nolibc-i386-linux.tar.xz | tar -C /opt -xJ
> -RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/9.2.0/x86_64-gcc-9.2.0-nolibc-m68k-linux.tar.xz | tar -C /opt -xJ
> -RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/9.2.0/x86_64-gcc-9.2.0-nolibc-mips-linux.tar.xz | tar -C /opt -xJ
> -RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/9.2.0/x86_64-gcc-9.2.0-nolibc-microblaze-linux.tar.xz | tar -C /opt -xJ
> -RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/9.2.0/x86_64-gcc-9.2.0-nolibc-nios2-linux.tar.xz | tar -C /opt -xJ
> -RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/9.2.0/x86_64-gcc-9.2.0-nolibc-powerpc-linux.tar.xz | tar -C /opt -xJ
> -RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/9.2.0/x86_64-gcc-9.2.0-nolibc-riscv32-linux.tar.xz | tar -C /opt -xJ
> -RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/9.2.0/x86_64-gcc-9.2.0-nolibc-riscv64-linux.tar.xz | tar -C /opt -xJ
> -RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/9.2.0/x86_64-gcc-9.2.0-nolibc-sh2-linux.tar.xz | tar -C /opt -xJ
> +# Manually install the kernel.org "Crosstool" based toolchains for gcc-11.1.0
> +RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/11.1.0/x86_64-gcc-11.1.0-nolibc-aarch64-linux.tar.xz | tar -C /opt -xJ
> +RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/11.1.0/x86_64-gcc-11.1.0-nolibc-arm-linux-gnueabi.tar.xz | tar -C /opt -xJ
> +RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/11.1.0/x86_64-gcc-11.1.0-nolibc-i386-linux.tar.xz | tar -C /opt -xJ
> +RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/11.1.0/x86_64-gcc-11.1.0-nolibc-m68k-linux.tar.xz | tar -C /opt -xJ
> +RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/11.1.0/x86_64-gcc-11.1.0-nolibc-mips-linux.tar.xz | tar -C /opt -xJ
> +RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/11.1.0/x86_64-gcc-11.1.0-nolibc-microblaze-linux.tar.xz | tar -C /opt -xJ
> +RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/11.1.0/x86_64-gcc-11.1.0-nolibc-nios2-linux.tar.xz | tar -C /opt -xJ
> +RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/11.1.0/x86_64-gcc-11.1.0-nolibc-powerpc-linux.tar.xz | tar -C /opt -xJ
> +RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/11.1.0/x86_64-gcc-11.1.0-nolibc-riscv32-linux.tar.xz | tar -C /opt -xJ

I believe riscv32 toolchain is no longer needed given it's used for
the grub riscv32 build. U-Boot for riscv32 can be built with the
riscv64 toolchain IIRC.

> +RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/11.1.0/x86_64-gcc-11.1.0-nolibc-riscv64-linux.tar.xz | tar -C /opt -xJ
> +RUN wget -O - https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/11.1.0/x86_64-gcc-11.1.0-nolibc-sh2-linux.tar.xz | tar -C /opt -xJ
>
>  # Manually install other toolchains
> -RUN wget -O - https://github.com/foss-xtensa/toolchain/releases/download/2018.02/x86_64-2018.02-xtensa-dc233c-elf.tar.gz | tar -C /opt -xz
> -RUN wget -O - https://github.com/foss-for-synopsys-dwc-arc-processors/toolchain/releases/download/arc-2019.09-release/arc_gnu_2019.09_prebuilt_uclibc_le_archs_linux_install.tar.gz | tar --no-same-owner -C /opt -xz
> +RUN wget -O - https://github.com/foss-xtensa/toolchain/releases/download/2020.07/x86_64-2020.07-xtensa-dc233c-elf.tar.gz | tar -C /opt -xz
> +RUN wget -O - https://github.com/foss-for-synopsys-dwc-arc-processors/toolchain/releases/download/arc-2021.03-release/arc_gnu_2021.03_prebuilt_uclibc_le_archs_linux_install.tar.gz | tar --no-same-owner -C /opt -xz
>  RUN wget -O - https://github.com/vincentzwc/prebuilt-nds32-toolchain/releases/download/20180521/nds32le-linux-glibc-v3-upstream.tar.gz | tar -C /opt -xz
>
>  # Update and install things from apt now
> @@ -100,9 +100,6 @@ RUN apt-get update && apt-get install -y \
>         zip \
>         && rm -rf /var/lib/apt/lists/*
>
> -# Manually install libmpfr4 for the toolchains
> -RUN wget http://mirrors.kernel.org/ubuntu/pool/main/m/mpfr4/libmpfr4_3.1.4-1_amd64.deb && dpkg -i libmpfr4_3.1.4-1_amd64.deb && rm libmpfr4_3.1.4-1_amd64.deb
> -
>  # Manually install a new enough version of efitools (must be v1.5.2 or later)
>  RUN wget http://mirrors.kernel.org/ubuntu/pool/universe/e/efitools/efitools_1.8.1-0ubuntu2_amd64.deb && sudo dpkg -i efitools_1.8.1-0ubuntu2_amd64.deb && rm efitools_1.8.1-0ubuntu2_amd64.deb
>
> @@ -119,16 +116,16 @@ RUN git clone https://git.kernel.org/pub/scm/linux/kernel/git/jejb/sbsigntools.g
>  # Build GRUB UEFI targets for ARM & RISC-V, 32-bit and 64-bit
>  RUN git clone git://git.savannah.gnu.org/grub.git /tmp/grub && \
>         cd /tmp/grub && \
> -       git checkout grub-2.04 && \
> +       git checkout grub-2.06 && \
>         ./bootstrap && \
>         mkdir -p /opt/grub && \
>         ./configure --target=aarch64 --with-platform=efi \
>         CC=gcc \
> -       TARGET_CC=/opt/gcc-9.2.0-nolibc/aarch64-linux/bin/aarch64-linux-gcc \
> -       TARGET_OBJCOPY=/opt/gcc-9.2.0-nolibc/aarch64-linux/bin/aarch64-linux-objcopy \
> -       TARGET_STRIP=/opt/gcc-9.2.0-nolibc/aarch64-linux/bin/aarch64-linux-strip \
> -       TARGET_NM=/opt/gcc-9.2.0-nolibc/aarch64-linux/bin/aarch64-linux-nm \
> -       TARGET_RANLIB=/opt/gcc-9.2.0-nolibc/aarch64-linux/bin/aarch64-linux-ranlib && \
> +       TARGET_CC=/opt/gcc-11.1.0-nolibc/aarch64-linux/bin/aarch64-linux-gcc \
> +       TARGET_OBJCOPY=/opt/gcc-11.1.0-nolibc/aarch64-linux/bin/aarch64-linux-objcopy \
> +       TARGET_STRIP=/opt/gcc-11.1.0-nolibc/aarch64-linux/bin/aarch64-linux-strip \
> +       TARGET_NM=/opt/gcc-11.1.0-nolibc/aarch64-linux/bin/aarch64-linux-nm \
> +       TARGET_RANLIB=/opt/gcc-11.1.0-nolibc/aarch64-linux/bin/aarch64-linux-ranlib && \
>         make && \
>         ./grub-mkimage -O arm64-efi -o /opt/grub/grubaa64.efi --prefix= -d \
>         grub-core cat chain configfile echo efinet ext2 fat halt help linux \
> @@ -138,11 +135,11 @@ RUN git clone git://git.savannah.gnu.org/grub.git /tmp/grub && \
>         make clean && \
>         ./configure --target=arm --with-platform=efi \
>         CC=gcc \
> -       TARGET_CC=/opt/gcc-9.2.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-gcc \
> -       TARGET_OBJCOPY=/opt/gcc-9.2.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objcopy \
> -       TARGET_STRIP=/opt/gcc-9.2.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-strip \
> -       TARGET_NM=/opt/gcc-9.2.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-nm \
> -       TARGET_RANLIB=/opt/gcc-9.2.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-ranlib && \
> +       TARGET_CC=/opt/gcc-11.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-gcc \
> +       TARGET_OBJCOPY=/opt/gcc-11.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-objcopy \
> +       TARGET_STRIP=/opt/gcc-11.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-strip \
> +       TARGET_NM=/opt/gcc-11.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-nm \
> +       TARGET_RANLIB=/opt/gcc-11.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-ranlib && \
>         make && \
>         ./grub-mkimage -O arm-efi -o /opt/grub/grubarm.efi --prefix= -d \
>         grub-core cat chain configfile echo efinet ext2 fat halt help linux \
> @@ -152,31 +149,17 @@ RUN git clone git://git.savannah.gnu.org/grub.git /tmp/grub && \
>         make clean && \
>         ./configure --target=riscv64 --with-platform=efi \
>         CC=gcc \
> -       TARGET_CC=/opt/gcc-9.2.0-nolibc/riscv64-linux/bin/riscv64-linux-gcc \
> -       TARGET_OBJCOPY=/opt/gcc-9.2.0-nolibc/riscv64-linux/bin/riscv64-linux-objcopy \
> -       TARGET_STRIP=/opt/gcc-9.2.0-nolibc/riscv64-linux/bin/riscv64-linux-strip \
> -       TARGET_NM=/opt/gcc-9.2.0-nolibc/riscv64-linux/bin/riscv64-linux-nm \
> -       TARGET_RANLIB=/opt/gcc-9.2.0-nolibc/riscv64-linux/bin/riscv64-linux-ranlib && \
> +       TARGET_CC=/opt/gcc-11.1.0-nolibc/riscv64-linux/bin/riscv64-linux-gcc \
> +       TARGET_OBJCOPY=/opt/gcc-11.1.0-nolibc/riscv64-linux/bin/riscv64-linux-objcopy \
> +       TARGET_STRIP=/opt/gcc-11.1.0-nolibc/riscv64-linux/bin/riscv64-linux-strip \
> +       TARGET_NM=/opt/gcc-11.1.0-nolibc/riscv64-linux/bin/riscv64-linux-nm \
> +       TARGET_RANLIB=/opt/gcc-11.1.0-nolibc/riscv64-linux/bin/riscv64-linux-ranlib && \
>         make && \
>         ./grub-mkimage -O riscv64-efi -o /opt/grub/grubriscv64.efi --prefix= -d \
>         grub-core cat chain configfile echo efinet ext2 fat halt help linux \
>         lsefisystab loadenv lvm minicmd normal part_msdos part_gpt reboot \
>         search search_fs_file search_fs_uuid search_label serial sleep test \
>         true && \
> -       make clean && \
> -       ./configure --target=riscv32 --with-platform=efi \
> -       CC=gcc \
> -       TARGET_CC=/opt/gcc-9.2.0-nolibc/riscv32-linux/bin/riscv32-linux-gcc \
> -       TARGET_OBJCOPY=/opt/gcc-9.2.0-nolibc/riscv32-linux/bin/riscv32-linux-objcopy \
> -       TARGET_STRIP=/opt/gcc-9.2.0-nolibc/riscv32-linux/bin/riscv32-linux-strip \
> -       TARGET_NM=/opt/gcc-9.2.0-nolibc/riscv32-linux/bin/riscv32-linux-nm \
> -       TARGET_RANLIB=/opt/gcc-9.2.0-nolibc/riscv32-linux/bin/riscv32-linux-ranlib && \
> -       make && \
> -       ./grub-mkimage -O riscv32-efi -o /opt/grub/grubriscv32.efi --prefix= -d \
> -       grub-core cat chain configfile echo efinet ext2 fat halt help linux \
> -       lsefisystab loadenv lvm minicmd normal part_msdos part_gpt reboot \
> -       search search_fs_file search_fs_uuid search_label serial sleep test \
> -       true && \
>         rm -rf /tmp/grub
>
>  RUN git clone git://git.qemu.org/qemu.git /tmp/qemu && \
> @@ -194,9 +177,9 @@ USER uboot:uboot
>
>  # Create the buildman config file
>  RUN /bin/echo -e "[toolchain]\nroot = /usr" > ~/.buildman
> -RUN /bin/echo -e "kernelorg = /opt/gcc-9.2.0-nolibc/*" >> ~/.buildman
> -RUN /bin/echo -e "arc = /opt/arc_gnu_2019.09_prebuilt_uclibc_le_archs_linux_install" >> ~/.buildman
> -RUN /bin/echo -e "\n[toolchain-prefix]\nxtensa = /opt/2018.02/xtensa-dc233c-elf/bin/xtensa-dc233c-elf-" >> ~/.buildman;
> +RUN /bin/echo -e "kernelorg = /opt/gcc-11.1.0-nolibc/*" >> ~/.buildman
> +RUN /bin/echo -e "arc = /opt/arc_gnu_2021.03_prebuilt_uclibc_le_archs_linux_install" >> ~/.buildman
> +RUN /bin/echo -e "\n[toolchain-prefix]\nxtensa = /opt/2020.07/xtensa-dc233c-elf/bin/xtensa-dc233c-elf-" >> ~/.buildman;
>  RUN /bin/echo -e "\nnds32 = /opt/nds32le-linux-glibc-v3-upstream/bin/nds32le-linux-" >> ~/.buildman;
>  RUN /bin/echo -e "\n[toolchain-alias]\nsh = sh2" >> ~/.buildman
>  RUN /bin/echo -e "\nriscv = riscv64" >> ~/.buildman

Reviewed-by: Bin Meng <bmeng.cn at gmail.com>