[PATCH v6 00/21] binman: Rework compression support

[Stefan Herbrechtsmeier]

From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier at weidmueller.com>

Rework the compression support of binman. Add a length header attribute
to dtb entry, manage the compression bintool in the entry class and add
support for several compression tools.

Changes in v6:
- Set uncomp_size after decompress in DecompressData function
- Set data with tools.get_bytes(0, 1024) instead returning

Changes in v5:
- Add commit to avoid duplicates in bintool lists
- Add commit to forward AddBintools calls
- Add commit to forward AddBintools calls to base class
- Add commit to collect bintools before usage
- Add commit to check only section data in multi section test
- Add commit to add DecompressData function to entry class
- Fix decompress
- Disable header in testInvalidCompress function
- Remove header value in testInvalidCompress
- Add commit to move compression bintools creation into test setup
- Add commit to select compression bintools in cbfs_util class
- Add commit to move compression bintool management into entry class
- Use record_missing_bintool function
- Use tools.get_bytes function
- Add support to DecompressData function
- Reuse 85_compress_section.dts file
- Remove 236_compress_dtb_missing_bintool.dts file
- Use self.comp_bintools dict
- Use _CheckBintool(bintool) function
- Remove self.fetch_package not None check in fetch function
- Add algorithm name to entry.py
- Add tool name to ftest.py
- Add algorithm name to entry.py
- Add tool name to ftest.py
- Add algorithm name to entry.py
- Add tool name to entry.py and ftest.py
- Add algorithm name to entry.py
- Add tool name to ftest.py
- Test that zstd compress of device-tree files failed
- Add algorithm name to entry.py
- Add tool name to ftest.py

Changes in v4:
- Add missing 236_compress_dtb_missing_bintool.dts file

Changes in v3:
- Move comp_util.py changes (drop with_header) into separate commits.
- Add commit to disable compressed data header
- Add commit to remove obsolete compressed data header handling
- Add commit to support missing compression tools
- Use 'tools.get_bytes(0, 64)' instead of 'bytes([0]) * 64'
- Check if tool is present
- Rename tests
- Document class properties

Changes in v2:
- Add commit to skip elf tests
- Reworked to make the feature optional.
- Add commit to add compression tests
- Add commit to add BintoolPacker class to bintool
- Add commit to add bzip2 bintool
- Add commit to add gzip bintool
- Add commit to add lzop bintool
- Add commit to add xz bintool
- Add commit to add zstd bintool

Stefan Herbrechtsmeier (21):
  binman: Skip elf tests if python elftools is not available
  binman: Avoid duplicates in bintool lists
  binman: Forward AddBintools calls to sub entries in cbfs_util
  binman: Forward AddBintools calls to base class
  binman: Collect bintools before usage
  binman: Check only section data in multi section test
  binman: Add DecompressData function to entry class
  binman: Add length header attribute to dtb entry
  binman: Disable compressed data header
  binman: Remove obsolete compressed data header handling
  binman: Move compression bintools creation into test setup
  binman: Select compression bintools in cbfs_util class
  binman: Move compression bintool management into entry class
  binman: Support missing compression tools
  binman: Add compression tests
  binman: Add BintoolPacker class to bintool
  binman: Add bzip2 bintool
  binman: Add gzip bintool
  binman: Add lzop bintool
  binman: Add xz bintool
  binman: Add zstd bintool

 tools/binman/bintool.py                       | 106 ++++++++++++
 tools/binman/btool/bzip2.py                   |  30 ++++
 tools/binman/btool/gzip.py                    |  31 ++++
 tools/binman/btool/lzop.py                    |  30 ++++
 tools/binman/btool/xz.py                      |  31 ++++
 tools/binman/btool/zstd.py                    |  30 ++++
 tools/binman/cbfs_util.py                     |  20 +--
 tools/binman/cbfs_util_test.py                |   4 +-
 tools/binman/comp_util.py                     |  76 ---------
 tools/binman/control.py                       |   5 +-
 tools/binman/elf_test.py                      |  14 ++
 tools/binman/entries.rst                      |   3 +
 tools/binman/entry.py                         |  54 ++++++-
 tools/binman/entry_test.py                    |   9 ++
 tools/binman/etype/blob_dtb.py                |  31 ++++
 tools/binman/etype/cbfs.py                    |   5 +
 tools/binman/etype/gbb.py                     |   1 +
 tools/binman/etype/intel_ifwi.py              |   1 +
 tools/binman/etype/mkimage.py                 |   1 +
 tools/binman/etype/section.py                 |   4 +-
 tools/binman/etype/vblock.py                  |   1 +
 tools/binman/ftest.py                         | 152 +++++++++++++++---
 .../test/235_compress_dtb_prepend_invalid.dts |  17 ++
 .../test/236_compress_dtb_prepend_length.dts  |  19 +++
 .../binman/test/237_compress_dtb_invalid.dts  |  16 ++
 tools/binman/test/238_compress_dtb_zstd.dts   |  16 ++
 26 files changed, 590 insertions(+), 117 deletions(-)
 create mode 100644 tools/binman/btool/bzip2.py
 create mode 100644 tools/binman/btool/gzip.py
 create mode 100644 tools/binman/btool/lzop.py
 create mode 100644 tools/binman/btool/xz.py
 create mode 100644 tools/binman/btool/zstd.py
 delete mode 100644 tools/binman/comp_util.py
 create mode 100644 tools/binman/test/235_compress_dtb_prepend_invalid.dts
 create mode 100644 tools/binman/test/236_compress_dtb_prepend_length.dts
 create mode 100644 tools/binman/test/237_compress_dtb_invalid.dts
 create mode 100644 tools/binman/test/238_compress_dtb_zstd.dts

-- 
2.30.2