[U-Boot] [PATCH v2 00/31] binman: Allow reading of images to list contents

[Simon Glass]

This series adds features to allow binman to read its own images. Using
an 'FDT map' inside the image (basically a cut-down copy of the input
config with some pieces added) it can reverse-engineer the image and
figure out what is where.

Uses of this feature are

- a new 'ls' command, which allows listing the entries in an image
- a new 'extract' command, which allows entry contents to be extracted and
    written to one or more files

Other improvements along the way include:
- Support for FDT map
- Support for an image header, which points to the FDT map
- Ability to deal with entries which change size after packing
- Additional error checking and FDT features for CBFS
- Convert from OptionParser to ArgumentParser
- Refactor to reduce code differences between entries and sections

Future work will allow reading files from the image, but I've decided to
put that in the next series.

Changes in v2:
- Add patches to enhance the 'ls' command to filter which entries are shown
- Add patches to implement the 'extract' command
- Add the Entry object to EntryInfo as well
- Adjust LocateFdtmap() to return the position of the header
- Adjust the test to not rely on lz4 compression version
- Allow listing subsets of the image
- Avoid dual assignment in ResetForPack()
- Change Image.ListEntries() to BuildEntryList() since signature differs
- Change list command from 'list' to 'ls'
- Deal with travis's old lz4 version by skipping tests as necessary
- Don't allow global arguments after the command
- Fix up arguments in main Makefile
- Move 129_list_fdtmap.dts into the next commit
- Move patch 'Add a comment about CBFS test structure' to earlier series
- Move patch 'Allow cbfstool to be optional with tests' to earlier series
- Pass verbosity to tests via the setup_test_args() method
- Put --toolpath in the right place in functional tests
- Rename 128_decode_image_no_hdr.dts to 129_decode_image_nohdr.dts
- Show a message when a section's size changes
- Update Image for LocateFdtmap() returning the position of the header
- Update commit subject to more accurately describe this patch
- Update output from 'list' command
- Update test to use _DoReadFileRealDtb() helper
- Use 3 packing passes instead of two, and add a comment
- Use bytearray for Python 3

Simon Glass (31):
  binman: Simplify the entry test
  binman: Update future features
  binman: Update help for new features
  binman: Add a convenience functions for real-DTB tests
  binman: Add an FDT map
  binman: Add an image header
  binman: Convert to use ArgumentParser
  binman: Move compression into the Entry base class
  binman: Drop an unused arg in Entry.Lookup()
  binman: Allow easy importing of entry modules
  binman: Fix up ProcessUpdateContents error and comments
  binman: Call ProcessUpdateContents() consistently
  binman: Add a return value to ProcessContentsUpdate()
  binman: Add a control for post-pack entry expansion
  binman: Allow entries to expand after packing
  binman: Allow device-tree entries to be compressed
  binman: Provide the actual data address for cbfs files
  binman: Use the cbfs memlen field only for uncompressed length
  binman: Support FDT update for CBFS
  binman: Detect bad CBFS file types
  binman: Allow listing the entries in an image
  binman: Support locating an FDT map
  binman: Support locating an image header
  binman: Support reading an image into an Image object
  binman: Convert Image to a subclass of Entry
  binman: Support listing an image
  binman: Allow for logging information to be displayed
  binman: Allow reading an entry from an image
  binman: Support reading from CBFS entries
  binman: Add an 'extract' command
  binman: Add a test for nested and aligned sections

 .travis.yml                                   |   2 +-
 Makefile                                      |   4 +-
 test/run                                      |   4 +-
 tools/binman/README                           | 129 ++-
 tools/binman/README.entries                   |  78 +-
 tools/binman/binman.py                        |  47 +-
 tools/binman/bsection.py                      | 496 -----------
 tools/binman/cbfs_util.py                     |  56 +-
 tools/binman/cmdline.py                       | 101 ++-
 tools/binman/control.py                       | 213 ++++-
 tools/binman/entry.py                         | 149 +++-
 tools/binman/entry_test.py                    |  32 +-
 tools/binman/etype/__init__.py                |   0
 tools/binman/etype/_testing.py                |  14 +-
 tools/binman/etype/blob.py                    |  51 +-
 tools/binman/etype/blob_dtb.py                |  10 +-
 tools/binman/etype/cbfs.py                    |  64 +-
 tools/binman/etype/fdtmap.py                  | 130 +++
 tools/binman/etype/files.py                   |   3 +-
 tools/binman/etype/fmap.py                    |   4 +-
 tools/binman/etype/image_header.py            |  99 +++
 tools/binman/etype/section.py                 | 439 +++++++++-
 tools/binman/etype/u_boot_with_ucode_ptr.py   |   8 +-
 tools/binman/ftest.py                         | 811 ++++++++++++++++--
 tools/binman/image.py                         | 315 +++++--
 tools/binman/image_test.py                    |  18 +-
 tools/binman/state.py                         |  26 +-
 tools/binman/test/115_fdtmap.dts              |  13 +
 tools/binman/test/116_fdtmap_hdr.dts          |  17 +
 tools/binman/test/117_fdtmap_hdr_start.dts    |  19 +
 tools/binman/test/118_fdtmap_hdr_pos.dts      |  19 +
 tools/binman/test/119_fdtmap_hdr_missing.dts  |  16 +
 tools/binman/test/120_hdr_no_location.dts     |  16 +
 tools/binman/test/121_entry_expand.dts        |  20 +
 tools/binman/test/122_entry_expand_twice.dts  |  21 +
 .../binman/test/123_entry_expand_section.dts  |  22 +
 tools/binman/test/124_compress_dtb.dts        |  14 +
 tools/binman/test/125_cbfs_update.dts         |  21 +
 tools/binman/test/126_cbfs_bad_type.dts       |  17 +
 tools/binman/test/127_list.dts                |  33 +
 tools/binman/test/128_decode_image.dts        |  36 +
 tools/binman/test/129_decode_image_nohdr.dts  |  33 +
 tools/binman/test/130_list_fdtmap.dts         |  36 +
 tools/binman/test/131_pack_align_section.dts  |  28 +
 tools/patman/test_util.py                     |   6 +-
 tools/patman/tout.py                          |  10 +-
 46 files changed, 2787 insertions(+), 913 deletions(-)
 delete mode 100644 tools/binman/bsection.py
 create mode 100644 tools/binman/etype/__init__.py
 create mode 100644 tools/binman/etype/fdtmap.py
 create mode 100644 tools/binman/etype/image_header.py
 create mode 100644 tools/binman/test/115_fdtmap.dts
 create mode 100644 tools/binman/test/116_fdtmap_hdr.dts
 create mode 100644 tools/binman/test/117_fdtmap_hdr_start.dts
 create mode 100644 tools/binman/test/118_fdtmap_hdr_pos.dts
 create mode 100644 tools/binman/test/119_fdtmap_hdr_missing.dts
 create mode 100644 tools/binman/test/120_hdr_no_location.dts
 create mode 100644 tools/binman/test/121_entry_expand.dts
 create mode 100644 tools/binman/test/122_entry_expand_twice.dts
 create mode 100644 tools/binman/test/123_entry_expand_section.dts
 create mode 100644 tools/binman/test/124_compress_dtb.dts
 create mode 100644 tools/binman/test/125_cbfs_update.dts
 create mode 100644 tools/binman/test/126_cbfs_bad_type.dts
 create mode 100644 tools/binman/test/127_list.dts
 create mode 100644 tools/binman/test/128_decode_image.dts
 create mode 100644 tools/binman/test/129_decode_image_nohdr.dts
 create mode 100644 tools/binman/test/130_list_fdtmap.dts
 create mode 100644 tools/binman/test/131_pack_align_section.dts

-- 
2.22.0.410.gd8fdbe21b5-goog