[U-Boot] [PATCH 6/6] moveconfig: Support looking for implied CONFIG options
Heiko Schocher
hs at denx.de
Tue May 16 03:40:37 UTC 2017
Hello Simon,
Am 15.05.2017 um 13:47 schrieb Simon Glass:
> Some CONFIG options can be implied by others and this can help to reduce
> the size of the defconfig files. For example, CONFIG_X86 implies
> CONFIG_CMD_IRQ, so we can put 'imply CMD_IRQ' under 'config X86' and
> all x86 boards will have that option, avoiding adding CONFIG_CMD_IRQ to
> each of the x86 defconfig files.
>
> Add a -i option which searches for such options.
>
> Signed-off-by: Simon Glass <sjg at chromium.org>
> ---
>
> tools/moveconfig.py | 215 +++++++++++++++++++++++++++++++++++++++++++++++++++-
> 1 file changed, 214 insertions(+), 1 deletion(-)
Thanks!
Reviewed-by: Heiko Schocher <hs at denx.de>
bye,
Heiko
>
> diff --git a/tools/moveconfig.py b/tools/moveconfig.py
> index ed576f4b83..f33203d51b 100755
> --- a/tools/moveconfig.py
> +++ b/tools/moveconfig.py
> @@ -128,6 +128,69 @@ To process CONFIG_CMD_FPGAD only for a subset of configs based on path match:
> ./tools/moveconfig.py -Cy CONFIG_CMD_FPGAD -d -
>
>
> +Finding implied CONFIGs
> +-----------------------
> +
> +Some CONFIG options can be implied by others and this can help to reduce
> +the size of the defconfig files. For example, CONFIG_X86 implies
> +CONFIG_CMD_IRQ, so we can put 'imply CMD_IRQ' under 'config X86' and
> +all x86 boards will have that option, avoiding adding CONFIG_CMD_IRQ to
> +each of the x86 defconfig files.
> +
> +This tool can help find such configs. To use it, first build a database:
> +
> + ./tools/moveconfig.py -b
> +
> +Then try to query it:
> +
> + ./tools/moveconfig.py -i CONFIG_CMD_IRQ
> + CONFIG_CMD_IRQ found in 311/2384 defconfigs
> + 44 : CONFIG_SYS_FSL_ERRATUM_IFC_A002769
> + 41 : CONFIG_SYS_FSL_ERRATUM_A007075
> + 31 : CONFIG_SYS_FSL_DDR_VER_44
> + 28 : CONFIG_ARCH_P1010
> + 28 : CONFIG_SYS_FSL_ERRATUM_P1010_A003549
> + 28 : CONFIG_SYS_FSL_ERRATUM_SEC_A003571
> + 28 : CONFIG_SYS_FSL_ERRATUM_IFC_A003399
> + 25 : CONFIG_SYS_FSL_ERRATUM_A008044
> + 22 : CONFIG_ARCH_P1020
> + 21 : CONFIG_SYS_FSL_DDR_VER_46
> + 20 : CONFIG_MAX_PIRQ_LINKS
> + 20 : CONFIG_HPET_ADDRESS
> + 20 : CONFIG_X86
> + 20 : CONFIG_PCIE_ECAM_SIZE
> + 20 : CONFIG_IRQ_SLOT_COUNT
> + 20 : CONFIG_I8259_PIC
> + 20 : CONFIG_CPU_ADDR_BITS
> + 20 : CONFIG_RAMBASE
> + 20 : CONFIG_SYS_FSL_ERRATUM_A005871
> + 20 : CONFIG_PCIE_ECAM_BASE
> + 20 : CONFIG_X86_TSC_TIMER
> + 20 : CONFIG_I8254_TIMER
> + 20 : CONFIG_CMD_GETTIME
> + 19 : CONFIG_SYS_FSL_ERRATUM_A005812
> + 18 : CONFIG_X86_RUN_32BIT
> + 17 : CONFIG_CMD_CHIP_CONFIG
> + ...
> +
> +This shows a list of config options which might imply CONFIG_CMD_EEPROM along
> +with how many defconfigs they cover. From this you can see that CONFIG_X86
> +implies CONFIG_CMD_EEPROM. Therefore, instead of adding CONFIG_CMD_EEPROM to
> +the defconfig of every x86 board, you could add a single imply line to the
> +Kconfig file:
> +
> + config X86
> + bool "x86 architecture"
> + ...
> + imply CMD_EEPROM
> +
> +That will cover 20 defconfigs. Many of the options listed are not suitable as
> +they are not related. E.g. it would be odd for CONFIG_CMD_GETTIME to imply
> +CMD_EEPROM.
> +
> +Using this search you can reduce the size of moveconfig patches.
> +
> +
> Available options
> -----------------
>
> @@ -191,6 +254,7 @@ To see the complete list of supported options, run
>
> """
>
> +import collections
> import copy
> import difflib
> import filecmp
> @@ -1395,6 +1459,148 @@ def move_config(configs, options, db_queue):
> slots.show_failed_boards()
> slots.show_suspicious_boards()
>
> +def imply_config(config_list, find_superset=False):
> + """Find CONFIG options which imply those in the list
> +
> + Some CONFIG options can be implied by others and this can help to reduce
> + the size of the defconfig files. For example, CONFIG_X86 implies
> + CONFIG_CMD_IRQ, so we can put 'imply CMD_IRQ' under 'config X86' and
> + all x86 boards will have that option, avoiding adding CONFIG_CMD_IRQ to
> + each of the x86 defconfig files.
> +
> + This function uses the moveconfig database to find such options. It
> + displays a list of things that could possibly imply those in the list.
> + The algorithm ignores any that start with CONFIG_TARGET since these
> + typically refer to only a few defconfigs (often one). It also does not
> + display a config with less than 5 defconfigs.
> +
> + The algorithm works using sets. For each target config in config_list:
> + - Get the set 'defconfigs' which use that target config
> + - For each config (from a list of all configs):
> + - Get the set 'imply_defconfig' of defconfigs which use that config
> + -
> + - If imply_defconfigs contains anything not in defconfigs then
> + this config does not imply the target config
> +
> + Params:
> + config_list: List of CONFIG options to check (each a string)
> + find_superset: True to look for configs which are a superset of those
> + already found. So for example if CONFIG_EXYNOS5 implies an option,
> + but CONFIG_EXYNOS covers a larger set of defconfigs and also
> + implies that option, this will drop the former in favour of the
> + latter. In practice this option has not proved very used.
> +
> + Note the terminoloy:
> + config - a CONFIG_XXX options (a string, e.g. 'CONFIG_CMD_EEPROM')
> + defconfig - a defconfig file (a string, e.g. 'configs/snow_defconfig')
> + """
> + # key is defconfig name, value is dict of (CONFIG_xxx, value)
> + config_db = {}
> +
> + # Holds a dict containing the set of defconfigs that contain each config
> + # key is config, value is set of defconfigs using that config
> + defconfig_db = collections.defaultdict(set)
> +
> + # Set of all config options we have seen
> + all_configs = set()
> +
> + # Set of all defconfigs we have seen
> + all_defconfigs = set()
> +
> + # Read in the database
> + configs = {}
> + with open(CONFIG_DATABASE) as fd:
> + for line in fd.readlines():
> + line = line.rstrip()
> + if not line: # Separator between defconfigs
> + config_db[defconfig] = configs
> + all_defconfigs.add(defconfig)
> + configs = {}
> + elif line[0] == ' ': # CONFIG line
> + config, value = line.strip().split('=', 1)
> + configs[config] = value
> + defconfig_db[config].add(defconfig)
> + all_configs.add(config)
> + else: # New defconfig
> + defconfig = line
> +
> + # Work through each target config option in tern, independently
> + for config in config_list:
> + defconfigs = defconfig_db.get(config)
> + if not defconfigs:
> + print '%s not found in any defconfig' % config
> + continue
> +
> + # Get the set of defconfigs without this one (since a config cannot
> + # imply itself)
> + non_defconfigs = all_defconfigs - defconfigs
> + num_defconfigs = len(defconfigs)
> + print '%s found in %d/%d defconfigs' % (config, num_defconfigs,
> + len(all_configs))
> +
> + # This will hold the results: key=config, value=defconfigs containing it
> + imply_configs = {}
> + rest_configs = all_configs - set([config])
> +
> + # Look at every possible config, except the target one
> + for imply_config in rest_configs:
> + if 'CONFIG_TARGET' in imply_config:
> + continue
> +
> + # Find set of defconfigs that have this config
> + imply_defconfig = defconfig_db[imply_config]
> +
> + # Get the intersection of this with defconfigs containing the
> + # target config
> + common_defconfigs = imply_defconfig & defconfigs
> +
> + # Get the set of defconfigs containing this config which DO NOT
> + # also contain the taret config. If this set is non-empty it means
> + # that this config affects other defconfigs as well as (possibly)
> + # the ones affected by the target config. This means it implies
> + # things we don't want to imply.
> + not_common_defconfigs = imply_defconfig & non_defconfigs
> + if not_common_defconfigs:
> + continue
> +
> + # If there are common defconfigs, imply_config may be useful
> + if common_defconfigs:
> + skip = False
> + if find_superset:
> + for prev in imply_configs.keys():
> + prev_count = len(imply_configs[prev])
> + count = len(common_defconfigs)
> + if (prev_count > count and
> + (imply_configs[prev] & common_defconfigs ==
> + common_defconfigs)):
> + # skip imply_config because prev is a superset
> + skip = True
> + break
> + elif count > prev_count:
> + # delete prev because imply_config is a superset
> + del imply_configs[prev]
> + if not skip:
> + imply_configs[imply_config] = common_defconfigs
> +
> + # Now we have a dict imply_configs of configs which imply each config
> + # The value of each dict item is the set of defconfigs containing that
> + # config. Rank them so that we print the configs that imply the largest
> + # number of defconfigs first.
> + ranked_configs = sorted(imply_configs,
> + key=lambda k: len(imply_configs[k]), reverse=True)
> + for config in ranked_configs:
> + num_common = len(imply_configs[config])
> +
> + # Don't bother if there are less than 5 defconfigs affected.
> + if num_common < 5:
> + continue
> + missing = defconfigs - imply_configs[config]
> + missing_str = ', '.join(missing) if missing else 'all'
> + missing_str = ''
> + print ' %d : %-30s%s' % (num_common, config.ljust(30),
> + missing_str)
> +
> +
> def main():
> try:
> cpu_count = multiprocessing.cpu_count()
> @@ -1413,6 +1619,8 @@ def main():
> help='a file containing a list of defconfigs to move, '
> "one per line (for example 'snow_defconfig') "
> "or '-' to read from stdin")
> + parser.add_option('-i', '--imply', action='store_true', default=False,
> + help='find options which imply others')
> parser.add_option('-n', '--dry-run', action='store_true', default=False,
> help='perform a trial run (show log with no changes)')
> parser.add_option('-e', '--exit-on-error', action='store_true',
> @@ -1437,7 +1645,8 @@ def main():
>
> (options, configs) = parser.parse_args()
>
> - if len(configs) == 0 and not any((options.force_sync, options.build_db)):
> + if len(configs) == 0 and not any((options.force_sync, options.build_db,
> + options.imply)):
> parser.print_usage()
> sys.exit(1)
>
> @@ -1447,6 +1656,10 @@ def main():
>
> check_top_directory()
>
> + if options.imply:
> + imply_config(configs)
> + return
> +
> config_db = {}
> db_queue = Queue.Queue()
> t = DatabaseThread(config_db, db_queue)
>
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
More information about the U-Boot
mailing list