[PATCH v5 0/2] binman: add PKCS#11/HSM signing support for X509 certificates

[Sergio Prado]

Motivation
----------

TI K3 secure boot requires X509 certificates to be signed with a private
key at build time. Until now, binman expected this key to be present as
a plain PEM file on the build host. For production use, however, private
keys should never exist unprotected on a build machine — they belong
inside a Hardware Security Module (HSM) that enforces access control and
keeps the key material unexportable.

This series adds support for signing X509 certificates via any PKCS#11-
capable HSM (YubiKey, TPM, network HSM, SoftHSM2 for development, etc.).

Design
------

The 'keyfile' entry argument of x509_cert (and its TI K3 subclasses)
now accepts either a filesystem path to a PEM key, as before, or a
PKCS#11 URI referring to a key stored in an HSM. The URI is forwarded
as-is to 'openssl -key', which resolves it via the pkcs11 provider (or
engine) configured externally through OPENSSL_CONF. This keeps binman
itself free of any PKCS#11-specific knowledge.

A new make variable, BINMAN_X509_KEY_URI, overrides the 'keyfile' entry
argument for all x509 certificate signing operations:

    make BINMAN_X509_KEY_URI="pkcs11:token=mytk;object=mykey;type=private" \
         OPENSSL_CONF=/path/to/openssl.cnf

The recommended way to deliver the PIN for non-interactive signing is
to configure pkcs11-module-token-pin in openssl.cnf under the pkcs11
provider section. As a convenience fallback, the PKCS11_PIN environment
variable can be set; its value is appended to the URI as a percent-
encoded ?pin-value=<pin> (per RFC 7512) so the PIN does not have to be
embedded in BINMAN_X509_KEY_URI.

Two URI forms are supported on OpenSSL 3.x:

  - Provider path (recommended), via the pkcs11-provider package
  - Engine path, prefixed with org.openssl.engine:<engine_name>: so
    OpenSSL 3.x's STORE API routes it to the engine

Testing
-------

Tested on a Toradex Verdin AM62 (verdin-am62_a53_defconfig) board with
both the engine path and the pkcs11 provider path, using SoftHSM2 and
YubiKey 5 NFC.

A SoftHSM2-based integration test (testX509CertPkcs11) exercises the
full signing path end-to-end against an in-tree test key; it skips
cleanly when the OpenSSL pkcs11 provider is not installed. A unit test
(testX509CertBuildPkcs11Key) covers the URI/PIN combiner including
percent-encoding of reserved characters.

Changes in v5:
- Split the Entry_x509_cert docstring expansion into a separate
  preparatory commit (Simon)
- Unify URI support under the existing 'keyfile' entry argument
  rather than introducing a new 'x509-key-uri' arg; drop the
  Entry_x509_cert state and override logic that v4 added (Quentin)
- Percent-encode the PIN before appending to the URI using
  urllib.parse.quote(), per RFC 7512, and add a unit test
  covering PINs with reserved characters (Simon)
- Detect the OpenSSL pkcs11 provider via
  'openssl list -providers -provider pkcs11' rather than discovering
  the provider .so path manually; drops the 'openssl version -m'
  MODULESDIR lookup (Quentin)
- Drop the 'p11-kit print-config' / softhsm2 .so discovery from the
  test; the simpler openssl.cnf relies on softhsm2 being registered
  with p11-kit globally (Quentin). As a side effect this also fixes
  the test on Ubuntu 22.04, where 'p11-kit print-config' is not a
  valid subcommand
- Move the test openssl.cnf in-tree as
  tools/binman/test/fit/openssl_provider.conf and trim it to the
  minimum needed (no 'module = ', no 'pkcs11-module-path = ') (Quentin)
- Replace 'pkcs11-tool --keypairgen' with 'softhsm2-util --import' of
  tools/binman/test/fit/rsa2048.key so the test runs faster and drops
  the pkcs11-tool dependency (Quentin)
- Recommend pkcs11-module-token-pin in openssl.cnf as the primary
  way to deliver the PIN; PKCS11_PIN env var is now documented as
  the convenience fallback (Quentin)
- Drop the incorrect claim that PKCS11_PIN keeps the PIN out of
  shell history (Quentin)
- Rephrase the URI intro in binman.rst and clarify that
  'pkcs11-provider' is a Debian package name, not a path (Quentin)
- Drop the inheritance notes added to Entry_ti_secure and
  Entry_ti_secure_rom in v4; with the v5 keyfile unification the
  original motivation (an inherited x509-key-uri property) no
  longer applies

Changes in v4:
- Drop the v3 bintool extra_env commit entirely; binman no longer
  sets any PKCS#11-related environment variables (Quentin)
- Drop BINMAN_PKCS11_MODULE / pkcs11-module entry argument; the
  PKCS#11 module path must be configured externally via openssl.cnf
  (Quentin)
- Drop provider/engine auto-detection (_pkcs11_use_provider,
  _build_key_args, _run_cmd_pkcs11) along with the threading.Lock;
  the user selects provider or engine via OPENSSL_CONF and the URI
  form (Quentin)
- Rename the v3 BINMAN_PKCS11_URI / pkcs11-uri to BINMAN_X509_KEY_URI
  / x509-key-uri to scope the names to x509 certificate entries
  without locking them to a specific URI scheme (Quentin)
- Document that the engine path is supported on OpenSSL 3.x by
  prefixing the URI with org.openssl.engine:<engine_name>: (Quentin)
- Replace the mocked openssl test with a real SoftHSM2-based
  integration test using the provider path and OPENSSL_CONF (Quentin)
- Use 'p11-kit print-config' to locate the softhsm2 library at test
  time instead of hardcoding a distro-specific path (Quentin)
- Use 'openssl version -m' (MODULESDIR) to locate the OpenSSL pkcs11
  provider .so file, so multiarch paths like
  /usr/lib/x86_64-linux-gnu/ossl-modules on Debian/Ubuntu are handled
  correctly
- Generate the test RSA keypair with pkcs11-tool; softhsm2-util has
  no key-generation action (only --import) and silently exits 0 on an
  unknown --generate-keypair option, which would leave the token
  empty and make the openssl step fail with 'Could not read private
  key'
- Add self._CheckBintool() to all PKCS#11 test paths so tests skip
  cleanly when bintools are missing (Quentin)
- Extract the URI/PIN combiner into Entry_x509_cert._build_pkcs11_key()
  and add a unit test for it
- Document that PKCS11_PIN keeps the PIN out of the make command line
  but is still visible in 'ps' output via the openssl invocation; for
  improved isolation, configure the PIN in openssl.cnf
- Document all Entry_x509_cert properties (content, keyfile,
  x509-key-uri, cert-ca, cert-revision-int, sw-rev) in its docstring
  (Quentin)
- Add inheritance notes to Entry_ti_secure and Entry_ti_secure_rom
  docstrings, pointing out that they extend Entry_x509_cert via
  super() and therefore accept its properties (notably x509-key-uri)
  (Quentin)

Changes in v3:
- Split into two patches: bintool infrastructure (1/2) and x509_cert
  feature (2/2)
- Fix global environment mutation: _run_cmd_pkcs11() no longer writes
  to os.environ directly; it now uses the new extra_env parameter so
  module paths are scoped to the subprocess only, which is both
  cleaner and safe under concurrent execution
- Add module-level threading.Lock to serialise concurrent PKCS#11
  signing calls and fix intermittent login failures caused by binman's
  ThreadPoolExecutor
- Fix URI query string separator: use '&' when the URI already
  contains '?' (e.g. module-path already present), '?' otherwise
- Test cases updated

Changes in v2:
- Add tests for _build_key_args() (PEM path, PKCS#11 provider, PKCS#11
  engine, PIN appending), _pkcs11_use_provider() (caching),
  _run_cmd_pkcs11() (with and without module path), and end-to-end
  x509_cert signing with a PKCS#11 URI (testX509CertPkcs11), ensuring
  btool/openssl.py and etype/x509_cert.py have 100% test coverage

Sergio Prado (2):
  binman: x509_cert: document Entry_x509_cert properties
  binman: x509_cert: support PKCS#11 URI in keyfile for HSM signing

 Makefile                                    |  1 +
 tools/binman/binman.rst                     | 55 ++++++++++++
 tools/binman/etype/x509_cert.py             | 45 ++++++++--
 tools/binman/ftest.py                       | 92 +++++++++++++++++++++
 tools/binman/test/fit/openssl_provider.conf | 14 ++++
 5 files changed, 202 insertions(+), 5 deletions(-)
 create mode 100644 tools/binman/test/fit/openssl_provider.conf

-- 
2.34.1