Skip to content

rsa: keep RSA-PSS keys without parameters unrestricted - #483

Open
yosuke-wolfssl wants to merge 1 commit into
wolfSSL:masterfrom
yosuke-wolfssl:fix/f_12483
Open

yosuke-wolfssl wants to merge 1 commit into
wolfSSL:masterfrom
yosuke-wolfssl:fix/f_12483

Conversation

@yosuke-wolfssl

Copy link
Copy Markdown
Contributor

Problem

An RSA-PSS SubjectPublicKeyInfo or PKCS#8 PrivateKeyInfo whose
AlgorithmIdentifier carries no RSASSA-PSS-params is unrestricted: any
supported digest may be used with it. wp_rsa_pss_get_params() had no way to
tell "parameters absent" from "parameters present" — it never learned where the
AlgorithmIdentifier ended — so it installed the RFC 8017 defaults and set
pssDefSet unconditionally. An unrestricted key decoded as restricted to
SHA-1/MGF1-SHA-1/salt 20.

Path Before After
d2i_PUBKEY_ex on absent-params SPKI loads, pinned to SHA-1 loads, unrestricted
d2i_AutoPrivateKey_ex on absent-params PKCS#8 decode fails outright loads, unrestricted
i2d_PUBKEY of such a key injects RSASSA-PSS-params bare OID, byte-identical

Valid signatures over another supported digest were rejected after decoding.

Fix (src/wp_rsa_kmgmt.c)

  • algIdEnd takes the AlgorithmIdentifier SEQUENCE's own length octet as
    the end of its content, making presence decidable. This replaces a walk that
    stepped over the SPKI BIT STRING and landed inside RSAPublicKey, matched no
    parameter tag, and still reported success.
  • wp_rsa_pss_encode_key_alg_id() encodes the OID alone for an unrestricted
    key. Key encoders use it; the signature AlgorithmIdentifier keeps carrying the
    digest, MGF and salt the signature was actually made with.
  • NULL parameters are rejected, matching ossl_rsa_param_decode().

In src/wp_rsa_sig.c, minSaltLen now uses -1 for "unrestricted" — the
sentinel wp_rsa_setup_md() already tested for but which was never assigned, so
the restriction check fired for every PSS key.

Empty parameters (30 00) still decode as restricted to SHA-1, per the ASN.1
defaults. Closes f-12483.

Tests

  • test_rsa_pss_no_params — decodes both formats, checks the default digest
    is SHA256 and no salt length is reported, signs and verifies under SHA-256/384/512
    in both directions against OpenSSL, asserts byte-exact round-trips, and confirms a
    restricted key is still pinned.
  • test_rsa_sig_alg_id — now A/Bs the PSS signature AlgorithmIdentifier
    against OpenSSL across three digests.

Verification

  • 217/217 unit tests; RSA CLI suite green; ASan + UBSan clean.
  • Interops with the upstream evppkey_rsa.txt unrestricted PSS vector.
  • Negative controls: on base, test_rsa_pss_no_params fails; reverting the
    encoder split makes the alg-id test report 13 bytes against OpenSSL's 67.

@yosuke-wolfssl yosuke-wolfssl self-assigned this Sep 2, 2026
Copilot AI lite review requested due to automatic review settings September 2, 2026 23:57

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

It changes security- and interoperability-critical DER parsing/encoding paths for RSA-PSS keys, so a final human review is warranted even with strong tests.

Pull request overview

This PR fixes RSA-PSS key decode/encode behavior so that keys whose AlgorithmIdentifier omits RSASSA-PSS-params remain unrestricted (rather than being implicitly pinned to RFC 8017 defaults like SHA-1), and ensures the encoded AlgorithmIdentifier round-trips byte-identically for those keys.

Changes:

  • Update RSA-PSS parameter detection/decoding to distinguish “params absent” vs “params present” and only apply defaults when params are actually present (src/wp_rsa_kmgmt.c).
  • Split key AlgorithmIdentifier encoding so unrestricted RSA-PSS keys encode as a bare OID (no injected params), while signature alg-ids still carry digest/MGF/salt (src/wp_rsa_kmgmt.c).
  • Fix PSS restriction handling by initializing/copying minSaltLen with the intended “unrestricted” sentinel and add unit tests covering unrestricted PSS keys and signature alg-id parity with OpenSSL (src/wp_rsa_sig.c, test/test_rsa.c, test/unit.*).
File summaries
File Description
test/unit.h Adds the new unit test prototype to the test registry header.
test/unit.c Registers the new RSA-PSS “no params” unit test in the test list.
test/test_rsa.c Adds DER fixtures and new test logic for unrestricted RSA-PSS keys and PSS signature AlgorithmIdentifier A/B vs OpenSSL.
src/wp_rsa_sig.c Initializes and propagates minSaltLen so unrestricted PSS keys don’t incorrectly trigger restriction checks.
src/wp_rsa_kmgmt.c Corrects RSA-PSS param presence detection and adjusts key AlgorithmIdentifier encoding for unrestricted keys.
Review details
  • Files reviewed: 5/5 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/wp_rsa_kmgmt.c Outdated

@wolfSSL-Fenrir-bot wolfSSL-Fenrir-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fenrir Automated Review — PR #483

Scan targets checked: wolfprovider-bugs, wolfprovider-src

Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Reported findings require changes before merge.

Comment thread src/wp_rsa_kmgmt.c
- wp_rsa_pss_get_params() takes the AlgorithmIdentifier's length
  octet as the end of its content, reads a SEQUENCE after the OID as
  parameters present and nothing at all as absent, and rejects any
  other tag. The defaults, the hash, MGF and salt blocks and
  rsa->pssDefSet apply only when parameters are present. The BIT
  STRING step-over is removed.
- wp_rsa_pss_encode_key_alg_id() encodes the OID alone for a key with
  no PSS restrictions, else defers to wp_rsa_pss_encode_alg_id(); the
  six call sites in the SPKI and PKCS#8 encoders use it. rsa_pss_oid
  moves to file scope and both encoders compute their length delta
  with signed operands.
- wp_rsa_ctx_new() sets minSaltLen to -1 and wp_rsa_ctx_dup() copies
  it. For a key with no PSS restrictions wp_rsa_signverify_init()
  uses minSaltLen -1, and saltLen RSA_PSS_SALTLEN_AUTO where
  RSA_PSS_SALTLEN_AUTO_DIGEST_MAX is unavailable.
- test_rsa.c gains rsa_pss_key_der_2048_pkcs8_noparams and
  test_rsa_pss_no_params(), covering an empty parameters SEQUENCE
  restricting the key and a duplicated context keeping the salt
  length restriction; test_rsa_sig_alg_id() compares the PSS
  signature AlgorithmIdentifier against OpenSSL.

Issue: F-12483
Comment thread src/wp_rsa_kmgmt.c

@wolfSSL-Fenrir-bot wolfSSL-Fenrir-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fenrir Automated Review — PR #483

Scan targets checked: wolfprovider-bugs, wolfprovider-src

Fenrir result: Approved ✅

No new issues found in the changed files.

Advisory only — this automated result does not count as a GitHub approval.

@wolfSSL-Fenrir-bot
wolfSSL-Fenrir-bot dismissed their stale review September 3, 2026 03:08

Fenrir's latest completed scan found no issues; clearing the prior automated change request.

@ColtonWilley ColtonWilley added the ci:all PR OSP toggle: run all label Sep 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:all PR OSP toggle: run all

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants