TLS Fenrir fixes by Frauschi · Pull Request #11094 · wolfSSL/wolfssl

Frauschi · 2026-08-06T12:47:46Z

Three independent Fenrir fixes, each with a regression test that fails without its source change.

1. Honor the per-context X509_STORE_CTX verify callback (F-7341)

wolfSSL_X509_STORE_CTX_set_verify_cb() stored the callback in ctx->verify_cb, but every verification site read ctx->store->verify_cb, so the field was never consulted - a restrictive callback installed on the store context was silently ignored, and wolfSSL_X509_verify_cert() could succeed on a chain the callback would have rejected.

New X509StoreGetVerifyCb() prefers the context callback and falls back to the store's, matching OpenSSL precedence; it is used at all four call sites. wolfSSL_X509_STORE_CTX_init() now clears ctx->verify_cb so a reused context carries no stale callback.

Two notes: the pathLen and INVALID_CA overrides become reachable in OPENSSL_EXTRA builds, not just OPENSSL_ALL/WOLFSSL_QT, since the setter is guarded that way - this is the documented OpenSSL semantic. Rejection (callback returns 0) is honored only for the per-context callback; widening the store callback would be a separate behavior change.

2. Fix double free in `wolfSSL_X509_EXTENSION_set_data()` (F-7340)

The function freed the extension's dynamic ASN.1 string buffer but left value.data and value.isDynamic pointing at it. The following wolfSSL_ASN1_STRING_copy() snapshots those fields and frees the old buffer itself once the copy completes, so the stale pointer was freed twice. Any second call on a value of at least CTC_NAME_SIZE bytes hit this; passing the extension its own value also read freed memory. wolfSSL_ASN1_STRING_copy() is alias-safe, so the manual free is dropped.

3. Send a protocol_version alert when no version can be negotiated (F-7568)

DoClientHello() has four exits that fail with VERSION_ERROR when runtime version restrictions leave nothing acceptable at or below the offered version. Three sent no alert, the fourth only under WOLFSSL_EXTRA_ALERTS, so a default build just dropped the connection - the client could not tell a version mismatch from a network failure. RFC 5246 7.2.2 and RFC 8446 4.1.3 require a fatal protocol_version alert.

The generic fallback did not help: SendFatalAlertOnly() is a no-op without WOLFSSL_EXTRA_ALERTS, and where defined it grouped VERSION_ERROR with MATCH_SUITE_ERROR and sent handshake_failure, disagreeing with the TLS 1.3 mapping. All four branches now alert regardless of WOLFSSL_EXTRA_ALERTS (propagating SOCKET_ERROR_E), and VERSION_ERROR gets its own case in SendFatalAlertOnly(). Only observable on the TLS 1.2 path; TLS 1.3 already translates the error correctly.

Tests

Test	Group	Covers
`test_wolfSSL_X509_STORE_CTX_verify_cb`	`ossl_x509_store`	Context callback is consulted, wins over the store's, and is cleared by `X509_STORE_CTX_init()`
`test_wolfSSL_X509_EXTENSION_set_data`	`ossl_x509_ext`	Replacing a dynamic value twice, and setting a value from itself
`test_tls_no_acceptable_version_alert`	`tls`	Fatal `protocol_version` alert on the wire (no `WOLFSSL_EXTRA_ALERTS`)
`test_tls_version_error_alert_mapping`	`tls`	`SendFatalAlertOnly()` maps `VERSION_ERROR` to `protocol_version` (with `WOLFSSL_EXTRA_ALERTS`)

The two TLS tests are complementary: with WOLFSSL_EXTRA_ALERTS the fallback emits a byte-identical alert, so the first would pass either way.

wolfSSL-Fenrir-bot

Fenrir Automated Review — PR #11094

Scan targets checked: wolfcrypt-rs-bugs, wolfssl-bugs, wolfssl-src

Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Findings are non-blocking.

github-actions · 2026-08-06T13:08:36Z

MemBrowse Memory Report

gcc-arm-cortex-m4-openssl-compat

FLASH: .text +128 B (+0.0%, 773,164 B / 1,048,576 B, total: 74% used)

gcc-arm-cortex-m4-rsa-only

FLASH: .text +64 B (+0.0%, 327,392 B / 1,048,576 B, total: 31% used)
No memory changes detected for:
gcc-arm-cortex-m0plus
gcc-arm-cortex-m3
gcc-arm-cortex-m4
gcc-arm-cortex-m4-baremetal
gcc-arm-cortex-m4-crypto-only
gcc-arm-cortex-m4-dtls13
gcc-arm-cortex-m4-min-ecc
gcc-arm-cortex-m4-pkcs7
gcc-arm-cortex-m4-pq
gcc-arm-cortex-m4-sp-math
gcc-arm-cortex-m4-tls12
gcc-arm-cortex-m4-tls13
gcc-arm-cortex-m7
gcc-arm-cortex-m7-pq
gcc-arm-cortex-m7-tls13
linuxkm-pie
linuxkm-standard
stm32-sim-stm32h753

wolfSSL-Fenrir-bot

Fenrir Automated Review — PR #11094

Scan targets checked: wolfcrypt-rs-bugs, wolfssl-bugs, wolfssl-src

Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Findings are non-blocking.

wolfSSL-Fenrir-bot

Fenrir Automated Review — PR #11094

Scan targets checked: wolfcrypt-rs-bugs, wolfssl-bugs, wolfssl-src

Findings: 6
6 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Findings are non-blocking.

DoClientHello has four exits that fail with VERSION_ERROR when runtime version restrictions leave nothing acceptable at or below the version the client offered. Three of them sent no alert at all, and the fourth sent one only when WOLFSSL_EXTRA_ALERTS was defined, so a default build simply dropped the connection. The client could not tell a version mismatch from a network failure. The generic fallback did not help. SendFatalAlertOnly is a no-op unless WOLFSSL_EXTRA_ALERTS is defined, and where it is defined it grouped VERSION_ERROR with MATCH_SUITE_ERROR and sent handshake_failure. That also disagreed with the TLS 1.3 mapping, which already resolves VERSION_ERROR to protocol_version. Send a fatal protocol_version alert from all four branches regardless of WOLFSSL_EXTRA_ALERTS, and give VERSION_ERROR its own case in SendFatalAlertOnly so the generic path agrees. Note that this is only observable on the TLS 1.2 message path. A TLS 1.3 capable server routes the ClientHello through DoTls13HandShakeMsgType, which already translates the error into the right alert. Fixes F-7568.

The function freed the extension's dynamically allocated ASN.1 string buffer but left value.data and value.isDynamic pointing at it. The subsequent wolfSSL_ASN1_STRING_copy() call snapshots those fields before copying and frees the old buffer once the copy is complete, so the stale pointer was freed a second time. Any second call to wolfSSL_X509_EXTENSION_set_data() on an extension holding a value of at least CTC_NAME_SIZE bytes hit this, and passing the extension its own value made the copy read freed memory as well. wolfSSL_ASN1_STRING_set() already performs an alias safe replacement and disposes of the previous buffer itself, so drop the manual free. Add a regression test that replaces a dynamically allocated extension value and then sets the value from itself. Fixes F-7340.

wolfSSL_X509_STORE_CTX_set_verify_cb stored the application callback in ctx->verify_cb, but every verification site read ctx->store->verify_cb instead, so the field was never consulted. An application installing a restrictive callback on the store context, which is the OpenSSL documented way to enforce extra policy during verification, had it silently ignored, and wolfSSL_X509_verify_cert could report success on a chain the callback would have rejected. Add X509StoreGetVerifyCb, which prefers the context callback and falls back to the store one, and use it at all four call sites in X509StoreVerifyCert, X509StoreCheckPathLen and wolfSSL_X509_verify_cert. The store fallback keeps its OPENSSL_ALL or WOLFSSL_QT guard because the store field only exists there, while the call sites now follow the OPENSSL_EXTRA guard of the setter. Clear ctx->verify_cb in wolfSSL_X509_STORE_CTX_init along with the other per-verification state so a reused context does not carry a stale callback. A rejection also has to be reportable. When the certificate manager accepts a chain, ctx->error is X509_V_OK, so a callback that rejects it without recording an error of its own left wolfSSL_X509_verify_cert returning failure while X509_STORE_CTX_get_error still said the chain was fine. Record WOLFSSL_X509_V_ERR_UNSPECIFIED in that case, matching what OpenSSL reports, and only when the callback set no error itself. Add that value to the X509 error enum, where the openssl compatibility header already had the define. Feeding the rejection marker to SetupStoreCtxError is not an option there, since GetX509Error has no X509_V_ error for it and would pass the negative value through as the reported error. The OPENSSL_ALL date recheck did exactly that after a rejection, so skip that block once the callback has rejected, which also stops it from consulting the callback a second time. Add a regression test that verifies a good chain twice, once bare and once with a rejecting context callback, requires the second attempt to fail, and checks the reported error both when the callback records one and when it does not. Fixes F-7341.

dgarske · 2026-08-06T20:34:12Z

                WOLFSSL_MSG("Client trying to connect with lesser version");
                ret = VERSION_ERROR;
+                /* propagate socket errors to avoid re-calling send alert */
+                if (SendAlert(ssl, alert_fatal,


This can produce a duplicate fatal alert.
Recommendation: Add the ssl->alert_history.last_tx.level != alert_fatal guard — preferably at src/tls13.c:14896, since that fixes the whole class of problem for every error DoClientHello alerts on directly (it already double-sends for decode_error, illegal_parameter and inappropriate_fallback via the same route); alternatively guard the four new SendAlert() calls in DoClientHello.

dgarske · 2026-08-06T20:35:09Z

     */
 enum {
    WOLFSSL_X509_V_OK                                    = 0,
+    WOLFSSL_X509_V_ERR_UNSPECIFIED                       = 1,


Consider fixing GetX509Error() -> 'case 1: return 0' success mapping

dgarske · 2026-08-06T20:36:16Z

+     * the application the chain was fine. Fill in the generic error OpenSSL
+     * uses for this, without touching an error the callback did set. */
+    if (ret != WOLFSSL_SUCCESS && ctx->error == WOLFSSL_X509_V_OK) {
+        ctx->error = WOLFSSL_X509_V_ERR_UNSPECIFIED;


Blanket X509_V_ERR_UNSPECIFIED fallback applies to every failure path, including allocation failures.
Recommendation: Decide which behavior is intended. If the blanket fallback is deliberate, widen the comment so it does not read as callback-specific. If not, record a specific error (OpenSSL's X509_V_ERR_OUT_OF_MEM) at the allocation-failure goto exit sites and keep the fallback scoped to the veto case.

dgarske · 2026-08-06T20:37:03Z

-                                                        WOLFSSL_SUCCESS : ret;
-    #endif
+        if (verifyCb != NULL) {
+            if (verifyCb(ret >= 0 ? 1 : 0, ctx) == 1) {


Recommendation: Snapshot ctx->error before calling the callback and compare afterwards, so only an error the callback genuinely recorded is preserved.

dgarske · 2026-08-06T20:37:55Z

+ * (WOLFSSL_FAILURE / WOLFSSL_FATAL_ERROR) because chain building must stop on
+ * it: a veto must not be turned into a retry with another issuer, and must not
+ * be cleared by the partial-chain fallback in wolfSSL_X509_verify_cert(). */
+#define X509_STORE_CB_REJECTED (-2)


Recommendation: Pick a value that cannot collide with wolfSSL_ErrorCodes/WOLFSSL_CBIO_ERR_*, or return the veto through an out-parameter instead of in band. Either way, move the #define up with the file's other constants.

dgarske · 2026-08-06T20:38:21Z

-        XFREE(current->data, NULL, DYNAMIC_TYPE_OPENSSL);
-    }
-
+    /* wolfSSL_ASN1_STRING_copy() frees any existing dynamic buffer itself,


Recommendation: Reword to name wolfSSL_ASN1_STRING_set() as the owner of the free.

dgarske · 2026-08-06T20:39:21Z

                WOLFSSL_MSG("\tversion below minimum allowed, fatal error");
                ret = VERSION_ERROR;
+                /* propagate socket errors to avoid re-calling send alert */
+                if (SendAlert(ssl, alert_fatal,


Recommendation: Consider restoring ssl->version to the client's offered version (or the server's own maximum) around the SendAlert() call at exits 3 and 4, so the alert record carries a version the peer will accept. At minimum, note the intent in a comment at the source rather than only in the test.

Frauschi requested a review from wolfSSL-Fenrir-bot August 6, 2026 12:47

Frauschi self-assigned this Aug 6, 2026

wolfSSL-Fenrir-bot reviewed Aug 6, 2026

View reviewed changes

Comment thread src/x509_str.c Outdated

Frauschi force-pushed the fenrir_tls branch from 1f4a6cc to ba85fb1 Compare August 6, 2026 14:01

Frauschi commented Aug 6, 2026

View reviewed changes

Comment thread src/x509_str.c Outdated

Frauschi requested a review from wolfSSL-Fenrir-bot August 6, 2026 14:04

wolfSSL-Fenrir-bot reviewed Aug 6, 2026

View reviewed changes

Comment thread tests/api/test_ossl_x509_str.c

Frauschi force-pushed the fenrir_tls branch from ba85fb1 to be96e8a Compare August 6, 2026 15:52

Frauschi commented Aug 6, 2026

View reviewed changes

Comment thread tests/api/test_ossl_x509_str.c

Frauschi requested a review from wolfSSL-Fenrir-bot August 6, 2026 16:46

wolfSSL-Fenrir-bot reviewed Aug 6, 2026

View reviewed changes

Comment thread src/x509_str.c Outdated

Comment thread src/x509_str.c Outdated

Comment thread wolfssl/ssl.h

Comment thread src/internal.c

Comment thread src/x509_str.c Outdated

Comment thread src/internal.c

Frauschi added 3 commits August 6, 2026 19:58

Frauschi force-pushed the fenrir_tls branch from be96e8a to 196dd9f Compare August 6, 2026 17:59

Frauschi assigned wolfSSL-Bot and unassigned Frauschi Aug 6, 2026

dgarske requested changes Aug 6, 2026

View reviewed changes

dgarske assigned Frauschi Aug 6, 2026

Uh oh!

Conversation

Frauschi commented Aug 6, 2026

1. Honor the per-context X509_STORE_CTX verify callback (F-7341)

2. Fix double free in wolfSSL_X509_EXTENSION_set_data() (F-7340)

3. Send a protocol_version alert when no version can be negotiated (F-7568)

Tests

Uh oh!

wolfSSL-Fenrir-bot left a comment

Choose a reason for hiding this comment

Fenrir Automated Review — PR #11094

Uh oh!

Uh oh!

github-actions Bot commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

MemBrowse Memory Report

Uh oh!

Uh oh!

wolfSSL-Fenrir-bot left a comment

Choose a reason for hiding this comment

Fenrir Automated Review — PR #11094

Uh oh!

Uh oh!

Uh oh!

wolfSSL-Fenrir-bot left a comment

Choose a reason for hiding this comment

Fenrir Automated Review — PR #11094

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

2. Fix double free in `wolfSSL_X509_EXTENSION_set_data()` (F-7340)

github-actions Bot commented Aug 6, 2026 •

edited

Loading