Fail fast on invalid config regexes and enabled config

[prk-Jr]

Summary

• Fail startup instead of panicking or silently disabling when config-derived regexes or enabled integration/provider configs are invalid.
• Prepare handler and Next.js rewrite artifacts before request handling so invalid config returns descriptive TrustedServerError responses.
• Add regression coverage for handler overrides, disabled-config bypasses, Next.js RSC/__NEXT_DATA__ rewrites, and publisher fallback encoding behavior.

Changes

File	Change
.claude/agents/pr-creator.md	Require PRs touching config-derived regex or pattern compilation to document startup hardening and validation coverage.
.claude/agents/pr-reviewer.md	Treat panic-prone config compilation and silent invalid-enabled-config disablement as blocking review findings.
crates/common/src/auction/mod.rs	Make auction provider construction fallible and propagate startup configuration errors.
crates/common/src/auth.rs	Make basic-auth evaluation fallible so invalid handler regexes return config errors instead of panicking.
crates/common/src/integrations/adserver_mock.rs	Return startup errors for invalid enabled provider config instead of logging and skipping it.
crates/common/src/integrations/aps.rs	Return startup errors for invalid enabled provider config instead of logging and skipping it.
crates/common/src/integrations/datadome.rs	Make enabled config registration fail fast with descriptive validation errors.
crates/common/src/integrations/didomi.rs	Make enabled config registration fail fast with descriptive validation errors.
crates/common/src/integrations/google_tag_manager.rs	Make enabled config registration fail fast with descriptive validation errors.
crates/common/src/integrations/gpt.rs	Make enabled config registration fail fast with descriptive validation errors.
crates/common/src/integrations/lockr.rs	Make enabled config registration fail fast with descriptive validation errors.
crates/common/src/integrations/mod.rs	Update shared integration builder signatures and helpers for fallible startup registration.
crates/common/src/integrations/nextjs/html_post_process.rs	Pass request host and scheme through the post-processor RSC rewrite path.
crates/common/src/integrations/nextjs/mod.rs	Build Next.js rewriters during registration and add fixture-style regressions for RSC and __NEXT_DATA__ coverage.
crates/common/src/integrations/nextjs/rsc.rs	Reuse the new RSC rewrite helpers while preserving payload sizing and chunk handling.
crates/common/src/integrations/nextjs/script_rewriter.rs	Replace config-derived expect() regex compilation with fallible rewriters and add hostname, port, whitespace, and metacharacter regressions.
crates/common/src/integrations/nextjs/shared.rs	Replace origin-specific regex construction with static generic patterns and safe hostname-boundary helpers.
crates/common/src/integrations/permutive.rs	Make enabled config registration fail fast with descriptive validation errors.
crates/common/src/integrations/prebid.rs	Fail startup on invalid enabled config, including empty server_url, and return fallible provider builders.
crates/common/src/integrations/registry.rs	Propagate integration registration errors during startup instead of silently dropping invalid enabled integrations.
crates/common/src/integrations/testlight.rs	Make enabled config registration fail fast with descriptive validation errors.
crates/common/src/publisher.rs	Restrict publisher fallback Accept-Encoding to codecs the rewrite pipeline supports and add regression coverage.
crates/common/src/settings.rs	Add runtime preparation for handler regexes, short-circuit raw enabled = false configs, and add startup hardening regressions.
crates/common/src/settings_data.rs	Run runtime preparation when loading baked settings data so invalid handler regexes fail during startup.
crates/fastly/src/main.rs	Surface orchestrator and auth configuration errors through the existing Fastly error-response path.

Hardening note

Invalid handlers[].path regexes now fail during startup preparation and still return descriptive configuration errors if request-time code encounters unprepared settings. Enabled integrations and auction providers now fail registry/orchestrator startup instead of logging and disabling themselves, while raw enabled = false configs still short-circuit before validation. Regression coverage includes invalid handler TOML and env overrides, disabled-invalid config bypasses, enabled-invalid integration/provider startup failures, empty prebid.server_url, Next.js __NEXT_DATA__ and RSC hostname/port/metacharacter rewrites, and the publisher fallback encoding path.

Closes

Closes #403

Test plan

• cargo test --workspace
• cargo clippy --all-targets --all-features -- -D warnings
• cargo fmt --all -- --check
• JS tests: cd crates/js/lib && npx vitest run
• JS format: cd crates/js/lib && npm run format
• Docs format: cd docs && npm run format
• WASM build: cargo build --bin trusted-server-fastly --release --target wasm32-wasip1
• Manual testing via fastly compute serve
• Other: npx vitest run was skipped per explicit user instruction because of the unrelated repo-wide ERR_REQUIRE_ESM failure in html-encoding-sniffer.

Checklist

• Changes follow CLAUDE.md conventions
• No unwrap() in production code — use expect(\"should ...\")
• Uses tracing macros (not println!)
• New code has tests
• No secrets or credentials committed

[ChristianPavilonis]

PR Review: "Fail fast on invalid config regexes and enabled config"

Verdict: Looks good — no blockers.

---

Findings

P2 — Medium (Informational)

1. RscUrlRewriter now uses a generic regex (crates/common/src/integrations/nextjs/shared.rs:22-27) — The static regex matches all URLs instead of per-origin URLs, so every URL in an RSC payload goes through the closure. The existing early-exit check (input.contains(&self.origin_host) at line 139) short-circuits the common case, so performance impact is negligible in practice. No action needed.

2. Prebid server_url validation behavioral change (crates/common/src/integrations/prebid.rs:210-219) — Empty server_url now causes a hard error instead of silently disabling. This is the stated intent of the PR and is covered by the new empty_prebid_server_url_fails_orchestrator_startup test. Acceptable.

---

Highlights

• is_explicitly_disabled short-circuit — Inspects raw JSON before deserialization so enabled: false configs with otherwise invalid fields don't cause spurious startup failures. Smart design, well-tested.
• Comprehensive test coverage — Invalid handler regexes (TOML + env var), disabled+invalid config short-circuits for integrations/providers/auction providers, enabled+invalid startup failures, __NEXT_DATA__ rewrite fixtures with ports/metacharacters/whitespace/hostname boundaries.
• Consistent fallible builder pattern — Every integration's build() uniformly returns Result<Option<T>, Report<TrustedServerError>>, making the codebase very predictable.
• Lockr regex moved to Lazy<Regex> static (crates/common/src/integrations/lockr.rs:37-42) — Avoids recompiling on every request.
• strip_origin_host_with_optional_port boundary helper (crates/common/src/integrations/nextjs/shared.rs:67-100) — Properly prevents origin.example.com.evil from matching while allowing origin.example.com:8443/path. Edge cases handled correctly and tested.

---

All CI checks pass. High-quality hardening PR with thorough, consistent changes across all 25 files.

[aram356]

Summary

Solid defensive hardening that converts config-derived regex compilation and integration registration from panic/log-and-swallow to Result-based propagation with fail-fast at startup. The is_explicitly_disabled short-circuit is a well-targeted addition. Consistent application across all 10+ integrations with thorough test coverage.

Non-blocking

🤔 thinking

• Inconsistent parameter storage across rewriters: RscUrlRewriter stores origin_host, UrlRewriter is stateless — sibling types with different conventions (shared.rs:118, script_rewriter.rs:113)
• restrict_accept_encoding unconditionally overwrites client preferences: prevents unsupported encodings from origin but does not honor clients that only accept identity or a subset (publisher.rs:19)

♻️ refactor

• rewrite_nextjs_values_with_rewriter is a pass-through wrapper: delegates to rewriter.rewrite_embedded() with no added logic — can be inlined (script_rewriter.rs:74)
• validate_path compiles regex redundantly: from_toml now calls prepare_runtime() which compiles via OnceLock, making the validation-time compile redundant (settings.rs:473)
• Redundant prepare_runtime() call in get_settings: from_toml already calls it internally (settings_data.rs:37)

🌱 seedling

• once_cell::sync::Lazy → std::sync::LazyLock: project uses Rust 2024 edition where LazyLock is stable — opportunity to drop the once_cell dep in a follow-up
• strip_origin_host_with_optional_port does not handle IPv6: RSC_URL_PATTERN captures bracketed IPv6 but this helper cannot match them — worth a doc comment (shared.rs:67)

👍 praise

• is_explicitly_disabled short-circuit: checking raw JSON before deserialization means disabled integrations with placeholder config do not blow up startup (settings.rs:123)
• Excellent hardening test matrix: disabled-invalid-skips, enabled-invalid-fails, provider configs, registry/orchestrator startup, handler regex, env var overrides, and a real-world-shaped __NEXT_DATA__ fixture
• Consistent Result-based registration: uniform pattern across all integrations with no stragglers

CI Status

• cargo fmt: PASS
• cargo test: PASS
• vitest: PASS
• CodeQL/Analyze: PASS
• format-docs: PASS
• format-typescript: PASS

[aram356]

🤔 thinking — Inconsistent parameter storage across the two sibling rewriters

RscUrlRewriter stores origin_host in self but takes request_host/request_scheme as method args. Meanwhile UrlRewriter in script_rewriter.rs stores nothing and takes origin_host as a method arg too.

The two types model the same conceptual inputs with different ownership conventions. Consider picking one: either both store origin_host (it's per-config, not per-request) or both are fully stateless.

[aram356]

🤔 thinking — restrict_accept_encoding unconditionally overwrites client preferences

The fix correctly prevents the pipeline from receiving encodings it can't decode (e.g., zstd). But a client that only accepts identity or a strict subset will now receive content encoded with gzip/br it didn't ask for.

Consider computing the intersection of client capabilities and pipeline capabilities, falling back to identity if the intersection is empty.

[aram356]

♻️ refactor — rewrite_nextjs_values_with_rewriter is a pass-through wrapper

This function just delegates to rewriter.rewrite_embedded(...) with no additional logic. The call site in rewrite_structured (line 42) could call self.rewriter.rewrite_embedded(...) directly and this intermediary could be removed.

[aram356]

♻️ refactor — validate_path compiles the regex a second time

validate_path compiles the regex during serde validation, then prepare_runtime() → compiled_regex() compiles it again into the OnceLock. Since from_toml now calls prepare_runtime(), the validation-time compile is redundant work.

Consider deferring regex validation to prepare_runtime only and removing the validate_path custom validator (or having validate_path just check non-empty, letting prepare_runtime handle compilation).

[aram356]

♻️ refactor — Redundant prepare_runtime() call

Settings::from_toml already calls prepare_runtime() internally. This second call is idempotent (due to OnceLock) but misleading — it suggests from_toml doesn't do preparation. Consider removing this or adding a comment explaining the belt-and-suspenders intent.

[aram356]

🌱 seedling — strip_origin_host_with_optional_port doesn't handle IPv6 brackets

RSC_URL_PATTERN captures bracketed IPv6 hosts (\\[[^\\]]+\\]), but this function uses plain strip_prefix which won't match them. Fine if IPv6 origins are not a real use case — worth a doc comment noting the limitation so future readers don't assume full coverage.

[aram356]

👍 praise — is_explicitly_disabled short-circuit is well-designed

Checking the raw JSON for "enabled": false before attempting deserialization+validation means disabled integrations with intentionally invalid config (e.g., placeholder URLs) don't blow up startup. Clean, minimal, solves the real problem.