fix(react-compiler-analyzer): clarify evidence and suppression diagnostics by Hotell · Pull Request #37001 · microsoft/fluentui

Martin Hochel (Hotell) · 2026-10-26T12:31:03Z

Previous Behavior

React Compiler Analyzer 0.0.3 already supports annotation dry-run planning, protected justified
directive revalidation, and syntax-only TypeScript parsing. However:

Analysis-only memo statistics were labeled as emitted-cache evidence.
Suppression-related Babel rejection was not a focused, enumerable exception category.
Risk/bailout wording could imply unsafe caching had actually occurred.
Revalidation compared acceptance without structured before/after statistics.
There were no compiler integration tests checking relevant native-SWC output and runtime exports.

New Behavior

Same Babel backend; clearer evidence, focused exception reporting, stronger tests.

                         Your source
                              |
                   Babel-based RCA analysis
                              |
           +------------------+-------------------+
           v                  v                   v
    Compiler outcome    Reported cache       Detected risks
    accepted / rejected statistics           independent of
    skipped / unknown   positive / zero / ?  compiler outcome
           +------------------+-------------------+
                              v
                    Evidence-aware report
                              |
           +------------------+-------------------+
           v                  v                   v
    Suppression-related Preventive bailout  Profile-specific
    exception list      rationale           guard revalidation

    Actual emitted caching: UNVERIFIED in normal analysis

Separately, in tests:

                  Small compiler fixture corpus
                              |
                 +------------+------------+
                 v                         v
           Babel emission           Native SWC emission
                 +------------+------------+
                              v
                 Compare relevant behavior
                 - cache placement
                 - diagnostics
                 - runtime exports
                 - gated overload behavior

                 Differences can be expected.
                 Universal parity is not required.

1. Suppression rejection becomes a reviewable exception

For valid hooks with an unnecessary suppression:

function Example({ value }: { value: string }) {
  // eslint-disable-next-line react-hooks/exhaustive-deps
  useEffect(() => {
    consume(value);
  }, [value]);
  return <span>{value}</span>;
}

The pinned corpus verifies the bounded difference: Babel emits no target cache while native
SWC emits caches. This is not a claim about every suppression or every SWC version.

Analyze output identifies compiler-backed rejection with
functions[].rejectionKind: "suppression-related", retains diagnostic reasons/categories
and any risk findings, and includes a focused human-report section.

The exception population is measurable using summary.suppressionRejected,
summary.suppressionRejectedWithRisks, and the summary.functions denominator.

react-compiler-analyzer analyze ./src --format json \
  | jq '.functions[] | select(.rejectionKind == "suppression-related") | {file, line, function, diagnostics}'

Another compiler/profile may optimize these functions. RCA still does not automatically
annotate Babel-rejected functions.

2. A preventive bailout does not require emitted-cache proof

function SelectedLabel() {
  const label = appStore.getState().label;
  return <span>{label}</span>;
}

With the relevant risk rule enabled, the evidence dimensions remain separate:

Evidence	Example observation	Meaning
Compiler outcome	Accepted	Babel accepted the function
Reported memo slots	`0`	Analysis reported zero slots
Detected risk	Imperative store snapshot	A configured risk pattern exists
Actual emitted caching	Unverified	Normal analysis does not inspect output
Bailout rationale	Preventive	Guard against potential memoization interactions

This is an illustrative scenario, not a promise that this exact source always reports zero.
Zero or unavailable statistics do not disqualify preventive all-safe/bailout-only targets.
Positive statistics also do not prove that the particular risky operation was cached.

Generated justifications use "preventive opt-out" rather than "unsafe to memoize".
Preview and write JSON share target metadata:

{
  "directive": "use no memo",
  "rationale": "preventive-risk-opt-out",
  "reportedMemoCache": "zero",
  "emittedCaching": "unverified",
  "riskRules": ["nonreactive-store-read"]
}

The excerpt omits the target's relative file, line, and function name.

3. Revalidation says what changed, and under which profile

react-compiler-analyzer lint ./src --mode infer --revalidate-justified

Revalidated JSON rows include evidence.before, evidence.after, acceptanceEffect,
statisticsEffect, and emittedOptimization: "not-measured".
Human output presents the same evidence alongside the recorded profile.

An illustrative observation:

                             With guard       Without guard
Compiler outcome             Skipped          Accepted
Reported memo slots          Unavailable      0

Acceptance effect:           Changed
Emitted optimization effect: Not measured
Automatic action:            None - justified guard is protected

Missing events/counters and failed probes are unavailable, not evidence of equivalence.
No observed effect is profile-specific, not proof of universal redundancy or a refutation
of a preventive justification.

Guard situation	Behavior
Has a `justified:` comment	Never automatically removed
Un-justified and currently considered redundant	Existing removal behavior remains
Babel-rejected function with a detected risk	Report it; do not automatically add a guard

This improves the claims, not the agreed deletion policy.

4. JSON stops calling statistics "emission"

Schema v3 replaces summary.memoCacheEmitted with:

reportedMemoCachePositive
reportedMemoCacheZero
reportedMemoCacheUnavailable

The package upgrade is 0.0.3 to 0.0.4. The JSON contract change is breaking for automation
and strict validators, even though this is a pre-1.0 patch-version increment. CLI flags,
configuration, and source-editing policies remain unchanged. The migration guide lists
required JSON consumer updates separately from additive review information.

Both commands include an analysis profile with resolved Babel/compiler versions, mode,
extra parser plugins, explicit no-emit/panic settings, and inherited target/suppression defaults.
The top-level emittedCaching: "unverified" makes the evidence boundary explicit.
Strict analyze and lint schemas, the migration guide, all human formats, and shipped agent
guidance are updated together.

Acceptance != positive statistics != emitted caching != the risky operation being cached.

5. Integration tests verify differences instead of demanding equality

Fixture group	What the tests establish
Valid hooks + unnecessary suppression	Preserve the known Babel/SWC difference
Invalid conditional hooks with/without suppression	Missing diagnostics are not proof of valid or optimized code
Malformed, misplaced, and unknown-rule suppressions	Record precise conditions and outcomes
TypeScript syntax and computed keys	Preserve syntax fixes and bounded compiler-specific differences
Exported overloads + gating	Check runtime exports, callability, and gate outcomes
Hidden selector hooks	Inspect relevant cache placement while retaining RCA risk findings
Positive and module-opt-out controls	Distinguish actual emission from analysis statistics

Compiler comparisons and packaged-CLI checks share tests/integration/ and one
test-integration target, which builds the CLI before running both suites. The target has
24 passing tests and 21 compiler behavior baselines. It records source hashes, resolved
versions, options, diagnostics, generated output, and runtime export evidence.
SWC per-function acceptance stays unavailable where its API does not expose it.

The measured gated-overload case exposes a Babel divergence: the intended useSample export
is lost in favor of useSample_unoptimized. SWC preserves the intended export. This is
explicitly baselined, not repaired by RCA.

Tests use normal workspace-resolved @swc/core, @babel/core, and
babel-plugin-react-compiler imports without npm aliases or extra compiler installations.
All 21 behavior baselines pass against the workspace versions without regeneration.
Native SWC remains test-only; the production backend is unchanged.

Scope at a glance

CHANGES                              STAYS THE SAME
---------------------------------    ---------------------------------
Suppression exception diagnostics    Babel production backend
Evidence and rationale terminology   Annotation eligibility
Profile provenance                   Zero-statistics preventive bailouts
Revalidation detail                  Justified-guard protection
JSON schema v3                       Existing un-justified fix policy
Compiler/CLI integration tests        Application tests as final validation

Validation

Commands below ran in the isolated implementation worktree with NX_DAEMON=false.

Command	Result
`yarn nx run react-compiler-analyzer:test --runInBand`	Passed: 540 tests, 28 suites, 23 snapshots
`yarn nx run react-compiler-analyzer:test-integration --skipNxCache`	Passed: 24 tests, 2 suites, 21 compiler baselines; shipped CLI schema/profile/counters, dry-run/no writes, and no SWC loading
`yarn nx run react-compiler-analyzer:type-check`	Passed: production and integration TypeScript
`yarn nx run react-compiler-analyzer:lint`	Passed
`yarn nx run react-compiler-analyzer:format --configuration=check`	Passed
`git diff --check`	Passed

The production build checks that test-only SWC comparison code does not leak into the CLI bundle.
PR CI runs both suites through yarn nx affected -t test-integration --nxBail.
The target depends on build and sets passWithNoTests: false. Ordinary :test remains
the unit/workflow suite and does not duplicate integration tests.
One patch Beachball change file targets 0.0.4.
Application behavior/performance tests and remote CI have not been run. Isolated compiler and
module-export tests are not application-runtime safety guarantees.
The branch was eight commits behind upstream master when validated; none of those commits
changed the analyzer.

Related Issue(s)

No linked issue; implements the consumer feedback on React Compiler Analyzer 0.0.3.

…stics Separate compiler acceptance, reported memo statistics, and preventive risk rationale. Add profile-specific guard evidence, schema v3 migration for 0.0.4, and pinned differential and packaged-CLI regression checks. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-10-26T12:39:32Z

📊 Bundle size report

✅ No changes found

Copilot

🟡 Changes recommended

Differential snapshots omit Babel and analyzer diagnostic reasons, allowing rejection-cause regressions to pass undetected.

1 open issue

Include normalized diagnostic reasons in differential snapshots · New

What changed in this PR

This PR clarifies React Compiler Analyzer’s evidence reporting while retaining the Babel backend and existing source-editing policies.

Changes:

Introduces schema v3 with analysis profiles, reported memo statistics, suppression rejection details, and annotation rationale.
Adds before/after evidence for justified-directive revalidation and updates reports and migration guidance.
Adds pinned Babel/SWC differential tests, built-CLI smoke checks, and CI integration.

File	Description
yarn.lock	Locks isolated compiler comparison dependencies.
tools/react-compiler-analyzer/tsconfig.differential.json	Configures differential-test type checking.
tools/react-compiler-analyzer/src/types.ts	Defines schema v3 and evidence types.
tools/react-compiler-analyzer/src/serializer.ts	Serializes profiles, diagnostics, and target evidence.
tools/react-compiler-analyzer/src/risk-plugin.ts	Clarifies risk-attribution comments.
tools/react-compiler-analyzer/src/risk-patterns.ts	Qualifies potential stale-cache risk wording.
tools/react-compiler-analyzer/src/revalidation-evidence.ts	Compares acceptance and reported statistics.
tools/react-compiler-analyzer/src/reporter.ts	Displays profile-specific revalidation evidence.
tools/react-compiler-analyzer/src/memo-cache-outcome.ts	Classifies reported slots without claiming emission.
tools/react-compiler-analyzer/src/coverage-reporter.ts	Updates evidence labels and suppression reporting.
tools/react-compiler-analyzer/src/coverage-fixer.ts	Returns annotation evidence and preventive rationale.
tools/react-compiler-analyzer/src/compiler.ts	Extracts suppression diagnostic categories.
tools/react-compiler-analyzer/src/compiler-events.ts	Preserves categories and classifies suppression rejection.
tools/react-compiler-analyzer/src/commands/shared.ts	Displays analysis profiles in human reports.
tools/react-compiler-analyzer/src/commands/lint.ts	Passes parser settings to JSON serialization.
tools/react-compiler-analyzer/src/commands/analyze.ts	Reports annotation evidence and parser settings.
tools/react-compiler-analyzer/src/analyzer.ts	Attaches evidence to justified revalidation.
tools/react-compiler-analyzer/src/analysis-profile.ts	Builds compiler-version and settings metadata.
tools/react-compiler-analyzer/src/__tests__/source-functions.test.ts	Accommodates expanded annotation results.
tools/react-compiler-analyzer/src/__tests__/reporter.test.ts	Updates lint-report snapshots.
tools/react-compiler-analyzer/src/__tests__/lint-command.test.ts	Updates profile and scan-log expectations.
tools/react-compiler-analyzer/src/__tests__/json-format.test.ts	Updates schema and statistics assertions.
tools/react-compiler-analyzer/src/__tests__/evidence-contract.test.ts	Tests suppression, annotation, and revalidation evidence.
tools/react-compiler-analyzer/src/__tests__/coverage-reporter.test.ts	Tests revised statistics and risk wording.
tools/react-compiler-analyzer/src/__tests__/coverage-fixer.test.ts	Accommodates annotation target metadata.
tools/react-compiler-analyzer/src/__tests__/compatibility-workflows.test.ts	Tests profile reporting and preview/write consistency.
tools/react-compiler-analyzer/src/__tests__/analyze-schema.test.ts	Validates renamed required counters.
tools/react-compiler-analyzer/src/__tests__/analyze-command.test.ts	Updates human-output snapshots.
tools/react-compiler-analyzer/smoke/cli.cjs	Checks built CLI contracts and SWC isolation.
tools/react-compiler-analyzer/skills/react-compiler-analyzer/SKILL.md	Updates agent guidance on evidence boundaries.
tools/react-compiler-analyzer/skills/react-compiler-analyzer/references/cli-reference.md	Documents schema migration and preventive annotations.
tools/react-compiler-analyzer/README.md	Documents evidence fields and validation commands.
tools/react-compiler-analyzer/rca.lint.schema.json	Adds a strict lint-output schema.
tools/react-compiler-analyzer/rca.analyze.schema.json	Defines the schema v3 analysis contract.
tools/react-compiler-analyzer/project.json	Adds differential, smoke, and type-check targets.
tools/react-compiler-analyzer/package.json	Ships the lint schema and pins test dependencies.
tools/react-compiler-analyzer/MIGRATION.md	Documents breaking JSON consumer updates.
tools/react-compiler-analyzer/jest.differential.config.ts	Configures the differential corpus suite.
tools/react-compiler-analyzer/esbuild.config.mjs	Guards against test-only SWC bundle leakage.
tools/react-compiler-analyzer/docs/runtime-risks.md	Separates detected risks from proven caching.
tools/react-compiler-analyzer/docs/migration-workflow.md	Explains preventive guards and suppression review.
tools/react-compiler-analyzer/docs/how-it-works.md	Describes evidence collection and schema changes.
tools/react-compiler-analyzer/differential/harness.ts	Collects compiler output and runtime evidence.
tools/react-compiler-analyzer/differential/fixtures.ts	Defines bounded compiler comparison cases.
tools/react-compiler-analyzer/differential/corpus.ts	Checks expected compiler behavior and divergences.
tools/react-compiler-analyzer/differential/__snapshots__/corpus.ts.snap	Records per-fixture behavior baselines.
change/@fluentui-react-compiler-analyzer-d394e0dc-6b09-4369-a085-235ded8f759a.json	Records the patch release and JSON migration warning.
.github/workflows/pr.yml	Runs new checks and uploads failure evidence.

_{Review effort: Balanced}

+    },
+    babel: {
+      error: evidence.babel.error,
+      events: eventSummary(evidence.babel.events),


github-actions · 2026-10-26T12:48:29Z

Pull request demo site: URL

github-actions · 2026-10-26T13:23:44Z

      - run: yarn install --immutable

      - name: React Compiler analysis (affected)
        run: |


🕵🏾‍♀️ visual changes to review in the Visual Change Report

vr-tests-react-components/Positioning 2 screenshots

Image Name Diff(in Pixels) Image Type

vr-tests-react-components/Positioning.Positioning end.chromium.png 731 Changed

vr-tests-react-components/Positioning.Positioning end.updated 2 times.chromium.png 138 Changed

vr-tests-react-components/ProgressBar converged 3 screenshots

Image Name Diff(in Pixels) Image Type

vr-tests-react-components/ProgressBar converged.Indeterminate + thickness - Dark Mode.default.chromium.png 23 Changed

vr-tests-react-components/ProgressBar converged.Indeterminate + thickness.default.chromium.png 50 Changed

vr-tests-react-components/ProgressBar converged.Indeterminate + thickness - High Contrast.default.chromium.png 205 Changed

vr-tests-react-components/TagPicker 1 screenshots

Image Name Diff(in Pixels) Image Type

vr-tests-react-components/TagPicker.disabled - Dark Mode.disabled input hover.chromium.png 658 Changed

vr-tests-web-components/MenuList 2 screenshots

Image Name Diff(in Pixels) Image Type

vr-tests-web-components/MenuList. - Dark Mode.normal.chromium.png 498 Changed

vr-tests-web-components/MenuList. - RTL.2nd selected.chromium.png 17 Changed

vr-tests-web-components/TextInput 1 screenshots

Image Name Diff(in Pixels) Image Type

vr-tests-web-components/TextInput. - Dark Mode.normal.chromium_1.png 288 Changed

vr-tests/react-charting-LineChart 1 screenshots

Image Name Diff(in Pixels) Image Type

vr-tests/react-charting-LineChart.Multiple - RTL.default.chromium.png 200 Changed

There were 1 duplicate changes discarded. Check the build logs for more information.

Use workspace compiler dependencies instead of aliased installations. Run compiler behavior and packaged CLI coverage together under tests/integration through one build-dependent Nx target and CI entry point. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Correct analysis profile documentation and run integration tests only in the main CI job, keeping failure evidence uploads alongside their execution. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions Bot added CI NX: core labels Oct 26, 2026

Martin Hochel (Hotell) requested a balanced review from Copilot October 26, 2026 12:39

Copilot started reviewing on behalf of Martin Hochel (Hotell) October 26, 2026 12:40

View session

Copilot AI reviewed Oct 26, 2026

View reviewed changes

Comment thread tools/react-compiler-analyzer/tests/integration/compiler/harness.ts

},

babel: {

error: evidence.babel.error,

events: eventSummary(evidence.babel.events),

github-actions Bot reviewed Oct 26, 2026

View reviewed changes

Martin Hochel (Hotell) and others added 2 commits October 26, 2026 16:06

fix(react-compiler-analyzer): address adversarial review findings

99677c8

Correct analysis profile documentation and run integration tests only in the main CI job, keeping failure evidence uploads alongside their execution. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Navigation Menu

Repository navigation

fix(react-compiler-analyzer): clarify evidence and suppression diagnostics - #37001

fix(react-compiler-analyzer): clarify evidence and suppression diagnostics#37001
Martin Hochel (Hotell) wants to merge 3 commits into
microsoft:masterfrom
Hotell:feat/rca-evidence-alignment

Martin Hochel (Hotell) commented Oct 26, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 26, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

github-actions Bot commented Oct 26, 2026

Uh oh!

github-actions Bot Oct 26, 2026 •

edited

Loading

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Image Name	Diff(in Pixels)	Image Type
vr-tests-react-components/Positioning.Positioning end.chromium.png	731	Changed
vr-tests-react-components/Positioning.Positioning end.updated 2 times.chromium.png	138	Changed

Image Name	Diff(in Pixels)	Image Type
vr-tests-react-components/ProgressBar converged.Indeterminate + thickness - Dark Mode.default.chromium.png	23	Changed
vr-tests-react-components/ProgressBar converged.Indeterminate + thickness.default.chromium.png	50	Changed
vr-tests-react-components/ProgressBar converged.Indeterminate + thickness - High Contrast.default.chromium.png	205	Changed

Image Name	Diff(in Pixels)	Image Type
vr-tests-web-components/MenuList. - Dark Mode.normal.chromium.png	498	Changed
vr-tests-web-components/MenuList. - RTL.2nd selected.chromium.png	17	Changed

Conversation

Martin Hochel (Hotell) commented Oct 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Previous Behavior

New Behavior

1. Suppression rejection becomes a reviewable exception

2. A preventive bailout does not require emitted-cache proof

3. Revalidation says what changed, and under which profile

4. JSON stops calling statistics "emission"

5. Integration tests verify differences instead of demanding equality

Scope at a glance

Validation

Related Issue(s)

Uh oh!

github-actions Bot commented Oct 26, 2026

📊 Bundle size report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

github-actions Bot commented Oct 26, 2026

Uh oh!

github-actions Bot Oct 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

🕵🏾‍♀️ visual changes to review in the Visual Change Report

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Martin Hochel (Hotell) commented Oct 26, 2026 •

edited

Loading

github-actions Bot Oct 26, 2026 •

edited

Loading