fix(mocha-test-setup): Disambiguate JUnit test names by extending mocha's xunit reporter

[Alex Villarreal (alexvy86)]

Description

Azure DevOps Test Results shows test names that are ambiguous or outright duplicated: mocha's built-in xunit reporter puts the full describe-block path into the JUnit classname attribute and only the innermost it() title into name. ADO's PublishTestResults@2 JUnit parser uses name as the test's display title in the Tests tab (classname is not shown as the title), so tests with the same leaf title under different suites are indistinguishable in the UI.

This PR fixes that — and a related "Test file" grouping issue — with a small subclass of mocha's own xunit reporter that overrides just the name and classname fields on each <testcase>.

An earlier iteration of this PR instead adopted the third-party mocha-junit-reporter package. It was dropped: it emits one <testsuite> per describe block instead of one per file, which caused ADO to fall back to a generic, non-per-package Test Run name (see below), and it has no option to set classname to the spec file path, so it would have needed the same kind of subclassing anyway — with none of xunit's single-suite guarantee.

Evidence

Real example from @fluidframework/counter's fuzz tests, captured from the actual report files produced by pnpm run test:mocha:esm before and after this change.

Before — report XML:

<testcase classname="Counter fuzz testing default configuration"
          name="workload: default configuration seed: 0" .../>
<testcase classname="Counter fuzz testing with rebasing default configuration"
          name="workload: default configuration seed: 0" .../>

Both belong to different fuzz suites, but name — the field ADO displays — is identical. classname (fully qualified, but not shown as the title) is also the only field with any file/suite-path information — there's nothing at all pointing back to a source file.

After — report XML:

<testcase classname="packages/dds/counter/lib/test/counter.spec.js"
          name="Counter fuzz testing default configuration workload: default configuration seed: 0" .../>
<testcase classname="packages/dds/counter/lib/test/counter.spec.js"
          name="Counter fuzz testing with rebasing default configuration workload: default configuration seed: 0" .../>

name is now the fully-qualified suite path, so it's unique, and classname is the actual spec file the test lives in.

Across the whole @fluidframework/counter suite (214 tests), 204/214 (95%) of name values collided with another test's name before this change; after, there are zero collisions. The trade-off is longer names (avg 90 chars, max 142 chars in this package, vs. max 67 chars before) — an acceptable cost for uniquely identifiable results, and one already anticipated in AB#4462.

Before — ADO:

And grouping by "Test file", where some test names were being used in the field that ADO parses as test file:

After — ADO:

And grouping by "Test file", now correct;

"Test Run" naming, unaffected: the report file still has exactly one <testsuite name="<package name>"> element per file — unchanged from before this PR — so ADO's Tests tab still shows a clean per-package Test Run name. (Root-caused against ADO's actual open-source JUnit importer, JunitResultReader.cs: the run name falls back to a generic JUnit_<file name> whenever a report file has more than one <testsuite> element, regardless of any name attributes or file-naming scheme — this is what mocha-junit-reporter ran into, and why it was dropped in favor of extending xunit, which never emits more than one <testsuite> per file.)

"Test file" grouping, fixed: ADO's Tests tab can also be grouped by "Test file" — this uses the JUnit <testcase classname="..."> attribute directly. The old xunit reporter set classname to the fully-qualified describe-block path, never the spec file, so that grouping showed test names instead of file names. classname is now the test's spec file path, relative to the repo root (e.g. packages/dds/counter/lib/test/counter.spec.js), so that grouping is meaningful.

(Filed AB#83326 as a follow-up to separately investigate an unrelated "Test file" grouping oddity seen for the Playwright JUnit reporter, used in a few packages' end-to-end suites — unaffected by this PR, since it doesn't touch Playwright.)

Scope

This PR only touches the client release group (root pnpm workspace — packages/**, examples/**, experimental/**), matching AB#4462's acceptance criteria ("build - client" pipeline). build-tools/, server/routerlicious/, common/build/eslint-config-fluid/, and tools/test-tools/ are separate release groups/pipelines and are intentionally left on the old reporter pending a follow-up decision.

Changes

• @fluid-internal/mocha-test-setup: added FluidXunitReporter (src/xunitReporter.ts, exported as @fluid-internal/mocha-test-setup/xunit-reporter), a subclass of mocha's built-in xunit reporter (used by nearly all client packages via getFluidTestMochaConfig()) that overrides the <testcase> name (fully-qualified title) and classname (repo-relative spec file path) attributes. No new dependency — mocha's own reporter is reused directly.
• @fluidframework/core-interfaces: same fix, applied directly since it can't depend on mocha-test-setup (circular dependency) — it carries its own copy of .mocharc.cjs/test-config.json/the reporter subclass (xunit-reporter-classname.cjs) for that reason.

Verified end-to-end (build + real test run) in @fluidframework/counter and @fluidframework/core-interfaces: each produces a single valid <testsuite name="<package name>"> element per report file, with fully-qualified, collision-free testcase names and repo-relative classnames.

AB#4462

Reviewer Guidance

The review process is outlined in the pull request guidelines.

• No changeset included: this only affects test-reporter config, not published package behavior.
• Would like opinions on whether the longer, fully-qualified test names are an acceptable trade-off repo-wide, especially for suites with deep nesting or long parameterized/fuzz test titles.
• Happy to follow up with the same change for build-tools/server/routerlicious/etc. in a separate PR if this is well-received.

[github-actions]

Hi! Thank you for opening this PR. Want me to review it?

Based on the diff (584 lines, 13 files), I've queued these reviewers:

• Correctness — logic errors, race conditions, lifecycle issues
• Security — vulnerabilities, secret exposure, injection
• API Compatibility — breaking changes, release tags, type design
• Performance — algorithmic regressions, memory leaks
• Testing — coverage gaps, hollow tests

How this works

• Adjust the reviewer set by ticking/unticking boxes above. Reviewer toggles alone don't trigger anything.

• Tick Start review below to dispatch the review fleet.

• After review finishes, tick Start review again to request another run — it auto-resets after each dispatch.

• This comment updates as new commits land; your reviewer selections are preserved.

• Start review

[Copilot]

🟡 Changes recommended

Skipped tests need includePending: true, and all shared-helper consumers must have the reporter dependency available.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This pull request replaces Mocha’s xunit reporter with mocha-junit-reporter for client-release-group test reporting, producing unique Azure DevOps test names.

Changes:

• Updates shared and core-interfaces reporter configuration.
• Adds the reporter dependency across client packages and updates the lockfile.
• Preserves existing report paths and supports CJS/ESM runs.

File summaries

File	Summary
pnpm-lock.yaml	Locks the new reporter and dependencies.
packages/utils/tool-utils/package.json	Adds the reporter devDependency.
packages/utils/telemetry-utils/package.json	Adds the reporter devDependency.
packages/utils/odsp-doclib-utils/package.json	Adds the reporter devDependency.
packages/tools/fluid-runner/package.json	Adds the reporter devDependency.
packages/tools/devtools/devtools/package.json	Adds the reporter devDependency.
packages/tools/devtools/devtools-view/package.json	Adds the reporter devDependency.
packages/tools/devtools/devtools-core/package.json	Adds the reporter devDependency.
packages/test/test-version-utils/package.json	Adds the reporter devDependency.
packages/test/test-utils/package.json	Adds the reporter devDependency.
packages/test/test-end-to-end-tests/package.json	Adds the reporter devDependency.
packages/test/stochastic-test-utils/package.json	Adds the reporter devDependency.
packages/test/snapshots/package.json	Adds the reporter devDependency.
packages/test/mocha-test-setup/test-config.json	Configures the new reporter; enable includePending: true to retain skipped tests (moderate, 1 vote).
packages/test/mocha-test-setup/src/mocharcCommon.ts	Selects the new reporter; ensure every helper consumer declares the dependency (moderate, 2 votes).
packages/test/local-server-tests/package.json	Adds the reporter devDependency.
packages/test/local-server-stress-tests/package.json	Adds the reporter devDependency.
packages/test/functional-tests/package.json	Adds the reporter devDependency.
packages/service-clients/end-to-end-tests/odsp-client/package.json	Adds the reporter devDependency.
packages/service-clients/end-to-end-tests/azure-client/package.json	Adds the reporter devDependency.
packages/runtime/test-runtime-utils/package.json	Adds the reporter devDependency.
packages/runtime/runtime-utils/package.json	Adds the reporter devDependency.
packages/runtime/id-compressor/package.json	Adds the reporter devDependency.
packages/runtime/datastore/package.json	Adds the reporter devDependency.
packages/runtime/container-runtime/package.json	Adds the reporter devDependency.
packages/loader/driver-utils/package.json	Adds the reporter devDependency.
packages/loader/container-loader/package.json	Adds the reporter devDependency.
packages/framework/undo-redo/package.json	Adds the reporter devDependency.
packages/framework/type-factory/package.json	Adds the reporter devDependency.
packages/framework/tree-agent/package.json	Adds the reporter devDependency.
packages/framework/tree-agent-ses/package.json	Adds the reporter devDependency.
packages/framework/tree-agent-langchain/package.json	Adds the reporter devDependency.
packages/framework/synthesize/package.json	Adds the reporter devDependency.
packages/framework/request-handler/package.json	Adds the reporter devDependency.
packages/framework/react/package.json	Adds the reporter devDependency.
packages/framework/quill-react/package.json	Adds the reporter devDependency.
packages/framework/presence-runtime/package.json	Adds the reporter devDependency.
packages/framework/fluid-static/package.json	Adds the reporter devDependency.
packages/framework/dds-interceptions/package.json	Adds the reporter devDependency.
packages/framework/attributor/package.json	Adds the reporter devDependency.
packages/framework/aqueduct/package.json	Adds the reporter devDependency.
packages/drivers/routerlicious-urlResolver/package.json	Adds the reporter devDependency.
packages/drivers/routerlicious-driver/package.json	Adds the reporter devDependency.
packages/drivers/odsp-urlResolver/package.json	Adds the reporter devDependency.
packages/drivers/odsp-driver/package.json	Adds the reporter devDependency.
packages/drivers/local-driver/package.json	Adds the reporter devDependency.
packages/drivers/driver-base/package.json	Adds the reporter devDependency.
packages/dds/tree/package.json	Adds the reporter devDependency.
packages/dds/test-dds-utils/package.json	Adds the reporter devDependency.
packages/dds/task-manager/package.json	Adds the reporter devDependency.
packages/dds/shared-summary-block/package.json	Adds the reporter devDependency.
packages/dds/shared-object-base/package.json	Adds the reporter devDependency.
packages/dds/sequence/package.json	Adds the reporter devDependency.
packages/dds/register-collection/package.json	Adds the reporter devDependency.
packages/dds/pact-map/package.json	Adds the reporter devDependency.
packages/dds/ordered-collection/package.json	Adds the reporter devDependency.
packages/dds/merge-tree/package.json	Adds the reporter devDependency.
packages/dds/matrix/package.json	Adds the reporter devDependency.
packages/dds/map/package.json	Adds the reporter devDependency.
packages/dds/legacy-dds/package.json	Adds the reporter devDependency.
packages/dds/ink/package.json	Adds the reporter devDependency.
packages/dds/counter/package.json	Adds the reporter devDependency.
packages/dds/claims/package.json	Adds the reporter devDependency.
packages/dds/cell/package.json	Adds the reporter devDependency.
packages/common/core-utils/package.json	Adds the reporter devDependency.
packages/common/core-interfaces/test-config.json	Configures the new reporter; enable includePending: true to retain skipped tests (moderate, 1 vote).
packages/common/core-interfaces/package.json	Adds the reporter devDependency.
packages/common/core-interfaces/.mocharc.cjs	Switches core-interfaces reporting to the new reporter.
packages/common/client-utils/package.json	Adds the reporter devDependency.
experimental/PropertyDDS/packages/property-properties/package.json	Adds the reporter devDependency.
experimental/PropertyDDS/packages/property-dds/package.json	Adds the reporter devDependency.
experimental/PropertyDDS/packages/property-common/package.json	Adds the reporter devDependency.
experimental/PropertyDDS/packages/property-changeset/package.json	Adds the reporter devDependency.
experimental/dds/tree/package.json	Adds the reporter devDependency.
experimental/dds/sequence-deprecated/package.json	Adds the reporter devDependency.
experimental/dds/ot/sharejs/json1/package.json	Adds the reporter devDependency.
experimental/dds/ot/ot/package.json	Adds the reporter devDependency.
examples/utils/webpack-fluid-loader/package.json	Adds the reporter devDependency.
examples/utils/import-testing/package.json	Adds the reporter devDependency.
examples/external-data/package.json	Adds the reporter devDependency.
examples/data-objects/webflow/package.json	Adds the reporter devDependency.
examples/data-objects/table-document/package.json	Adds the reporter devDependency.
examples/benchmarks/tablebench/package.json	Adds the reporter devDependency.
examples/apps/tree-cli-app/package.json	Adds the reporter devDependency.

Review details

Files not reviewed (1)

• pnpm-lock.yaml: Generated file

Suppressed comments (2)

packages/common/core-interfaces/test-config.json:5

• mocha-junit-reporter only registers its pending handler when includePending is true, so this configuration drops every it.skip()/this.skip() case from the XML even though the previous Mocha xunit reporter emitted each as <skipped/>. Core-interfaces has skipped tests as well, so its ADO report will omit those test results. Please set includePending to true in these reporter options.

	"mochaJunitReporterReporterOptions": {
		"mochaFile": "nyc/{id}junit-report.xml",
		"testsuitesTitle": "{id}"

packages/test/mocha-test-setup/test-config.json:5

• mocha-junit-reporter only registers its pending handler when includePending is true, so this configuration drops every it.skip()/this.skip() case from the XML even though the previous Mocha xunit reporter emitted each as <skipped/>. Affected suites contain skipped tests, so the report will omit those ADO test results (and its test counts can no longer match the emitted cases). Please set includePending to true in these reporter options.

• Files reviewed: 83/84 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

🔭 PR Review Fleet Report

Note

This report is generated by an experimental AI review fleet and is provided as a beta feature. Findings are a starting point for discussion, not a gate. Use your own judgement.

Verdict: ⚠️ Approve with Suggestions

0 Exterminate, 0 Squash, 2 Investigate

Findings

Sev	#	Area	File	What	Fix
🐜 Investigate	M1	Testing	packages/common/core-interfaces/FluidMochaReporter.cjs:58	The duplicated CJS reporter's test() override is only exercised by fluidMochaReporter.spec.ts via a single passing test that has a file set. Unlike the canonical FluidXunitReporter in mocha-test-setup (whose spec covers failed/skipped tests and the test.file === undefined fallback branch), this duplicate has no test for a test with no file (fallback to test.fullTitle() for classname) or for failed/skipped tests reaching the wrapped super.test() call. Since the file's own comment says it 'must be kept in sync' with the TS original via manual duplication, a future edit that breaks the fallback or crashes on failed/skipped tests here would ship undetected even though the canonical copy's tests would still pass.	Extend fluidMochaReporter.spec.ts (or add a second it) to run a fixture suite with a failing test, a skipped test, and a test added via Mocha.Suite.create/Mocha.Test (no file set) through the package's real multi-reporter config, then assert the resulting JUnit XML contains <failure>, <skipped/>, and a classname equal to the fully qualified title for the no-file case — mirroring xunitReporter.spec.ts's 'falls back to the fully qualified title...' test.
🐜 Investigate	M2	Testing	packages/test/mocha-test-setup/src/xunitReporter.ts:19	findRepoRoot has an untested fallback branch: when no ancestor directory contains pnpm-workspace.yaml (e.g. this reporter is used from a package installed outside the monorepo, or the marker file is renamed/moved), it silently returns startDir instead of the real root, which would make classname fall back to an incorrect relative (or absolute) path. Both copies of this logic (here and the duplicate in packages/common/core-interfaces/FluidMochaReporter.cjs) only have tests that run inside the actual repo tree, so the parent === dir fallback path is never exercised.	Add a unit test that calls findRepoRoot directly with a temp directory tree that has no pnpm-workspace.yaml anywhere up to the filesystem root (e.g. mock/stub existsSync to always return false, or use a directory outside the repo), and assert it returns the original startDir unchanged.

---

View workflow run

[Alex Villarreal (alexvy86)]

[Agent-generated] Addressing the review feedback about includePending: without it, mocha-junit-reporter never registers a listener for Mocha's pending event, so skipped tests are silently dropped from the JUnit XML entirely — unlike the old xunit reporter, which always emitted a entry. Set includePending: true in both est-config.json files (mocha-test-setup and core-interfaces) in 64a1458 to restore that behavior.

[Tony Murphy (anthony-murphy)]

Deep Review

Reviewed commit 25227d9 on 2026-09-11.

Readiness: 3/10 — GETTING STARTED

Not ready for sign-off. The reporter choice and package-aware filename fix are supported by representative ESM/CJS runs and live Azure DevOps evidence, but the shared reporting contract lacks durable regression coverage across the client release group.

Path to Ready

• Add automated coverage for package-aware filenames, the mocha-junit-reporter option mapping, testsuitesTitle, includePending, and the default, CJS, and custom testReportPrefix variants; exercise duplicate leaf titles and a pending test in a generated report

Context for Reviewers

• The new reporter preserves the historical output invariants: {id} substitution and testReportPrefix still produce distinct files, generated names still match **/*junit-report.xml, testsuitesTitle retains suite identity, and includePending: true retains skipped tests.
• The client-only rollout is deliberate; build-tools/, server/routerlicious/, common/build/eslint-config-fluid/, and tools/test-tools/ remain outside this PR, consistent with the staged test-tooling precedent in PR refactor: Add new simple type test generator #14334.
• PR Fix reporting in e2e tests #16098 and PR Set suiteName on FLUID_TEST_MULTIREPORT #16289 establish the load-bearing requirements for distinct multi-configuration report files and useful suite identity; Abe27342 authored both changes.

For human reviewer

• Needs human judgment — Decide whether fully qualified names averaging 90 characters and reaching 142 characters in the cited package are an acceptable Azure DevOps usability tradeoff.
• Requires live verification — Inspect a representative compat/e2e PublishTestResults@2 result to confirm package, driver, and version identity remains useful after moving package identity into the report basename and root suite metadata.
• Relevant area expertise — Abe27342 or the current test-infrastructure/ADO-reporting owner can assess the distinct-file and suite-identity expectations established in PR Fix reporting in e2e tests #16098 and PR Set suiteName on FLUID_TEST_MULTIREPORT #16289.

Review history (1 prior review)

• 64a1458 2026-09-11 · 10/10 — Ready for sign-off.

[Copilot]

🟡 Changes recommended

Identical suite paths across files remain ambiguous, and the duplicated reporter lacks integration coverage.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (1)

packages/common/core-interfaces/FluidMochaReporter.cjs:71

• This duplicated implementation has the same remaining collision: fullTitle() omits the source file, so matching describe/it chains in separate core-interfaces specs still have identical ADO display names. Include the repo-relative file path in name too so the core-interfaces copy provides the promised disambiguation.

			title: { value: test.fullTitle(), enumerable: true },

• Files reviewed: 10/10 changed files
• Comments generated: 3
• Review effort level: Balanced

[Copilot]

🟢 Approval recommended

The implementation and coverage are sound; only two non-blocking stale source links remain.

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

packages/common/core-interfaces/.mocharc.cjs:15

• This source link returns 404 because the project is hosted under stanleyhlng, not stevemao. Update the URL so the implementation detail documented here remains verifiable.
  packages/test/mocha-test-setup/src/mocharcCommon.ts:133
• This source link returns 404 because the project is hosted under stanleyhlng, not stevemao. Update the URL so the implementation detail documented here remains verifiable.

• Files reviewed: 11/11 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Copilot]

🟢 Approval recommended

The intended behavior is covered by integration tests; the remaining comments are non-blocking dependency-hardening suggestions.

Review details

• Files reviewed: 11/11 changed files
• Comments generated: 2
• Review effort level: Balanced

[Copilot]

🟢 Approval recommended

The reporter behavior, configuration paths, CommonJS compatibility, and generated output are adequately covered without unresolved correctness issues.

Review details

• Files reviewed: 11/11 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Abram Sanderson (Abe27342)]

implementation looks fine, lmk when you address the couple comments I left and I can approve

[github-actions]

Bundle size comparison

Base commit: 66d337c9fc338cbb3d7db50e0797b9bd62ca7ba7
Head commit: d54185faf96a86ee560206f320027a89b53e9be0

Pending — Build - client packages is running. Results will appear here when the build completes.