Show failure details in GitHub Actions step-summary collapsible sections

[Azat Mukhametshin (azat-msft)]

Fixes #10591

What

The GitHub Actions job summary previously listed only the fully-qualified name of each failed test, so investigating a failure meant leaving the summary page for the Annotations tab (which has no stack trace) or the raw workflow log.

Each failed test is now expanded into a collapsible <details> section:

<details>
<summary><code>Namespace.TestClass.TestMethod</code> — 2.40s</summary>

**Exception:** `System.InvalidOperationException`

**Location:** `src/Calc.cs:42`

```text
Expected: 42
Actual:   41

   at Calc.Add() in Calc.cs:line 42

```

The summary line reuses the test name — duration presentation and duration formatting of the existing "Slowest tests" section, so the two are visually consistent.

How

• Capture — GitHubActionsSummaryReporter now records the failure explanation (or exception message), exception type, stack trace and source location for each failing node. The location is resolved exactly the way GitHubActionsAnnotationReporter resolves it: prefer the exception's call site, then fall back to the test's own TestFileLocationProperty, so frameworks without a usable stack trace still get a location.
• Aggregation — the diagnostics flow through the CiRunSummaryTest fragment model, so aggregated multi-module dotnet test runs render them too. They are attached only to the failures list, never to slowest-tests entries, so fragments don't carry duplicate stack traces.
• Option — --report-gh-failure-details on|off, defaulting to on, restores the previous compact list.

Bounding the output

Jakub Jareš (@nohwnd)'s point on the issue about the job-summary size limit is handled with two independent bounds, both stated explicitly in the rendered output rather than silently applied:

Bound	Limit	On overflow
Single failure message	2,000 chars	value clipped, [... truncated] appended
Single stack trace	4,000 chars	value clipped, [... truncated] appended
Expanded detail per module section	60,000 chars	remaining failures degrade to compact lines + a [!NOTE] callout saying how many were omitted
Failure list	existing 20-failure cap	a [!NOTE] callout now states Showing the first 20 of N failed tests

Clipping happens at capture time, not render time, so an enormous stack trace never reaches the fragment written to disk. A failure that carries no diagnostics at all falls back to the compact line instead of rendering an empty disclosure.

Injection safety

• Test-provided values in <summary> are HTML-encoded — a generic test name like T.Map<string,int> would otherwise be parsed as a tag and swallow the rest of the line.
• The code fence around the body is chosen to be longer than the longest backtick run inside it, so a failure message that itself contains a ``` fence cannot terminate our block and leak raw markdown.

Existing GitHub error/warning annotations are unchanged.

Testing

• 10 new unit tests in GitHubActionsSummaryReporterTests covering the rendered section, the off-switch, the no-details fallback, HTML encoding, fence escaping, both truncation paths, Clip, and the aggregate/fragment path.
• 2 new acceptance tests in GitHubActionsReportTests driving a real MTP session with an exception-carrying failure, asserting the collapsible section end-to-end and that --report-gh-failure-details off keeps the compact list.
• HelpInfoAllExtensionsTests --help / --info expectations updated for the new option.
• Full Microsoft.Testing.Extensions.UnitTests suite passes (1,100 tests).

Docs (PACKAGE.md, docs/glossary.md) and the .xlf localization files are updated.

Left as draft pending validation of the rendered output in real workflow runs (see linked validation PRs).

[Copilot]

Pull request overview

Adds actionable failure diagnostics to GitHub Actions job summaries, including aggregated multi-module runs.

Changes:

• Captures and renders failure details in collapsible, injection-safe sections.
• Adds --report-gh-failure-details on|off and output-size controls.
• Updates tests, documentation, API baselines, and localization resources.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/GitHubActionsSummaryReporterTests.cs	Tests failure-detail rendering and limits.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/HelpInfoAllExtensionsTests.cs	Updates CLI help expectations.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/GitHubActionsReportTests.cs	Adds end-to-end summary tests.
src/Platform/SharedExtensionHelpers/SummaryReporterHelpers.cs	Adds failure diagnostics to test records.
src/Platform/SharedExtensionHelpers/CiRunSummaryAggregation.cs	Persists diagnostics through aggregation.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hant.xlf	Adds Traditional Chinese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.zh-Hans.xlf	Adds Simplified Chinese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.tr.xlf	Adds Turkish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ru.xlf	Adds Russian localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pt-BR.xlf	Adds Brazilian Portuguese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.pl.xlf	Adds Polish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ko.xlf	Adds Korean localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.ja.xlf	Adds Japanese localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.it.xlf	Adds Italian localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.fr.xlf	Adds French localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.es.xlf	Adds Spanish localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.de.xlf	Adds German localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/xlf/GitHubActionsResources.cs.xlf	Adds Czech localization entries.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/Resources/GitHubActionsResources.resx	Defines new localized messages.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/PACKAGE.md	Documents the new option.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/InternalAPI/InternalAPI.Unshipped.txt	Updates GitHub reporter API baseline.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryReporter.cs	Captures and renders failure diagnostics.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsSummaryArtifactPostProcessor.cs	Applies the option during aggregation.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsFailureDetails.cs	Implements bounded collapsible rendering.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineProvider.cs	Registers and validates the option.
src/Platform/Microsoft.Testing.Extensions.GitHubActionsReport/GitHubActionsCommandLineOptions.cs	Defines the option name.
src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/InternalAPI/InternalAPI.Unshipped.txt	Updates shared internal API baseline.
docs/glossary.md	Documents detailed failure summaries.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Azat Mukhametshin (azat-msft)]

Validation in real GitHub Actions runs

Validated end-to-end in azat-msft/gh-report-validation with this build packed into that repo's local feed (extension 1.1.0-dev, platform 2.4.0-dev). Each PR's workflow echoes the summary size and the markers that prove which rendering path was taken, so the evidence is in the run log rather than a manual read of the Summary page.

PR	Pipeline	Summary	Result
#2 green run	✅ green	1,640 B	0 collapsible sections, 0 clips, no truncation notes — the new rendering adds nothing when there is nothing to report
#3 short failure details	❌ red (deliberate)	6,309 B	Every failure fully expanded, 0 clips, no truncation notes
#4 oversized failure details	❌ red (deliberate)	76,355 B	18 values clipped, list capped at 20 of 31, detail budget exhausted after 10 — all reported explicitly

The headline number for the size concern: in #4, 31 failures each carrying a ~6 KB message and a 40-frame stack trace produce a 76 KB summary — roughly 7% of GitHub's 1 MiB job-summary limit — with both truncation notes rendered:

> Showing the first 20 of 31 failed tests. See the workflow log or the test report for the remaining failures.
> Failure details for 10 listed test(s) were omitted because the job summary size limit was reached.

Those validation PRs also fix a pre-existing bug in that repo's workflow, unrelated to this change: it passed --report-gh-slow-test-threshold without the --report-gh master switch, which the reporter correctly rejects as an invalid configuration.

[Azat Mukhametshin (azat-msft)]

Fourth validation run: the failure-count axis

Added azat-msft/gh-report-validation#5, which applies the opposite pressure from the oversized-details run: 5,000 failing tests with tiny diagnostics rather than a few with enormous ones.

Summary size: 27749 bytes
Collapsible failure sections: 21
Clipped values: 0
> Showing the first 20 of 5000 failed tests. See the workflow log or the test report for the remaining failures.

5,000 failures produce a 27 KB summary — about 2.6% of GitHub's 1 MiB limit. Varying only the failure count (measured locally):

Failing tests	Summary size
600	17,754 B
5,000	17,809 B

The size is flat; the 55-byte delta is just the wider count in the text.

Notable result: I could not construct a summary that overflows purely from failure count. Both reporters bound their own sections — this one at 20 failures (12,750 B), TUnit's own block at a 50-row table (4,848 B). So failure count cannot push a run past the 1 MiB limit; only per-failure size can, which is exactly what the per-value clips and the per-section budget exist to contain.

The two runs bracket the design: #4 shows the size axis is bounded at runtime, #5 shows the count axis is bounded by construction.

One design question before this leaves draft

MaxFailures is a fixed 20. At 5,000 failures you see 20, with the note pointing at the workflow log and the test report for the rest. That seems like the right default for a 1 MiB page, but it is worth deciding explicitly whether the cap should be configurable — it is a small follow-up on top of this PR if so.