Skip to content

.NET: Python: Add dotnet integration test report to CI #5515

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
giles17 wants to merge 6 commits into
base: main
Choose a base branch
from
+278 −45
Draft

.NET: Python: Add dotnet integration test report to CI #5515

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
aec47dd
Add dotnet integration test report to CI
giles17 Apr 27, 2026
450eab4
chore: trigger dotnet CI for report validation
giles17 Apr 27, 2026
f48c8b3
fix: use .junit extension (not .junit.xml) for xunit v3 output
giles17 Apr 27, 2026
df49b91
fix: use deterministic provider-qualified keys for dotnet tests
giles17 Apr 27, 2026
4ff5130
fix: match Python report summary format (Total, passed/total, etc.)
giles17 Apr 27, 2026
2aee1d8
feat: split dotnet report into per-framework tables
giles17 Apr 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions .github/workflows/dotnet-build-and-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -257,6 +257,8 @@ jobs:
-c ${{ matrix.configuration }} `
--no-build -v Normal `
--report-xunit-trx `
--report-junit `
--results-directory ../IntegrationTestResults/ `
--ignore-exit-code 8 `
--filter-not-trait "Category=IntegrationDisabled" `
--parallel-algorithm aggressive `
Expand Down Expand Up @@ -299,6 +301,14 @@ jobs:
shell: pwsh
run: ./dotnet/eng/scripts/dotnet-check-coverage.ps1 -JsonReportPath "TestResults/Reports/Summary.json" -CoverageThreshold $env:COVERAGE_THRESHOLD

- name: Upload integration test results
if: always() && github.event_name != 'pull_request' && matrix.integration-tests
uses: actions/upload-artifact@v7
with:
name: dotnet-test-results-${{ matrix.targetFramework }}-${{ matrix.os }}
path: IntegrationTestResults/**/*.junit
if-no-files-found: ignore

# This final job is required to satisfy the merge queue. It must only run (or succeed) if no tests failed
dotnet-build-and-test-check:
if: always()
Expand Down Expand Up @@ -341,3 +351,59 @@ jobs:
uses: actions/github-script@v8
with:
script: core.setFailed('Integration Tests Cancelled!')

# Integration test trend report (aggregates JUnit XML results from dotnet test jobs)
dotnet-integration-test-report:
name: Integration Test Report
if: >
always() &&
github.event_name != 'pull_request' &&
(contains(join(needs.*.result, ','), 'success') ||
contains(join(needs.*.result, ','), 'failure'))
needs: [dotnet-test]
runs-on: ubuntu-latest
defaults:
run:
working-directory: python
steps:
- uses: actions/checkout@v6
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this workflow, other checkout steps explicitly set persist-credentials: false to avoid leaving the GITHUB_TOKEN credentials in the local git config. This new job’s checkout doesn’t set it, so credentials will persist by default. Consider adding with: persist-credentials: false here for consistency and to reduce the blast radius of any later step that might invoke git.

Suggested change
- uses: actions/checkout@v6
- uses: actions/checkout@v6
with:
persist-credentials: false

Copilot uses AI. Check for mistakes.
- name: Set up python and install the project
uses: ./.github/actions/python-setup
with:
python-version: "3.13"
os: ${{ runner.os }}
- name: Download all test results from current run
uses: actions/download-artifact@v4
with:
pattern: dotnet-test-results-*
path: dotnet-test-results/
- name: Restore report history cache
uses: actions/cache/restore@v4
with:
path: python/dotnet-integration-report-history.json
key: dotnet-integration-report-history-${{ github.run_id }}
restore-keys: |
dotnet-integration-report-history-
- name: Generate trend report
run: >
uv run python scripts/flaky_report/aggregate.py
../dotnet-test-results/
dotnet-integration-report-history.json
dotnet-integration-test-report.md
- name: Post to Job Summary
if: always()
run: cat dotnet-integration-test-report.md >> $GITHUB_STEP_SUMMARY
- name: Save report history cache
if: always()
uses: actions/cache/save@v4
with:
path: python/dotnet-integration-report-history.json
key: dotnet-integration-report-history-${{ github.run_id }}
- name: Upload trend report
if: always()
uses: actions/upload-artifact@v7
with:
name: dotnet-integration-test-report
path: |
python/dotnet-integration-test-report.md
python/dotnet-integration-report-history.json
1 change: 1 addition & 0 deletions dotnet/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,3 +33,4 @@ Console.WriteLine(await agent.RunAsync("Write a haiku about Microsoft Agent Fram
- [Design Documents](../docs/design)
- [Architectural Decision Records](../docs/decisions)
- [MSFT Learn Docs](https://learn.microsoft.com/agent-framework/overview/agent-framework-overview)

Loading
Loading