Add ADO CI, SDL, and release pipelines with e2e test enablement by RyAuld · Pull Request #890 · AzureAD/microsoft-authentication-library-for-python

RyAuld · 2026-03-20T21:58:30Z

Summary

Sets up the full ADO CI/release pipeline infrastructure for the \msal\ Python package in IDDP. Previously, releases were managed manually. This PR covers three objectives:

PR / CI pipeline (\�zure-pipelines.yml) — automated pipeline in IDDP that runs SDL scans (PoliCheck + CredScan) and the full test matrix (Python 3.9–3.14) on every PR and push to \dev.
Release pipeline (\pipeline-publish.yml) — manually-triggered pipeline for publishing \msal\ to test.pypi.org (Preview/RC) or PyPI (production via ESRP).
E2E test enablement — CI stage retrieves the MSID Lab certificate from Key Vault and wires it into the test runner so integration tests run in ADO.

Following the same template convention as MSAL.NET, shared stages live in \ emplate-pipeline-stages.yml\ and are called from thin top-level wrappers.

Pipeline Files

File	Purpose
\�zure-pipelines.yml\	PR gate and post-merge CI — calls shared template with \
unPublish: false\
.Pipelines/pipeline-publish.yml\	Release pipeline — manually queued, builds and publishes to PyPI
.Pipelines/template-pipeline-stages.yml\	Shared stages: PreBuildCheck (SDL), Validate, CI
.Pipelines/credscan-exclusion.json\	CredScan suppressions for known test fixtures
.Pipelines/CI-AND-RELEASE-PIPELINES.md\	Public-facing pipeline documentation

ADO Pipelines in IDDP

Pipeline	ID	Purpose
\MSAL.Python-PR-OneBranch-Official\	3064	PR gate and post-merge CI (this file: \�zure-pipelines.yml)
\MSAL.Python-Publish\	3067	Release pipeline (.Pipelines/pipeline-publish.yml)

Latest successful builds:

PR/CI: 20260330.4 — 270 passed, all 6 Python versions ✅
Publish (ESRP path): 1613569 — PreBuildCheck + Validate + CI + Build passed; EsrpRelease@9 reached ESRP gateway (blocked pending ESRP portal approval — see below)

Stage Flow

PR / CI build (\�zure-pipelines.yml)

\
PreBuildCheck (SDL) ─► CI (Python 3.9–3.14)
\\

Release build (\pipeline-publish.yml)

\
PreBuildCheck (SDL) ─► Validate ─► CI ─► Build ─┬─► PublishMSALPython (test.pypi.org)
└─► PublishPyPI (pypi.org via ESRP)
\\

Other Changes

\msal/sku.py\ — bumped version to \1.35.2rc1\

Pending Before Release Can Publish

The pipeline infrastructure is complete. The following external approvals are still needed before a release can fully publish packages:

test.pypi.org (Preview / RC path)

Create \MSAL-Test-Python-Upload\ ADO service connection in IDDP with a test.pypi.org API token scoped to the \msal\ project
Uncomment \TwineAuthenticate@1\ + \ wine upload\ steps in \PublishMSALPython\ stage (currently commented out with a placeholder \�cho\ step)

pypi.org (ESRP Production path)

ESRP portal approval: add PyPI content type (\msal\ package) to the \MSALJavaReleaseBuilds\ app in ESRP — status currently shows \UpdateInProgress; pipeline reaches the ESRP gateway but is rejected until approved

Post-Merge Actions (after this PR merges to \dev)

Disable pipeline 769 (\MSAL-Python-SDL-CI) — SDL scanning is now covered by the \PreBuildCheck\ stage in pipeline 3064, which triggers on PRs, pushes to \dev, and on a daily schedule. Pipeline 769 is fully redundant.
Update pipeline 3064 triggers — add scheduled daily trigger to \�zure-pipelines.yml\ to match the schedule currently on pipeline 769 (11:45 PM Pacific, \dev\ branch)
Authorize new publish pipeline (3067) for any additional service connections once they are created (e.g. \MSAL-Test-Python-Upload)

.Pipelines/pipeline-publish.yml

gladjohn

Approved with comments. Please Add Python 3.14 to the matrix in .Pipelines/pipeline-publish.yml

Copilot

Pull request overview

Adds a manually triggered Azure DevOps publish pipeline (via .Pipelines/) intended to validate the release version, run the existing test matrix, build sdist/wheel, and publish msal to either test.pypi.org or pypi.org with environment-based approval gating for production releases.

Changes:

Added a new multi-stage ADO pipeline (pipeline-publish.yml) composed of reusable step templates for test, build, and publish.
Added documentation (ADO-PUBLISH-SETUP.md) describing one-time Azure DevOps setup and release walkthrough steps.
Bumped msal package version in msal/sku.py to 1.35.2rc1.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
`msal/sku.py`	Updates the library version used by packaging metadata (`setup.cfg` reads `msal.__version__`).
`.Pipelines/pipeline-publish.yml`	Defines Validate → CI → Build → Publish stages and routes publish target via a parameter.
`.Pipelines/template-run-tests.yml`	Installs dependencies and runs pytest, publishing JUnit results.
`.Pipelines/template-build-package.yml`	Builds sdist/wheel, runs `twine check`, and publishes `dist/` as a pipeline artifact.
`.Pipelines/template-publish-package.yml`	Authenticates to a PyPI-compatible feed and uploads built artifacts via twine.
`.Pipelines/template-install-lab-cert.yml`	Retrieves and materializes a lab certificate from Key Vault (currently always invoked by test template).
`.Pipelines/ADO-PUBLISH-SETUP.md`	Documents required ADO service connections/environments and how to run the manual publish pipeline.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.Pipelines/template-run-tests.yml

.Pipelines/pipeline-publish.yml

.Pipelines/ADO-PUBLISH-SETUP.md

.Pipelines/pipeline-publish.yml

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.Pipelines/template-install-lab-cert.yml

.Pipelines/ADO-PUBLISH-SETUP.md

.Pipelines/pipeline-publish.yml

.Pipelines/template-publish-package.yml

msal/sku.py

.Pipelines/template-run-tests.yml

.Pipelines/ADO-PUBLISH-SETUP.md

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.Pipelines/template-run-tests.yml

.Pipelines/pipeline-publish.yml

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.Pipelines/template-pipeline-stages.yml

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.Pipelines/template-pipeline-stages.yml

.Pipelines/ADO-PUBLISH-SETUP.md

.Pipelines/template-pipeline-stages.yml

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.Pipelines/template-pipeline-stages.yml

Copilot

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.Pipelines/template-pipeline-stages.yml

.Pipelines/pipeline-publish.yml

.Pipelines/ADO-PUBLISH-SETUP.md

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 10 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.Pipelines/ADO-PUBLISH-SETUP.md

.Pipelines/template-pipeline-stages.yml

.Pipelines/pipeline-publish.yml

.Pipelines/ADO-PUBLISH-SETUP.md

azure-pipelines.yml

.Pipelines/ADO-PUBLISH-SETUP.md

.Pipelines/credscan-exclusion.json

.Pipelines/ADO-PUBLISH-SETUP.md

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.Pipelines/pipeline-publish.yml

.Pipelines/ADO-PUBLISH-SETUP.md

tests/test_e2e.py

…pport

…p/sed

…stead

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

tests/test_e2e.py:62

_clean_env() is implemented here and also in tests/lab_config.py with nearly identical logic. To avoid drift and inconsistent behavior, consider centralizing this helper (e.g., make it a shared utility in tests/ or expose a single public helper from lab_config and reuse it here).


def _get_app_and_auth_code(
        client_id,
        client_secret=None,
        authority="https://login.microsoftonline.com/common",
        port=44331,

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.Pipelines/CI-AND-RELEASE-PIPELINES.md

.Pipelines/pipeline-publish.yml

… undefined variable

… cert is absent

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.Pipelines/CI-AND-RELEASE-PIPELINES.md

tests/test_e2e.py

tests/lab_config.py

… dev only)

…NES.md

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tests/lab_config.py

azure-pipelines.yml

… fix doc diagram, drop redundant import os

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.Pipelines/template-pipeline-stages.yml

RyAuld changed the base branch from main to dev March 20, 2026 22:03

RyAuld requested a review from a team as a code owner March 20, 2026 22:03

gladjohn reviewed Mar 21, 2026

View reviewed changes

.Pipelines/pipeline-publish.yml Outdated Show resolved Hide resolved

gladjohn approved these changes Mar 21, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings March 23, 2026 16:43

Copilot started reviewing on behalf of RyAuld March 23, 2026 16:45 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings March 23, 2026 16:55

Copilot started reviewing on behalf of RyAuld March 23, 2026 16:58 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

bgavrilMS reviewed Mar 23, 2026

View reviewed changes

.Pipelines/template-run-tests.yml Outdated Show resolved Hide resolved

bgavrilMS requested changes Mar 23, 2026

View reviewed changes

bgavrilMS reviewed Mar 23, 2026

View reviewed changes

.Pipelines/ADO-PUBLISH-SETUP.md Outdated Show resolved Hide resolved

Copilot AI review requested due to automatic review settings March 23, 2026 17:07

Copilot started reviewing on behalf of RyAuld March 23, 2026 17:08 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

.Pipelines/template-run-tests.yml Outdated Show resolved Hide resolved

.Pipelines/pipeline-publish.yml Outdated Show resolved Hide resolved

Copilot AI review requested due to automatic review settings March 23, 2026 18:22

Copilot AI reviewed Mar 23, 2026

View reviewed changes

Copilot started reviewing on behalf of RyAuld March 23, 2026 18:34 View session

Copilot AI review requested due to automatic review settings March 23, 2026 18:44

Copilot started reviewing on behalf of RyAuld March 23, 2026 18:46 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings March 26, 2026 20:26

Copilot started reviewing on behalf of RyAuld March 26, 2026 20:27 View session

Copilot AI reviewed Mar 26, 2026

View reviewed changes

.Pipelines/template-pipeline-stages.yml Outdated Show resolved Hide resolved

.Pipelines/template-pipeline-stages.yml Outdated Show resolved Hide resolved

.Pipelines/template-pipeline-stages.yml Outdated Show resolved Hide resolved

Copilot AI review requested due to automatic review settings March 30, 2026 16:08

Copilot started reviewing on behalf of RyAuld March 30, 2026 16:09 View session

Copilot AI reviewed Mar 30, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings March 30, 2026 16:38

Copilot started reviewing on behalf of RyAuld March 30, 2026 16:38 View session

Copilot AI reviewed Mar 30, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings March 30, 2026 21:49

Copilot started reviewing on behalf of RyAuld March 30, 2026 21:50 View session

Copilot AI reviewed Mar 30, 2026

View reviewed changes

.Pipelines/pipeline-publish.yml Show resolved Hide resolved

.Pipelines/ADO-PUBLISH-SETUP.md Outdated Show resolved Hide resolved

tests/test_e2e.py Outdated Show resolved Hide resolved

Add ADO CI/release pipelines with SDL, e2e tests, and ESRP publish su…

3821a67

…pport

RyAuld force-pushed the RyAuld/PyPI-ADO-PackagePublish branch from 70c7ec2 to 3821a67 Compare March 30, 2026 22:25

RyAuld changed the title ~~Add ADO publish pipeline for PyPI releases~~ Add ADO CI, SDL, and release pipelines with e2e test enablement Mar 30, 2026

Use Python runpy to read __version__ in Validate stage instead of gre…

9dbad21

…p/sed

Copilot AI review requested due to automatic review settings March 30, 2026 22:44

Copilot started reviewing on behalf of RyAuld March 30, 2026 22:45 View session

Remove duplicate _clean_env in test_e2e.py; import from lab_config in…

7f60411

…stead

Copilot AI reviewed Mar 30, 2026

View reviewed changes

.Pipelines/CI-AND-RELEASE-PIPELINES.md Show resolved Hide resolved

.Pipelines/pipeline-publish.yml Outdated Show resolved Hide resolved

RyAuld added 2 commits March 30, 2026 15:49

Pass LabAuth secret via env var to avoid bash command substitution on…

e8e4d55

… undefined variable

Skip Key Vault and cert steps on forked PRs; e2e tests self-skip when…

42aa35a

… cert is absent

Copilot AI review requested due to automatic review settings March 30, 2026 22:51

Copilot started reviewing on behalf of RyAuld March 30, 2026 22:52 View session

RyAuld added 2 commits March 30, 2026 15:52

Fix header comment: production path uses EsrpRelease@9 not Twine SC

ff11b30

Fix CredScan suppression paths: prefix with tests/

5ce2b22

Copilot AI reviewed Mar 30, 2026

View reviewed changes

.Pipelines/CI-AND-RELEASE-PIPELINES.md Outdated Show resolved Hide resolved

tests/test_e2e.py Show resolved Hide resolved

tests/test_e2e.py Outdated Show resolved Hide resolved

tests/lab_config.py Show resolved Hide resolved

RyAuld added 2 commits March 30, 2026 15:57

Add daily scheduled trigger to azure-pipelines.yml (11:45 PM Pacific,…

b2015a7

… dev only)

Fix dead doc reference: ADO-PUBLISH-SETUP.md -> CI-AND-RELEASE-PIPELI…

3fe050b

…NES.md

Copilot AI review requested due to automatic review settings March 30, 2026 22:58

Copilot started reviewing on behalf of RyAuld March 30, 2026 22:59 View session

Copilot AI reviewed Mar 30, 2026

View reviewed changes

tests/lab_config.py Outdated Show resolved Hide resolved

azure-pipelines.yml Outdated Show resolved Hide resolved

RyAuld added 2 commits March 30, 2026 16:02

Address review comments: rename _clean_env to clean_env, fix __all__,…

eddfc53

… fix doc diagram, drop redundant import os

Clarify cron comment: 07:45 UTC = 11:45 PM PST / 12:45 AM PDT

ef6220c

Copilot AI review requested due to automatic review settings March 30, 2026 23:03

Copilot started reviewing on behalf of RyAuld March 30, 2026 23:04 View session

Copilot AI reviewed Mar 30, 2026

View reviewed changes

.Pipelines/template-pipeline-stages.yml Show resolved Hide resolved

.Pipelines/template-pipeline-stages.yml Show resolved Hide resolved

RyAuld requested a review from bgavrilMS March 30, 2026 23:40

Conversation

RyAuld commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Pipeline Files

ADO Pipelines in IDDP

Stage Flow

PR / CI build (\�zure-pipelines.yml)

Release build (\pipeline-publish.yml)

Other Changes

Pending Before Release Can Publish

test.pypi.org (Preview / RC path)

pypi.org (ESRP Production path)

Post-Merge Actions (after this PR merges to \dev)

Uh oh!

Uh oh!

gladjohn left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

RyAuld commented Mar 20, 2026 •

edited

Loading