diff --git a/.Pipelines/CI-AND-RELEASE-PIPELINES.md b/.Pipelines/CI-AND-RELEASE-PIPELINES.md new file mode 100644 index 00000000..d8d18d00 --- /dev/null +++ b/.Pipelines/CI-AND-RELEASE-PIPELINES.md @@ -0,0 +1,126 @@ +# CI/CD Pipelines + +This document describes the pipeline structure for the `msal` Python package, +including what each pipeline does, when it runs, and how to trigger a release. + +--- + +## Pipeline Files + +| File | Purpose | +|------|---------| +| [`azure-pipelines.yml`](../azure-pipelines.yml) | PR gate and post-merge CI — calls the shared template with `runPublish: false` | +| [`pipeline-publish.yml`](pipeline-publish.yml) | Release pipeline — manually queued, builds and publishes to PyPI | +| [`template-pipeline-stages.yml`](template-pipeline-stages.yml) | Shared stages template — PreBuildCheck, Validate, and CI stages reused by both pipelines | +| [`credscan-exclusion.json`](credscan-exclusion.json) | CredScan suppression file for known test fixtures | + +--- + +## PR / CI Pipeline (`azure-pipelines.yml`) + +### Triggers + +| Event | Branches | +|-------|----------| +| Pull request opened / updated | all branches | +| Push / merge | `dev`, `azure-pipelines` | +| Scheduled | Daily at 11:45 PM Pacific, `dev` branch (only when there are new changes) | + +### Stages + +``` +PreBuildCheck ─► CI +``` + +| Stage | What it does | +|-------|-------------| +| **PreBuildCheck** | Runs SDL security scans: PoliCheck (policy/offensive content), CredScan (leaked credentials), and PostAnalysis (breaks the build on findings) | +| **CI** | Runs the full test suite on Python 3.9, 3.10, 3.11, 3.12, 3.13, and 3.14 | + +The Validate stage is **skipped** on PR/CI runs (it only applies to release builds). + +> **SDL coverage:** The PreBuildCheck stage satisfies the OneBranch SDL requirement. +> It runs on every PR, every merge to `dev`, and on the daily schedule — ensuring +> continuous security scanning without a separate dedicated SDL pipeline. + +--- + +## Release Pipeline (`pipeline-publish.yml`) + +### Triggers + +**Manual only** — no automatic branch or tag triggers. Must be queued explicitly +with both parameters filled in. + +### Parameters + +| Parameter | Description | Example values | +|-----------|-------------|----------------| +| **Package version to publish** | Must exactly match `msal/sku.py __version__`. [PEP 440](https://peps.python.org/pep-0440/) format. | `1.36.0`, `1.36.0rc1`, `1.36.0b1` | +| **Publish target** | Destination for this release. | `test.pypi.org (Preview / RC)` or `pypi.org (ESRP Production)` | + +### Stage Flow + +``` +PreBuildCheck ─► Validate ─► CI ─► Build ─┬─► PublishMSALPython (publishTarget == 'test.pypi.org (Preview / RC)') + └─► PublishPyPI (publishTarget == 'pypi.org (ESRP Production)') +``` + +| Stage | What it does | Condition | +|-------|-------------|-----------| +| **PreBuildCheck** | PoliCheck + CredScan scans | Always | +| **Validate** | Asserts the `packageVersion` parameter matches `msal/sku.py __version__` | Always (release runs only) | +| **CI** | Full test matrix (Python 3.9–3.14) | After Validate passes | +| **Build** | Builds `sdist` and `wheel` via `python -m build`; publishes `python-dist` artifact | After CI passes | +| **PublishMSALPython** | Uploads to test.pypi.org | `publishTarget == test.pypi.org (Preview / RC)` | +| **PublishPyPI** | Uploads to PyPI via ESRP; requires manual approval | `publishTarget == pypi.org (ESRP Production)` | + +--- + +## How to Publish a Release + +### Step 1 — Update the version + +Edit `msal/sku.py` and set `__version__` to the target version: + +```python +__version__ = "1.36.0rc1" # RC / preview +__version__ = "1.36.0" # production release +``` + +Push the change to the branch you intend to release from. + +### Step 2 — Queue the pipeline + +1. Go to the **MSAL.Python-Publish** pipeline in ADO. +2. Click **Run pipeline**. +3. Select the branch to release from. +4. Enter the **Package version to publish** (must match `msal/sku.py` exactly). +5. Select the **Publish target**: + - `test.pypi.org (Preview / RC)` — for release candidates and previews + - `pypi.org (ESRP Production)` — for final releases (requires approval gate) +6. Click **Run**. + +### Step 3 — Approve (production releases only) + +The `pypi.org (ESRP Production)` path includes a required manual approval before +the package is uploaded. An approver must review and approve in the ADO +**Environments** panel before the `PublishPyPI` stage proceeds. + +### Step 4 — Verify + +- **test.pypi.org:** https://test.pypi.org/project/msal/ +- **PyPI:** https://pypi.org/project/msal/ + +--- + +## Version Format + +PyPI enforces [PEP 440](https://peps.python.org/pep-0440/). Versions with `-` (e.g. `1.36.0-Preview`) are rejected at upload time. Use standard suffixes: + +| Release type | Format | +|-------------|--------| +| Production | `1.36.0` | +| Release candidate | `1.36.0rc1` | +| Beta | `1.36.0b1` | +| Alpha | `1.36.0a1` | diff --git a/.Pipelines/credscan-exclusion.json b/.Pipelines/credscan-exclusion.json new file mode 100644 index 00000000..4defd2b8 --- /dev/null +++ b/.Pipelines/credscan-exclusion.json @@ -0,0 +1,13 @@ +{ + "tool": "Credential Scanner", + "suppressions": [ + { + "file": "tests/certificate-with-password.pfx", + "_justification": "Self-signed certificate used only in unit tests. Not a production credential." + }, + { + "file": "tests/test_mi.py", + "_justification": "WWW-Authenticate challenge header value used as a mock HTTP response fixture in unit tests. Not a real credential." + } + ] +} diff --git a/.Pipelines/pipeline-publish.yml b/.Pipelines/pipeline-publish.yml new file mode 100644 index 00000000..818dfddb --- /dev/null +++ b/.Pipelines/pipeline-publish.yml @@ -0,0 +1,179 @@ +# pipeline-publish.yml +# +# Release pipeline for the msal Python package — manually triggered only. +# Source: https://github.com/AzureAD/microsoft-authentication-library-for-python +# +# Publish targets: +# test.pypi.org (Preview / RC) — preview releases via MSAL-Test-Python-Upload SC +# (SC creation pending test.pypi.org API token) +# pypi.org (ESRP Production) — production releases via ESRP (EsrpRelease@9) using MSAL-ESRP-AME SC +# +# For pipeline documentation, see .Pipelines/CI-AND-RELEASE-PIPELINES.md. + +parameters: +- name: packageVersion + displayName: 'Package version to publish (must match msal/sku.py, e.g. 1.36.0 or 1.36.0rc1)' + type: string + +- name: publishTarget + displayName: 'Publish target' + type: string + values: + - 'test.pypi.org (Preview / RC)' + - 'pypi.org (ESRP Production)' + +trigger: none # manual runs only — no automatic branch or tag triggers +pr: none + +# Stage flow: +# +# PreBuildCheck ─► Validate ─► CI ─► Build ─► PublishMSALPython (publishTarget == Preview) +# └─► PublishPyPI (publishTarget == ESRP Production) + +stages: + +# PreBuildCheck, Validate, and CI stages are defined in the shared template. +- template: template-pipeline-stages.yml + parameters: + packageVersion: ${{ parameters.packageVersion }} + runPublish: true + +# ══════════════════════════════════════════════════════════════════════════════ +# Stage 3 · Build — build sdist + wheel +# ══════════════════════════════════════════════════════════════════════════════ +- stage: Build + displayName: 'Build package' + dependsOn: CI + condition: eq(dependencies.CI.result, 'Succeeded') + jobs: + - job: BuildDist + displayName: 'Build sdist + wheel (Python 3.12)' + pool: + vmImage: ubuntu-latest + steps: + - task: UsePythonVersion@0 + inputs: + versionSpec: '3.12' + displayName: 'Use Python 3.12' + + - script: | + python -m pip install --upgrade pip build twine + displayName: 'Install build toolchain' + + - script: | + python -m build + displayName: 'Build sdist and wheel' + + - script: | + python -m twine check dist/* + displayName: 'Verify distribution (twine check)' + + - task: PublishPipelineArtifact@1 + displayName: 'Publish dist/ as pipeline artifact' + inputs: + targetPath: dist/ + artifact: python-dist + +# ══════════════════════════════════════════════════════════════════════════════ +# Stage 4a · Publish to test.pypi.org (Preview / RC) +# Note: requires MSAL-Test-Python-Upload SC in ADO (pending test.pypi.org API token) +# ══════════════════════════════════════════════════════════════════════════════ +- stage: PublishMSALPython + displayName: 'Publish to test.pypi.org (Preview)' + dependsOn: Build + condition: > + and( + eq(dependencies.Build.result, 'Succeeded'), + eq('${{ parameters.publishTarget }}', 'test.pypi.org (Preview / RC)') + ) + jobs: + - deployment: DeployMSALPython + displayName: 'Upload to test.pypi.org' + pool: + vmImage: ubuntu-latest + environment: MSAL-Python + strategy: + runOnce: + deploy: + steps: + - task: DownloadPipelineArtifact@2 + displayName: 'Download python-dist artifact' + inputs: + artifactName: python-dist + targetPath: $(Pipeline.Workspace)/python-dist + + - task: UsePythonVersion@0 + inputs: + versionSpec: '3.12' + displayName: 'Use Python 3.12' + + - script: | + python -m pip install --upgrade pip twine + displayName: 'Install twine' + + # TODO: create MSAL-Test-Python-Upload SC with test.pypi.org API token, then uncomment: + # - task: TwineAuthenticate@1 + # displayName: 'Authenticate with MSAL-Test-Python-Upload' + # inputs: + # pythonUploadServiceConnection: MSAL-Test-Python-Upload + + # - script: | + # python -m twine upload \ + # -r "MSAL-Test-Python-Upload" \ + # --config-file $(PYPIRC_PATH) \ + # --skip-existing \ + # $(Pipeline.Workspace)/python-dist/* + # displayName: 'Upload to test.pypi.org' + + - script: echo "Publish to test.pypi.org skipped — MSAL-Test-Python-Upload SC not yet created." + displayName: 'Skip upload (SC pending)' + +# ══════════════════════════════════════════════════════════════════════════════ +# Stage 4b · Publish to PyPI (ESRP Production) +# Uses EsrpRelease@9 via the MSAL-ESRP-AME service connection. +# IMPORTANT: configure a required manual approval on this environment in +# ADO → Pipelines → Environments → MSAL-Python-Release → Approvals and checks. +# IMPORTANT: EsrpRelease@9 requires a Windows agent. +# ══════════════════════════════════════════════════════════════════════════════ +- stage: PublishPyPI + displayName: 'Publish to PyPI (ESRP Production)' + dependsOn: Build + condition: > + and( + eq(dependencies.Build.result, 'Succeeded'), + eq('${{ parameters.publishTarget }}', 'pypi.org (ESRP Production)') + ) + jobs: + - deployment: DeployPyPI + displayName: 'Upload to PyPI via ESRP' + pool: + vmImage: windows-latest + environment: MSAL-Python-Release + strategy: + runOnce: + deploy: + steps: + - task: DownloadPipelineArtifact@2 + displayName: 'Download python-dist artifact' + inputs: + artifactName: python-dist + targetPath: $(Pipeline.Workspace)/python-dist + + - task: EsrpRelease@9 + displayName: 'Publish to PyPI via ESRP' + inputs: + connectedservicename: 'MSAL-ESRP-AME' + usemanagedidentity: true + keyvaultname: 'MSALVault' + signcertname: 'MSAL-ESRP-Release-Signing' + clientid: '8650ce2b-38d4-466a-9144-bc5c19c88112' + intent: 'PackageDistribution' + contenttype: 'PyPi' + contentsource: 'Folder' + folderlocation: '$(Pipeline.Workspace)/python-dist' + waitforreleasecompletion: true + owners: 'ryauld@microsoft.com,avdunn@microsoft.com' + approvers: 'avdunn@microsoft.com,bogavril@microsoft.com' + serviceendpointurl: 'https://api.esrp.microsoft.com' + mainpublisher: 'ESRPRELPACMAN' + domaintenantid: '33e01921-4d64-4f8c-a055-5bdaffd5e33d' diff --git a/.Pipelines/template-pipeline-stages.yml b/.Pipelines/template-pipeline-stages.yml new file mode 100644 index 00000000..972121c3 --- /dev/null +++ b/.Pipelines/template-pipeline-stages.yml @@ -0,0 +1,204 @@ +# template-pipeline-stages.yml +# +# Shared stages template for the msal Python package. +# +# Called from: +# pipeline-publish.yml — release build (runPublish: true) +# azure-pipelines.yml — PR gate and post-merge CI (runPublish: false) +# +# Parameters: +# packageVersion - Version to validate against msal/sku.py +# Required when runPublish is true; unused otherwise. +# runPublish - When true: also runs the Validate stage before CI. +# When false (PR / merge builds): only PreBuildCheck + CI run. +# +# Stage flow: +# +# runPublish: true → PreBuildCheck ─► Validate ─► CI +# runPublish: false → PreBuildCheck ─► CI (Validate is skipped) +# +# Build and Publish stages are defined in pipeline-publish.yml (not here), +# so that the PR build never references PyPI service connections. + +parameters: +- name: packageVersion + type: string + default: '' +- name: runPublish + type: boolean + default: false + +stages: + +# ══════════════════════════════════════════════════════════════════════════════ +# Stage 0 · PreBuildCheck — SDL security scans (PoliCheck + CredScan) +# Always runs, mirrors MSAL.NET pre-build analysis. +# ══════════════════════════════════════════════════════════════════════════════ +- stage: PreBuildCheck + displayName: 'Pre-build security checks' + jobs: + - job: SecurityScan + displayName: 'PoliCheck + CredScan' + pool: + vmImage: windows-latest + variables: + Codeql.SkipTaskAutoInjection: true + steps: + - task: NodeTool@0 + displayName: 'Install Node.js (includes npm)' + inputs: + versionSpec: '20.x' + + - task: securedevelopmentteam.vss-secure-development-tools.build-task-policheck.PoliCheck@2 + displayName: 'Run PoliCheck' + inputs: + targetType: F + continueOnError: true + + - task: securedevelopmentteam.vss-secure-development-tools.build-task-credscan.CredScan@3 + displayName: 'Run CredScan' + inputs: + suppressionsFile: '$(Build.SourcesDirectory)/.Pipelines/credscan-exclusion.json' + toolMajorVersion: V2 + debugMode: false + + - task: securedevelopmentteam.vss-secure-development-tools.build-task-postanalysis.PostAnalysis@2 + displayName: 'Post Analysis' + inputs: + GdnBreakGdnToolCredScan: true + GdnBreakGdnToolPoliCheck: true + +# ══════════════════════════════════════════════════════════════════════════════ +# Stage 1 · Validate — verify packageVersion matches msal/sku.py __version__ +# Skipped when runPublish is false (PR / merge builds). +# ══════════════════════════════════════════════════════════════════════════════ +- stage: Validate + displayName: 'Validate version' + dependsOn: PreBuildCheck + condition: and(${{ parameters.runPublish }}, eq(dependencies.PreBuildCheck.result, 'Succeeded')) + jobs: + - job: ValidateVersion + displayName: 'Check version matches source' + pool: + vmImage: ubuntu-latest + steps: + - task: UsePythonVersion@0 + inputs: + versionSpec: '3.12' + displayName: 'Set up Python' + + - script: | + python - <<'EOF' + import sys, runpy + ns = runpy.run_path("msal/sku.py") + sku_ver = ns.get("__version__", "") + param_ver = "${{ parameters.packageVersion }}" + if not param_ver: + print("##vso[task.logissue type=error]packageVersion is required. Enter the version to publish (must match msal/sku.py __version__).") + sys.exit(1) + elif param_ver != sku_ver: + print(f"##vso[task.logissue type=error]Version mismatch: parameter '{param_ver}' != msal/sku.py '{sku_ver}'") + print("Update msal/sku.py __version__ to match the packageVersion parameter, or correct the parameter.") + sys.exit(1) + else: + print(f"Version validated: {param_ver}") + EOF + displayName: 'Verify version parameter matches msal/sku.py' + +# ══════════════════════════════════════════════════════════════════════════════ +# Stage 2 · CI — run the full test matrix across all supported Python versions. +# Always runs. Waits for Validate when runPublish is true; +# runs immediately when Validate is skipped (PR / merge builds). +# ══════════════════════════════════════════════════════════════════════════════ +- stage: CI + displayName: 'Run tests' + dependsOn: + - PreBuildCheck + - Validate + condition: | + and( + eq(dependencies.PreBuildCheck.result, 'Succeeded'), + in(dependencies.Validate.result, 'Succeeded', 'Skipped') + ) + jobs: + - job: Test + displayName: 'Run unit tests' + pool: + vmImage: ubuntu-latest + strategy: + matrix: + Python39: + python.version: '3.9' + Python310: + python.version: '3.10' + Python311: + python.version: '3.11' + Python312: + python.version: '3.12' + Python313: + python.version: '3.13' + Python314: + python.version: '3.14' + steps: + # Retrieve the MSID Lab certificate from Key Vault (via AuthSdkResourceManager SC). + # Matches the pattern used by MSAL.js (install-keyvault-secrets.yml) and MSAL Java. + # Skipped on forked PRs — service connections are not available to forks. + # E2E tests self-skip when LAB_APP_CLIENT_CERT_PFX_PATH is unset. + - task: AzureKeyVault@2 + displayName: 'Retrieve lab certificate from Key Vault' + condition: ne(variables['System.PullRequest.IsFork'], 'True') + inputs: + azureSubscription: 'AuthSdkResourceManager' + KeyVaultName: 'msidlabs' + SecretsFilter: 'LabAuth' + RunAsPreJob: false + + - bash: | + set -euo pipefail + if [ -z "${LAB_AUTH_B64:-}" ]; then + echo "##vso[task.logissue type=error]LabAuth secret is empty or was not injected — Key Vault retrieval may have failed." + exit 1 + fi + CERT_PATH="$(Agent.TempDirectory)/lab-auth.pfx" + printf '%s' "$LAB_AUTH_B64" | base64 -d > "$CERT_PATH" + echo "##vso[task.setvariable variable=LAB_APP_CLIENT_CERT_PFX_PATH]$CERT_PATH" + echo "Lab cert written to: $CERT_PATH ($(wc -c < "$CERT_PATH") bytes)" + displayName: 'Write lab certificate to disk' + condition: ne(variables['System.PullRequest.IsFork'], 'True') + env: + LAB_AUTH_B64: $(LabAuth) + + - task: UsePythonVersion@0 + inputs: + versionSpec: '$(python.version)' + displayName: 'Set up Python' + + - script: | + python -m pip install --upgrade pip + pip install -r requirements.txt + displayName: 'Install dependencies' + + # Use bash: explicitly; set -o pipefail so that pytest failures aren't hidden by the pipe to tee. + # Without pipefail, tee exits 0 and the step can succeed even when tests fail. + # (set -o pipefail also works in script: steps, but bash: makes the shell choice explicit.) + - bash: | + pip install pytest pytest-azurepipelines + mkdir -p test-results + set -o pipefail + pytest -vv --junitxml=test-results/junit.xml 2>&1 | tee test-results/pytest.log + displayName: 'Run tests' + env: + LAB_APP_CLIENT_CERT_PFX_PATH: $(LAB_APP_CLIENT_CERT_PFX_PATH) + + - task: PublishTestResults@2 + displayName: 'Publish test results' + condition: succeededOrFailed() + inputs: + testResultsFormat: 'JUnit' + testResultsFiles: 'test-results/junit.xml' + failTaskOnFailedTests: true + testRunTitle: 'Python $(python.version)' + + - bash: rm -f "$(Agent.TempDirectory)/lab-auth.pfx" + displayName: 'Clean up lab certificate' + condition: always() diff --git a/azure-pipelines.yml b/azure-pipelines.yml index 0426b1eb..b800165e 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -1,59 +1,26 @@ -# Derived from the default YAML generated by Azure DevOps for a Python package -# Create and test a Python package on multiple Python versions. -# Add steps that analyze code, save the dist with the build record, publish to a PyPI-compatible index, and more: -# https://docs.microsoft.com/azure/devops/pipelines/languages/python +# PR gate and branch CI for the msal Python package. +# Runs on pushes to dev/azure-pipelines, on all pull requests, and on a daily schedule. +# Delegates all stages to .Pipelines/template-pipeline-stages.yml with +# runPublish: false — PreBuildCheck (SDL scans) + CI (test matrix) only. trigger: - dev - azure-pipelines -pool: - vmImage: ubuntu-latest -strategy: - matrix: - Python39: - python.version: '3.9' - Python310: - python.version: '3.10' - Python311: - python.version: '3.11' - Python312: - python.version: '3.12' - Python313: - python.version: '3.13' - Python314: - python.version: '3.14' +pr: + branches: + include: + - '*' -steps: -- task: UsePythonVersion@0 - inputs: - versionSpec: '$(python.version)' - displayName: 'Use Python $(python.version)' +schedules: +- cron: '45 7 * * *' # 07:45 UTC daily (11:45 PM PST / 12:45 AM PDT) — matches legacy MSAL-Python-SDL-CI schedule + displayName: 'Daily SDL + CI (dev)' + branches: + include: + - dev + always: false # only run when there are new changes -- script: | - python -m pip install --upgrade pip - pip install -r requirements.txt - displayName: 'Install dependencies' - -- script: | - pip install pytest pytest-azurepipelines - mkdir -p test-results - set -o pipefail - pytest -vv --junitxml=test-results/junit.xml 2>&1 | tee test-results/pytest.log - displayName: 'pytest (verbose + junit + log)' - -- task: PublishTestResults@2 - displayName: 'Publish test results' - condition: succeededOrFailed() - inputs: - testResultsFormat: 'JUnit' - testResultsFiles: 'test-results/junit.xml' - failTaskOnFailedTests: true - testRunTitle: 'Python $(python.version) pytest' - -- task: PublishPipelineArtifact@1 - displayName: 'Publish pytest log artifact' - condition: succeededOrFailed() - inputs: - targetPath: 'test-results' - artifact: 'pytest-logs-$(python.version)' +stages: +- template: .Pipelines/template-pipeline-stages.yml + parameters: + runPublish: false diff --git a/msal/sku.py b/msal/sku.py index 01751048..19ff0138 100644 --- a/msal/sku.py +++ b/msal/sku.py @@ -1,6 +1,6 @@ -"""This module is from where we recieve the client sku name and version. +"""This module is from where we receive the client sku name and version. """ # The __init__.py will import this. Not the other way around. -__version__ = "1.35.0" +__version__ = "1.35.2rc1" SKU = "MSAL.Python" diff --git a/tests/lab_config.py b/tests/lab_config.py index f99b1683..8b692e3f 100644 --- a/tests/lab_config.py +++ b/tests/lab_config.py @@ -20,7 +20,6 @@ app = get_app_config(AppSecrets.PCA_CLIENT) Environment Variables: - LAB_APP_CLIENT_ID: Client ID for Key Vault authentication (required) LAB_APP_CLIENT_CERT_PFX_PATH: Path to .pfx certificate file (required) """ @@ -37,6 +36,7 @@ __all__ = [ # Constants + "LAB_APP_CLIENT_ID", "UserSecrets", "AppSecrets", # Data classes @@ -48,6 +48,7 @@ "get_app_config", "get_user_password", "get_client_certificate", + "clean_env", ] # ============================================================================= @@ -57,6 +58,12 @@ _MSID_LAB_VAULT = "https://msidlabs.vault.azure.net" _MSAL_TEAM_VAULT = "https://id4skeyvault.vault.azure.net" +# Client ID for the RequestMSIDLAB app used to authenticate against the lab +# Key Vaults. Hardcoded here following the same pattern as MSAL.NET +# (see build/template-install-keyvault-secrets.yaml in that repo). +# See https://docs.msidlab.com/accounts/confidentialclient.html +LAB_APP_CLIENT_ID = "f62c5ae3-bf3a-4af5-afa8-a68b800396e9" + # ============================================================================= # Secret Name Constants # ============================================================================= @@ -164,6 +171,21 @@ class AppConfig: _msal_team_client: Optional[SecretClient] = None +def clean_env(name: str) -> Optional[str]: + """Return the env var value, or None if unset or it contains an unexpanded + ADO pipeline variable literal such as ``$(VAR_NAME)``. + + Azure DevOps injects the literal string ``$(VAR_NAME)`` when a ``$(...)`` + reference in a step ``env:`` block refers to a variable that has not been + defined at runtime. That literal is truthy, so a plain ``os.getenv()`` + check would incorrectly proceed as if the variable were set. + """ + value = os.getenv(name) + if value and value.startswith("$("): + return None + return value or None + + def _get_credential(): """ Create an Azure credential for Key Vault access. @@ -177,19 +199,14 @@ def _get_credential(): Raises: EnvironmentError: If required environment variables are not set. """ - client_id = os.getenv("LAB_APP_CLIENT_ID") - cert_path = os.getenv("LAB_APP_CLIENT_CERT_PFX_PATH") + cert_path = clean_env("LAB_APP_CLIENT_CERT_PFX_PATH") tenant_id = "72f988bf-86f1-41af-91ab-2d7cd011db47" # Microsoft tenant - - if not client_id: - raise EnvironmentError( - "LAB_APP_CLIENT_ID environment variable is required for Key Vault access") - + if cert_path: logger.debug("Using certificate credential for Key Vault access") return CertificateCredential( tenant_id=tenant_id, - client_id=client_id, + client_id=LAB_APP_CLIENT_ID, certificate_path=cert_path, send_certificate_chain=True, ) @@ -396,7 +413,7 @@ def get_client_certificate() -> Dict[str, object]: Raises: EnvironmentError: If LAB_APP_CLIENT_CERT_PFX_PATH is not set. """ - cert_path = os.getenv("LAB_APP_CLIENT_CERT_PFX_PATH") + cert_path = clean_env("LAB_APP_CLIENT_CERT_PFX_PATH") if not cert_path: raise EnvironmentError( "LAB_APP_CLIENT_CERT_PFX_PATH environment variable is required " @@ -407,4 +424,4 @@ def get_client_certificate() -> Dict[str, object]: return { "private_key_pfx_path": cert_path, "public_certificate": True, # Enable SNI (send certificate chain) - } \ No newline at end of file + } diff --git a/tests/test_e2e.py b/tests/test_e2e.py index 1202d443..37632ee7 100644 --- a/tests/test_e2e.py +++ b/tests/test_e2e.py @@ -1,5 +1,4 @@ """If the following ENV VAR were available, many end-to-end test cases would run. -LAB_APP_CLIENT_ID=... LAB_APP_CLIENT_CERT_PFX_PATH=... """ try: @@ -29,7 +28,7 @@ from tests.broker_util import is_pymsalruntime_installed from tests.lab_config import ( get_user_config, get_app_config, get_user_password, get_secret, - UserSecrets, AppSecrets, + UserSecrets, AppSecrets, LAB_APP_CLIENT_ID, clean_env, ) @@ -44,7 +43,17 @@ _PYMSALRUNTIME_INSTALLED = is_pymsalruntime_installed() _AZURE_CLI = "04b07795-8ddb-461a-bbee-02f9e1bf7b46" -_SKIP_UNATTENDED_E2E_TESTS = os.getenv("TRAVIS") or not os.getenv("CI") +# Skip interactive / browser-dependent tests when: +# - on Travis CI (TRAVIS), or +# - on Azure DevOps (TF_BUILD) where there is no display/browser on the agent, or +# - not running in any CI environment at all (not CI). +# Service-principal and ROPC tests are NOT gated on this flag; only tests that +# call acquire_token_interactive() or acquire_token_by_device_flow() are. +_SKIP_UNATTENDED_E2E_TESTS = ( + os.getenv("TRAVIS") or os.getenv("TF_BUILD") or not os.getenv("CI") +) + + def _get_app_and_auth_code( client_id, @@ -329,13 +338,16 @@ def test_access_token_should_be_obtained_for_a_supported_scope(self): self.assertIsNotNone(result.get("access_token")) -@unittest.skipIf(os.getenv("TF_BUILD"), "Skip PublicCloud scenarios on Azure DevOps") class PublicCloudScenariosTestCase(E2eTestCase): # Historically this class was driven by tests/config.json for semi-automated runs. - # It now uses lab config + env vars so it can run automatically without local files. + # It now uses lab config + env vars so it can run automatically on any CI + # (including Azure DevOps) as long as LAB_APP_CLIENT_CERT_PFX_PATH is set. @classmethod def setUpClass(cls): + if not clean_env("LAB_APP_CLIENT_CERT_PFX_PATH"): + raise unittest.SkipTest( + "LAB_APP_CLIENT_CERT_PFX_PATH not set; skipping PublicCloud e2e tests") pca_app = get_app_config(AppSecrets.PCA_CLIENT) user = get_user_config(UserSecrets.PUBLIC_CLOUD) cls.config = { @@ -416,13 +428,11 @@ def test_client_secret(self): def test_subject_name_issuer_authentication(self): from tests.lab_config import get_client_certificate - - client_id = os.getenv("LAB_APP_CLIENT_ID") - if not client_id: - self.skipTest("LAB_APP_CLIENT_ID environment variable is required") + if not clean_env("LAB_APP_CLIENT_CERT_PFX_PATH"): + self.skipTest("LAB_APP_CLIENT_CERT_PFX_PATH not set") self.app = msal.ConfidentialClientApplication( - client_id, + LAB_APP_CLIENT_ID, authority="https://login.microsoftonline.com/microsoft.onmicrosoft.com", client_credential=get_client_certificate(), http_client=MinimalHttpClient()) @@ -447,7 +457,6 @@ def manual_test_device_flow(self): def get_lab_app( - env_client_id="LAB_APP_CLIENT_ID", env_client_cert_path="LAB_APP_CLIENT_CERT_PFX_PATH", authority="https://login.microsoftonline.com/" "72f988bf-86f1-41af-91ab-2d7cd011db47", # Microsoft tenant ID @@ -455,19 +464,20 @@ def get_lab_app( **kwargs): """Returns the lab app as an MSAL confidential client. - Get it from environment variables if defined, otherwise fall back to use MSI. + Uses the hardcoded lab app client ID (RequestMSIDLAB) and a certificate + from the LAB_APP_CLIENT_CERT_PFX_PATH env var. """ logger.info( - "Reading ENV variables %s and %s for lab app defined at " + "Reading ENV variable %s for lab app defined at " "https://docs.msidlab.com/accounts/confidentialclient.html", - env_client_id, env_client_cert_path) - if os.getenv(env_client_id) and os.getenv(env_client_cert_path): + env_client_cert_path) + cert_path = clean_env(env_client_cert_path) + if cert_path: # id came from https://docs.msidlab.com/accounts/confidentialclient.html - client_id = os.getenv(env_client_id) client_credential = { "private_key_pfx_path": # Cert came from https://ms.portal.azure.com/#@microsoft.onmicrosoft.com/asset/Microsoft_Azure_KeyVault/Certificate/https://msidlabs.vault.azure.net/certificates/LabAuth - os.getenv(env_client_cert_path), + cert_path, "public_certificate": True, # Opt in for SNI } else: @@ -475,7 +485,7 @@ def get_lab_app( # See also https://microsoft.sharepoint-df.com/teams/MSIDLABSExtended/SitePages/Programmatically-accessing-LAB-API's.aspx raise unittest.SkipTest("MSI-based mechanism has not been implemented yet") return msal.ConfidentialClientApplication( - client_id, + LAB_APP_CLIENT_ID, client_credential=client_credential, authority=authority, http_client=MinimalHttpClient(timeout=timeout), @@ -831,7 +841,6 @@ def test_user_account(self): class WorldWideTestCase(LabBasedTestCase): - _ADFS_LABS_UNAVAILABLE = "ADFS labs were temporarily down since July 2025 until further notice" def test_aad_managed_user(self): # Pure cloud """Test username/password flow for a managed AAD user.""" @@ -846,7 +855,6 @@ def test_aad_managed_user(self): # Pure cloud scope=["https://graph.microsoft.com/.default"], ) - @unittest.skip(_ADFS_LABS_UNAVAILABLE) def test_adfs2022_fed_user(self): """Test username/password flow for a federated user via ADFS 2022.""" app = get_app_config(AppSecrets.PCA_CLIENT) @@ -1159,18 +1167,15 @@ def _test_acquire_token_for_client(self, configured_region, expected_region): Uses the lab app certificate for authentication. """ - import os from tests.lab_config import get_client_certificate - # Get client ID from environment and certificate from lab_config - client_id = os.getenv("LAB_APP_CLIENT_ID") - if not client_id: - self.skipTest("LAB_APP_CLIENT_ID environment variable is required") - + # Get client ID from lab_config constant and certificate from lab_config + if not clean_env("LAB_APP_CLIENT_CERT_PFX_PATH"): + self.skipTest("LAB_APP_CLIENT_CERT_PFX_PATH is required") client_credential = get_client_certificate() self.app = msal.ConfidentialClientApplication( - client_id, + LAB_APP_CLIENT_ID, client_credential=client_credential, authority="https://login.microsoftonline.com/microsoft.onmicrosoft.com", azure_region=configured_region,