Skip to content

Refactor Python package release workflow #69

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
MMathisLab wants to merge 6 commits into from
Closed

Refactor Python package release workflow #69

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
a8ae666
Refactor Python package release workflow
MMathisLab Mar 23, 2026
99fb5fb
Update python-package.yml
MMathisLab Mar 23, 2026
309ce83
Update .github/workflows/python-package.yml
MMathisLab Mar 23, 2026
c971bf3
Fix pip cache ordering: move checkout first, use setup-python built-i…
Copilot Mar 23, 2026
bd3cd2f
Fix apt get install (#70)
Copilot Mar 23, 2026
6aba304
Add manual release_tag input and validation
C-Achard Mar 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
193 changes: 38 additions & 155 deletions .github/workflows/python-package.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,45 +1,31 @@
name: Build, validate & Release

# Usage:
# - For PRs: this workflow runs automatically to validate the package builds and installs correctly on multiple Python versions. No artifacts are published for PRs.
# - For releases: when you push a tag like v1.2.3, this workflow runs the full matrix validation, then builds the release artifacts, and finally publishes to PyPI if all checks pass.
# - For manual re-runs: use "Run workflow" and provide a tag-like value such as v2.0.0rc1.
name: Update pypi release

on:
workflow_dispatch:
inputs:
release_tag:
description: 'Tag to build/publish (e.g. v1.2.3 or v2.0.0rc1)'
required: true
type: string
# Release pipeline: run only when pushing a version-like tag (e.g. v1.2.3)
# Test pipeline: run tests on main/master pushes & pull requests AND tags.
push:
tags:
- "v*.*.*"
- 'v*.*.*'

pull_request:
branches:
- main
- master
types:
- labeled
- opened
- edited
- synchronize
- reopened

# Validation pipeline: run on PRs targeting main/master (no publishing)
pull_request:
branches: [main, master]
types: [opened, edited, synchronize, reopened]

# This workflow only needs to read repo contents
permissions:
contents: read
workflow_dispatch:
inputs:
release_tag:
description: 'Tag to build/publish (e.g. v1.2.3 or v2.0.0rc1)'
required: true
type: string

jobs:
test_matrix:
# PR + tag validation: ensure the project builds and installs on multiple Pythons
name: Test install & smoke (Py ${{ matrix.python-version }})
release:
runs-on: ubuntu-latest
strategy:
# Run all versions even if one fails (helps spot version-specific issues)
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12"]

steps:
- name: Validate manual tag input
Expand All @@ -56,15 +42,19 @@ jobs:
;;
esac

- name: Checkout sources
- name: Checkout code
uses: actions/checkout@v6
with:
ref: ${{ github.event_name == 'workflow_dispatch' && format('refs/tags/{0}', inputs.release_tag) || github.ref }}

- name: Set up Python
- name: Setup Python
id: setup-python
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}
python-version: '3.x'
cache: 'pip'
cache-dependency-path: |
pyproject.toml

- name: Install Qt/OpenGL runtime deps (Ubuntu)
run: |
Expand All @@ -76,128 +66,21 @@ jobs:
libxkbcommon-x11-0 \
libxcb-cursor0

# Install packaging toolchain:
# - build: creates wheel + sdist
# - twine: validates metadata and can upload (upload only happens in publish job)
- name: Install build tools
run: python -m pip install -U pip build twine

# Build distributions just to verify packaging config works on this Python
- name: Build (for validation only)
run: python -m build

# Validate dist metadata (README rendering, required fields, etc.)
- name: Twine check
run: python -m twine check dist/*

# Smoke test: install the built wheel and verify the package imports
- name: Install from wheel & smoke test
run: |
WHEEL=$(ls -1 dist/*.whl | head -n 1)
echo "Using wheel: $WHEEL"
python -m pip install \
--extra-index-url https://download.pytorch.org/whl/cpu \
"deeplabcut-live-gui[pytorch] @ file://$(pwd)/${WHEEL}"
python -c "import dlclivegui; print('Imported dlclivegui OK')"
QT_QPA_PLATFORM=offscreen dlclivegui --help

build_release:
# Tag-only build: produce the "official" release artifacts once matrix passed
name: Build release artifacts
runs-on: ubuntu-latest
needs: test_matrix
# Safety gate: only run for version tags, never for PRs/branches
if: ${{ startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch' }}

steps:
- name: Validate manual tag input
if: ${{ github.event_name == 'workflow_dispatch' }}
shell: bash
run: |
case "${{ inputs.release_tag }}" in
v*.*.*)
echo "Manual release tag accepted: ${{ inputs.release_tag }}"
;;
*)
echo "release_tag must look like v1.2.3 (or similar, e.g. v2.0.0rc1)"
exit 1
;;
esac
# Fetch sources for the tagged revision
- name: Checkout sources
uses: actions/checkout@v6
with:
# For a manual run, we want to check out the commit at the provided tag
ref: ${{ github.event_name == 'workflow_dispatch' && format('refs/tags/{0}', inputs.release_tag) || github.ref }}

# Use a single, modern Python for the canonical release build
- name: Set up Python (release build)
uses: actions/setup-python@v6
with:
python-version: "3.12"

# Install build + validation tooling
- name: Install build tools
run: python -m pip install -U pip build twine

# Produce both sdist and wheel in dist/
- name: Build distributions
run: python -m build

# Re-check metadata on the final artifacts we intend to publish
- name: Twine check
run: python -m twine check dist/*

# Store dist/ outputs so the publish job uploads exactly what we built here
- name: Upload dist artifacts
uses: actions/upload-artifact@v4
with:
name: dist
path: dist/*

publish:
# Tag-only publish: download built artifacts and upload them to PyPI
name: Publish to PyPI (API token)
runs-on: ubuntu-latest
needs: build_release
# Safety gate: only run for version tags
if: ${{ startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch' }}

steps:
# Retrieve the exact distributions produced in build_release
- name: Download dist artifacts
uses: actions/download-artifact@v4
with:
name: dist
path: dist

# Set up Python (only needed to run Twine)
- name: Set up Python (publish)
uses: actions/setup-python@v6
with:
python-version: "3.12"

# Install twine for uploading
- name: Install Twine
run: python -m pip install -U twine

# Check that the PyPI API token is present before attempting upload (fails fast if not set)
- name: Check PyPI credential presence
shell: bash
env:
TWINE_PASSWORD: ${{ secrets.TWINE_API_KEY }}
- name: Install dependencies
run: |
if [ -z "$TWINE_PASSWORD" ]; then
echo "TWINE_PASSWORD is empty"
exit 1
else
echo "TWINE_PASSWORD is present"
fi

# Upload to PyPI using an API token stored in repo secrets.
# --skip-existing avoids failing if you re-run a workflow for the same version.
- name: Publish to PyPI
pip install --upgrade pip
pip install wheel
pip install "packaging>=24.2"
pip install build
pip install twine

- name: Build and publish to PyPI
if: ${{ github.event_name == 'push' }}
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.TWINE_API_KEY }}
run: python -m twine upload --non-interactive --skip-existing dist/*
run: |
python -m build
ls dist/
tar tvf dist/deeplabcut-live-gui-*.tar.gz
python -m twine upload --verbose dist/*
Loading