diff --git a/.github/workflows/basic-tests.yml b/.github/workflows/basic-tests.yml index 211d77e48c8be..f2f2198c8e6d4 100644 --- a/.github/workflows/basic-tests.yml +++ b/.github/workflows/basic-tests.yml @@ -95,7 +95,7 @@ jobs: - name: "Install Breeze" uses: ./.github/actions/breeze - name: "Run unit tests" - run: uv tool run --from apache-airflow-breeze pytest -n auto --color=yes + run: uv run --locked pytest -n auto --color=yes working-directory: ./dev/breeze/ run-breeze-integration-tests: timeout-minutes: 120 @@ -122,7 +122,7 @@ jobs: - name: "Install hatch" run: uv tool install hatch - name: "Run integration tests" - run: uv tool run --from apache-airflow-breeze pytest -v --color=yes -m integration_tests + run: uv run --locked pytest -v --color=yes -m integration_tests working-directory: ./dev/breeze/ tests-shared-distributions: timeout-minutes: 10 diff --git a/AGENTS.md b/AGENTS.md index c414057ac1fef..46c1ab0f3ddf3 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -22,7 +22,7 @@ Don't spell out **Directed Acyclic Graph** except for historical context. - Install prek: `uv tool install prek` - Enable commit hooks: `prek install` -- Install breeze shim (one-time, per machine): `scripts/tools/setup_breeze` — installs `~/.local/bin/breeze` that runs breeze via `uvx` from the current git worktree's `dev/breeze` (so each worktree, including ephemeral agent worktrees, gets its own breeze tied to its sources). See [ADR 0017](dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md). +- Install breeze shim (one-time, per machine): `scripts/tools/setup_breeze` — installs `~/.local/bin/breeze` that runs breeze via `uv run --locked` from the current git worktree's `dev/breeze`, with dependencies pinned by `dev/breeze/uv.lock` (so each worktree, including ephemeral agent worktrees, gets its own breeze tied to its sources). See [ADR 0017](dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md). - **Never run pytest, python, or airflow commands directly on the host** — always use `breeze`. - Place temporary scripts in `dev/` (mounted as `/opt/airflow/dev/` inside Breeze). diff --git a/contributing-docs/03_contributors_quick_start.rst b/contributing-docs/03_contributors_quick_start.rst index f08fef2deec1b..e0ac6c59b4739 100644 --- a/contributing-docs/03_contributors_quick_start.rst +++ b/contributing-docs/03_contributors_quick_start.rst @@ -414,9 +414,10 @@ see in CI in your local environment. ``uv`` is the recommended general-purpose Python development environment for Airflow. 2. Run ``./scripts/tools/setup_breeze`` in your checked-out repository. This installs a small shim - at ``~/.local/bin/breeze`` that runs Breeze via ``uvx`` from the current git worktree's - ``dev/breeze`` folder, so each worktree (including ephemeral ones used by coding agents) gets - its own Breeze tied to that worktree's sources. See + at ``~/.local/bin/breeze`` that runs Breeze via ``uv run --locked`` from the current git + worktree's ``dev/breeze`` folder, so each worktree (including ephemeral ones used by coding + agents) gets its own Breeze, tied to that worktree's sources and to the dependency versions + pinned in ``dev/breeze/uv.lock``. See `ADR 0017 <../dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md>`_ for the rationale. diff --git a/contributing-docs/quick-start-ide/contributors_quick_start_gitpod.rst b/contributing-docs/quick-start-ide/contributors_quick_start_gitpod.rst index 5f8694cbeece0..b3e49802a6516 100644 --- a/contributing-docs/quick-start-ide/contributors_quick_start_gitpod.rst +++ b/contributing-docs/quick-start-ide/contributors_quick_start_gitpod.rst @@ -65,8 +65,9 @@ the shim installer, which works the same in Gitpod as on a local machine: pip install uv ./scripts/tools/setup_breeze -This installs ``~/.local/bin/breeze`` as a small shim that runs Breeze via ``uvx`` from the -current git worktree's ``dev/breeze`` folder. See +This installs ``~/.local/bin/breeze`` as a small shim that runs Breeze via ``uv run --locked`` +from the current git worktree's ``dev/breeze`` folder, with dependencies pinned by +``dev/breeze/uv.lock``. See `ADR 0017 <../../dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md>`_ for the rationale. diff --git a/dev/README_AIRFLOW3_DEV.md b/dev/README_AIRFLOW3_DEV.md index a947d5d01cc09..d99a81a60f7a1 100644 --- a/dev/README_AIRFLOW3_DEV.md +++ b/dev/README_AIRFLOW3_DEV.md @@ -130,7 +130,8 @@ To test your changes locally, check out the `v2-11-test` branch. Breeze on Airfl not compatible with Airflow 2.11, so you need a Breeze that matches the branch you're on. If you installed Breeze via the recommended shim (`./scripts/tools/setup_breeze`), nothing extra -is needed — the shim runs Breeze via `uvx` from the current git worktree's `dev/breeze`, so +is needed — the shim runs Breeze via `uv run --locked` from the current git worktree's +`dev/breeze`, so checking out a different branch (or using a separate git worktree) automatically picks up that branch's Breeze: diff --git a/dev/README_RELEASE_AIRFLOW.md b/dev/README_RELEASE_AIRFLOW.md index 6f140f3126ebc..3fd3cf6e89f54 100644 --- a/dev/README_RELEASE_AIRFLOW.md +++ b/dev/README_RELEASE_AIRFLOW.md @@ -388,7 +388,7 @@ export AIRFLOW_REPO_ROOT=$(pwd) ``` - Install `breeze` command (recommended — installs a shim at `~/.local/bin/breeze` that runs - breeze via `uvx` from the current git worktree's `dev/breeze`; see + breeze via `uv run --locked` from the current git worktree's `dev/breeze`; see [ADR 0017](breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md)): ```shell script diff --git a/dev/breeze/README.md b/dev/breeze/README.md index f7cc35c1d59cb..992be53f91cf1 100644 --- a/dev/breeze/README.md +++ b/dev/breeze/README.md @@ -40,7 +40,8 @@ out, in [editable/development mode](https://packaging.python.org/en/latest/guide The recommended way to make `breeze` available is to install a small **shim script** at `~/.local/bin/breeze` that runs breeze from the `dev/breeze` folder of the *current* git -worktree via `uvx`. This avoids a single global install and means each git worktree +worktree via `uv run --locked`, with every dependency pinned by the committed +`dev/breeze/uv.lock`. This avoids a single global install and means each git worktree (including ephemeral worktrees used by coding agents) gets its own breeze, tied to that worktree's sources. Because the shim is a real file on `PATH`, subprocesses (pre-commit hooks, CI scripts, dev tools) see it just like a `uv tool`-installed binary. See @@ -84,12 +85,12 @@ else exit 1 fi exec env AIRFLOW_ROOT_PATH="${breeze_root}" SKIP_BREEZE_SELF_UPGRADE_CHECK=1 \ - uvx --from "${breeze_root}/dev/breeze" --quiet breeze "$@" + uv run --project "${breeze_root}/dev/breeze" --locked --quiet breeze "$@" ``` Then `breeze` invoked from any Airflow checkout uses that checkout's source, and from anywhere else it uses `$AIRFLOW_REPO_ROOT` or the baked-in fallback. The first call in a -fresh worktree pays a one-time `uvx` resolve/install; subsequent calls hit the cache. +fresh worktree pays a one-time sync of `dev/breeze/.venv`; subsequent calls reuse it. The legacy global-install path (`uv tool install -e ./dev/breeze --force` or `pipx install -e ./dev/breeze --force`) still works for users who explicitly want a single diff --git a/dev/breeze/doc/01_installation.rst b/dev/breeze/doc/01_installation.rst index eb337d29aa498..c54a535076869 100644 --- a/dev/breeze/doc/01_installation.rst +++ b/dev/breeze/doc/01_installation.rst @@ -269,7 +269,8 @@ Set your working directory to the root of this cloned repository. The recommended way to make ``breeze`` available is to install a small **shim script** at ``~/.local/bin/breeze`` that runs breeze from the ``dev/breeze`` folder of the current git -worktree via ``uvx``. This avoids a single global install and means each git worktree +worktree via ``uv run --locked``, with every dependency pinned by the committed +``dev/breeze/uv.lock``. This avoids a single global install and means each git worktree (including ephemeral worktrees used by coding agents) gets its own breeze, tied to that worktree's sources. Because the shim is a real file on ``PATH``, subprocesses (pre-commit hooks, CI scripts, dev tools) see it just like a ``uv tool``-installed binary. See @@ -300,10 +301,10 @@ marks it executable. To do it manually, write this file to ``~/.local/bin/breeze exit 1 fi exec env AIRFLOW_ROOT_PATH="${repo_root}" SKIP_BREEZE_SELF_UPGRADE_CHECK=1 \ - uvx --from "${repo_root}/dev/breeze" --quiet breeze "$@" + uv run --project "${repo_root}/dev/breeze" --locked --quiet breeze "$@" Then ``breeze`` invoked from any Airflow checkout uses that checkout's source. The first call in -a fresh worktree pays a one-time ``uvx`` resolve/install; subsequent calls hit the cache. +a fresh worktree pays a one-time sync of ``dev/breeze/.venv``; subsequent calls reuse it. Alternative: legacy global install (``uv tool`` or ``pipx``) ------------------------------------------------------------ diff --git a/dev/breeze/doc/adr/0016-use-uv-tool-to-install-breeze.md b/dev/breeze/doc/adr/0016-use-uv-tool-to-install-breeze.md index 21e21b335c17b..696bad76987b1 100644 --- a/dev/breeze/doc/adr/0016-use-uv-tool-to-install-breeze.md +++ b/dev/breeze/doc/adr/0016-use-uv-tool-to-install-breeze.md @@ -35,7 +35,7 @@ Date: 2024-11-11 ## Status -Superseded by [17. Use `uvx` to run breeze from local sources](0017-use-uvx-to-run-breeze-from-local-sources.md) +Superseded by [17. Run breeze from the current worktree's locked sources](0017-use-uvx-to-run-breeze-from-local-sources.md) Supersedes [10. Use pipx to install breeze](0010-use-pipx-to-install-breeze.md) diff --git a/dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md b/dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md index 017fd6d1d1854..8f2a06e2071f2 100644 --- a/dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md +++ b/dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md @@ -21,7 +21,7 @@ **Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* -- [17. Use `uvx` to run breeze from local sources](#17-use-uvx-to-run-breeze-from-local-sources) +- [17. Run breeze from the current worktree's locked sources](#17-run-breeze-from-the-current-worktrees-locked-sources) - [Status](#status) - [Context](#context) - [Decision](#decision) @@ -29,9 +29,9 @@ -# 17. Use `uvx` to run breeze from local sources +# 17. Run breeze from the current worktree's locked sources -Date: 2026-04-26 +Date: 2026-04-26 (amended 2026-08-26: dispatch moved from ``uvx`` to ``uv run --locked``) ## Status @@ -65,12 +65,23 @@ Two patterns have made that single-install model awkward: the same ``~/.local/bin/breeze`` symlink, and an agent that does ``uv tool install --force`` to "fix" itself silently sabotages every other worktree on the machine. -``uv`` ships a tool — ``uvx`` — that runs a command from a project directory in an -ephemeral, cached environment without installing anything globally. ``uvx --from -./dev/breeze breeze ...`` resolves dependencies once per ``pyproject.toml`` /``uv.lock`` -hash, caches the resulting environment, and reuses it on subsequent calls. The first -call in a fresh worktree is slow (one resolve + install); every call after that is -fast. +``uv`` ships a way to run a command from a project directory without installing anything +globally: ``uv run --project ./dev/breeze --locked breeze ...`` syncs that project's own +environment (``dev/breeze/.venv``) to exactly what ``dev/breeze/uv.lock`` pins, then runs the +command in it. The first call in a fresh worktree pays for the sync; every call after that +reuses the environment. + +The ``--locked`` part matters as much as the per-worktree part. The two alternatives that +install from a path — ``uvx --from ./dev/breeze`` and ``uv tool install -e ./dev/breeze`` — +re-resolve breeze's requirements against the package index on every fresh environment and read +neither ``dev/breeze/uv.lock`` nor the ``[tool.uv] exclude-newer`` buffer declared next to it +(that setting governs project operations such as ``uv lock`` and ``uv sync`` only). Under those, +the committed lock recorded nothing and an unrelated upstream release could change breeze with +no commit in this repository. click 8.5.0 showed the cost: released on 2026-08-26, it added a +``help`` field to ``click.Argument.to_info_dict()`` — the dict breeze hashes to detect command +drift — so within the hour every CI job resolved the new click, every command taking a +positional argument hashed differently from its committed value, and static checks went red on +every open PR regardless of what it touched. The lock said click 8.4.2 throughout. That gives us a way to make ``breeze`` always run from the *current* worktree's source without ever touching a shared global install — but the dispatch mechanism has to be @@ -87,9 +98,9 @@ The recommended way to run breeze is via a small **shim script** at ```shell #!/usr/bin/env bash # Apache Airflow breeze shim — managed by scripts/tools/setup_breeze (ADR 0017). -# Runs breeze from the dev/breeze folder of the current git worktree via 'uvx', -# so each worktree (e.g. parallel agentic runs) gets its own ephemerally-installed -# breeze tied to that worktree's source. +# Runs breeze from the dev/breeze folder of the current git worktree via 'uv run', +# so each worktree (e.g. parallel agentic runs) gets its own environment tied to +# that worktree's source, with dependencies resolved from dev/breeze/uv.lock. # # Resolution order for the Airflow sources breeze runs from: # 1. the current git worktree (per-worktree isolation — see above); @@ -115,7 +126,7 @@ else exit 1 fi exec env AIRFLOW_ROOT_PATH="${breeze_root}" SKIP_BREEZE_SELF_UPGRADE_CHECK=1 \ - uvx --from "${breeze_root}/dev/breeze" --quiet breeze "$@" + uv run --project "${breeze_root}/dev/breeze" --locked --quiet breeze "$@" ``` ``scripts/tools/setup_breeze`` writes this file (replacing any previous @@ -128,8 +139,9 @@ The user-facing command stays the same — they still type ``breeze`` — but ea invocation: * resolves ``$(git rev-parse --show-toplevel)`` from the current working directory, -* dispatches to ``uvx --from /dev/breeze breeze``, -* and therefore always runs the breeze code that belongs to that worktree. +* dispatches to ``uv run --project /dev/breeze --locked breeze``, +* and therefore always runs the breeze code that belongs to that worktree, with the + dependencies that worktree's ``uv.lock`` pins. Because the shim is a real file on ``PATH`` (not a shell function), it is also visible to subprocesses — pre-commit hooks, CI scripts, dev tools, and anything @@ -137,11 +149,17 @@ else that does ``subprocess.run(["breeze", ...])`` will pick it up exactly like they picked up the old ``uv tool``-installed binary. The two ``env`` variables matter: ``AIRFLOW_ROOT_PATH`` short-circuits breeze's -installation-source detection (which walks up from ``__file__`` and would -otherwise misfire because ``__file__`` lives inside the uvx cache, not the -source tree), and ``SKIP_BREEZE_SELF_UPGRADE_CHECK=1`` disables the "your -install is older than your sources" nag — moot under uvx, which auto-rebuilds -the env when ``pyproject.toml`` / ``uv.lock`` change. +installation-source detection (which walks up from ``__file__`` and would otherwise +misfire on installs that do not live in the source tree), and +``SKIP_BREEZE_SELF_UPGRADE_CHECK=1`` disables the "your install is older than your +sources" nag — moot here, since ``uv run`` re-syncs the environment whenever +``pyproject.toml`` / ``uv.lock`` change and installs the sources as editable. + +CI installs breeze the same way: ``scripts/ci/install_breeze.sh`` runs +``uv sync --project ./dev/breeze/ --locked`` and puts ``dev/breeze/.venv/bin`` on ``PATH`` +rather than installing a global ``uv tool``. Dependency upgrades reach breeze only through a +change to ``dev/breeze/uv.lock`` — in practice the scheduled ``breeze ci upgrade`` PR, which +regenerates the lock and the command-output files together, as one reviewable commit. ``uv tool install -e ./dev/breeze`` and ``pipx install -e ./dev/breeze`` remain supported as alternatives for users who explicitly want the old single-install @@ -159,6 +177,10 @@ behaviour, but they are no longer the recommended path. checked out — not whatever was current the last time someone reinstalled. The "your installed breeze is older than your sources" warning class largely goes away. +* **Reproducible dependencies.** Two checkouts of the same commit run breeze with the + same dependency versions, whatever the index served that day, so the command hashes + under ``dev/breeze/doc/images/`` are a property of the repository rather than of the + calendar — and the ``exclude-newer`` buffer around lock upgrades finally applies. * **Cheap setup in fresh worktrees.** Spinning up a new worktree (manually or via an agent) needs no extra install step; ``breeze`` works the moment ``cd`` lands in the tree. @@ -175,11 +197,15 @@ behaviour, but they are no longer the recommended path. **Costs** -* **First call in a new worktree is slow.** ``uvx`` has to resolve and install - breeze's dependencies the first time it sees a given ``pyproject.toml`` / - ``uv.lock``. Subsequent calls hit the cache and are fast. +* **First call in a new worktree is slow.** ``uv run`` has to populate + ``dev/breeze/.venv`` (~275 MB, mostly hardlinked into the uv cache; ignored by both + ``.gitignore`` and ``.dockerignore``) the first time. Subsequent calls reuse it. +* **A stale lock blocks breeze.** Editing ``dev/breeze/pyproject.toml`` without re-running + ``uv lock`` makes every breeze call fail until the lock is refreshed. The error names the + fix, and the alternative — silently running dependencies nobody recorded — is the failure + mode this dispatch removes. * **Adds a small bash startup overhead.** The shim is a tiny bash script that - runs ``git rev-parse`` and ``uvx`` for every invocation. Negligible at the + runs ``git rev-parse`` and ``uv run`` for every invocation. Negligible at the command line, but noticeable inside tight loops or shell completion that re-invokes ``breeze`` many times. * **Resolution is current-worktree-first, with two fallbacks.** ``breeze`` diff --git a/dev/breeze/doc/images/output_build-docs.txt b/dev/breeze/doc/images/output_build-docs.txt index c532c1df7a877..f2015f69a55d7 100644 --- a/dev/breeze/doc/images/output_build-docs.txt +++ b/dev/breeze/doc/images/output_build-docs.txt @@ -1 +1 @@ -43e56152396620517257cbcd0ce28ccd +f941b35af1d912596b27d89d4863dd0a diff --git a/dev/breeze/doc/images/output_ci-image_pull.txt b/dev/breeze/doc/images/output_ci-image_pull.txt index 8c10da62effb3..fb2ef908512cc 100644 --- a/dev/breeze/doc/images/output_ci-image_pull.txt +++ b/dev/breeze/doc/images/output_ci-image_pull.txt @@ -1 +1 @@ -d23c74c8aa244d8b8cd55c9182049f10 +fc42909eb684d7821ff2a3575a7e1073 diff --git a/dev/breeze/doc/images/output_ci-image_verify.txt b/dev/breeze/doc/images/output_ci-image_verify.txt index 874ad0f29917e..d62405260cc9d 100644 --- a/dev/breeze/doc/images/output_ci-image_verify.txt +++ b/dev/breeze/doc/images/output_ci-image_verify.txt @@ -1 +1 @@ -3f40148d24cfcc454145bc9971902ead +59daf613dc467f3f1f923bcf7d24eb88 diff --git a/dev/breeze/doc/images/output_exec.txt b/dev/breeze/doc/images/output_exec.txt index a7708253212fc..b3ad835922b46 100644 --- a/dev/breeze/doc/images/output_exec.txt +++ b/dev/breeze/doc/images/output_exec.txt @@ -1 +1 @@ -08e27060ce2c204f4a2b5b1d0c326c74 +0c19dd1538b155da498dd37640c9327e diff --git a/dev/breeze/doc/images/output_k8s_deploy-airflow.txt b/dev/breeze/doc/images/output_k8s_deploy-airflow.txt index fc6808f34b1d8..f4c4c5620afd0 100644 --- a/dev/breeze/doc/images/output_k8s_deploy-airflow.txt +++ b/dev/breeze/doc/images/output_k8s_deploy-airflow.txt @@ -1 +1 @@ -bcbd14d163c29520124a0049a1b084dd +3ce1a62e91cffceefefd09619cb6eb52 diff --git a/dev/breeze/doc/images/output_k8s_dev.txt b/dev/breeze/doc/images/output_k8s_dev.txt index 2269c7f1dfc4d..fc220faba9788 100644 --- a/dev/breeze/doc/images/output_k8s_dev.txt +++ b/dev/breeze/doc/images/output_k8s_dev.txt @@ -1 +1 @@ -ba04bbdd68c6b991e906625178b3478f +012f58c236bd03a3dd0a1578af2a4d85 diff --git a/dev/breeze/doc/images/output_k8s_k9s.txt b/dev/breeze/doc/images/output_k8s_k9s.txt index a0b5531337ce5..eb49e665e9243 100644 --- a/dev/breeze/doc/images/output_k8s_k9s.txt +++ b/dev/breeze/doc/images/output_k8s_k9s.txt @@ -1 +1 @@ -83f4fffcd60ffdf1711983ffa43a5eee +81eeaa4e9ab9ab7ec358057d6e4e0968 diff --git a/dev/breeze/doc/images/output_k8s_run-complete-tests.txt b/dev/breeze/doc/images/output_k8s_run-complete-tests.txt index 971ed6e834e77..932dbefc44e0c 100644 --- a/dev/breeze/doc/images/output_k8s_run-complete-tests.txt +++ b/dev/breeze/doc/images/output_k8s_run-complete-tests.txt @@ -1 +1 @@ -abc593ccc646cf825fab808db673a269 +bbff103f342a54b5f22b0a3577d3a632 diff --git a/dev/breeze/doc/images/output_k8s_shell.txt b/dev/breeze/doc/images/output_k8s_shell.txt index d4eaa5a3ee396..84c5b435e6f32 100644 --- a/dev/breeze/doc/images/output_k8s_shell.txt +++ b/dev/breeze/doc/images/output_k8s_shell.txt @@ -1 +1 @@ -f75f1f3af7d1bc1e4b7cdd5fd1747ee9 +2b3987215e9ca3cf742199fba144c0e2 diff --git a/dev/breeze/doc/images/output_k8s_smoke-test-overlay.txt b/dev/breeze/doc/images/output_k8s_smoke-test-overlay.txt index 087918e8b3de1..fb2b987eceb90 100644 --- a/dev/breeze/doc/images/output_k8s_smoke-test-overlay.txt +++ b/dev/breeze/doc/images/output_k8s_smoke-test-overlay.txt @@ -1 +1 @@ -912bce7922f7ce90391f7af648ac4636 +c9fae60d9f4e72e390a08d53366d20f9 diff --git a/dev/breeze/doc/images/output_k8s_tests.txt b/dev/breeze/doc/images/output_k8s_tests.txt index 4fcde22a9ddb3..459e3cd1d2df1 100644 --- a/dev/breeze/doc/images/output_k8s_tests.txt +++ b/dev/breeze/doc/images/output_k8s_tests.txt @@ -1 +1 @@ -737cf1595fa7afd91a23ef79f46eff16 +7a1170264859cd3fc078681755907622 diff --git a/dev/breeze/doc/images/output_prod-image_pull.txt b/dev/breeze/doc/images/output_prod-image_pull.txt index b55181ed864b8..fa188f44b6160 100644 --- a/dev/breeze/doc/images/output_prod-image_pull.txt +++ b/dev/breeze/doc/images/output_prod-image_pull.txt @@ -1 +1 @@ -d7ed9abca3180bd5a1648f95a2624413 +943b61b60f81476638e74ab0e5ba451e diff --git a/dev/breeze/doc/images/output_prod-image_verify.txt b/dev/breeze/doc/images/output_prod-image_verify.txt index f124b84cb84d5..f431e6eae9daa 100644 --- a/dev/breeze/doc/images/output_prod-image_verify.txt +++ b/dev/breeze/doc/images/output_prod-image_verify.txt @@ -1 +1 @@ -a2f42c9ae4c6b66967588bd5ab538031 +f95887aff9ae25fe203629e01c2fe25d diff --git a/dev/breeze/doc/images/output_release-management_add-back-references.txt b/dev/breeze/doc/images/output_release-management_add-back-references.txt index 4cdc1d4f62f23..ad57d752e8b7d 100644 --- a/dev/breeze/doc/images/output_release-management_add-back-references.txt +++ b/dev/breeze/doc/images/output_release-management_add-back-references.txt @@ -1 +1 @@ -3162d4ae447d02bdddf9d67590fb4134 +bc1e2c3d12063d93fa3f3605b4f42de1 diff --git a/dev/breeze/doc/images/output_release-management_check-release-files.txt b/dev/breeze/doc/images/output_release-management_check-release-files.txt index 2a16aae00bfc2..e3b08d31a5e3e 100644 --- a/dev/breeze/doc/images/output_release-management_check-release-files.txt +++ b/dev/breeze/doc/images/output_release-management_check-release-files.txt @@ -1 +1 @@ -465b1fffb9a13eb9a97ffa2dcadd4c58 +4d7d540413a5955a51eaf7d035ff31bc diff --git a/dev/breeze/doc/images/output_release-management_classify-provider-changes.txt b/dev/breeze/doc/images/output_release-management_classify-provider-changes.txt index a78f36b3ac39c..44b287551c6e1 100644 --- a/dev/breeze/doc/images/output_release-management_classify-provider-changes.txt +++ b/dev/breeze/doc/images/output_release-management_classify-provider-changes.txt @@ -1 +1 @@ -ce0ed1333a3f38983043d4eb04f0bf1c +0f0f937f1a0d86a58147723b1400b3ac diff --git a/dev/breeze/doc/images/output_release-management_generate-issue-content-providers.txt b/dev/breeze/doc/images/output_release-management_generate-issue-content-providers.txt index a249fc1a2edf6..91f9d9e152d75 100644 --- a/dev/breeze/doc/images/output_release-management_generate-issue-content-providers.txt +++ b/dev/breeze/doc/images/output_release-management_generate-issue-content-providers.txt @@ -1 +1 @@ -ec8538954daf8fe21156a7ad72992e85 +465c7719885f58cdf5fc8bd0592ba7de diff --git a/dev/breeze/doc/images/output_release-management_prepare-provider-distributions.txt b/dev/breeze/doc/images/output_release-management_prepare-provider-distributions.txt index 5203efb313d60..4d4703a71ae30 100644 --- a/dev/breeze/doc/images/output_release-management_prepare-provider-distributions.txt +++ b/dev/breeze/doc/images/output_release-management_prepare-provider-distributions.txt @@ -1 +1 @@ -b635f3eb5c8e04be0937b9e85dee856a +b937b6a253ce2ab2cbabb9cd8b7369d4 diff --git a/dev/breeze/doc/images/output_release-management_prepare-provider-documentation.txt b/dev/breeze/doc/images/output_release-management_prepare-provider-documentation.txt index e7ca97a104b0f..5b7184cf9c60b 100644 --- a/dev/breeze/doc/images/output_release-management_prepare-provider-documentation.txt +++ b/dev/breeze/doc/images/output_release-management_prepare-provider-documentation.txt @@ -1 +1 @@ -a94ab59178e740921f451bdc68bfb445 +57343f19b12400ac43ea8fc4b892ac99 diff --git a/dev/breeze/doc/images/output_release-management_publish-docs.txt b/dev/breeze/doc/images/output_release-management_publish-docs.txt index ba3f69ebcb5a0..2e522c29cf776 100644 --- a/dev/breeze/doc/images/output_release-management_publish-docs.txt +++ b/dev/breeze/doc/images/output_release-management_publish-docs.txt @@ -1 +1 @@ -f08a2861a8c88397caa6ea8471819456 +1a68af03bc2f8fae0d46e4a2a2266eb0 diff --git a/dev/breeze/doc/images/output_run.txt b/dev/breeze/doc/images/output_run.txt index 4a274308775f7..c107633ee5ac7 100644 --- a/dev/breeze/doc/images/output_run.txt +++ b/dev/breeze/doc/images/output_run.txt @@ -1 +1 @@ -e850a14ddf66f700b2595d65d7511869 +d4841617ec2b8d96643f6695adcb68d6 diff --git a/dev/breeze/doc/images/output_shell.txt b/dev/breeze/doc/images/output_shell.txt index 8e1ee65300e86..796f1d521e18f 100644 --- a/dev/breeze/doc/images/output_shell.txt +++ b/dev/breeze/doc/images/output_shell.txt @@ -1 +1 @@ -0278a7f8d4d43cf91efbaaae1368326e +cfa0588bb3e466155cd793b4528ac28c diff --git a/dev/breeze/doc/images/output_start-airflow.txt b/dev/breeze/doc/images/output_start-airflow.txt index fa1d44f74c75c..e2dc6a5a11e87 100644 --- a/dev/breeze/doc/images/output_start-airflow.txt +++ b/dev/breeze/doc/images/output_start-airflow.txt @@ -1 +1 @@ -b4b01862a54de30d7df835ee6f9150df +3cdf6229fb227fdbf3f7b0524b4a7f75 diff --git a/dev/breeze/doc/images/output_testing_airflow-ctl-integration-tests.txt b/dev/breeze/doc/images/output_testing_airflow-ctl-integration-tests.txt index 446ec42dd1f5c..36147a39cccf1 100644 --- a/dev/breeze/doc/images/output_testing_airflow-ctl-integration-tests.txt +++ b/dev/breeze/doc/images/output_testing_airflow-ctl-integration-tests.txt @@ -1 +1 @@ -4c341666e70a017acd38edd879adb595 +e302335890458a12a28a52ed51b6337f diff --git a/dev/breeze/doc/images/output_testing_airflow-ctl-tests.txt b/dev/breeze/doc/images/output_testing_airflow-ctl-tests.txt index 5799a6650b8ee..98f03c94d2357 100644 --- a/dev/breeze/doc/images/output_testing_airflow-ctl-tests.txt +++ b/dev/breeze/doc/images/output_testing_airflow-ctl-tests.txt @@ -1 +1 @@ -f11826c57955e3a6bef2f3c95989cacc +caf04a20ceb4b3a426e095860f8a800f diff --git a/dev/breeze/doc/images/output_testing_airflow-e2e-tests.txt b/dev/breeze/doc/images/output_testing_airflow-e2e-tests.txt index d2467d70674af..31336273b3b1a 100644 --- a/dev/breeze/doc/images/output_testing_airflow-e2e-tests.txt +++ b/dev/breeze/doc/images/output_testing_airflow-e2e-tests.txt @@ -1 +1 @@ -2bbd90928e950df8cf78abd0aff50f0b +3005f4343ce3d9c33960042d8d7e2d20 diff --git a/dev/breeze/doc/images/output_testing_core-integration-tests.txt b/dev/breeze/doc/images/output_testing_core-integration-tests.txt index cde65e2f90c8d..aad3803373ceb 100644 --- a/dev/breeze/doc/images/output_testing_core-integration-tests.txt +++ b/dev/breeze/doc/images/output_testing_core-integration-tests.txt @@ -1 +1 @@ -ccda221c325794a1a29ff0b77159dbeb +56f677845a9c84dbdc1131c51b9af83a diff --git a/dev/breeze/doc/images/output_testing_core-tests.txt b/dev/breeze/doc/images/output_testing_core-tests.txt index a4246da8dce5f..ce17b293f3685 100644 --- a/dev/breeze/doc/images/output_testing_core-tests.txt +++ b/dev/breeze/doc/images/output_testing_core-tests.txt @@ -1 +1 @@ -2d18c088667df831820438a03ba9654a +e89451fe0b493819e12e78316a070cee diff --git a/dev/breeze/doc/images/output_testing_docker-compose-tests.txt b/dev/breeze/doc/images/output_testing_docker-compose-tests.txt index 46a207d87962d..2c906c51d5de1 100644 --- a/dev/breeze/doc/images/output_testing_docker-compose-tests.txt +++ b/dev/breeze/doc/images/output_testing_docker-compose-tests.txt @@ -1 +1 @@ -04af62df42438f286d861f3f1956e5e4 +e7f68555c25e802df53097e5e7558088 diff --git a/dev/breeze/doc/images/output_testing_helm-tests.txt b/dev/breeze/doc/images/output_testing_helm-tests.txt index fb44f419c4d5e..5331937a401e7 100644 --- a/dev/breeze/doc/images/output_testing_helm-tests.txt +++ b/dev/breeze/doc/images/output_testing_helm-tests.txt @@ -1 +1 @@ -43eb5774bbdef528e443613d1d8048d9 +02ba09ea9213ae046fe814563630054d diff --git a/dev/breeze/doc/images/output_testing_providers-integration-tests.txt b/dev/breeze/doc/images/output_testing_providers-integration-tests.txt index 8626377a0200c..da0296790d848 100644 --- a/dev/breeze/doc/images/output_testing_providers-integration-tests.txt +++ b/dev/breeze/doc/images/output_testing_providers-integration-tests.txt @@ -1 +1 @@ -fc0029baa8c28609f1e6972829465401 +1ab3cf1b8c5a5deb4181110bb41fb811 diff --git a/dev/breeze/doc/images/output_testing_providers-tests.txt b/dev/breeze/doc/images/output_testing_providers-tests.txt index 770a5cb7598c9..4efab5ed2813a 100644 --- a/dev/breeze/doc/images/output_testing_providers-tests.txt +++ b/dev/breeze/doc/images/output_testing_providers-tests.txt @@ -1 +1 @@ -776d6d9819a1351ff4f96f3884f9e039 +9372d4ac91f1f8aec84665bd5bbc7dc6 diff --git a/dev/breeze/doc/images/output_testing_python-api-client-tests.txt b/dev/breeze/doc/images/output_testing_python-api-client-tests.txt index 57a62e8cd409b..ca9a2d70afa80 100644 --- a/dev/breeze/doc/images/output_testing_python-api-client-tests.txt +++ b/dev/breeze/doc/images/output_testing_python-api-client-tests.txt @@ -1 +1 @@ -64c20fc3168ebec7147f51be8e323040 +0d407ea088f01326dfc1104f12a3017d diff --git a/dev/breeze/doc/images/output_testing_system-tests.txt b/dev/breeze/doc/images/output_testing_system-tests.txt index b9ebe1ecd1370..ecb78ce4d7fd3 100644 --- a/dev/breeze/doc/images/output_testing_system-tests.txt +++ b/dev/breeze/doc/images/output_testing_system-tests.txt @@ -1 +1 @@ -3a8636011e5f5d6090c61ec0433575e5 +0cdb967ba346e6663ac663acf599b5ae diff --git a/dev/breeze/doc/images/output_testing_task-sdk-integration-tests.txt b/dev/breeze/doc/images/output_testing_task-sdk-integration-tests.txt index 1f0b6d1bf5b3e..6e754162d012b 100644 --- a/dev/breeze/doc/images/output_testing_task-sdk-integration-tests.txt +++ b/dev/breeze/doc/images/output_testing_task-sdk-integration-tests.txt @@ -1 +1 @@ -4d9236fb254edf6d1f6e8ec815b50223 +76d2a06163a1c3f4dbc54f2e4aa13ade diff --git a/dev/breeze/doc/images/output_testing_task-sdk-tests.txt b/dev/breeze/doc/images/output_testing_task-sdk-tests.txt index 66725084dfd33..475499ef8a804 100644 --- a/dev/breeze/doc/images/output_testing_task-sdk-tests.txt +++ b/dev/breeze/doc/images/output_testing_task-sdk-tests.txt @@ -1 +1 @@ -7135479eff2be2409fad7759de789087 +58ecc16666e1b4f5c5572533e7ccc0a7 diff --git a/dev/breeze/doc/images/output_testing_ui-e2e-tests.txt b/dev/breeze/doc/images/output_testing_ui-e2e-tests.txt index 40d9b34fd05f2..69046ce01ebfe 100644 --- a/dev/breeze/doc/images/output_testing_ui-e2e-tests.txt +++ b/dev/breeze/doc/images/output_testing_ui-e2e-tests.txt @@ -1 +1 @@ -07feb53b4eeacb942d45ad1a70dc8740 +109d8450ccc1fa74ebc2547ae2db41d5 diff --git a/dev/breeze/doc/images/output_workflow-run_publish-docs.txt b/dev/breeze/doc/images/output_workflow-run_publish-docs.txt index 9dd84999bbf79..bd1e1fde13747 100644 --- a/dev/breeze/doc/images/output_workflow-run_publish-docs.txt +++ b/dev/breeze/doc/images/output_workflow-run_publish-docs.txt @@ -1 +1 @@ -65b7b00a5792d185aee3628fdc2227de +c4e8633fee01c127eb062f8ccfac1c5c diff --git a/dev/breeze/src/airflow_breeze/utils/path_utils.py b/dev/breeze/src/airflow_breeze/utils/path_utils.py index 533e741c61ec2..9183d1fb96e02 100644 --- a/dev/breeze/src/airflow_breeze/utils/path_utils.py +++ b/dev/breeze/src/airflow_breeze/utils/path_utils.py @@ -252,9 +252,9 @@ def warn_if_shim_outdated(airflow_sources: Path, shim_text: str | None = None) - setup_script = airflow_sources / "scripts" / "tools" / "setup_breeze" installed_text = installed_version if installed_version is not None else "unknown (pre-versioning)" console_print( - f"\n[warning]Your breeze shim at {BREEZE_SHIM_PATH} is out of date " + f"\n[warning]Your breeze shim at {BREEZE_SHIM_PATH} needs to be upgraded " f"(installed: {installed_text}, current: {expected_version}).[/]\n" - "[warning]Re-run the setup script to refresh it:[/]\n\n" + "[warning]Re-run the setup script to upgrade it:[/]\n\n" f" {setup_script}\n" ) return True @@ -314,9 +314,9 @@ def warn_if_breeze_launcher_outdated(airflow_sources: Path) -> bool: ) setup_script = airflow_sources / "scripts" / "tools" / "setup_breeze" console_print( - f"\n[warning]Breeze is installed as a legacy global '{legacy}' install, which still works " - "but is no longer the recommended setup (see ADR 0017).[/]\n" - "[warning]Migrate to the per-worktree uvx shim by uninstalling the global install and " + f"\n[warning]Breeze is installed as a legacy global '{legacy}' install, which resolves its " + "dependencies against the package index rather than dev/breeze/uv.lock (see ADR 0017).[/]\n" + "[warning]Migrate to the per-worktree shim by uninstalling the global install and " "running the setup script:[/]\n\n" f" {uninstall_cmd}\n" f" {setup_script}\n" diff --git a/dev/breeze/src/airflow_breeze/utils/reinstall.py b/dev/breeze/src/airflow_breeze/utils/reinstall.py index e6ec61c4cd70a..d594bbbce7944 100644 --- a/dev/breeze/src/airflow_breeze/utils/reinstall.py +++ b/dev/breeze/src/airflow_breeze/utils/reinstall.py @@ -76,14 +76,15 @@ def reinstall_breeze(breeze_sources: Path, re_run: bool = True): ["pipx", "install", "-e", breeze_sources.as_posix(), "--force"], stderr=subprocess.STDOUT ) else: - # Recommended setup: breeze is invoked via the `uvx`-based shell function - # (see ADR 0017). There is no global install to reinstall — uvx will - # rebuild the cached env on next call when pyproject.toml / uv.lock change. + # Recommended setup: breeze is invoked through the shim (see ADR 0017), which runs + # `uv run --locked` against the worktree. There is no global install to reinstall — + # the next call re-syncs the environment whenever pyproject.toml / uv.lock change. console_print( "[info]No global breeze install detected (uv tool / pipx). " - "Assuming the recommended uvx-based setup — nothing to reinstall.[/]\n" - "[info]If you suspect a stale cached env, clear it with:[/]\n" - " uv cache clean apache-airflow-breeze\n" + "Assuming the recommended shim-based setup — nothing to reinstall.[/]\n" + "[info]If you suspect a broken environment, remove it and let the next call " + "rebuild it:[/]\n" + " rm -rf dev/breeze/.venv\n" ) if re_run: diff --git a/dev/breeze/tests/test_shim_version_check.py b/dev/breeze/tests/test_shim_version_check.py index b027955a1d449..671a1a09689ff 100644 --- a/dev/breeze/tests/test_shim_version_check.py +++ b/dev/breeze/tests/test_shim_version_check.py @@ -61,7 +61,7 @@ def test_parse_shim_version(version_line, expected): def test_get_expected_shim_version_reads_real_setup_script(): # The real setup_breeze in the sources is the source of truth — keep this test in sync # with the SHIM_VERSION it declares. - assert get_expected_shim_version(ACTUAL_AIRFLOW_SOURCES) == 1 + assert get_expected_shim_version(ACTUAL_AIRFLOW_SOURCES) == 2 def test_get_expected_shim_version_from_fake_sources(tmp_path): @@ -123,7 +123,7 @@ def test_warn_if_shim_outdated_older_installed(tmp_path, monkeypatch, capsys): sources = _fake_sources_with_version(tmp_path, 2) assert warn_if_shim_outdated(sources) is True output = capsys.readouterr().out - assert "out of date" in output + assert "needs to be upgraded" in output assert "setup_breeze" in output @@ -135,7 +135,7 @@ def test_warn_if_shim_outdated_pre_versioning_shim(tmp_path, monkeypatch, capsys sources = _fake_sources_with_version(tmp_path, 1) assert warn_if_shim_outdated(sources) is True output = capsys.readouterr().out - assert "out of date" in output + assert "needs to be upgraded" in output assert "pre-versioning" in output @@ -181,7 +181,7 @@ def test_launcher_check_prefers_shim_version(tmp_path, monkeypatch, capsys): assert warn_if_breeze_launcher_outdated(sources) is True detect.assert_not_called() output = capsys.readouterr().out - assert "out of date" in output + assert "needs to be upgraded" in output @pytest.mark.parametrize( diff --git a/dev/skill-evals/last-eval-hash.txt b/dev/skill-evals/last-eval-hash.txt index 4e72dc42b313c..f44ab7e72f8a3 100644 --- a/dev/skill-evals/last-eval-hash.txt +++ b/dev/skill-evals/last-eval-hash.txt @@ -1,3 +1,3 @@ # Generated by dev/skill-evals/eval.py — do not edit or resolve conflicts by hand. # Run `prek run run-skill-eval --hook-stage manual --all-files` to regenerate. -8cfe725b810c4c2d364d99f63b27ec739545e40032195624c218b0eb23d75859 +f74d85cc733701dd0b39c1024b68868be0b739931613b268dd76a8e0d4d18110 diff --git a/scripts/ci/install_breeze.sh b/scripts/ci/install_breeze.sh index 1f37d0cf32e60..088072d627d55 100755 --- a/scripts/ci/install_breeze.sh +++ b/scripts/ci/install_breeze.sh @@ -27,7 +27,12 @@ if [[ ${PYTHON_VERSION=} != "" ]]; then fi python -m pip install --upgrade "pip==${PIP_VERSION}" +# A leftover global tool install would shadow the venv script below via ~/.local/bin. uv tool uninstall apache-airflow-breeze >/dev/null 2>&1 || true +# `uv sync --locked` installs exactly what dev/breeze/uv.lock pins. `uv tool install` re-resolved +# against the index instead, so any third-party release landing mid-day silently changed breeze's +# dependencies — and with them the breeze command hashes every PR is checked against. # shellcheck disable=SC2086 -uv tool install ${PYTHON_ARG} --force --editable ./dev/breeze/ +uv sync ${PYTHON_ARG} --project ./dev/breeze/ --locked +echo "$(pwd)/dev/breeze/.venv/bin" >> "${GITHUB_PATH}" echo '/home/runner/.local/bin' >> "${GITHUB_PATH}" diff --git a/scripts/ci/prek/breeze_cmd_line.py b/scripts/ci/prek/breeze_cmd_line.py index 7a694073a42dd..aa2d3577cd669 100755 --- a/scripts/ci/prek/breeze_cmd_line.py +++ b/scripts/ci/prek/breeze_cmd_line.py @@ -28,7 +28,13 @@ import subprocess import sys -from common_prek_utils import AIRFLOW_ROOT_PATH, console, initialize_breeze_prek +from common_prek_utils import ( + AIRFLOW_ROOT_PATH, + SETUP_BREEZE_PATH, + console, + describe_breeze_not_running_from_lock, + initialize_breeze_prek, +) BREEZE_INSTALL_DIR = AIRFLOW_ROOT_PATH / "dev" / "breeze" BREEZE_DOC_DIR = BREEZE_INSTALL_DIR / "doc" @@ -40,12 +46,10 @@ def breeze_env_with_local_sources() -> dict[str, str]: """Return an environment that forces breeze to import the local worktree sources. - The ``breeze`` command on PATH is normally the uvx shim (ADR 0017), which runs - breeze from a *cached* build. That cache does not always reflect uncommitted edits - to ``dev/breeze``: ``uvx --refresh`` / ``--reinstall`` do not rebuild a local path - dependency, only ``uvx --no-cache`` does. When the cache is stale this hook computes - the command hashes / option groups from old code and then either misses a needed - regeneration or *reverts* a correctly regenerated image back to the stale version. + The ``breeze`` command on PATH is not guaranteed to import this worktree's sources: a + legacy global install (``uv tool`` / ``pipx``) stays bound to whichever checkout it was + installed from. Computing the command hashes / option groups from that other checkout's + code either misses a needed regeneration or *reverts* a correctly regenerated image. Prepending the local breeze sources to ``PYTHONPATH`` makes the in-process computation (and the help rendering it spawns) always reflect the current source, @@ -100,12 +104,34 @@ def is_regeneration_needed() -> bool: return result.returncode != 0 +def fail_if_breeze_does_not_run_from_lock() -> None: + """Stop before regenerating anything from a breeze whose dependencies are not the locked ones. + + The stored hashes cover the command definitions *as rendered by the locked click*, so + regenerating from an unlocked install rewrites every one of them and the drift comes back on + the next run from a locked install. + """ + reason = describe_breeze_not_running_from_lock() + if reason is None: + return + console.print( + f"\n[red]Cannot regenerate breeze command output: {reason}.[/]\n\n" + "[yellow]Install or upgrade the breeze shim, which runs breeze from " + "`dev/breeze/uv.lock` (ADR 0017):[/]\n\n" + f" {SETUP_BREEZE_PATH}\n\n" + "[yellow]A legacy global install has to go first — it owns the same path:[/]\n\n" + " uv tool uninstall apache-airflow-breeze # or: pipx uninstall apache-airflow-breeze\n" + ) + sys.exit(1) + + def main() -> int: initialize_breeze_prek(__name__, __file__) return_code = 0 verify_all_commands_described_in_docs() if is_regeneration_needed(): + fail_if_breeze_does_not_run_from_lock() console.print( "\n[bright_blue]Some of the commands changed since last time images were generated. " "Regenerating.\n" diff --git a/scripts/ci/prek/common_prek_utils.py b/scripts/ci/prek/common_prek_utils.py index 40cd8ce583873..3a799b2dd196d 100644 --- a/scripts/ci/prek/common_prek_utils.py +++ b/scripts/ci/prek/common_prek_utils.py @@ -346,6 +346,59 @@ def check_uv_version(uv_bin: str = "uv") -> None: sys.exit(1) +BREEZE_SHIM_MARKER = "Apache Airflow breeze shim — managed by scripts/tools/setup_breeze" +BREEZE_SHIM_VERSION_PREFIX = "# breeze-shim-version:" +SETUP_BREEZE_SHIM_VERSION_PREFIX = "SHIM_VERSION=" +SETUP_BREEZE_PATH = AIRFLOW_ROOT_PATH / "scripts" / "tools" / "setup_breeze" +BREEZE_LOCKED_VENV_PATH = AIRFLOW_BREEZE_SOURCES_PATH / ".venv" + + +def _read_shim_version(text: str, prefix: str) -> int | None: + for line in text.splitlines(): + stripped = line.strip() + if stripped.startswith(prefix): + try: + return int(stripped[len(prefix) :].strip().strip("\"'")) + except ValueError: + return None + return None + + +def describe_breeze_not_running_from_lock() -> str | None: + """Describe why the ``breeze`` on PATH does not run from ``dev/breeze/uv.lock``. + + Only the current shim (which dispatches to ``uv run --locked``) and the locked venv CI + syncs run the versions the lock pins. An older shim and a legacy ``uv tool`` / ``pipx`` + install both resolve breeze's dependencies against the index instead, so anything derived + from them — command hashes above all — reflects whatever the index served that day. + + :return: a description of the offending install, or None when breeze runs from the lock. + """ + breeze_bin = shutil.which("breeze") + if breeze_bin is None: + return None + resolved = Path(breeze_bin).resolve() + if resolved.is_relative_to(BREEZE_LOCKED_VENV_PATH.resolve()): + return None + try: + text = Path(breeze_bin).read_text() + except (OSError, UnicodeDecodeError): + return None + if BREEZE_SHIM_MARKER not in text: + return f"`{breeze_bin}` is a legacy global install, which ignores the lock" + expected_version = _read_shim_version(SETUP_BREEZE_PATH.read_text(), SETUP_BREEZE_SHIM_VERSION_PREFIX) + if expected_version is None: + return None + installed_version = _read_shim_version(text, BREEZE_SHIM_VERSION_PREFIX) + if installed_version is not None and installed_version >= expected_version: + return None + installed_text = installed_version if installed_version is not None else "pre-versioning" + return ( + f"the shim at `{breeze_bin}` needs to be upgraded " + f"(installed: {installed_text}, current: {expected_version})" + ) + + def initialize_breeze_prek(name: str, file: str): if name not in ("__main__", "__mp_main__"): raise SystemExit( @@ -363,7 +416,7 @@ def initialize_breeze_prek(name: str, file: str): "[red]The `breeze` command is not on path.[/]\n\n" "[yellow]Please install breeze. Recommended: run `./scripts/tools/setup_breeze` " "from the repo root — it installs a shim at `~/.local/bin/breeze` that runs breeze " - "via `uvx` from the current git worktree (see ADR 0017).\n" + "via `uv run --locked` from the current git worktree (see ADR 0017).\n" "Legacy global install (`uv tool install -e ./dev/breeze` or " "`pipx install -e ./dev/breeze`) still works but is no longer recommended.[/]\n\n" "[bright_blue]You can also set SKIP_BREEZE_PREK_HOOKS env variable to non-empty " diff --git a/scripts/tests/ci/prek/test_breeze_cmd_line.py b/scripts/tests/ci/prek/test_breeze_cmd_line.py index 5e83c718e6161..941c3fff1cb30 100644 --- a/scripts/tests/ci/prek/test_breeze_cmd_line.py +++ b/scripts/tests/ci/prek/test_breeze_cmd_line.py @@ -24,11 +24,11 @@ class TestBreezeEnvWithLocalSources: - """The hook must run breeze against the local worktree sources, not a stale cached build. + """The hook must run breeze against the local worktree sources, not another checkout's. - The ``breeze`` shim (ADR 0017) runs from a uvx cache that does not always reflect - uncommitted ``dev/breeze`` edits. Prepending the local sources to ``PYTHONPATH`` - makes the in-process command-hash / option-group computation use the current code. + A legacy global breeze install stays bound to the checkout it was installed from. + Prepending the local sources to ``PYTHONPATH`` makes the in-process command-hash / + option-group computation use the current worktree's code. """ def test_sets_pythonpath_to_breeze_sources_when_unset(self, monkeypatch): @@ -40,7 +40,7 @@ def test_prepends_breeze_sources_before_existing_pythonpath(self, monkeypatch): existing = f"/some/path{os.pathsep}/other/path" monkeypatch.setenv("PYTHONPATH", existing) env = breeze_env_with_local_sources() - # Local breeze sources must come first so they win over the cached build. + # Local breeze sources must come first so they win over any other install. assert env["PYTHONPATH"] == f"{BREEZE_SOURCES_DIR}{os.pathsep}{existing}" assert env["PYTHONPATH"].split(os.pathsep)[0] == str(BREEZE_SOURCES_DIR) diff --git a/scripts/tests/ci/prek/test_common_prek_utils.py b/scripts/tests/ci/prek/test_common_prek_utils.py index b169b9b90dc05..9632091cd47f9 100644 --- a/scripts/tests/ci/prek/test_common_prek_utils.py +++ b/scripts/tests/ci/prek/test_common_prek_utils.py @@ -651,6 +651,53 @@ def test_exits_when_breeze_not_found(self, monkeypatch): assert exc_info.value.code == 1 +class TestDescribeBreezeNotRunningFromLock: + @staticmethod + def _install(tmp_path, monkeypatch, breeze_body: str, setup_version: str = "2"): + breeze_bin = tmp_path / "breeze" + breeze_bin.write_text(breeze_body) + setup_breeze = tmp_path / "setup_breeze" + setup_breeze.write_text(f'SHIM_VERSION="{setup_version}"\n') + monkeypatch.setattr(common_prek_utils, "SETUP_BREEZE_PATH", setup_breeze) + monkeypatch.setattr(common_prek_utils, "BREEZE_LOCKED_VENV_PATH", tmp_path / "locked" / ".venv") + monkeypatch.setattr(common_prek_utils.shutil, "which", lambda _: str(breeze_bin)) + return breeze_bin + + def _shim(self, version: int | None) -> str: + version_line = f"# breeze-shim-version: {version}\n" if version is not None else "" + return f"#!/usr/bin/env bash\n# {common_prek_utils.BREEZE_SHIM_MARKER}\n{version_line}" + + def test_reports_an_outdated_shim(self, tmp_path, monkeypatch): + self._install(tmp_path, monkeypatch, self._shim(1)) + assert "needs to be upgraded" in common_prek_utils.describe_breeze_not_running_from_lock() + + def test_reports_a_shim_predating_versioning(self, tmp_path, monkeypatch): + self._install(tmp_path, monkeypatch, self._shim(None)) + assert "pre-versioning" in common_prek_utils.describe_breeze_not_running_from_lock() + + def test_reports_a_legacy_global_install(self, tmp_path, monkeypatch): + self._install( + tmp_path, monkeypatch, "#!/usr/bin/env python\nfrom airflow_breeze.breeze import main\n" + ) + assert "legacy global install" in common_prek_utils.describe_breeze_not_running_from_lock() + + def test_accepts_a_current_shim(self, tmp_path, monkeypatch): + self._install(tmp_path, monkeypatch, self._shim(2)) + assert common_prek_utils.describe_breeze_not_running_from_lock() is None + + def test_accepts_the_locked_venv_ci_syncs(self, tmp_path, monkeypatch): + venv_bin = tmp_path / "locked" / ".venv" / "bin" + venv_bin.mkdir(parents=True) + (venv_bin / "breeze").write_text("#!/usr/bin/env python\n") + monkeypatch.setattr(common_prek_utils, "BREEZE_LOCKED_VENV_PATH", tmp_path / "locked" / ".venv") + monkeypatch.setattr(common_prek_utils.shutil, "which", lambda _: str(venv_bin / "breeze")) + assert common_prek_utils.describe_breeze_not_running_from_lock() is None + + def test_accepts_a_missing_breeze(self, monkeypatch): + monkeypatch.setattr(common_prek_utils.shutil, "which", lambda _: None) + assert common_prek_utils.describe_breeze_not_running_from_lock() is None + + class TestTemporaryTscProject: def test_creates_temp_tsconfig(self, tmp_path): tsconfig = tmp_path / "tsconfig.json" diff --git a/scripts/tools/setup_breeze b/scripts/tools/setup_breeze index 45742b43a97b3..3d946a3f07154 100755 --- a/scripts/tools/setup_breeze +++ b/scripts/tools/setup_breeze @@ -41,11 +41,13 @@ SHIM_MARKER="# Apache Airflow breeze shim — managed by scripts/tools/setup_bre # into the installed shim; if the installed shim is older it warns the user to # re-run this script. See warn_if_shim_outdated() in # dev/breeze/src/airflow_breeze/utils/path_utils.py. -SHIM_VERSION="1" +SHIM_VERSION="2" -# The shim itself. Runs breeze via 'uvx' against the dev/breeze folder of the +# The shim itself. Runs breeze via 'uv run' against the dev/breeze folder of the # *current* git worktree, so multiple checkouts / agentic worktrees never -# share a single global install. See ADR 0017. +# share a single global install. '--locked' pins every dependency to the +# committed dev/breeze/uv.lock, so a fresh third-party release cannot change +# what breeze runs with until that lock is upgraded. See ADR 0017. # # When invoked outside any Airflow worktree (e.g. from an SVN release checkout # such as asf-dist during a provider release), the shim falls back to, in order: @@ -55,9 +57,9 @@ read -r -d '' BREEZE_SHIM_BODY <