Skip to content

#2561: Make python a pristine versioned tool with per-project venv is… - #2623

Open
krystynaShatkovska wants to merge 10 commits into
devonfw:mainfrom
krystynaShatkovska:feature/2561-python-pristine-per-project-isolation
Open

krystynaShatkovska wants to merge 10 commits into
devonfw:mainfrom
krystynaShatkovska:feature/2561-python-pristine-per-project-isolation

Conversation

@krystynaShatkovska

@krystynaShatkovska krystynaShatkovska commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

This PR fixes #2561

Implemented changes:

  • Change 1
    Install python as a pristine, versioned interpreter once into the shared software repository at $IDE_ROOT/_ide/software/default/python/python/«version», using the same default-repository layout as other pristine versioned tool node. One copy of a version is now shared across all projects instead of each project owning its own copy.

  • Change 2
    Route each project's global packages (pip, ruff, …) into a per-project virtual environment at IDE_HOME/.venv (created with uv venv backed by the pristine interpreter and seeded with pip), so two projects on the same interpreter no longer collide, addresses Design for tools like node and python that hack their own installation #352.

  • Change 3
    Move the uv tool store to be per-project as well (IDE_HOME/.uv-tools) instead of living under shared software/python, so uv-managed tools also don't leak between projects.

  • Change 4
    Add a python launcher in cli/src/main/package/functions (Bash) and functions.ps1 (PowerShell), mirroring the existing node/npm/claude functions: inside a project it runs ide python, otherwise it falls back to the system python.

  • Change 5
    Update the uv/pip test-project fixtures and rewrite the Python/Uv tests to assert the new pristine + per-project-venv layout.


Testing instructions

  1. Unit tests. From the cli module run:
    cd cli && mvn
    -Dtest='PythonTest,UvTest,PipTest,PipBasedIdeToolCommandletTest,RuffTest,SpyderTest,JustTest' test
    All should pass. PythonTest.testInstallCreatesPristineInterpreterAndPerProjectVenv asserts the interpreter lands in the software repo (real path ends in 3.14.6) and the venv exists under IDE_HOME/.venv. testInstallKeepsPerProjectVenvIsolated asserts re-installing the shared interpreter does not wipe a project's installed packages (the Design for tools like node and python that hack their own installation #352 fix).

  2. Manual (inside a project). With two separate projects configured to use python:
    ide python --version # uses the pristine interpreter via the new launcher
    python --version # in a project: routes to ide python; outside a project: system python
    pip install requests # installs into IDE_HOME/.venv (per-project)
    Then in a second project run pip install requests. Verify it does not see the first project's packages (isolation). The pristine interpreter path should be $IDE_ROOT/_ide/software/default/python/python/«version» and be identical (one shared copy) for both projects.

  3. Full quality gates. mvn -q verify -pl '!documentation' from the repo root (build + tests) and
    mvn checkstyle:check on the changed files both pass.

Checklist for this PR

Make sure everything is checked before merging this PR. For further info please also see
our DoD.

  • When running mvn clean test locally all tests pass and build is successful
  • PR title is of the form #«issue-id»: «brief summary» (e.g. #921: fixed setup.bat and not feature/921 fixed setup.bat).
    If no issue ID exists, title only.
  • PR top-level comment summarizes what has been done and contains link to addressed issue(s)
  • PR and issue(s) have suitable labels
  • Issue is set to In Progress and assigned to you or there is no issue (might happen for very small PRs)
  • You followed all coding conventions
  • You have added the issue implemented by your PR in CHANGELOG.adoc unless issue is labelled
    with internal
  • You have not changed any dependency in pom.xml files or otherwise if runtime dependencies changed,
    you have updated our LICENSE.adoc
  • You have formulated clear instructions on how to test your contribution under "Testing instructions"

…venv isolation

Install python as a pristine, versioned interpreter once into the shared
software repository ($IDE_ROOT/software/default/python/python/<version>),
shared across projects, and route each project's global packages into a
per-project virtual environment (IDE_HOME/.venv) so projects no longer
collide (devonfw#352). The uv tool store is likewise per-project (IDE_HOME/.uv-tools).

Add a python launcher (Bash and PowerShell) that routes to 'ide python'
inside a project and falls back to the system python outside, mirroring the
existing node/npm/claude functions.
@coveralls

coveralls commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Coverage Report for CI Build 41044692055

Coverage decreased (-0.04%) to 75.245%

Details

  • Coverage decreased (-0.04%) from the base build.
  • Patch coverage: No coverable lines changed in this PR.
  • 41 coverage regressions across 4 files.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

41 previously-covered lines in 4 files lost coverage.

File Lines Losing Coverage Coverage
com/devonfw/tools/ide/tool/python/Python.java 31 57.14%
com/devonfw/tools/ide/tool/uv/Uv.java 8 81.01%
com/devonfw/tools/ide/variable/IdeVariables.java 1 95.0%
com/devonfw/tools/ide/version/VersionSegment.java 1 90.29%

Coverage Stats

Coverage Status
Relevant Lines: 19963
Covered Lines: 15679
Line Coverage: 78.54%
Relevant Branches: 8868
Covered Branches: 6015
Branch Coverage: 67.83%
Branches in Coverage %: Yes
Coverage Strength: 3.38 hits per line

💛 - Coveralls

@krystynaShatkovska krystynaShatkovska added the python runtime for python language label Oct 8, 2026
…avadoc

Fix code-review findings on the python pristine tool:
- create the "Scripts"->"bin" symlink for the per-project virtual
  environment on Windows (previously only the pristine interpreter got
  it) so the venv "bin" PATH entry from setEnvironment resolves on all
  platforms.
- rewrite UvTest.testSetEnvironmentWithNullIdeHome to exercise the
  actual getIdeHome() guard instead of the obsolete getSoftwarePath()
  one.
- correct the stale "virtual environment" @PARAM javadoc to the Python
  installation.
@QuangAnhLe QuangAnhLe self-assigned this Oct 8, 2026

@QuangAnhLe QuangAnhLe left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@krystynaShatkovska Thanks for the PR! Looks good overall. I’ve left a few comments for you to take a look at.

Comment thread cli/src/main/package/functions Resolved
Comment thread cli/src/main/package/functions.ps1 Resolved
…, add python3

- Stop priming ruff as a side effect of `ide install python`: gate it behind a
  new opt-in variable PYTHON_INSTALL_RUFF (default off) so a linter is not
  pushed onto every project.
- Windows fix: a pristine interpreter keeps its executable in the installation
  root (Scripts/ is empty), so no "bin" symlink is created for the interpreter
  (which would otherwise shadow the real python.exe). The "bin" symlink is only
  created for the per-project venv, where Scripts/ genuinely holds the exe.
- Add a python3 wrapper (bash functions + PowerShell) mirroring python.
- Add a Windows-layout regression test (driven by a UV_LAYOUT=windows env var
  so the shared uv mock emits the Windows layout on any host).

@QuangAnhLe QuangAnhLe left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi @krystynaShatkovska, thanks for your fix. They look gut. There is only a problem in Uv.Java: Line 124-125. Can you maybe see it ?

The uv tool store was moved from the per-project software folder
(<IDE_HOME>/software/python/tools, see devonfw#1518) to <IDE_HOME>/.uv-tools to make
it per-project (see devonfw#352), but previously installed uv tools were left behind.

On setEnvironment, one-time migrate the legacy store into .uv-tools: move it if
the new store is free, back it up if the new store already holds tools. Best
effort and never fails the environment setup.
@QuangAnhLe QuangAnhLe moved this from Team Review to 👀 In review in IDEasy board Oct 23, 2026
Keep the per-project virtual environment design (devonfw#2561): setEnvironment points
VIRTUAL_ENV and UV_PROJECT_ENVIRONMENT at IDE_HOME/.venv (the venv created by
setupProjectEnvironment), not at the shared pristine interpreter from devonfw#2701.
Pointing uv at a per-project venv does not break uv projects, so the devonfw#2701
regression does not apply here; taking upstream's side would orphan the whole
per-project venv + Windows bin symlink logic this branch introduces.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python runtime for python language

Projects

Status: 👀 In review

Development

Successfully merging this pull request may close these issues.

python in pristine software repo with per-project isolation

3 participants