feat: Add Python Virtual Environment Support: Installing User Defined Packages

[SarahAsad23]

What changes were proposed in this PR?

This PR is an extension of PR #4484. Previously, we introduced support for creating Python Virtual Environments (PVEs) with system-level dependencies preinstalled. This PR builds on that foundation by enabling users to install custom Python packages within a PVE. In some cases, installing a user package may overwrite system-level packages. For this reason, System package versions are locked to prevent these conflicts.

Some duplicate lines of code (previously introduced in PR #4484) were removed from PveManager to improve clarity and reduce redundancy.

Any related issues, documentation, discussions?

This change is part of ongoing efforts to support environment isolation and reproducibility within Texera. Related issue includes #4296. This PR closes sub-issue #4465.

A GitHub CI sync check between requirements.txt and system-requirements-lock.txt will be added in a follow-up PR. Tracking issue: #5034.

How was this PR tested?

Tested Manually and PveResourceSpec test file updated.
To test:

1. On CU click "+" Python Environments.
2. Input environment name.
3. Input package name and version.
4. Click "OK" and wait for pip logs.

To test system dependency conflit management, install the following packages:

• Scanpy 1.11.1 -> Successful
• Tensorflow 2.16.1 -> Unsuccessful because of system level dependency conflict

Was this PR authored or co-authored using generative AI tooling?

Co-authored using: ChatGPT (OpenAI)

[kunwp1]

The feature is working well! I left a few comments to improve the codebase and user experience.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 26.25000% with 118 lines in your changes missing coverage. Please review.
✅ Project coverage is 42.72%. Comparing base (2b763e4) to head (e5546cd).

Files with missing lines	Patch %	Lines
...power-button/computing-unit-selection.component.ts	0.00%	62 Missing ⚠️
...wer-button/computing-unit-selection.component.html	0.00%	35 Missing ⚠️
...resource/pythonvirtualenvironment/PveManager.scala	76.36%	9 Missing and 4 partials ⚠️
...esource/pythonvirtualenvironment/PveResource.scala	0.00%	5 Missing ⚠️
...virtual-environment/virtual-environment.service.ts	0.00%	2 Missing ⚠️
...ythonvirtualenvironment/PveWebsocketResource.scala	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #4902      +/-   ##
============================================
- Coverage     42.78%   42.72%   -0.07%     
+ Complexity     2189     2186       -3     
============================================
  Files          1045     1045              
  Lines         39876    39979     +103     
  Branches       4205     4218      +13     
============================================
+ Hits          17062    17080      +18     
- Misses        21757    21838      +81     
- Partials       1057     1061       +4     

Flag	Coverage Δ		*Carryforward flag
access-control-service	39.53% <ø> (ø)		Carriedforward from 2b763e4
agent-service	33.72% <ø> (ø)		Carriedforward from 2b763e4
amber	43.34% <68.85%> (+0.04%)	⬆️
computing-unit-managing-service	0.00% <ø> (ø)		Carriedforward from 2b763e4
config-service	0.00% <ø> (ø)		Carriedforward from 2b763e4
file-service	32.18% <ø> (ø)		Carriedforward from 2b763e4
frontend	34.04% <0.00%> (-0.17%)	⬇️
python	88.90% <ø> (ø)		Carriedforward from 2b763e4
workflow-compiling-service	47.72% <ø> (ø)		Carriedforward from 2b763e4

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[kunwp1]

Left a few more comments.

[kunwp1]

Left more comments. Can you add more test cases that Codecov Report complains?

Also, this PR lacks sync check between requirements.txt and system-requirements-lock.txt. It's better to add a sync check in the github CI but totally up to you. If you are not going to add a github CI check on this PR, please mention it in the PR description and create an issue.

[SarahAsad23]

Thanks for the comments. I reviewed the Codecov report, and the current PveResourceSpec test file already covers the main PVE functionality and backend behavior. The remaining uncovered lines are mostly UI-related or exercised indirectly through the existing resource-level tests, so I did not add additional test cases at this stage.

I also created issue #5034 to track adding a CI sync check between requirements.txt and system-requirements-lock.txt, and updated the PR description to mention that it will be added in a follow-up PR.

[kunwp1]

LGTM!