Skip to content

ci: optimize self-hosted KubeVirt runners and CI pipeline#787

Draft
mangelajo wants to merge 5 commits into
mainfrom
kubevirt-runners
Draft

ci: optimize self-hosted KubeVirt runners and CI pipeline#787
mangelajo wants to merge 5 commits into
mainfrom
kubevirt-runners

Conversation

@mangelajo

@mangelajo mangelajo commented Jun 12, 2026

Copy link
Copy Markdown
Member

Summary

  • Split KubeVirt runners into small (4Gi) and large (16Gi) VM flavors for better concurrency
  • Route unit test / build jobs to arc-runner-kubevirt-small, E2E test jobs to arc-runner-kubevirt-large
  • Run package tests in parallel (make test -j4 --output-sync on GNU Make, -j4 on BSD)
  • Suppress log noise in CI with --log-level=CRITICAL --log-cli-level=CRITICAL via PYTEST_ADDOPTS (project defaults unchanged for local dev)
  • Skip apt-get install when CI dependencies are already pre-baked in the golden image
  • Increase Renode monitor connect timeout from 10s to 45s (smaller VMs need more startup time)
  • Add download logging/timeout to u-boot test fixture for CI debuggability
  • Detect --output-sync support via make --help instead of version string (works on macOS BSD make)

Test plan

  • All python-tests matrix jobs pass (Linux small + macOS, Python 3.11/3.12/3.13)
  • E2E tests pick up arc-runner-kubevirt-large label
  • Log output is suppressed in CI but not locally
  • macOS jobs run with -j4 without --output-sync (BSD make)
  • Pre-baked dependencies skip apt-get install (check "Setup Linux dependencies" step)

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: c04b78c0-052a-4c13-9150-ca547ba35bed

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This PR updates GitHub Actions workflow runner configurations across the E2E and Python test pipelines, migrating from ubuntu-24.04 to arc-runner-kubevirt for job execution while maintaining the existing matrix structures for architecture coverage.

Changes

Runner Infrastructure Migration

Layer / File(s) Summary
E2E workflow jobs runner migration
.github/workflows/e2e.yaml
Build controller and operator image jobs, build-python-wheels, e2e-tests, and compatibility jobs switch from ubuntu-24.04 to arc-runner-kubevirt in their job matrices and runner configurations.
Python tests workflow runner migration
.github/workflows/python-tests.yaml
Pytest-matrix job updates its runs-on strategy from ubuntu-24.04 to arc-runner-kubevirt while retaining macos-15.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Poem

🐰 In the CI clouds we hop and bound,
New runners found, a better ground,
Arc-runner-kubevirt, swift and true,
Let tests and builds run spry and new!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title 'ci: optimize self-hosted KubeVirt runners and CI pipeline' is partially related to the changeset, which focuses on switching to KubeVirt runners, but overstates the scope by claiming 'optimize CI pipeline' when the changes are primarily runner migrations.
Description check ✅ Passed The PR description comprehensively aligns with the changeset, detailing infrastructure changes (KubeVirt runner migration), performance improvements (parallel testing), and CI optimizations (log suppression, timeout adjustments).

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch kubevirt-runners

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@kirkbrauer kirkbrauer left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me, hopefully we get quite a speedup here!

@mangelajo mangelajo marked this pull request as draft June 15, 2026 07:31
@mangelajo

Copy link
Copy Markdown
Member Author

I am experimenting @kirkbrauer , the operator I am trying for the github actions in K8s + kubevirt seems to be a bit slow to rotate VMs, and grab jobs. Unless we get to improve that we would have to look for an alternative option

@mangelajo mangelajo changed the title ci: use self-hosted KubeVirt runners for amd64 jobs ci: optimize self-hosted KubeVirt runners and CI pipeline Jun 19, 2026
mangelajo and others added 5 commits June 22, 2026 09:10
Route build jobs to arc-runner-kubevirt-small (4Gi) and E2E tests
to arc-runner-kubevirt-large (16Gi) for better resource utilization.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use arc-runner-kubevirt-small for pytest matrix, fail-fast: false
- Run tests with make -j4 and LOGS_DIR for per-package log capture
- Makefile: LOGS_DIR captures stdout/stderr via tee, tracks failures
  with .failed markers, and test-report prints failed package logs
- Suppress log noise in CI with --log-level/--log-cli-level CRITICAL
- Upload per-package logs as artifacts (7-day retention)
- Consolidate Linux dependency setup into single idempotent step
- Exclude test files from diff-cover to avoid false coverage failures

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Resolve sysctl full path via shutil.which with /usr/sbin fallback
- Capture exporter stderr for debugging
- Fix test mocks to match new _resolve_tool and shutil.which patterns

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Renode can take longer to start under parallel CI load on shared VMs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
rpmfile decompresses the entire 572MB CPIO payload into memory. Use
rpm2cpio | cpio for streaming extraction with rpmfile as fallback.
Also adds download progress logging and a 120s timeout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants