Skip to content

Activate Docker container lifecycle with skip-rebuild - #95

Merged
igmarin merged 4 commits into
mainfrom
feat/issue-88-activate-lifecycle
Aug 8, 2026
Merged

Activate Docker container lifecycle with skip-rebuild#95
igmarin merged 4 commits into
mainfrom
feat/issue-88-activate-lifecycle

Conversation

@igmarin

@igmarin igmarin commented Aug 8, 2026

Copy link
Copy Markdown
Owner

Summary

  • Prefer Docker isolation when context + daemon are available
  • ensure_image builds only when versioned tag is missing (skip rebuild)
  • Use evaluator-sandbox:<VERSION> for run; tag :latest on build
  • container_id stays nil when Docker is unavailable (fail-closed host unchanged)
  • Unit tests cover availability, skip-build, security flags, and run paths

Closes #88

Base: #93 (image contract).
Milestone: container-isolation-v1

Stack

  1. BE | Package Docker build context in gem (#86) #92 packaging (BE | Package Docker build context in gem #86)
  2. Define evaluator-sandbox image contract and docker:build #93 image contract (BE | Define evaluator-sandbox image contract + build #87)
  3. This PR activation (BE | Activate container lifecycle with safe fallbacks #88)
  4. BE | Live Docker integration tests (opt-in) #89 live tests (next)
  5. BE | CI job for container integration #90 CI
  6. BE | Docs + CHANGELOG for shipped container isolation #91 docs

Pre-push

  • Lifecycle unit tests green (9)
  • Related sandbox / run_command / packaging tests green
  • RuboCop + YARD coverage clean on touched files
  • rs-guard: deepseek call hung; self-review: security flags preserved, fail-closed host, no allow_host default change

Test plan

  • lifecycle unit tests
  • sandbox + packaging + run_command
  • CI green

@coderabbitai

coderabbitai Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@igmarin, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 13 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Repository UI (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: 130e8fd0-f3b2-444b-8ec2-00e5e931a9e4

📥 Commits

Reviewing files that changed from the base of the PR and between 6b913f7 and dbce369.

📒 Files selected for processing (9)
  • .github/workflows/ci.yml
  • Rakefile
  • lib/skill_bench/clients/provider_config.rb
  • lib/skill_bench/constants.rb
  • lib/skill_bench/execution/sandbox.rb
  • test/cli/validate_command_test.rb
  • test/evaluator/clients/provider_config_test.rb
  • test/evaluator/sandbox_container_lifecycle_test.rb
  • test/evaluator/sandbox_docker_live_test.rb

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@igmarin
igmarin marked this pull request as ready for review August 8, 2026 05:07
@igmarin
igmarin force-pushed the feat/issue-88-activate-lifecycle branch from 2daad4b to 85c78a8 Compare August 8, 2026 15:26
Base automatically changed from feat/issue-87-image-contract to main August 8, 2026 15:27
Prefer versioned evaluator-sandbox image when Docker is available,
build only if the image is missing, and leave container_id nil when
Docker is unavailable so host execution stays fail-closed. Align rake
docker:build with image_ref/latest tags (Closes #88).
@igmarin
igmarin force-pushed the feat/issue-88-activate-lifecycle branch from 85c78a8 to 137d6af Compare August 8, 2026 15:27
igmarin added 2 commits August 8, 2026 09:32
Summary
Opt-in live Docker tests behind SKILL_BENCH_DOCKER_TESTS=1
Skip cleanly without flag or without daemon (default CI unit path)
Assert RunCommand works with a real container_id and container is cleaned up after Sandbox.run
Closes #89

Stacked on #95 / #88.

Test plan
 Without env flag: 2 skips
 With Docker + flag: live tests pass (CI job in BE | CI job for container integration #90)
 CI unit matrix still green (skips)
fetch_config used || so options with api_key: nil fell through to
SkillBench::Config.for_provider, making validate-key checks and clients
inherit polluted/global keys (flaky CI on ValidateCommandTest). Prefer
key presence including explicit nil. Harden validate test Config.reset.
@igmarin

igmarin commented Aug 8, 2026

Copy link
Copy Markdown
Owner Author

CI failure fix landed

Root cause: ProviderConfig#fetch_config used ||, so explicit api_key: nil fell through to global Config.for_provider (order-dependent / home config leak).

Fix: honor key presence including nil; regression test + Config.reset in validate setup.

CI after fix: test (3.3) ✅ test (3.4) ✅

Summary
Add container-integration job: build image + live Docker tests
Keep existing Ruby matrix hermetic (no Docker required for unit path)
Open pull_request triggers so stacked PRs (non-main bases) still get CI
Closes #90

Stacked on #96 / #89.
@igmarin
igmarin merged commit e2b8a28 into main Aug 8, 2026
6 checks passed
@igmarin
igmarin deleted the feat/issue-88-activate-lifecycle branch August 8, 2026 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BE | Activate container lifecycle with safe fallbacks

1 participant