Repository navigation
CI: hermetic spawns in scan/policy/network suites (#823) - #1515
Merged
Mikola Lysenko (mikolalysenko) merged 3 commits intoOct 14, 2026
Merged
Mikola Lysenko (mikolalysenko) merged 3 commits into
Mikola Lysenko (mikolalysenko) merged 3 commits into
Conversation
Assisted-by: Claude Code:claude-opus-5-5
This was referenced Oct 14, 2026
e2e_scan, e2e_socket_yml_policy and apply/apply_network started socket-patch with a bare Command::new and their own SOCKET_* sweeps. apply_network only removed SOCKET_API_TOKEN in most tests, so a developer shell exporting SOCKET_DRY_RUN or SOCKET_ECOSYSTEMS failed 9 of its tests. All three sweeps let an ambient SOCKET_NO_CONFIG=0 through, so a real `socket login` could reach the child, and none forced the update check off. All three now spawn through hermetic::command. The policy suite also scrubs Bundler env, since its fixture includes a gem. The npm children in e2e_scan drop ambient SOCKET_* too. The files leave PENDING_RAW_SPAWNS, so the spawn_env_hygiene ratchet keeps them hermetic. Refs #823 Assisted-by: Claude Code:claude-opus-5-5
The shared builder forces telemetry off and removes the API token. The offline airgap test then no longer showed that --offline alone keeps telemetry away from the mock. It turns telemetry back on, so it fails again if offline runs ever POST telemetry (checked by removing the offline term from is_telemetry_disabled). The cached-blob test only pointed SOCKET_API_URL at a dead port. With no token the child uses the public proxy, so a regressed fetch could reach the real proxy and still pass. Pin SOCKET_PROXY_URL too. The pypi test uses Extra::Venv instead of a hand-written VIRTUAL_ENV removal. Refs #823 Assisted-by: Claude Code:claude-opus-5-5
Mikola Lysenko (mikolalysenko)
marked this pull request as ready for review
October 14, 2026 02:05
Collaborator
Author
|
BugBot review Generated by Claude Code |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 88407b9. Configure here.
Tanmay Singla (Tanmay182003)
approved these changes
Oct 14, 2026
Collaborator
Author
|
Ready for review at
Generated by Claude Code |
Mikola Lysenko (mikolalysenko)
deleted the
agent/fix-ci-hermetic-scan-policy-network
branch
October 14, 2026 03:57
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LLM Description written by Claude Code:claude-opus-5-5
Refs #823 (slice 16: scan, socket.yml policy and apply-network suites). Uses
Refs, so #823 stays open for the remainingPENDING_RAW_SPAWNSfiles.Summary
e2e_scan.rs,e2e_socket_yml_policy.rsandapply/apply_network.rsnow spawn socket-patch throughtests/common/hermetic.rsand leavePENDING_RAW_SPAWNS(12 → 9 on main; 6 once #1510 also lands, and the two edits don't touch adjacent lines). The ratchet inspawn_env_hygiene.rskeeps them hermetic. Test-only; no product code and no workflow changes.Root cause
Each file started the CLI with a bare
Command::newand its ownSOCKET_*handling, which had drifted from the shared builder:apply_networkremoved onlySOCKET_API_TOKENin 8 of its 9 spawns. On main, a shell that exportsSOCKET_DRY_RUN=true SOCKET_ECOSYSTEMS=cargofails 9 of its tests.SOCKET_NO_CONFIG=0through, so a developer's realsocket logincould reach the child, and none of them forced the update check off.BUNDLE_*. It now usesExtra::Venv+Extra::Bundle.e2e_scan'snpmchildren now drop ambientSOCKET_*too.Two coverage fixes so the move doesn't weaken guards (from /code-review):
offline_apply_with_token_makes_zero_network_requests: the builder forces telemetry off, which would hide a telemetry POST under--offline. The test setsSOCKET_TELEMETRY_DISABLED=0. Checked by removingis_offline_env()fromis_telemetry_disabled: the test fails withmock server saw: ["POST /patch/telemetry"].apply_uses_locally_cached_blob_without_fetchingpinned onlySOCKET_API_URLto a dead port, but with no token the child uses the public proxy. It now pinsSOCKET_PROXY_URLas well.Test evidence
PENDING_RAW_SPAWNSremoval) and with a hostile shell (SOCKET_DRY_RUN=true SOCKET_ECOSYSTEMS=cargo SOCKET_ORG_SLUG=hostile SOCKET_NO_CONFIG=0):spawn_env_hygienefailsno_new_bare_binary_spawnsfor all 3 files, andapplyfails 9apply_networktests.VIRTUAL_ENV=/nonexistent):apply135/135,e2e_socket_yml_policy26/26,spawn_env_hygiene16/16,e2e_scannon-ignored 20/20.e2e_scan's 11#[ignore]d live-API tests need patches-api.socket.dev, which the agent sandbox can't reach, so they weren't run. CI doesn't run them on PRs either.cargo fmt --all -- --checkandcargo clippy --locked --workspace --all-features -- -D warningsare clean. The test-target clippy notes are all pre-existing inprebuilt_common./code-review high
Fixed: the telemetry guard, the proxy pin,
Extra::Venvin the pypi test, and the e2e_scan comment. Not fixed:run/npm_run/assert_run_okonto thecommoncopies, and scrubbingSOCKET_*insidecommon::run_toolchainfor every package-manager child: both are refactors ofcommonbeyond this slice. They're worth a follow-up slice under Spawn every CLI test child through one hermetic Command builder; 10 test files inherit ambient SOCKET_* today #823.CI cost
No workflow changes. A lean PR run is the same before and after (~28 Linux jobs, ~8 min).
🤖 Generated with Claude Code
https://claude.ai/code/session_01NSWn8iEQppkpg6sqyjhEx6
Generated by Claude Code