Skip to content

test: expand credential hiding tests to all 14 protected paths#1163

Merged
Mossaka merged 5 commits intomainfrom
fix/052-credential-hiding-tests
Mar 11, 2026
Merged

test: expand credential hiding tests to all 14 protected paths#1163
Mossaka merged 5 commits intomainfrom
fix/052-credential-hiding-tests

Conversation

@Mossaka
Copy link
Collaborator

@Mossaka Mossaka commented Mar 5, 2026

Summary

  • Adds integration tests for the 11 previously untested credential file paths (out of 14 total)
  • Tests cover SSH keys (id_rsa, id_ed25519, id_ecdsa, id_dsa), AWS credentials/config, Kube config, Azure credentials, GCloud credentials.db, Cargo credentials, and Composer auth.json
  • Verifies each path returns 0 bytes at both direct home path and /host chroot path

Test plan

  • npm run build passes
  • npm test passes (831 tests)
  • npm run lint passes (0 errors)
  • CI integration tests verify all 14 paths are hidden

Fixes #761

🤖 Generated with Claude Code

Copilot AI review requested due to automatic review settings March 5, 2026 20:02
@github-actions
Copy link
Contributor

github-actions bot commented Mar 5, 2026

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 82.37% 82.64% 📈 +0.27%
Statements 82.27% 82.63% 📈 +0.36%
Functions 82.60% 82.69% 📈 +0.09%
Branches 74.21% 74.87% 📈 +0.66%
📁 Per-file Coverage Changes (3 files)
File Lines (Before → After) Statements (Before → After)
src/ssl-bump.ts 90.5% → 90.5% (+0.00%) 88.8% → 90.7% (+1.87%)
src/cli.ts 46.6% → 47.0% (+0.43%) 47.0% → 47.5% (+0.42%)
src/docker-manager.ts 83.4% → 84.5% (+1.08%) 82.8% → 83.8% (+1.04%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Expands the integration test suite to verify credential-hiding behavior across all 14 protected credential paths (home and /host chroot), ensuring the files are effectively empty when mounted from /dev/null.

Changes:

  • Updates the existing “Test 4” to more explicitly target the original 3 credential paths and adjusts its shell check.
  • Adds a new test group that covers the 11 previously untested credential file paths.
  • Verifies 0-byte size for direct home paths and /host paths, plus a cat-content check.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +250 to +251
const homeDir = os.homedir();
const paths = untestedPaths.map(p => `${homeDir}/${p.path}`).join(' ');
Copy link

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Building a shell command by interpolating paths directly into sh -c '...' is unsafe and can break when homeDir contains spaces or shell-special characters (and is also an injection risk since this runs with sudo). Prefer passing file paths as positional args to sh -c and iterating over "$@" (or otherwise robustly quoting/escaping each path) so the loop receives the exact paths regardless of characters.

Suggested change
const homeDir = os.homedir();
const paths = untestedPaths.map(p => `${homeDir}/${p.path}`).join(' ');
const paths = untestedPaths.map(p => `"$HOME/${p.path}"`).join(' ');

Copilot uses AI. Check for mistakes.
// Check all credential files in a single container run for efficiency.
// wc -c reports byte count; /dev/null-mounted files should be 0 bytes.
const result = await runner.runWithSudo(
`sh -c 'for f in ${paths}; do wc -c "$f" 2>/dev/null; done' 2>&1 | grep -v "^\\["`,
Copy link

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Building a shell command by interpolating paths directly into sh -c '...' is unsafe and can break when homeDir contains spaces or shell-special characters (and is also an injection risk since this runs with sudo). Prefer passing file paths as positional args to sh -c and iterating over "$@" (or otherwise robustly quoting/escaping each path) so the loop receives the exact paths regardless of characters.

Copilot uses AI. Check for mistakes.

// cat all files and concatenate output - should be empty
const result = await runner.runWithSudo(
`sh -c 'for f in ${paths}; do cat "$f" 2>/dev/null; done' 2>&1 | grep -v "^\\["`,
Copy link

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test can produce a false pass if one or more files don’t exist or aren’t mounted: cat errors are suppressed (2>/dev/null), so the combined stdout can be empty even when paths are missing. To make the assertion meaningful, ensure each file is actually present/checked (e.g., fail the loop if a path is missing, or emit/validate per-file markers), or remove this test and rely on the wc -c tests that already validate counts.

Suggested change
`sh -c 'for f in ${paths}; do cat "$f" 2>/dev/null; done' 2>&1 | grep -v "^\\["`,
`sh -c 'for f in ${paths}; do cat "$f"; done' 2>&1 | grep -v "^\\["`,

Copilot uses AI. Check for mistakes.
const lines = result.stdout.split('\n').filter(l => l.match(/^\s*\d+/));
// Each file should be 0 bytes (hidden via /dev/null)
lines.forEach(line => {
const size = parseInt(line.trim().split(/\s+/)[0]);
Copy link

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parseInt should be called with an explicit radix to avoid edge-case parsing issues. Use parseInt(value, 10) here (and in the similar parsing block in the /host test) to make the intent unambiguous.

Copilot uses AI. Check for mistakes.
Comment on lines +102 to +107
test('Test 4: Original 3 credential files are mounted from /dev/null', async () => {
const homeDir = os.homedir();

// Check multiple credential files in one command
// Check the originally-tested credential files in one command
const result = await runner.runWithSudo(
`sh -c 'for f in ${homeDir}/.docker/config.json ${homeDir}/.npmrc ${homeDir}/.config/gh/hosts.yml; do if [ -f "$f" ]; then wc -c "$f"; fi; done' 2>&1 | grep -v "^\\["`,
`sh -c 'for f in ${homeDir}/.docker/config.json ${homeDir}/.npmrc ${homeDir}/.config/gh/hosts.yml; do wc -c "$f" 2>/dev/null; done' 2>&1 | grep -v "^\\["`,
Copy link

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With 2>/dev/null added and no follow-up assertion that all three files produced output, this test can now silently skip missing/unmounted files (no wc output) and potentially still pass depending on downstream expectations. To preserve the “mounted from /dev/null” guarantee, parse the wc output and assert you got exactly 3 size lines and that each is 0 (or make the shell loop fail if any file is missing).

Copilot uses AI. Check for mistakes.
@github-actions
Copy link
Contributor

github-actions bot commented Mar 5, 2026

🧪 Build Test: Bun

Project Install Tests Status
elysia 1/1 PASS
hono 1/1 PASS

Overall: ✅ PASS

Bun v1.3.10 — all tests passed

Generated by Build Test Bun for issue #1163

@github-actions
Copy link
Contributor

github-actions bot commented Mar 5, 2026

Build Test: Node.js Results ✅

Project Install Tests Status
clsx All passed PASS
execa All passed PASS
p-limit All passed PASS

Overall: PASS

Generated by Build Test Node.js for issue #1163

@github-actions
Copy link
Contributor

github-actions bot commented Mar 5, 2026

Go Build Test Results

Project Download Tests Status
color 1/1 PASS
env 1/1 PASS
uuid 1/1 PASS

Overall: PASS

Generated by Build Test Go for issue #1163

@github-actions
Copy link
Contributor

github-actions bot commented Mar 5, 2026

C++ Build Test Results

Project CMake Build Status
fmt PASS
json PASS

Overall: PASS

Generated by Build Test C++ for issue #1163

@github-actions
Copy link
Contributor

github-actions bot commented Mar 5, 2026

Smoke test results for @Mossaka's PR:

✅ GitHub MCP: #1159 fix(security): eliminate TOCTOU race conditions in ssl-bump.ts, #1158 fix(security): stop logging partial token values
✅ Playwright: github.com title contains "GitHub"
✅ File write: /tmp/gh-aw/agent/smoke-test-copilot-22734391352.txt created
✅ Bash: file verified via cat

Overall: PASS

📰 BREAKING: Report filed by Smoke Copilot for issue #1163

@github-actions
Copy link
Contributor

github-actions bot commented Mar 5, 2026

Build Test: Deno ✅

Project Tests Status
oak 1/1 ✅ PASS
std 1/1 ✅ PASS

Overall: PASS

Generated by Build Test Deno for issue #1163

@github-actions
Copy link
Contributor

github-actions bot commented Mar 5, 2026

Java Build Test Results ✅

Project Compile Tests Status
gson 1/1 PASS
caffeine 1/1 PASS

Overall: PASS

Generated by Build Test Java for issue #1163

@github-actions
Copy link
Contributor

github-actions bot commented Mar 5, 2026

Test results:

  1. GitHub MCP merged PRs: ✅ fix(security): eliminate TOCTOU race conditions in ssl-bump.ts; fix(security): stop logging partial token values
  2. Safeinputs GH PR list: ✅ test: expand credential hiding tests to all 14 protected paths; test: add chroot escape vector test coverage
  3. Playwright github title: ✅
  4. Tavily search: ❌ (Tavily MCP unavailable)
  5. File write: ✅
  6. Bash cat: ✅
  7. Discussion query+comment: ✅ (discussion 1149)
  8. Build: ✅
    Overall: FAIL

🔮 The oracle has spoken through Smoke Codex for issue #1163

@github-actions
Copy link
Contributor

github-actions bot commented Mar 5, 2026

.NET Build Test Results

Project Restore Build Run Status
hello-world PASS
json-parse PASS

Overall: PASS

Run output

hello-world:

Hello, World!
```

**json-parse:**
```
{
  "Name": "AWF Test",
  "Version": 1,
  "Success": true
}
Name: AWF Test, Success: True

Generated by Build Test .NET for issue #1163

@github-actions
Copy link
Contributor

github-actions bot commented Mar 5, 2026

🦀 Rust Build Test Results

Project Build Tests Status
fd 1/1 PASS
zoxide 1/1 PASS

Overall: ✅ PASS

Generated by Build Test Rust for issue #1163

@github-actions
Copy link
Contributor

github-actions bot commented Mar 5, 2026

Smoke Test Results

Overall: PASS

💥 [THE END] — Illustrated by Smoke Claude for issue #1163

Add 3 new integration tests covering all 11 untested credential paths:
SSH keys (4), AWS creds/config, Kube config, Azure creds, GCloud creds,
Cargo creds, Composer auth. Tests verify 0 bytes at both direct home
and /host chroot paths. Uses robust patterns (if -f, || true,
extractCommandOutput) consistent with existing tests.

Fixes #761

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@Mossaka Mossaka force-pushed the fix/052-credential-hiding-tests branch from c28ea23 to e10ad83 Compare March 11, 2026 00:54
@github-actions
Copy link
Contributor

Smoke Test Results — PASS

💥 [THE END] — Illustrated by Smoke Claude for issue #1163

@github-actions
Copy link
Contributor

GitHub MCP merged PRs: ✅
Merged PR titles: fix(squid): block direct IP connections that bypass domain filtering; feat: combine all build-test workflows into single build-test.md
safeinputs-gh PR list: ✅
Playwright title check: ✅
Tavily search: ❌ (tool unavailable)
File write: ✅
Bash cat verify: ✅
Discussion query + comment: ✅
Build (npm ci && npm run build): ✅
Overall status: FAIL

🔮 The oracle has spoken through Smoke Codex for issue #1163

@github-actions

This comment has been minimized.

@github-actions
Copy link
Contributor

✅ GitHub MCP: fix(squid): block direct IP connections that bypass domain filtering | feat: combine all build-test workflows into single build-test.md
✅ safeinputs-gh: test: add --allow-host-ports validation tests | test: add --proxy-logs-dir edge case coverage
✅ Playwright: GitHub · Change is constant. GitHub keeps you ahead. · GitHub
❌ Tavily search: tool unavailable
✅ File write: /tmp/gh-aw/agent/smoke-test-codex-22932059934.txt
✅ Bash cat: file read ok
✅ Discussion comment: posted on #1203
✅ Build: npm ci && npm run build
Overall: FAIL

🔮 The oracle has spoken through Smoke Codex for issue #1163

@github-actions
Copy link
Contributor

Smoke test results for @Mossaka:

Overall: PASS

📰 BREAKING: Report filed by Smoke Copilot for issue #1163

Replace existsSync+writeFileSync with writeFileSync({flag:'wx'}) to
eliminate the file-system-race CodeQL alert in the test beforeAll hook.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Contributor

Smoke test results for run 22932379687 — @Mossaka

Test Result
GitHub MCP: Last 2 merged PRs #1219 "test: add workDir tmpfs hiding integration tests", #1160 "fix(squid): block direct IP connections that bypass domain filtering"
Playwright: github.com title ✅ "GitHub · Change is constant. GitHub keeps you ahead. · GitHub"
File write /tmp/gh-aw/agent/smoke-test-copilot-22932379687.txt created
Bash verify ✅ File contents confirmed

Overall: PASS

📰 BREAKING: Report filed by Smoke Copilot for issue #1163

@github-actions
Copy link
Contributor

Smoke Test Results — PASS

💥 [THE END] — Illustrated by Smoke Claude for issue #1163

@github-actions

This comment has been minimized.

@github-actions
Copy link
Contributor

PR titles:
docs: sync version references and add missing CLI flags
test: add --skip-pull integration test
Tests: GitHub MCP ✅, safeinputs-gh ✅, Playwright ✅, Tavily ❌, file write ✅, bash cat ✅, discussion comment ✅, build ✅
Overall status: FAIL

🔮 The oracle has spoken through Smoke Codex for issue #1163

/dev/null-mounted credential files are character special devices, not
regular files. [ -f ] returns false for them, causing wc -c to produce
no output. Use [ -e ] (exists) to correctly detect these mounts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Contributor

Smoke Test Results

Overall: PASS

💥 [THE END] — Illustrated by Smoke Claude for issue #1163

@github-actions
Copy link
Contributor

Smoke test results for @Mossaka:

✅ GitHub MCP — Last 2 merged PRs: #1222 "test: add --skip-pull integration test", #1221 "test: add --allow-host-ports validation tests"
✅ Playwright — github.com title contains "GitHub"
✅ File write — /tmp/gh-aw/agent/smoke-test-copilot-22965779024.txt created and verified
✅ Bash — cat confirmed file contents

Overall: PASS

📰 BREAKING: Report filed by Smoke Copilot for issue #1163

@github-actions

This comment has been minimized.

@github-actions
Copy link
Contributor

Merged PRs: test: add --skip-pull integration test | test: add --allow-host-ports validation tests ✅
GitHub MCP merged PR review ✅
Safeinputs GH PR list ✅
Playwright github title ✅
Tavily search ❌
File write + cat ✅
Build (npm ci && npm run build) ✅
Overall: FAIL

🔮 The oracle has spoken through Smoke Codex for issue #1163

AWF always runs in chroot mode (chroot /host), so /host$HOME/... paths
don't exist inside the container. Changed the test from expecting 0-byte
files at /host paths to verifying those paths are inaccessible, which
is the correct security assertion for chroot mode.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Contributor

Smoke Test Results@Mossaka

✅ GitHub MCP: Last 2 merged PRs: #1229 "feat(cli): add short flags for frequently used options", #1228 "docs: clarify --image-tag behavior with agent-image presets" (both by @Mossaka)
✅ Playwright: github.com title contains "GitHub"
✅ File Write: /tmp/gh-aw/agent/smoke-test-copilot-22966932922.txt created
✅ Bash: File verified via cat

Overall: PASS

📰 BREAKING: Report filed by Smoke Copilot for issue #1163

@github-actions
Copy link
Contributor

Smoke Test Results

GitHub MCP: feat(cli): add short flags for frequently used options (#1229), docs: clarify --image-tag behavior with agent-image presets (#1228)
Playwright: github.com title contains "GitHub"
File Write: /tmp/gh-aw/agent/smoke-test-claude-22966932960.txt created
Bash Verify: File contents confirmed

Overall: PASS

💥 [THE END] — Illustrated by Smoke Claude for issue #1163

@github-actions
Copy link
Contributor

PR titles: #1240 test(docker): verify capsh execution chain after PR #715; #1234 fix(proxy): add lowercase proxy vars and NODE_EXTRA_CA_CERTS
GitHub MCP merged PR review: ✅
safeinputs-gh pr list: ✅
Playwright title check: ✅
Tavily search: ❌ (tool unavailable)
File write: ✅
Bash cat: ✅
Discussion comment: ✅
Build npm ci && npm run build: ✅
Overall: FAIL

🔮 The oracle has spoken through Smoke Codex for issue #1163

@github-actions
Copy link
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia ❌ CLONE_FAILED N/A ❌ FAIL
Bun hono ❌ CLONE_FAILED N/A ❌ FAIL
C++ fmt ❌ CLONE_FAILED N/A ❌ FAIL
C++ json ❌ CLONE_FAILED N/A ❌ FAIL
Deno oak ❌ CLONE_FAILED N/A ❌ FAIL
Deno std ❌ CLONE_FAILED N/A ❌ FAIL
.NET hello-world ❌ CLONE_FAILED N/A ❌ FAIL
.NET json-parse ❌ CLONE_FAILED N/A ❌ FAIL
Go color ❌ CLONE_FAILED N/A ❌ FAIL
Go env ❌ CLONE_FAILED N/A ❌ FAIL
Go uuid ❌ CLONE_FAILED N/A ❌ FAIL
Java gson ❌ CLONE_FAILED N/A ❌ FAIL
Java caffeine ❌ CLONE_FAILED N/A ❌ FAIL
Node.js clsx ❌ CLONE_FAILED N/A ❌ FAIL
Node.js execa ❌ CLONE_FAILED N/A ❌ FAIL
Node.js p-limit ❌ CLONE_FAILED N/A ❌ FAIL
Rust fd ❌ CLONE_FAILED N/A ❌ FAIL
Rust zoxide ❌ CLONE_FAILED N/A ❌ FAIL

Overall: 0/8 ecosystems passed — ❌ FAIL

❌ ALL_CLONES_FAILED

All 8 repository clones failed. The gh CLI is not authenticated — GH_TOKEN environment variable is not set in this environment.

Error:

gh: To use GitHub CLI in a GitHub Actions workflow, set the GH_TOKEN environment variable. Example:
  env:
    GH_TOKEN: ${{ github.token }}

Action required: Ensure GH_TOKEN is passed to the workflow job that runs this build test suite.

Generated by Build Test Suite for issue #1163 ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Testing] Credential hiding integration tests only cover 3 of 14 protected paths

2 participants