Skip to content

fix: require every test runner to report its scenario results - #1936

Closed
hmzakhalid wants to merge 3 commits into
mainfrom
fix/test-audit-runner-results
Closed

fix: require every test runner to report its scenario results#1936
hmzakhalid wants to merge 3 commits into
mainfrom
fix/test-audit-runner-results

Conversation

@hmzakhalid

@hmzakhalid hmzakhalid commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Fixes audit findings HAR-001, HAR-004, and RT-013.

  • Run only the existing default integration scenarios; preserve the CI-only prebuild entry point and reject unknown names before setup.
  • Require the named CRISP browser-test command to succeed.
  • Give each network run a unique Compose project. Require all six containers to finish with exit code zero, retain logs, and clean up only that project.
  • Run 22 local runner regressions in the CI dispatch job (pnpm test:harnesses locally).

Verification: all 22 regressions pass using command stand-ins; shell syntax, formatting, lint, and normal pre-push checks pass. The new prebuild regression first reproduced the CI compatibility failure, then passed with the restored entry point. Six controls with the installed concurrently confirm that an early clean server exit now fails, while completed test success/failure still behaves correctly. No Docker network or complete CRISP browser stack was run locally; the tests exercise the wrappers and result propagation.

@vercel

vercel Bot commented Sep 11, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
crisp Ready Ready Preview Sep 11, 2026 4:31am UTC
interfold-dashboard Ready Ready Preview Sep 11, 2026 4:31am UTC
interfold-docs Ready Ready Preview Sep 11, 2026 4:31am UTC

Request Review

@coderabbitai

coderabbitai Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Next included review available in 20 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 1f40e5ca-2955-44f5-ac04-451f88882176

📥 Commits

Reviewing files that changed from the base of the PR and between 03ed94e and 9473430.

📒 Files selected for processing (8)
  • .github/workflows/ci.yml
  • agent/CONTEXT.md
  • crates/net/tests/run.sh
  • examples/CRISP/scripts/test_e2e.sh
  • package.json
  • scripts/test-harnesses.test.mjs
  • tests/integration/README.md
  • tests/integration/test.sh

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@hmzakhalid

Copy link
Copy Markdown
Collaborator Author

Superseded by #1934 at the user’s request. The combined branch retains this PR’s commits, including 9473430. Full CI continues on the combined PR.

@hmzakhalid hmzakhalid closed this Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant