Skip to content

test(assail): check stripped output against an independent oracle - #306

Open
hyperpolymath wants to merge 1 commit into
mainfrom
claude/strip-oracle-test
Open

hyperpolymath wants to merge 1 commit into
mainfrom
claude/strip-oracle-test

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

Why

This is the fix for CodeRabbit's finding on #304 (thread discussion_r4590590229). It was pushed to #304's branch about five minutes after #304 merged, so it never reached main.

The exhaustive test on main checks length, the per-byte relation ("unchanged, or blanked") and idempotence. A scanner that blanks plain code passes those checks. For example, returning " " for "x" with block markers on is only caught if a targeted regression happens to cover that input. The ledger row claims exhaustive preservation, so the test should establish it.

What

  • oracle: the comment grammar as mutually recursive rules (code, line comment, block at depth d) over a char slice. It shares no code with the index-and-match scanner.
  • The exhaustive test now asserts that the scanner's full output equals oracle for every enumerated input, on top of the existing checks.
  • PROOF-NEEDS.adoc: the strip_proof_comments row names the oracle and lists the new planted fault.

Evidence

  • Planted fault, re-run on this main-based branch: blanking non-comment code (out.push_str(&" ".repeat(len)) in the code arm) fails the exhaustive test on its own, at the oracle assertion.
  • Tests: strip_proof_comments_tests and slash_comment_tests pass (13 tests).
  • Lints: cargo clippy --all-targets --all-features -D warnings and cargo fmt --check are clean.
  • Docs: PROOF-NEEDS.adoc renders clean.
  • Full suite: passed on fix(assail): strip proof comments in one pass, blanking in place #304's branch (895 tests). On this branch only the two test modules above were re-run. The code under test is unchanged from main, and only test code and the ledger change.

Tests and docs only; no production code changes.


Generated by Claude Code

The exhaustive test checked length, the per-byte relation and
idempotence, which a scanner that blanks plain code could still pass.
Add `oracle`, the comment grammar as mutually recursive rules over a
char slice (code, line comment, block at depth d), sharing no code with
the scanner, and assert the scanner's full output equals it for every
enumerated input. A planted fault that blanks non-comment code now
fails this test alone. PROOF-NEEDS names the oracle and the fault.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jma5ixtpeZyJRmfoSsC98u
@sonarqubecloud

Copy link
Copy Markdown

Repository owner deleted a comment from coderabbitai Bot Nov 16, 2026
@hyperpolymath
hyperpolymath marked this pull request as ready for review November 16, 2026 21:01
@hyperpolymath
hyperpolymath enabled auto-merge (squash) November 16, 2026 21:01
@coderabbitai

coderabbitai Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Summary

Summary by CodeRabbit

  • Tests
    • Expanded exhaustive checks for proof-comment stripping using an independent comment-grammar reference.
    • Added coverage to ensure code outside comments is not blanked.

The exhaustive bounded-input test for strip_proof_comments now compares results with an independent recursive oracle. The proof needs document describes the oracle and adds blanking code outside comments to the planted-fault list.

Changes: Proof comment stripper oracle tests

Layer / File(s) Summary
Oracle validation and coverage
src/assail/analyzer.rs, PROOF-NEEDS.adoc
The exhaustive test compares stripper output with an independent recursive oracle that models line and nested block comments, marker precedence, and UTF-8-byte-preserving blanking. The document records this oracle and notes that it catches blanking code outside comments.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other

Possibly related issues

  • hyperpolymath/panic-attack#280 — The PR tests strip_proof_comments, which issue #280 identifies as a building block. It does not add line anchoring or the remaining detectors described in the issue.

Possibly related PRs

Suggested reviewers: metadatastician

Merge Risk: Minimal · up to ef184

This change only strengthens a test and updates documentation, so it carries no known risk to users or production behavior.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the main change: a test in the assail module now checks the output of strip_proof_comments against an independent oracle. It matches the changeset, which adds an oracle and an exhausti…
Description check Passed The description explains the motivation (a follow-up to a CodeRabbit finding that never reached main), what the change adds (the recursive comment-grammar oracle, the exhaustive test assertion, and th…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: a test in the assail module now checks the output of strip_proof_comments against an independent oracle. It matches the changeset, which adds an oracle and an exhaustive assertion against it. It is specific and concise.

Full details: Description check

Explanation

The description explains the motivation (a follow-up to a CodeRabbit finding that never reached main), what the change adds (the recursive comment-grammar oracle, the exhaustive test assertion, and the PROOF-NEEDS.adoc update), and the evidence for it. It is directly related to the changeset.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: f5ae9263-24dd-4c2f-9f84-8465760079d7
Commits

Reviewing files that changed from the base of the PR and between 7b9f145 and ef1842e.

Files selected for processing: 2

  • PROOF-NEEDS.adoc
  • src/assail/analyzer.rs

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

Recent review details

Additional comments: 2

src/assail/analyzer.rs: 1

9012-9085: LGTM!

Also applies to: 9088-9090, 9097-9098

PROOF-NEEDS.adoc: 1

127-129: LGTM!

Also applies to: 140-140

Repository owner deleted a comment from chatgpt-codex-connector Bot Nov 16, 2026
Repository owner deleted a comment from chatgpt-codex-connector Bot Nov 16, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants