Skip to content

feat(contradictions): advisory scanner for conflicting approved claims#405

Open
jsdevninja wants to merge 1 commit into
vouchdev:testfrom
jsdevninja:feat/contradiction-scan
Open

feat(contradictions): advisory scanner for conflicting approved claims#405
jsdevninja wants to merge 1 commit into
vouchdev:testfrom
jsdevninja:feat/contradiction-scan

Conversation

@jsdevninja

@jsdevninja jsdevninja commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

closes #314

summary

  • adds src/vouch/contradictions.py: a read-only heuristic scanner that groups approved claims by shared Claim.entities and flags same-topic pairs whose text disagrees in polarity (one asserts, the other negates) as candidate contradictions.
  • adds vouch contradict-scan (cli-only, matching how vouch dedup ships without a dedicated kb.* method): --threshold, --entity, --limit, --dry-run/--no-dry-run (default dry-run).
  • dry-run only prints candidate pairs with a score and the shared entity. without --dry-run, each surviving pair files exactly one pending contradicts relation proposal via proposals.propose_relation, visible in kb.list_pending / vouch pending.
  • a pair already cross-linked via Claim.contradicts, already joined by an approved contradicts relation edge, or already carrying a pending contradicts proposal is skipped, so repeat scans don't pile up duplicates.
  • the scanner itself never sets ClaimStatus.CONTESTED, never writes a Relation, and never touches Claim.contradicts — all business logic lives in contradictions.py, storage.py is untouched.

design note (worth a second pair of eyes)

approving the resulting relation proposal currently lands a plain Relation(relation=CONTRADICTS) edge — it does not flip both claims to CONTESTED or cross-link Claim.contradicts. that full symmetric transition still only happens via the pre-existing, deliberately-manual lifecycle.contradict (kb.contradict), whose own docstring says it intentionally bypasses the proposal queue and warns against refactoring it for stricter gating. i kept proposals.approve() untouched rather than special-casing relation == "contradicts" there, since that's shared code for every relation kind and the issue's acceptance checklist only requires the proposal to land as a CONTRADICTS edge on approval, not the status flip. happy to wire that up instead if the intended reading of "the actual write still flows through lifecycle.contradict semantics on approval" was that kb.approve itself should trigger the full transition.

testing

  • tests/test_contradictions.py (15 tests): candidate scoring/grouping, entity/threshold/limit filters, inactive-claim exclusion, all three duplicate-skip paths (contradicts field / approved edge / pending proposal), dry-run writes nothing, non-dry-run files exactly one proposal per pair and never mutates claim status or writes a relation directly, repeat scans don't duplicate, and a full scan → propose → approve integration test landing a CONTRADICTS edge.
  • manually exercised vouch contradict-scan end-to-end against a scratch kb (dry-run, --no-dry-run, re-scan dedup, vouch approve) — matches the automated coverage.
  • ruff check src tests clean.
  • mypy src — same 3 pre-existing errors as main (sandbox.py os.getuid/os.getgid on non-posix, missing jinja2 stubs); none in the touched files.
  • pytest tests/ -q --ignore=tests/embeddings — ran locally on windows. two files (test_http_server.py, test_http_server_mcp.py) hang in this sandbox because they bind a real loopback socket, unrelated to this diff, so i excluded them locally; everything else passed except 8 pre-existing windows-only failures (posix os.getuid, unprivileged symlinks, / vs \ path assertions) untouched by this change. tests/test_contradictions.py is green.

test plan

  • vouch contradict-scan lists candidate pairs, writes nothing in --dry-run (default)
  • without --dry-run, each surviving pair produces exactly one pending contradicts proposal visible in kb.list_pending
  • scanner never sets CONTESTED, never writes a Relation, never touches Claim.contradicts
  • no duplicate proposals across repeat scans
  • --threshold, --entity, --limit, --dry-run/--no-dry-run behave as documented
  • scoring/grouping lives in src/vouch/contradictions.py, storage.py unchanged
  • cli-only for this first cut (no kb.* mcp/jsonl method), per the issue's stated allowance

Summary by CodeRabbit

  • New Features

    • Added vouch contradict-scan to scan approved claims for likely same-entity contradictions and rank candidate pairs.
    • Supports --dry-run (default) to preview candidates, or live mode to create pending contradiction relation proposals for later approval.
    • Added tunable --threshold, optional --entity scoping, and --limit to cap results.
  • Bug Fixes

    • Prevents duplicate suggestions when contradictions are already approved, linked, or previously proposed.
  • Tests

    • Added automated coverage for candidate detection, write vs preview behavior, idempotency, filtering, and the approve flow.

@github-actions github-actions Bot added docs documentation, specs, examples, and repo guidance cli command line interface tests tests and fixtures size: M 200-499 changed non-doc lines labels Jul 6, 2026
@coderabbitai

coderabbitai Bot commented Jul 6, 2026

Copy link
Copy Markdown

Review Change Stack

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e48824b6-5966-4f96-94f2-6c33d88e5a68

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds an advisory vouch contradict-scan feature that groups approved claims by shared entity, detects likely polarity conflicts, optionally files pending relation proposals, exposes CLI controls, and adds tests for discovery through approval.

Changes

Contradiction Scan Feature

Layer / File(s) Summary
Scoring model and entity grouping
src/vouch/contradictions.py
Adds tokenization, negation detection, Jaccard scoring, the Candidate dataclass, and shared-entity grouping.
Candidate discovery and proposal scanning
src/vouch/contradictions.py
Filters inactive or previously flagged pairs, identifies opposite-polarity candidates, and optionally creates pending contradicts proposals.
CLI command integration
src/vouch/cli.py
Adds contradict-scan with threshold, entity, dry-run, and limit options, then prints candidate results.
Pipeline validation and changelog
tests/test_contradictions.py, CHANGELOG.md
Tests discovery, filtering, proposal behavior, idempotency, limits, and scan-to-approve flow; documents the new command.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant CLI as contradict_scan
  participant Scanner as contradictions.scan
  participant Finder as find_candidates
  participant Store as KBStore
  participant Proposals as propose_relation

  User->>CLI: Run contradict-scan with options
  CLI->>Scanner: Call scan
  Scanner->>Finder: Find candidate pairs
  Finder->>Store: Read claims and existing contradiction records
  Finder-->>Scanner: Return scored candidates
  Scanner->>Proposals: Create pending proposal when writes are enabled
  Proposals-->>Scanner: Return proposal id or error
  Scanner-->>CLI: Return result rows
  CLI-->>User: Print candidates and proposal ids
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes match #314: they scan approved claims, score/group conflicts, avoid writes, and propose contradicts relations through the CLI.
Out of Scope Changes check ✅ Passed The modified files stay within the scanner feature, its CLI, tests, and changelog documentation.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: an advisory contradiction scanner for approved claims.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@CHANGELOG.md`:
- Line 16: Capitalize the word after the period in the changelog sentence so it
reads as a new sentence; update the text containing `--entity`, `--limit`, and
the following `scoring` phrase to start with an uppercase letter.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 66e8ef38-6b6c-401c-87d9-3049d7e67505

📥 Commits

Reviewing files that changed from the base of the PR and between 36f035e and 1364e17.

📒 Files selected for processing (4)
  • CHANGELOG.md
  • src/vouch/cli.py
  • src/vouch/contradictions.py
  • tests/test_contradictions.py

Comment thread CHANGELOG.md
@jsdevninja

Copy link
Copy Markdown
Contributor Author

@plind-junior Would you review please?

@plind-junior plind-junior changed the base branch from main to test July 13, 2026 05:42
@github-actions github-actions Bot added ci github actions and automation mcp mcp, jsonl, and http surfaces storage kb storage, migrations, schemas, and proposals retrieval context, search, synthesis, and evaluation embeddings embedding-backed retrieval size: XL 1000 or more changed non-doc lines and removed size: M 200-499 changed non-doc lines labels Jul 13, 2026
@jsdevninja jsdevninja force-pushed the feat/contradiction-scan branch 2 times, most recently from a673128 to c6eeb5a Compare July 13, 2026 15:49
@github-actions github-actions Bot added size: M 200-499 changed non-doc lines and removed ci github actions and automation mcp mcp, jsonl, and http surfaces storage kb storage, migrations, schemas, and proposals retrieval context, search, synthesis, and evaluation embeddings embedding-backed retrieval size: XL 1000 or more changed non-doc lines labels Jul 13, 2026
@jsdevninja

Copy link
Copy Markdown
Contributor Author

@plind-junior pleaes review

walk approved claims grouped by shared entity, flag same-topic pairs
that disagree in polarity, and file a pending `contradicts` relation
proposal per surviving pair via proposals.propose_relation. read-only
and advisory: the scanner never sets ClaimStatus.CONTESTED, never
writes a Relation, and never touches Claim.contradicts — those only
land once a human runs `vouch approve` (or the pre-existing manual
lifecycle.contradict). scoring/grouping lives in the new
src/vouch/contradictions.py, not storage.py.

`vouch contradict-scan` mirrors `vouch dedup`'s cli-only shape:
--threshold, --entity, --limit, and --dry-run/--no-dry-run (default
dry-run). a pair already cross-linked via Claim.contradicts, joined
by an approved contradicts edge, or with a pending contradicts
proposal is skipped so repeat scans don't duplicate proposals.

closes vouchdev#314
@jsdevninja jsdevninja force-pushed the feat/contradiction-scan branch from c6eeb5a to c9d6c9c Compare July 13, 2026 16:27
@jsdevninja

Copy link
Copy Markdown
Contributor Author

@plind-junior Please check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cli command line interface docs documentation, specs, examples, and repo guidance size: M 200-499 changed non-doc lines tests tests and fixtures

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: contradiction finder — scan the kb and propose contradicts links for conflicting claims

1 participant