Repository navigation
Preserve fallback duplicate matches while indexing MinHash candidates - #36
Conversation
There was a problem hiding this comment.
Validator approval after policy checks for exact head 7bef05ca07a391b278f7baecf8724c0e4c268718.
Ticket: ticket-027
Correlation ID: tellmesh-ticket-027-20261106
Model: openai/cursor-auto
Reviewed diff chunks: 2
Advisory LLM verdict: APPROVE
Advisory summary: Reviewed all 2 diff chunk(s). The visible implementation uses the pigeonhole property of d+1 disjoint bands to prune fallback MinHash candidates without changing the original Jaccard scoring, threshold comparison, result ordering, or duplicate semantics. It safely falls back to exhaustive comparison for unsupported thresholds and native scoring, and its cache key/invalidation strategy covers additions and threshold changes in the relevant fallback paths. No security defects or unresolved correctness issues are evident in this chunk. | The added regression tests provide strong coverage for fallback candidate selection, exhaustive-score parity, stable ordering, index refresh after additions and threshold changes, native scorer semantics, comparison pruning, and grouping parity. No unresolved correctness or security defects are visible in this chunk.
Advisory findings: none
The LLM output above is advisory and was not used as the approval trust root.
Semantic review prerequisite: not_required; policy 676cb4516bbfed2a000e40b9b1b6e4a430ecc761ec546aeb53d721a1905cfdd7.
Actual PR impact radar
Exact range: adba8c1a82468267e2d1902b8b68ff1fcf35378b...7bef05ca07a391b278f7baecf8724c0e4c268718
Change digest: c81d723e4b48d36037700e28025535da037444a34296ca0967e4c0b9b0630947
Score: 44/100 (M), estimated 50 min, split recommended: true
Affected services/components: repository-wide/unclassified
Machine-readable radar JSONL and SVG
{"actual_change":{"additions":287,"base_sha":"adba8c1a82468267e2d1902b8b68ff1fcf35378b","binary_files":0,"categories":{"code":1,"configuration":1,"docs":2,"tests":1},"change_digest":"c81d723e4b48d36037700e28025535da037444a34296ca0967e4c0b9b0630947","comparison":"adba8c1a82468267e2d1902b8b68ff1fcf35378b...7bef05ca07a391b278f7baecf8724c0e4c268718","deletions":11,"file_count":5,"files":["project/TICKETS.md","project/ticket-027/README.md","project/ticket-027/intent.json","src/redup/core/lsh_matcher.py","tests/test_lsh_matcher.py"],"head_sha":"7bef05ca07a391b278f7baecf8724c0e4c268718","service_count":0,"services":[]},"assessment_mode":"observed-pr","axes":{"coupling":3,"delivery":2,"scope":2,"uncertainty":3,"validation":1},"complexity":"M","confidence":0.9,"diagnostics":["RADAR-ACCEPTANCE-MISSING","RADAR-BUDGET-EXCEEDED"],"estimate":{"budget_minutes":30,"minutes":50,"within_budget":false},"impact":{"components":["project","src/redup","tests"],"files":["project/TICKETS.md","project/ticket-027/README.md","project/ticket-027/intent.json","src/redup/core/lsh_matcher.py","tests/test_lsh_matcher.py"],"public_interfaces":[],"runtime_dependencies":0},"schema":"subactor.ticket-radar/v1","score":44,"split":{"parts":[{"estimated_minutes":14,"name":"Implement project","scope":["project"]},{"estimated_minutes":14,"name":"Implement src/redup","scope":["src/redup"]},{"estimated_minutes":14,"name":"Implement tests","scope":["tests"]},{"estimated_minutes":15,"name":"Validate and project to trackers","scope":["tests","planfile","github/gitlab/jira projections"]}],"reason":"estimated_minutes_exceed_budget","recommended":true},"standards":[{"id":"wellmanifest/dsl","revision":"6c60fc4e0dd1f1bb74f46a7745e28019908d1203","version":"0.1.0-dev"},{"id":"wellmanifest/ticket-lifecycle","revision":"5bf581907a87b46a13a73e6c033d3abe4d9a306f","version":"0.1.0-dev"},{"id":"wellmanifest/git-lifecycle","revision":"7d77d4b7af57e69bc75c3a0290b3a4805c5c4438","version":"0.2.0-dev"},{"id":"wellmanifest/logs","revision":"48c284ef7a069055c0bcb6b900147ce5e65f8b43","version":"0.3.0"}],"ticket_ref":"ticket-027"}<svg xmlns="http://www.w3.org/2000/svg" width="128" height="128" viewBox="0 0 128 128" role="img"><title>ticket-027: Preserve fallback duplicate matches while indexing MinHash candidates</title><rect width="128" height="128" rx="12" fill="#f8fafc"/><g stroke-width="1"><polygon points="64,55 72,61 69,71 59,71 56,61" fill="none" stroke="#d7dde5"/><polygon points="64,47 80,59 74,78 54,78 48,59" fill="none" stroke="#d7dde5"/><polygon points="64,38 89,56 79,85 49,85 39,56" fill="none" stroke="#d7dde5"/><polygon points="64,30 97,53 84,92 44,92 31,53" fill="none" stroke="#d7dde5"/><polygon points="64,21 105,51 89,99 39,99 23,51" fill="none" stroke="#d7dde5"/><line x1="64" y1="64" x2="64" y2="21" stroke="#aab4c0"/><line x1="64" y1="64" x2="105" y2="51" stroke="#aab4c0"/><line x1="64" y1="64" x2="89" y2="99" stroke="#aab4c0"/><line x1="64" y1="64" x2="39" y2="99" stroke="#aab4c0"/><line x1="64" y1="64" x2="23" y2="51" stroke="#aab4c0"/></g><polygon points="64,47 89,56 79,85 59,71 48,59" fill="#fb923c" fill-opacity="0.45" stroke="#c2410c" stroke-width="2"/><circle cx="64" cy="64" r="3" fill="#c2410c"/><g font-family="sans-serif" font-size="7" fill="#334155"><text x="64" y="11" text-anchor="middle">SCO</text><text x="114" y="48" text-anchor="middle">COU</text><text x="95" y="107" text-anchor="middle">UNC</text><text x="33" y="107" text-anchor="middle">VAL</text><text x="14" y="48" text-anchor="middle">DEL</text></g><text x="64" y="124" text-anchor="middle" font-family="sans-serif" font-size="8" fill="#0f172a">M · 50m</text></svg>DECISION D-027-4726
TICKET ticket-027
HEAD_SHA 7bef05ca07a391b278f7baecf8724c0e4c268718
CORRELATION_ID tellmesh-ticket-027-20261106
ACTOR agent:ifuri-validator-agent[bot]
APPLIED_RULE P-CORE-015
INPUT author_login = "tom-sapletta-com"
INPUT observed_checks = ["governance / enforce=PASS","test (3.10)=PASS","governance / remote lifecycle=PASS","test (3.13)=PASS"]
INPUT required_checks = ["test (3.10)","test (3.13)"]
INPUT required_checks_source = "protected registry (env/request)"
INPUT reviewer_login = "ifuri-validator-agent[bot]"
INPUT semantic_review_assessment = {"schema":"subactor.validator/semantic-review-assessment/v1","subject":{"repository":"semcod/redup","pull_request":36,"head_sha":"7bef05ca07a391b278f7baecf8724c0e4c268718","base_sha":"adba8c1a82468267e2d1902b8b68ff1fcf35378b","diff_sha256":"ae5cfb6ca53be7a40e1f47c51c760c17a617cd142c7a05b64bf47c48f77ce6c7"},"policy":{"policy_schema":"subactor.validator/semantic-review-policy/v1","policy_version":1,"policy_sha256":"676cb4516bbfed2a000e40b9b1b6e4a430ecc761ec546aeb53d721a1905cfdd7","required":false,"critical_paths":[],"observed_paths":["project/TICKETS.md","project/ticket-027/README.md","project/ticket-027/intent.json","src/redup/core/lsh_matcher.py","tests/test_lsh_matcher.py"]},"grounding":"full-diff-not-per-finding-proof","execution_authority":false,"status":"not_required","reason":null,"review_sha256":null,"unresolved":[]}
INPUT superseded_checks = []
INPUT ticket_radar_receipt = {"schema":"subactor.ticket-radar/v1","base_sha":"adba8c1a82468267e2d1902b8b68ff1fcf35378b","head_sha":"7bef05ca07a391b278f7baecf8724c0e4c268718","change_digest":"c81d723e4b48d36037700e28025535da037444a34296ca0967e4c0b9b0630947","score":44,"complexity":"M","estimated_minutes":50,"split_recommended":true,"services":[],"authority":"ADVISORY","promotion":"FORBIDDEN"}
VERDICT APPROVE AUTHORITY DETERMINISTIC
REJECTED REQUEST_CHANGES BECAUSE NO_UNSAFE_CHANGE_REASON_FOUND
ADVISORY llm_verdict = "APPROVE" MODEL "openai/cursor-auto"
ASSERT VERDICT_AUTHORITY != "ADVISORY"
Without datasketch, workspace scans scored every MinHash against every stored signature, making the fallback quadratic. Cache disjoint signature bands to select every candidate that can pass the existing threshold, then preserve the original Jaccard scores, equal-score order and grouping. Rebuild after additions or threshold changes; retain exhaustive comparison with native scoring and unsupported thresholds.
Validation: full locked test suites on Python 3.10 and 3.13 each passed (294 passed, 14 optional-backend skips); affected tests also passed in the existing native-extension environment (63 passed, 10 skips). Ruff and native governance pass. New regressions cover exhaustive parity at floating-point boundaries, additions, threshold changes, native-scorer fallback and group parity. The comparison-budget test fails before this change (1000 comparisons for one match) and passes afterward (1). A bounded sparse-signature benchmark preserved all ordered results while reducing 1,000,000 comparisons to 1,000 (15.35 s to 0.46 s); dense inputs can still require exhaustive scoring.
Ticket: ticket-027
Coordination: PLF-19979 / PLF-19981. Independent protected approval and merge are required.