Skip to content

Feat: Add PR summary in review + Strict JSON schema + Persist previous review when possible#23

Merged
HungKNguyen merged 4 commits intomainfrom
austin/add-pr-summary
Feb 13, 2026
Merged

Feat: Add PR summary in review + Strict JSON schema + Persist previous review when possible#23
HungKNguyen merged 4 commits intomainfrom
austin/add-pr-summary

Conversation

@HungKNguyen
Copy link

Introduce PR Summary Table

Currently in the review comment, the bot will give automated one setence statement like "No issue found" or "Found issue ...". Instead we take inspiration from other review bot by asking Claude Code to also produce a pr_summary which describe what the PR is about as well as a table for detailing the changes per file.

We also remove analysis_summary from the Claude Code JSON since that can be derived from the rest of the JSON output (no need to waste token).

Increase reliability via Claude Code JSON Schema

This PR also enhances the quality of the Claude Code output by using the new --json-schema option introduced in 2.0.25 (Nov 2025). We still keep the old repair JSON logic as a safeguard

Change how the bot handle previous reviews

The current implementation use a flag to determine if we should dismiss the previous staled review. This PR removed the flag to opt for an automated behavior. Upon new reivews:

  • If the review decision remains the same, then we only update the review body
  • If the review decision is changed, we will dismiss the old review and write a new one

Misc

  • A bit of clean up regarding the parsing of JSON output in comment-pr-finding.js. Some of the fields like finding.extraMetadata and finding.extra is no longer being produced by Claude Code.
  • Fix up the eval code to work with the new summary

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📋 PR Summary:
This PR adds a PR summary feature to the code review output, implements strict JSON schema enforcement for Claude's responses, and refactors the review persistence logic to update existing reviews in place when possible rather than always dismissing stale reviews. These changes improve review output quality and reduce notification noise for PR authors.

13 files reviewed
File Changes
claudecode/review_schema.py New file defining JSON schema for review output validation
claudecode/github_action_audit.py Adds JSON schema enforcement, pr_summary extraction, updated validation
claudecode/prompts.py Adds pr_summary field and guidelines to review prompt
scripts/comment-pr-findings.js New review update logic, PR summary formatting, simplified finding format
action.yml Removes dismiss-stale-reviews input, adds pr-summary.json output
claudecode/evals/* Increased timeouts, added pr_summary/analysis_summary to eval results
claudecode/test_*.py Updated tests for new pr_summary field and schema changes
scripts/comment-pr-findings.bun.test.js Updated tests for new review update logic and PR summary formatting
.gitignore Added .env to gitignore

No issues found. Changes look good.

@HungKNguyen HungKNguyen merged commit bea9f7b into main Feb 13, 2026
2 checks passed
@HungKNguyen HungKNguyen deleted the austin/add-pr-summary branch February 13, 2026 02:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant