Skip to content

Improve changelog automation prompt#9070

Closed
amyblais wants to merge 5 commits into
mattermost:masterfrom
amyblais:amyblais/update-generate-changelog
Closed

Improve changelog automation prompt#9070
amyblais wants to merge 5 commits into
mattermost:masterfrom
amyblais:amyblais/update-generate-changelog

Conversation

@amyblais

Copy link
Copy Markdown
Member

Summary

  • Expanded system prompt with new instructions: blank-line rules between headings and content, DM/GM abbreviation expansion (DMDirect Message, GMGroup Message), one-section-per-note rule, and a new instruction Test image #10 ("Focus")
  • Added normalize_go_version() and extract_previous_go_version() helpers so the Go Version section always shows a concrete version number (looked up from the existing changelog when GO_VERSION is not provided)
  • Feature release anchor/heading now uses the -feature-release suffix to match the ESR pattern, and includes the full release-policy URL
  • CHANGELOG_PATH env var is now read in main() before building the Go section
  • go_section output changed from a bullet point to a bare sentence matching Mattermost changelog convention

Test plan

  • Run the script against a test milestone and verify the generated changelog entry matches the expected format
  • Verify Go version normalization handles both go1.22.5 and v1.22.5 input formats
  • Verify feature release heading includes the -feature-release anchor suffix

🤖 Generated with Claude Code

Copilot AI review requested due to automatic review settings June 25, 2026 12:10
@coderabbitai

coderabbitai Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Updates the changelog generation script's SYSTEM_PROMPT with stricter formatting/terminology rules, adds Go version normalization and extraction helpers, introduces a CHANGELOG_PATH environment variable override, and changes feature-release heading/anchor and Go Version section output logic.

Changes

Changelog Generator Updates

Layer / File(s) Summary
SYSTEM_PROMPT formatting and content rules
.github/scripts/generate_changelog.py
Tightens config.json section blank-line rules, adds "user interface" terminology enforcement, and extends Markdown/content constraints (bullet indentation, section completeness, proofreading, tone neutrality).
Go version normalization and extraction
.github/scripts/generate_changelog.py
Adds `normalize_go_version` to convert version strings into a consistent `v...` format and `extract_previous_go_version` to parse the most recent Go version from an existing changelog.
Changelog path override and release output wiring
.github/scripts/generate_changelog.py
Adds `CHANGELOG_PATH` environment variable support (defaulting to `CHANGELOG.md`), updates feature-release headings to use a feature-specific anchor with a "Feature Release" label, and changes the Go Version section to always output a concrete normalized version.

Estimated code review effort: 2 (Simple) | ~15 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Main as main()
  participant Env as Environment
  participant Changelog as CHANGELOG.md
  participant GoVer as Go Version Helpers

  Main->>Env: read CHANGELOG_PATH (default CHANGELOG.md)
  Main->>GoVer: resolve Go version (GO_VERSION or extract from changelog)
  GoVer->>Changelog: read previous changelog if GO_VERSION not set
  GoVer-->>Main: normalized Go version (v-prefixed)
  Main->>Changelog: write updated changelog entry with feature-release heading and Go Version section
Loading

Related Issues: None found in the provided context.

Related PRs: None found in the provided context.

Suggested labels: documentation, tooling

Suggested reviewers: None specified.

Poem:
A rabbit hopped through prompt and code,
Tightened rules where blank lines flowed,
Go versions normalized with care,
"v" prefixed and everywhere,
Changelog paths now free to roam. 🐇📜

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and matches the main changelog automation and prompt updates in this PR.
Description check ✅ Passed The description clearly summarizes the prompt, Go version, and changelog formatting changes.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@amyblais amyblais changed the title Update generate_changelog.py with improved system prompt and Go version handling Improve changelog automation prompt Jun 25, 2026
@amyblais

Copy link
Copy Markdown
Member Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Full review finished.

@amyblais amyblais added the Work In Progress Not yet ready for review label Jun 25, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the changelog generation script used by the docs repo workflow to (a) refine the AI system prompt used for release-note polishing and (b) improve Go version handling so the generated ### Go Version section includes a concrete version even when GO_VERSION isn’t provided.

Changes:

  • Expanded the Claude system prompt with stricter formatting/terminology rules (including DM/GM expansion and one-section-per-note).
  • Added helpers to normalize Go versions and to extract the previously documented Go version from the existing changelog when GO_VERSION is empty.
  • Updated feature-release anchors/headings to use the -feature-release suffix and adjusted how the Go Version section is rendered.

Comment thread .github/scripts/generate_changelog.py Outdated
Comment thread .github/scripts/generate_changelog.py Outdated
Comment thread .github/scripts/generate_changelog.py
Comment thread .github/scripts/generate_changelog.py Outdated
Comment thread .github/scripts/generate_changelog.py Outdated
@amyblais amyblais requested a review from NARSimoes July 7, 2026 10:37
@amyblais amyblais added 1: Dev Review Requires review by a core commiter and removed Work In Progress Not yet ready for review labels Jul 7, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
.github/scripts/generate_changelog.py (1)

282-296: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Regex is tightly coupled to the exact string this script emits; likely won't match the live changelog's actual historical phrasing.

Since CHANGELOG_PATH is wired to the published mattermost-v11-changelog.md (per the workflow), extract_previous_go_version scans real content. The actual published changelog historically phrases Go version updates as plain sentences, not "is built with Go \`vX``"— e.g. "Upgraded Go to 1.25.8." and "Updated Go to v1.25.10." Until the new### Go Versionphrasing from this script has been written into the file at least once, this regex will find no match and silently fall back to"(see previous release)"` with no warning printed.

Consider logging when the fallback path is hit so a missing/unexpected version isn't silently baked into a published release note.

💡 Suggested fallback logging
     if go_version:
         formatted_go = normalize_go_version(go_version)
     else:
-        formatted_go = extract_previous_go_version(changelog_path) or "(see previous release)"
+        formatted_go = extract_previous_go_version(changelog_path)
+        if not formatted_go:
+            print("⚠️  Could not determine Go version from GO_VERSION or previous changelog")
+            formatted_go = "(see previous release)"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/scripts/generate_changelog.py around lines 282 - 296, The fallback
in extract_previous_go_version is silent when the regex does not match the
existing changelog content, which can leave release notes with an unintended
“see previous release” version. Update generate_changelog.py so
extract_previous_go_version (or its caller in the changelog generation flow)
logs a clear warning when no Go version is found, using the existing
logger/print path used by the script, and keep the previous-version lookup
behavior unchanged otherwise.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In @.github/scripts/generate_changelog.py:
- Around line 282-296: The fallback in extract_previous_go_version is silent
when the regex does not match the existing changelog content, which can leave
release notes with an unintended “see previous release” version. Update
generate_changelog.py so extract_previous_go_version (or its caller in the
changelog generation flow) logs a clear warning when no Go version is found,
using the existing logger/print path used by the script, and keep the
previous-version lookup behavior unchanged otherwise.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: e4ca094c-923c-4b5f-95f0-4fc0c668a42c

📥 Commits

Reviewing files that changed from the base of the PR and between 65a2eb1 and 296f224.

📒 Files selected for processing (1)
  • .github/scripts/generate_changelog.py

@amyblais amyblais closed this Jul 10, 2026
@amyblais amyblais deleted the amyblais/update-generate-changelog branch July 10, 2026 11:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1: Dev Review Requires review by a core commiter

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants