Skip to content

fix(ci): make CodeRabbit review requester operational - #340

Merged
khaliqgant merged 5 commits into
mainfrom
fix/ai-review-request-permission
Aug 23, 2026
Merged

fix(ci): make CodeRabbit review requester operational#340
khaliqgant merged 5 commits into
mainfrom
fix/ai-review-request-permission

Conversation

@khaliqgant

@khaliqgant khaliqgant commented Aug 23, 2026

Copy link
Copy Markdown
Member

Summary

  • add the combined comment permissions that this repository's live GITHUB_TOKEN requires
  • narrow the automation to @coderabbitai review, the integration proven to accept a github-actions[bot] request
  • keep Codex as a mandatory human-authored exact-head request because Codex rejects the Actions bot identity

Live proof

Permission A/B against the same neutral POST /issues/340/comments:

  • run 32654217693, issues: write only: HTTP 403
  • run 32654246998, issues: write plus pull-requests: write: success

End-to-end bot-mention probe, run 32655006792:

  • the combined-scope workflow posted the exact-head request as github-actions[bot]
  • CodeRabbit accepted it and replied that it would review exact head 2211017...
  • Codex rejected that actor with “create a Codex account and connect to github”

The workflow therefore closes the real CodeRabbit push re-review gap without pretending it can replace the human-authored Codex gate.

Security boundary

The trusted-base pull_request_target workflow does not check out or execute contributor code, exposes no repository contents permission, and can only write issue/PR metadata needed for the review-request comment.

@khaliqgant

Copy link
Copy Markdown
Member Author

Exact-head review request for 4f4b0c2 (2026-08-23 17:11Z): @codex review

@coderabbitai review

One-line permission hotfix after live run 32653877712 proved issues:write alone receives HTTP 403 on the PR comment endpoint. Please verify the least-privilege boundary remains comment-only.

@coderabbitai

coderabbitai Bot commented Aug 23, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 78e7abb6-facc-4095-90d5-3e4be9e1e79e

📥 Commits

Reviewing files that changed from the base of the PR and between aa2a406 and 4f4b0c2.

📒 Files selected for processing (1)
  • .github/workflows/request-ai-reviews.yml

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The workflow job now has pull-requests: write permission in addition to issues: write.

Changes

Workflow Permissions

Layer / File(s) Summary
Configure pull-request write access
.github/workflows/request-ai-reviews.yml
The job permissions include pull-requests: write.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Merge Risk: ⚪ Minimal · up to 4f4b0

This narrowly grants the workflow permission needed to post pull request comments without changing its execution scope; no actionable merge-blocking risk remains beyond normal checks and review.

Poem

A rabbit checks the workflow gate,
Pull requests can now write their state.
Issues still hop along,
Permissions join the song.
One small line makes reviews great!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description check ✅ Passed The description directly explains the workflow permission change and its operational and security rationale.
Title check ✅ Passed The title clearly identifies the CI fix that makes CodeRabbit review requests operational.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/ai-review-request-permission

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai

coderabbitai Bot commented Aug 23, 2026

Copy link
Copy Markdown

@khaliqgant I will review PR #340 at the specified commit. I will verify that the workflow requires only comment-related write permissions and does not execute contributor code.

⚠️ Action not completed

Already reviewed the last commit. Use @coderabbitai full review to rerun a review of the entire changeset.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4f4b0c21da

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/request-ai-reviews.yml
@github-actions

Copy link
Copy Markdown
Contributor

Temporary GITHUB_TOKEN permission diagnostic for 2893e9a.

@khaliqgant

Copy link
Copy Markdown
Member Author

Exact-head review request for 22110176624c0a20e54dcb5149fb26c693412b35 (2026-08-23 17:16Z):

@coderabbitai review

@codex review

Empirical permission proof for the sole final-diff line:

  • temporary branch-only probe 93687ad used the same neutral POST /repos/AgentWorkforce/factory/issues/340/comments with only issues: write; run 32654217693 failed HTTP 403 and returned X-Accepted-Github-Permissions: issues=write; pull_requests=write
  • otherwise identical probe 2893e9a added pull-requests: write; run 32654246998 passed and posted the diagnostic comment
  • all diagnostic triggers/code were then removed; origin/main...2211017 is again exactly one added permission line

The current PR-target run still executes the workflow from base main, so it necessarily exercises the pre-fix single-scope version until this bootstrap patch lands.

@coderabbitai

coderabbitai Bot commented Aug 23, 2026

Copy link
Copy Markdown
🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

commit='22110176624c0a20e54dcb5149fb26c693412b35'

git cat-file -e "${commit}^{commit}"
printf '%s\n' '--- exact commit metadata ---'
git show -s --format='commit=%H%nparents=%P%nsubject=%s' "$commit"

printf '%s\n' '--- diff: origin/main...exact commit ---'
git diff --check "origin/main...$commit"
git diff --find-renames --find-copies --unified=80 "origin/main...$commit" -- .github/workflows/request-ai-reviews.yml

printf '%s\n' '--- changed-file list ---'
git diff --name-status "origin/main...$commit"

printf '%s\n' '--- workflow contents at exact commit ---'
git show "${commit}:.github/workflows/request-ai-reviews.yml" | cat -n

printf '%s\n' '--- security-relevant workflow constructs at exact commit ---'
git show "${commit}:.github/workflows/request-ai-reviews.yml" \
  | rg -n -i -C2 '^\s*permissions:|^\s*(issues|pull-requests|contents|actions|checks|deployments|id-token|packages|security-events|statuses):|checkout|pull_request_target|pull_request|workflow_run|actions/github-script|run:|uses:|github\.token|secrets\.|pulls/|issues/.*/comments' \
  || true

Length of output: 1670


🏁 Script executed:

#!/bin/bash
set -euo pipefail

repo='AgentWorkforce/factory'
pr='340'

printf '%s\n' '--- pull request metadata ---'
gh api "repos/$repo/pulls/$pr" \
  --jq '{number, state, head: {ref: .head.ref, sha: .head.sha, repo: .head.repo.full_name}, base: {ref: .base.ref, sha: .base.sha, repo: .base.repo.full_name}}'

printf '%s\n' '--- pull request changed files and patches ---'
gh api --paginate "repos/$repo/pulls/$pr/files?per_page=100" \
  --jq '.[] | {filename, status, additions, deletions, patch}'

printf '%s\n' '--- local checkout identity and available refs ---'
git status --short --branch
git rev-parse HEAD
git remote -v
git branch -a --contains HEAD || true
git show-ref | rg -n 'pull/340|main|HEAD' || true

workflow=$(fd -a '^request-ai-reviews\.yml$' .github 2>/dev/null | head -n 1 || true)
if [ -n "$workflow" ]; then
  printf '%s\n' '--- local workflow contents ---'
  cat -n "$workflow"
  printf '%s\n' '--- security-relevant workflow constructs ---'
  rg -n -i -C2 '^\s*permissions:|^\s*(issues|pull-requests|contents|actions|checks|deployments|id-token|packages|security-events|statuses):|checkout|pull_request_target|pull_request|workflow_run|actions/github-script|run:|uses:|github\.token|secrets\.|pulls/|issues/.*/comments' "$workflow" || true
fi

Length of output: 5197


@khaliqgant Reviewed PR #340 at 2211017e7b54ba596d3664f4e7c71e539ff73bad.

No findings.

The final diff changes only .github/workflows/request-ai-reviews.yml. It adds only pull-requests: write.

The workflow keeps top-level permissions: {}. The request job grants only issues: write and pull-requests: write.

The workflow does not check out repository content. It does not execute contributor code. Its only API operation is POST repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/comments. The stated bootstrap limitation is consistent with pull_request_target using the base-branch workflow before this change merges.

⚠️ Action not completed

No files to review.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: 2211017e7b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@github-actions

Copy link
Copy Markdown
Contributor

@coderabbitai review

@codex review

Requested for exact head 2211017e7b54ba596d3664f4e7c71e539ff73bad.

@chatgpt-codex-connector

Copy link
Copy Markdown

To use Codex here, create a Codex account and connect to github.

@coderabbitai

coderabbitai Bot commented Aug 23, 2026

Copy link
Copy Markdown

@github-actions[bot] I will review pull request #340 at exact head 2211017e7b54ba596d3664f4e7c71e539ff73bad.

⚠️ Action not completed

No files to review.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@khaliqgant khaliqgant changed the title fix(ci): grant PR comment permission to review requester fix(ci): make CodeRabbit review requester operational Aug 23, 2026
@khaliqgant

Copy link
Copy Markdown
Member Author

Exact-head review request for d0c63d0b06ac1a7c73c0bc5854a4897ea5807896 (2026-08-23 17:32Z):

@coderabbitai review

@codex review

The live bot-identity probe changed the contract: CodeRabbit accepts the Actions-authored request, while Codex rejects that actor. This head removes the unsupported/noisy Codex mention and accurately names/scopes the workflow while retaining the empirically required permission pair. Please review this exact head; all earlier review state is stale.

@coderabbitai

coderabbitai Bot commented Aug 23, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

repo='AgentWorkforce/factory'
pr='340'
commit='d0c63d0b06ac1a7c73c0bc5854a4897ea5807896'

printf '%s\n' '--- exact commit ---'
git cat-file -e "${commit}^{commit}"
git show -s --format='commit=%H%nparents=%P%nsubject=%s' "$commit"

printf '%s\n' '--- pull request head/base ---'
gh api "repos/$repo/pulls/$pr" \
  --jq '{number, state, head_sha: .head.sha, base_sha: .base.sha, changed_files, additions, deletions}'

printf '%s\n' '--- changed files from merge-base to exact commit ---'
base=$(git merge-base origin/main "$commit")
printf 'merge_base=%s\n' "$base"
git diff --check "$base...$commit"
git diff --name-status "$base...$commit"
git diff --find-renames --find-copies --unified=100 "$base...$commit" -- .github/workflows/request-ai-reviews.yml

printf '%s\n' '--- full workflow at exact commit ---'
git show "${commit}:.github/workflows/request-ai-reviews.yml" | cat -n

printf '%s\n' '--- security-relevant constructs at exact commit ---'
git show "${commit}:.github/workflows/request-ai-reviews.yml" \
  | rg -n -i -C3 '^\s*(name:|on:|permissions:|jobs:|[[:alnum:]_-]+:|issues:|pull-requests:|contents:|actions:|checks:|deployments:|id-token:|packages:|security-events:|statuses:|uses:|run:)|checkout|pull_request_target|pull_request|workflow_run|github-script|github\.token|secrets\.|curl|gh api|/issues/.*/comments|/pulls/' \
  || true

printf '%s\n' '--- PR changed-file API result ---'
gh api --paginate "repos/$repo/pulls/$pr/files?per_page=100" \
  --jq '.[] | {filename, status, additions, deletions, patch}'

Length of output: 8192


@khaliqgant Reviewed PR #340 at exact head d0c63d0b06ac1a7c73c0bc5854a4897ea5807896.

No findings.

The final diff changes only .github/workflows/request-ai-reviews.yml.

The workflow remains comment-only:

  • Top-level permissions remain empty.
  • The request job grants only issues: write and pull-requests: write.
  • The only token-authenticated operation is POST repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/comments.
  • The workflow does not check out repository content.
  • The workflow does not execute contributor code.

The workflow now requests only @coderabbitai review. The renamed workflow and concurrency group match that scope.


✏️ Learnings added
Learnt from: khaliqgant
URL: https://github.com/AgentWorkforce/factory/pull/340

Timestamp: 2026-08-23T17:31:41.519Z
Learning: In `.github/workflows/request-ai-reviews.yml`, the `pull_request_target` workflow posts an Actions-authored exact-head PR comment. CodeRabbit accepts the `coderabbitai review` request from this actor, while Codex rejects the actor. The workflow must request only CodeRabbit.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

You are interacting with an AI system.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: d0c63d0b06

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@khaliqgant

Copy link
Copy Markdown
Member Author

Independent corroboration, with the failure captured live — this is worth merging promptly.

I hit this from the other direction on #341 and diagnosed the same root cause before finding this PR. Run 32656894114, job request, failed in 4s:

HEAD_SHA: f8b7e13807f73be2e03a5409a298f4d6979e73ea
PR_NUMBER: 341
gh: Resource not accessible by integration (HTTP 403)
{"message":"Resource not accessible by integration",...,"status":"403"}
##[error]Process completed with exit code 1.

So the pull-requests: write line in this diff is confirmed as the fix by a real failing run, not only by reading the docs: a pull request is an issue, and this posts through /issues/{number}/comments, but the GITHUB_TOKEN scopes the two separately.

Why this is more urgent than a red check looks. Every PR in this repo is currently missing its automated AI review request, and the failure surfaces as a single red request check that reads like a flake. The result is PRs that appear reviewed — CodeRabbit renders a green check even when it reviewed nothing — while no review was ever requested. That is a silent review-coverage hole across the whole repo, and it is exactly the green-tick-over-nothing pattern this incident has already been bitten by.

I had a duplicate fix staged and dropped it in favour of this one.

One observation on the diff, take it or leave it: dropping @codex review narrows this to CodeRabbit only. CodeRabbit's free plan allows one review per hour per developer — I have watched it return Review rate limited while still rendering a green check — so losing the second reviewer removes the fallback precisely when the first is throttled. If dropping codex is deliberate, ignore this; if it was incidental to the rename, keeping both would leave more coverage.

@chatgpt-codex-connector

Copy link
Copy Markdown

To use Codex here, create an environment for this repo.

@khaliqgant

Copy link
Copy Markdown
Member Author

Merging. The red check on this PR's own head is not evidence against it — I confirmed it is the old workflow failing.

The failing run (32655081021) logs this body being built:

body=$(printf '@coderabbitai review\n\n@codex review\n\nRequested for exact head `%s`.\n<!-- factory-ai-review-request:%s -->' ...)

That is the pre-diff string. The trigger is pull_request_target, which by design always runs the workflow definition from the base branch, so this PR's version was never executed and cannot be until it is on main. The 403 in that log is the bug being fixed, observed in its natural habitat.

The fix itself is the standard remedy: Resource not accessible by integration on POST /issues/{n}/comments from pull_request_target with only issues: write is resolved by adding pull-requests: write. CI is 5/5 green at d0c63d0b, zero unresolved threads, mergeable. Blast radius is one workflow file that is currently failing 100% of the time, so it cannot regress.

One follow-up, not a blocker. The diff also drops @codex review from the requested body and renames the workflow to match. That is a real behaviour change — we stop auto-requesting codex — and it is not implied by "make the CodeRabbit requester operational". I am taking it because codex review is demonstrably still reaching PRs by other means today, and because a partly-working requester beats the current zero. But if dropping codex was incidental rather than intended, open a follow-up to put it back; it should be a deliberate call, not a side effect of a permissions fix.

Verification is immediate and free: the next PR event runs the new definition from main. If the next Request CodeRabbit review run is still 403, revert this rather than iterating on it.

@khaliqgant
khaliqgant merged commit 05552c9 into main Aug 23, 2026
8 of 9 checks passed
@chatgpt-codex-connector

Copy link
Copy Markdown

To use Codex here, create an environment for this repo.

@khaliqgant
khaliqgant deleted the fix/ai-review-request-permission branch August 23, 2026 18:15
@coderabbitai

coderabbitai Bot commented Aug 23, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

@khaliqgant Thanks for the verification details.

The failing run executed the base-branch workflow. Its body proves that it did not execute this PR’s workflow definition. The HTTP 403 is therefore evidence for the permission fix, not evidence against it.

The next PR event after merge is the correct verification. If Request CodeRabbit review still returns HTTP 403, revert the permission change as stated.

The removal of @codex review is a separate behavior change. Restoring it should be a deliberate follow-up decision.


✏️ Learnings added
Learnt from: khaliqgant
URL: https://github.com/AgentWorkforce/factory/pull/340

Timestamp: 2026-08-23T18:16:04.210Z
Learning: In `.github/workflows/request-ai-reviews.yml`, the `pull_request_target` trigger executes the workflow definition from the base branch. A check on a pull request can therefore run the pre-change workflow and cannot validate workflow changes in that same pull request before merge. Validate this workflow on the next pull request event after the change reaches `main`.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

You are interacting with an AI system.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant