From 43de66d1f098dc2f75f6fe902780cc487dcc6741 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=EC=98=A4=EC=A4=80=EC=84=9D=28Junseok=20Oh=29DevOps?= <52226147+Atom-oh@users.noreply.github.com> Date: Fri, 6 Nov 2026 03:59:29 +0000 Subject: [PATCH 1/2] ci: review up to 10 batches, five per wave Large promotions (dependency refresh plus a gated feature) need nine file-boundary batches. Raising max_batches to 10 and max_parallel_batches to 5 keeps every review within the existing two-wave, two-session workflow and its 190-minute timeout; each wave still runs one batch-length of model time. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/pr-review.yml | 4 ++-- CHANGELOG.md | 4 ++-- docs/runbooks/AGENTS.md | 4 ++-- docs/runbooks/CLAUDE.md | 2 +- docs/runbooks/branch-strategy.md | 2 +- docs/runbooks/dev-repo-setup.md | 2 +- docs/runbooks/pr-review-head-images.md | 16 ++++++++-------- scripts/AGENTS.md | 6 +++--- scripts/CLAUDE.md | 6 +++--- scripts/pr-review/review-limits.json | 4 ++-- 10 files changed, 25 insertions(+), 25 deletions(-) diff --git a/.github/workflows/pr-review.yml b/.github/workflows/pr-review.yml index a3e23e02..6857c7e7 100644 --- a/.github/workflows/pr-review.yml +++ b/.github/workflows/pr-review.yml @@ -49,8 +49,8 @@ jobs: # the runner pod for 6 hours. The longest panel cell is 2*(1200+10)=40m20s. # Chair primary/fallback total at most 2*(120+900+10)=34m20s, leaving 15m20s # for setup, publication, and cleanup. Mint a one-hour AWS session before each model phase. - # Batched runs execute at most max_parallel_batches (4) batches at once, so a review of up - # to 8 batches takes at most two waves per phase, each wave its own step and AWS session: + # Batched runs execute at most max_parallel_batches (5) batches at once, so a review of up + # to 10 batches takes at most two waves per phase, each wave its own step and AWS session: # 2*40m20s panel + 2*34m20s chair = 149m20s, plus 15m image staging, 15m20s setup/publish # and about 10 minutes of slack so the gate can still publish a diagnostic. timeout-minutes: 190 diff --git a/CHANGELOG.md b/CHANGELOG.md index 6d467c3d..ae7aca54 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -27,7 +27,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed - Bedrock models: chat answers with OpenAI GPT-6 Luna (without explicit Bedrock cache points, which GPT models reject); Explore "AI generate" writes queries with GPT-6.1 Terra and retries on Claude Sonnet 5.5 when Terra fails or its draft fails validation; primary, corrections and fallback share a 55-second request deadline with SDK cancellation; AI Diagnosis, routing, synthesis and the background workers move to Claude Sonnet 5.5, Opus 5.5 (AI Diagnosis Deep tier option) and Haiku 5.5. These models reject `temperature`, so requests no longer send it. Haiku calls and Sonnet calls with fixed output caps run without extended thinking so existing budgets still fit; Opus 5.5, which always thinks, gets extra output headroom for Deep report sections and invariant proposals. The web task now always holds a grant for its direct query-generation and assistant models (Terra, Sonnet 5.5, Haiku 5.5), so "AI generate" and the product-help assistant no longer fail with AccessDenied when hybrid routing is off; the chat runtime role additionally allows GPT-6 Luna. The Terraform variable `agentcore_chat_model_id = "global.anthropic.claude-sonnet-5-5"` rolls chat back to Sonnet 5.5 on every provisioning run, and the chat footer shows a model only when the answer carries its model provenance. -- PR review batching: a filtered diff larger than one review call's 6,000-line/128 KiB limit is reviewed in up to 8 complete batches split at file boundaries instead of being rejected outright. Workflow, review-script, Terraform/IAM, agent, AgentCore, secret-rotation, EKS, incident, API-route and AWS-client files, plus any path segment naming auth, secret, credential, assume-role, mutation or allowlist surfaces, are reviewed together in the first batch, and each batch receives the paths changed in the other batches as data, so a dependency no listed path provides stays blocking. Each batch passes the same input admission, gets both independent reviewers and its own chair in parallel, and the PR passes only if every batch passes; a later batch cannot clear an earlier batch's coverage or report failure. A single file over the per-batch budget, or a diff needing more than 8 batches, is still rejected before any model call, and diffs within the limit keep the single-review path. +- PR review batching: a filtered diff larger than one review call's 6,000-line/128 KiB limit is reviewed in up to 10 complete batches split at file boundaries instead of being rejected outright. Workflow, review-script, Terraform/IAM, agent, AgentCore, secret-rotation, EKS, incident, API-route and AWS-client files, plus any path segment naming auth, secret, credential, assume-role, mutation or allowlist surfaces, are reviewed together in the first batch, and each batch receives the paths changed in the other batches as data, so a dependency no listed path provides stays blocking. Each batch passes the same input admission, gets both independent reviewers and its own chair in parallel, and the PR passes only if every batch passes; a later batch cannot clear an earlier batch's coverage or report failure. A single file over the per-batch budget, or a diff needing more than 10 batches, is still rejected before any model call, and diffs within the limit keep the single-review path. ### Fixed @@ -702,7 +702,7 @@ First release of the **v2 line** (versioned independently from the v1 1.x line, ### Changed - Bedrock 모델: 채팅은 OpenAI GPT-6 Luna로 답합니다(GPT 모델이 거부하는 명시적 Bedrock 캐시 포인트는 끔). Explore "AI로 생성"은 GPT-6.1 Terra로 쿼리를 만들고, Terra 호출이 실패하거나 초안이 검증에서 거부되면 Claude Sonnet 5.5로 다시 시도합니다. 기본 모델·수정·폴백은 하나의 55초 요청 제한 시간과 SDK 취소 신호를 공유합니다. AI 진단, 라우팅, 합성, 백그라운드 워커는 Claude Sonnet 5.5, Opus 5.5(AI 진단 Deep 단계 선택), Haiku 5.5로 바뀝니다. 이 모델들은 `temperature`를 거부하므로 요청에서 더 이상 보내지 않습니다. Haiku 호출과 출력 상한이 고정된 Sonnet 호출은 기존 상한이 유지되도록 확장 사고 없이 실행하고, 항상 사고하는 Opus 5.5는 Deep 리포트 섹션과 불변식 제안에 출력 여유분을 더 받습니다. 웹 태스크는 직접 호출하는 쿼리 생성·도움말 모델(Terra, Sonnet 5.5, Haiku 5.5) 권한을 항상 가지므로, 하이브리드 라우팅이 꺼져 있어도 "AI로 생성"과 제품 도움말이 AccessDenied로 실패하지 않습니다. 채팅 런타임 역할은 GPT-6 Luna도 허용합니다. Terraform 변수 `agentcore_chat_model_id = "global.anthropic.claude-sonnet-5-5"`로 채팅을 Sonnet 5.5로 되돌리며 매 프로비저닝마다 유지되고, 채팅 하단의 모델 표시는 답변에 모델 출처가 있을 때만 나타납니다. -- PR 리뷰 분할 검토: 필터링된 diff가 리뷰 1회 한도(6,000줄/128 KiB)를 넘으면 바로 거부하지 않고, 파일 경계로 나눈 최대 8개의 완전한 배치로 검토합니다. 워크플로·리뷰 스크립트·Terraform/IAM·agent·AgentCore·시크릿 교체·EKS·인시던트·API 라우트·AWS 클라이언트 파일과, 경로 구간 이름이 인증·시크릿·자격 증명·역할 전환·변경·허용 목록을 가리키는 파일은 첫 배치에서 함께 검토하고, 각 배치는 다른 배치에서 바뀐 경로 목록을 데이터로 받아 어느 경로도 제공하지 않는 의존성은 차단합니다. 각 배치는 같은 입력 검사를 거쳐 두 독립 리뷰어와 자체 chair가 병렬로 검토하며, 모든 배치가 통과해야 PR이 통과합니다. 이후 배치가 앞선 배치의 검토 범위·보고서 실패를 지우지 못합니다. 파일 하나가 배치 한도를 넘거나 8개를 넘는 배치가 필요하면 모델 호출 전에 거부하고, 한도 안의 diff는 기존 단일 검토 경로를 그대로 씁니다. +- PR 리뷰 분할 검토: 필터링된 diff가 리뷰 1회 한도(6,000줄/128 KiB)를 넘으면 바로 거부하지 않고, 파일 경계로 나눈 최대 10개의 완전한 배치로 검토합니다. 워크플로·리뷰 스크립트·Terraform/IAM·agent·AgentCore·시크릿 교체·EKS·인시던트·API 라우트·AWS 클라이언트 파일과, 경로 구간 이름이 인증·시크릿·자격 증명·역할 전환·변경·허용 목록을 가리키는 파일은 첫 배치에서 함께 검토하고, 각 배치는 다른 배치에서 바뀐 경로 목록을 데이터로 받아 어느 경로도 제공하지 않는 의존성은 차단합니다. 각 배치는 같은 입력 검사를 거쳐 두 독립 리뷰어와 자체 chair가 병렬로 검토하며, 모든 배치가 통과해야 PR이 통과합니다. 이후 배치가 앞선 배치의 검토 범위·보고서 실패를 지우지 못합니다. 파일 하나가 배치 한도를 넘거나 10개를 넘는 배치가 필요하면 모델 호출 전에 거부하고, 한도 안의 diff는 기존 단일 검토 경로를 그대로 씁니다. ### 수정 diff --git a/docs/runbooks/AGENTS.md b/docs/runbooks/AGENTS.md index ac67c349..2173ba11 100644 --- a/docs/runbooks/AGENTS.md +++ b/docs/runbooks/AGENTS.md @@ -1,4 +1,4 @@ - + > You are an external reviewer for this repo — project context below, distilled from CLAUDE.md. This file is shared verbatim by the external review panel (not a per-AI copy). @@ -24,7 +24,7 @@ removed; flag any change that reintroduces a v1 procedure. `connector-target-sync.md` covers chat connector target registration (`connector_target_sync_enabled`), the ADR-005 owner-override exception: scope, one-stack-per-account caveat, enable/verify, reason codes and that disabling leaves already-written targets. `pr-review-head-images.md` defines two comprehensive reviewer reports, complete diff -admission (6,000 lines/128 KiB per call; larger diffs in at most 8 complete batches that must +admission (6,000 lines/128 KiB per call; larger diffs in at most 10 complete batches that must all pass), bounded chair input and distinct incomplete-review diagnostics, alongside staged HEAD image evidence and historical BASE context. diff --git a/docs/runbooks/CLAUDE.md b/docs/runbooks/CLAUDE.md index 9d2b4a00..d42cc338 100644 --- a/docs/runbooks/CLAUDE.md +++ b/docs/runbooks/CLAUDE.md @@ -47,7 +47,7 @@ erasing valid Read evidence. Source helper/tests and the role consumer catalog a - HEAD image review uses bounded static raster decoding from one shared format table, source/render lineage and a deterministic declaration gate. Two comprehensive vendor reports each attest L2–L5; - filtered diff admission is 6,000 lines/128 KiB per call (larger diffs: at most 8 complete + filtered diff admission is 6,000 lines/128 KiB per call (larger diffs: at most 10 complete file-boundary batches that must all pass) and chair stdin is at most 256 KiB. Missing panel coverage publishes bounded unadjudicated observations without a chair call. Response presence, image failure and unusable report output are separate. See `pr-review-head-images.md`; no tool or IAM permissions are added. diff --git a/docs/runbooks/branch-strategy.md b/docs/runbooks/branch-strategy.md index 1f020148..0f5341df 100644 --- a/docs/runbooks/branch-strategy.md +++ b/docs/runbooks/branch-strategy.md @@ -67,7 +67,7 @@ user's branch (or short-lived branches merged into it), then flows up via PR to The required AI review admits a complete filtered diff up to 6,000 lines/128 KiB per review call and up to 32 supported HEAD images under the existing byte/codec -bounds. A larger diff is reviewed in at most 8 complete file-boundary batches that +bounds. A larger diff is reviewed in at most 10 complete file-boundary batches that must all pass (`pr-review-head-images.md`). An accumulated dev-to-main promotion can still exceed that even if each feature PR was independently reviewed. It stays blocked; rerunning unchanged input cannot restore missing scope, and historical feature-review comments are not proof of diff --git a/docs/runbooks/dev-repo-setup.md b/docs/runbooks/dev-repo-setup.md index 11692011..b3701abf 100644 --- a/docs/runbooks/dev-repo-setup.md +++ b/docs/runbooks/dev-repo-setup.md @@ -325,7 +325,7 @@ target. See [GitHub's platform announcement](https://github.blog/changelog/2025- The workflow gives Codex and Claude one independent comprehensive review each, covering L2–L5, with at most two 1200-second attempts per vendor. Each report must attest all four checklists. Input admission precedes model credentials: complete -filtered diff up to 6,000 lines/128 KiB per call (larger diffs: at most 8 complete batches that +filtered diff up to 6,000 lines/128 KiB per call (larger diffs: at most 10 complete batches that must all pass) and complete hash-valid image evidence. Comprehensive reports allow 60,000 bytes each, 120,000 bytes combined, with a 256 KiB bound on actual chair stdin. Incomplete panel coverage skips chair calls diff --git a/docs/runbooks/pr-review-head-images.md b/docs/runbooks/pr-review-head-images.md index d912d931..20831047 100644 --- a/docs/runbooks/pr-review-head-images.md +++ b/docs/runbooks/pr-review-head-images.md @@ -116,7 +116,7 @@ limits, prompt propagation and missing-evidence failures. | One source or rendered file | 8 MiB | | Successfully staged source bytes / rendered bytes | 32 MiB each | | Complete filtered diff (raw and scrubbed) | 6,000 lines / 128 KiB per review call | -| Batched review of a larger diff | At most 8 file-boundary batches, each within 7/8 of the byte limit, at most 4 in flight per wave (two waves, fresh credentials each); every batch must pass | +| Batched review of a larger diff | At most 10 file-boundary batches, each within 7/8 of the byte limit, at most 5 in flight per wave (two waves, fresh credentials each); every batch must pass | | Published comment body | `comment_bytes` 60,000 (GitHub caps comments at 65,536 characters) | | Comprehensive reviewer report (scrubbed) | 60,000 bytes per vendor | | Panel bundle / envelope reserve | 120,000 bytes / 1,024 bytes | @@ -217,16 +217,16 @@ index or partial PASS. A filtered diff over one call's limit is reviewed in complete batches (`scripts/pr-review/review_batches.py`). Admission splits it at file boundaries into at most -8 batches, each within 7/8 of the byte limit less a 12 KiB paths header and within the line +10 batches, each within 7/8 of the byte limit less a 12 KiB paths header and within the line limit less the header's 400 lines, and admits every batch with the same `input_scope.py` checks. Each batch gets both vendor reviews and its own -chair inside the existing panel and chair steps. At most `max_parallel_batches` (4) batches run at -once, i.e. up to eight reviewer CLIs or four chairs, sized for the `sample-awsops` runner pod -(2 CPU, 8 GiB request, 12 GiB limit). Batches 5–8 form a second wave: each wave of each phase is +chair inside the existing panel and chair steps. At most `max_parallel_batches` (5) batches run at +once, i.e. up to ten reviewer CLIs or five chairs, sized for the `sample-awsops` runner pod +(2 CPU, 8 GiB request, 12 GiB limit). Batches 6–10 form a second wave: each wave of each phase is its own workflow step (`scripts/pr-review/run-batches.sh`) after a fresh one-hour AWS session, which is the `sample-awsops-ci-review` role's maximum, so no wave outlives its credentials. The job timeout (190 minutes) covers two worst-case waves of each phase plus staging and publication. -Admission refuses a filtered diff larger than 8 batch budgets plus one call limit (928 KiB) +Admission refuses a filtered diff larger than 10 batch budgets plus one call limit (1,128 KiB) before reading it. The per-batch scope note added to the trusted prompt contains only fixed text (batch number, rules), never PR file names. Batch 1 holds every file under `.github/`, `scripts/pr-review/`, `terraform/` (including the edge Lambda), `agent/`, `scripts/v2/agentcore/`, @@ -265,7 +265,7 @@ stays retrievable; each wave logs its batches before its exit status is checked. directories, the batch index and the gate's scratch file are removed at the end of every job, and admission writes batch copies owner-only. The combined comment has one section per batch, and the PR passes only if -every batch passes. One file over the per-batch budget, or a diff needing more than 8 batches, is +every batch passes. One file over the per-batch budget, or a diff needing more than 10 batches, is rejected before any model call. A diff within one call's limit keeps the single-review path. Input failures publish a distinct incomplete-input diagnosis; no panel or chair is called. Failed panel coverage skips the chair but publishes fixed diagnostics and unadjudicated severity-keyword presence booleans; it never asserts @@ -279,7 +279,7 @@ attestations remain incomplete coverage, separately from unreadable report bytes These changes reduce normal review calls from eight panel calls plus a chair to two panel calls plus a chair; existing retry/model/timeout choices remain. They do not promise a wall-clock speedup. A diff over one call's bounds is reviewed in the complete -batches described above; a promotion that needs more than 8 batches, or whose sensitive +batches described above; a promotion that needs more than 10 batches, or whose sensitive files exceed batch 1, is still rejected, and exact-commit coverage reuse is not implemented. Splitting feature PRs alone does not reduce an existing dev-to-main cumulative diff. Do not diff --git a/scripts/AGENTS.md b/scripts/AGENTS.md index 48363564..f5c8511f 100644 --- a/scripts/AGENTS.md +++ b/scripts/AGENTS.md @@ -1,4 +1,4 @@ - + > You are an external reviewer for this repo — project context below, distilled from CLAUDE.md. This file is shared verbatim by Kiro, Codex, and Agy (not a per-AI copy). @@ -33,11 +33,11 @@ that IO distinction, not model latency. and separates checklist/image/report diagnoses without implying code safety. Input admission requires complete filtered diffs within 6,000 lines/128 KiB and complete hash-valid image evidence before model credentials. No truncated reviews. -`review_batches.py`: larger diffs in ≤8 file batches, ≤4 per wave (`run-batches.sh`); all pass. +`review_batches.py`: larger diffs in ≤10 file batches, ≤5 per wave (`run-batches.sh`); all pass. Two independent Codex/Claude reports each cover all four checklists and must declare substantive sections for L2–L5, including security L3, plus plain `LENS_COVERAGE: L2,L3,L4,L5`; missing panel coverage skips chair calls. -Report truncation cannot approve unseen findings. Promotions beyond 8 batches (or whose +Report truncation cannot approve unseen findings. Promotions beyond 10 batches (or whose sensitive files exceed batch 1) are still rejected; no coverage reuse. Bounded full-report validation requires a plain `IMAGE_COVERAGE: COMPLETE` from both comprehensive vendor reports and chair for staged images. Explicit failure overrides PASS even without diff --git a/scripts/CLAUDE.md b/scripts/CLAUDE.md index d3031e1c..5e6d7954 100644 --- a/scripts/CLAUDE.md +++ b/scripts/CLAUDE.md @@ -296,13 +296,13 @@ secrets-manager) — installed by `make deps`. named missing vendors and separate lens/image/report diagnostics, never approval. `input_scope.py` admits only complete filtered diffs within 6,000 lines/128 KiB and complete hash-valid image evidence before model credentials; no partial review. - `review_batches.py` splits a larger diff at file boundaries into at most 8 batches, each + `review_batches.py` splits a larger diff at file boundaries into at most 10 batches, each admitted the same way and reviewed by both vendors plus its own chair; every batch must pass. - `run-batches.sh` runs each wave of at most 4 batches in its own step after fresh credentials. + `run-batches.sh` runs each wave of at most 5 batches in its own step after fresh credentials. Both panel reports require substantive sections for every L2–L5 checklist, including security L3, plus plain `LENS_COVERAGE: L2,L3,L4,L5`. Missing panel coverage skips chair calls; report truncation cannot approve unseen findings. Promotions beyond - 8 batches (or whose sensitive files exceed batch 1) are still rejected; no coverage reuse. + 10 batches (or whose sensitive files exceed batch 1) are still rejected; no coverage reuse. `review_context.py` pins the trusted CI checkout and reviewed PR/base metadata. `image-formats.json` selects detected extensions and the approved codec for both staging and `render_head_image.py`; unknown-codec entries remain explicit coverage failures. diff --git a/scripts/pr-review/review-limits.json b/scripts/pr-review/review-limits.json index ec5cf564..cc6851fb 100644 --- a/scripts/pr-review/review-limits.json +++ b/scripts/pr-review/review-limits.json @@ -5,7 +5,7 @@ "panel_bytes": 120000, "stdin_envelope_bytes": 1024, "chair_stdin_bytes": 262144, - "max_batches": 8, + "max_batches": 10, "comment_bytes": 60000, - "max_parallel_batches": 4 + "max_parallel_batches": 5 } From 30779492094ed4f06e234ddb6417cf100d35104d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=EC=98=A4=EC=A4=80=EC=84=9D=28Junseok=20Oh=29DevOps?= <52226147+Atom-oh@users.noreply.github.com> Date: Fri, 6 Nov 2026 04:09:21 +0000 Subject: [PATCH 2/2] docs: fix the remaining 8-batch promotion limit in branch strategy Co-Authored-By: Claude Opus 5.5 --- docs/runbooks/branch-strategy.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/runbooks/branch-strategy.md b/docs/runbooks/branch-strategy.md index 0f5341df..81e24b6b 100644 --- a/docs/runbooks/branch-strategy.md +++ b/docs/runbooks/branch-strategy.md @@ -81,7 +81,7 @@ main first. Do not assume this activation path for another repository whose defa branch is main. Its pre-provisioned, explicitly approved SHA-pinned recovery path must authorize the workflow change under the existing protections. -Complete bounded batching exists: a promotion diff up to 8 file-boundary batches, +Complete bounded batching exists: a promotion diff up to 10 file-boundary batches, whose batch-1 file set fits one batch, is reviewed in full by the same protected workflow on its current HEAD. A promotion beyond that still needs authenticated exact-commit coverage reuse, which is not implemented, through an ordinary PR to dev