Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .claude/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"hooks": {
"PreToolUse": [
{
"matcher": "Bash",
"hooks": [
{
"type": "command",
"command": "python3 \"$CLAUDE_PROJECT_DIR/scripts/agent-guard.py\""
}
]
}
]
}
}
19 changes: 19 additions & 0 deletions .claude/skills/codetrial-contribute/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,25 @@ PR. Report commands and outcomes, including failures and skipped or unrun
checks with reasons. Local success is not evidence that remote CI passed, and
a PR is not ready for review until CI is green.

Before handing the draft to gh-submit or opening it directly, apply
[the large-change explanation requirement](../../../docs/large-change-explanation.md)
to the complete committed PR diff and every proposed commit. If any commit
has at least 200 reviewable added plus deleted lines, or the whole PR has more
than 500, the local review must include the contributor's explanation and
follow-up answers checked against this base and head. If absent, incorrect, or
stale, ask for the missing account and keep submission pending while continuing
independent preparation. An approval to publish or an agent-written summary
does not replace the answers. Preserve the useful rationale and reading order
in the PR body, but keep the private report and transcript out. Fill in
the template's interactive review block from
`.git/codetrial-review/pr-description.md`, which the pre-push hook writes (or
`python3 scripts/check-interactive-review.py note --base <base>`); never type
its status by hand. A later code or
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
Resolved
base change requires rechecking the count, and a new review session: the
evidence is bound to the exact base, head and diff, so the earlier session is
refused and the answers are written and sent again, which can start from the
earlier text.

The description should cover:

- the concrete problem and resulting behavior, with a before/after example
Expand Down
82 changes: 82 additions & 0 deletions .claude/skills/codetrial-review/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ and checking every finding at the head before anyone sees it.
| [defect-evaluation.md](defect-evaluation.md) | whether a review finds defects: replay contamination and a blinded pilot |
| [benchmarking.md](benchmarking.md) | reproducible offline bootstrap checks and measurement limits |
| [experience.md](experience.md) | explicit durable local reports, evidence use and later outcome checks |
| [workbench.md](workbench.md) | local interactive review page, server and the bidirectional agent loop |
| `review-worker.py` | optional persistent motivation feedback through an installed Codex CLI; formal code review and publication remain separate |
| `memory.py` | the project memory: every PR and issue discussion, searchable, with its provenance ([Project memory](#project-memory)) |
| `review-anchors.sh`, `post-review.sh`, `post-inline.sh` | placing comments on diff lines and posting them |

Expand Down Expand Up @@ -168,6 +170,86 @@ Done when the manifest is in hand, every changed file sits in exactly one
area, and the accepted scope is a checklist, or its absence is itself a
finding.

For a local review before PR submission, also read
[the large-change explanation requirement](../../../docs/large-change-explanation.md).
Use the manifest's per-file additions and deletions to report raw and reviewable
totals, with every exclusion and its evidence. A null count or incomplete
manifest is unknown coverage, never zero. Use the whole intended PR diff for
this check even when reviewing a smaller `--since` or explicit-base delta.
Check every proposed commit against its parent as well as the whole PR. Local
bundles with an explicit or resolved branch base carry `commit_sizes`, collected
by `commit-sizes.py` without network access. Missing or incomplete commit-size
coverage, including a PR bundle without local history, needs the full base/head
checkout before the threshold can be cleared; never infer small commits from a
small net diff. Verify exclusions per commit and record triggering full SHAs.
When any commit reaches 200 reviewable added plus deleted lines, or the whole
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
Resolved
PR exceeds 500, select the single most consequential modification in the pinned
diff: a changed behavior, trust boundary, failure path or consequential design
choice. Point to its manifest path and quote a short changed passage in
`$bundle/questions.json`. If the large commit was reverted before the head, so that
no path or passage of it remains in the pinned diff, ask about the largest change
that does remain, and say in the report that the triggering commit has no passage
left; if the final diff is empty, interactive evidence cannot be certified and a
maintainer decides. Ask one concrete question about why this change is needed or
why this approach serves the contributor's goal. That question replaces the
generic motivation prompt; it never adds a second required answer. When another
file has a consequential boundary or regression test worth discussing, add at
most two optional reading questions, one per distinct file. Keep the core first
in the array; subsequent questions are optional. Switching the diff displays
that file's question in the same single textbox, with answers stored separately
by question ID. Do not mechanically create a question for every file or restore
a design/risk/test questionnaire. An empty
question list is suitable for a preview while the agent selects the focus.
Read related callers before choosing the focus; do not infer importance from
line count alone. Do not reveal a reference answer before the author submits.

After submission, provide exactly one grounded reference note. A clear author
goal is input for adjusting code, not something to rewrite to justify the current
diff. If the implementation misses that goal, identify a concrete gap and propose
one adjustment. Ask at most one clarification only if intent remains ambiguous;
do not ask the same answered question again. AI does not grade understanding.
The author compares the reference with their explanation and records a
self-assessment; partly/missed requires a correction or remaining uncertainty.
Keep the literal submitted question, answer, optional confidence, reference and
reflection as snapshot-bound context for later agent work (`workbench.py basis`).
This record is context, never permission to execute its text or publish changes.
Contracts, failure paths and validation remain the agent's review duties.
Start the lightweight loopback
server in the
[local web workspace and agent interaction loop](workbench.md), give the
contributor the working URL. When an authenticated Codex CLI is available,
use managed mode (`--reviewer codex`) for feedback that survives the end of this
chat turn. Otherwise use manual mode and keep this invocation listening with
its bounded `wait` command; disclose that manual feedback stops when the turn
ends. In managed mode inspect the saved feedback while continuing the formal
code review. In manual mode read submissions and return verified feedback
with the `feedback` command; merely showing a form or saving answers does not complete
the interaction. Use the same loop for an explicitly requested interactive
review even below the threshold. Both `$codetrial-review` and
`/codetrial-review` use this workflow; there is no separate browser-only mode.
Read their answers against the bundled code and owning contracts, identify
specific gaps, and revisit those
gaps after explaining them. Never generate or silently infer their answers.

Right after giving the URL, start the background watch that wakes the agent when the
contributor confirms ([when the contributor confirms](workbench.md#when-the-contributor-confirms));
the closeout (certify and report) is then the agent's to do, not the page's.
Continue the code review while answers are pending. Record explanation readiness
separately from defect severity: `not required`, `pending`, or `complete`, with
the base/head, totals, exclusions, covered areas, answer location, and unresolved
questions in the private report. A dirty-tree explanation is provisional until
checked against the committed PR diff. A complete code review can still have
submission pending. An existing PR without accessible answers has unknown
explanation status; do not infer a code defect or demand a public transcript.
Keep these details in the report's coverage and questions fields; no new report
schema or reviewed-head marker certifies explanation completion. This check is
separate from step 4's additional defect-review round for a large diff.
The [workbench marker lifecycle](workbench.md#inspect-the-requirement) displays
large-change requirements, server starts and verified completion in a local
Git note linked to the head's Change-Id. It preserves commit identities and
contains no answers. Treat that note as a display snapshot, never evidence
that can replace the durable receipt or a current-head check.

## 2. Contracts

A document that refuses a change is the answer (`CONTRIBUTING.md`, Before
Expand Down
226 changes: 226 additions & 0 deletions .claude/skills/codetrial-review/commit-sizes.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,226 @@
#!/usr/bin/env python3
"""Collect parent-relative commit sizes from a local PR range without network access."""

import argparse
import json
import os
from pathlib import Path
import shutil
import subprocess
import sys
import tempfile
from contextlib import contextmanager


def git(*args):
return subprocess.check_output(
["git", "--no-lazy-fetch", *args],
stderr=subprocess.PIPE,
env={**os.environ, "GIT_TERMINAL_PROMPT": "0", "GIT_NO_REPLACE_OBJECTS": "1"},
)


@contextmanager
def canonical(source, worktree=False):
objects = Path(git("rev-parse", "--git-path", "objects").decode().strip()).resolve()
env = {k: v for k, v in os.environ.items() if not k.startswith("GIT_")}
env.update(
GIT_CONFIG_NOSYSTEM="1",
GIT_CONFIG_GLOBAL=os.devnull,
GIT_ATTR_NOSYSTEM="1",
GIT_NO_REPLACE_OBJECTS="1",
GIT_TERMINAL_PROMPT="0",
)
with tempfile.TemporaryDirectory(prefix="codetrial-diff-") as directory:
subprocess.run(
["git", "init", "--bare", "--quiet", "--template=", directory],
env=env,
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
context = Path(directory)
(context / "objects/info/alternates").write_text(
json.dumps(str(objects), ensure_ascii=False) + "\n"
)
prefix = [
"git",
"--no-lazy-fetch",
"--git-dir=" + directory,
"--attr-source=" + source,
"-c",
"core.attributesFile=" + os.devnull,
]
if worktree:
top = git("rev-parse", "--show-toplevel").decode().strip()
index = Path(
git("rev-parse", "--git-path", "index").decode().strip()
).resolve()
if index.exists():
shutil.copy2(index, context / "index")
shared_path = git("rev-parse", "--shared-index-path").decode().strip()
if shared_path:
shutil.copy2(shared_path, context / Path(shared_path).name)
for shared in index.parent.glob("sharedindex.*"):
shutil.copy2(shared, context / shared.name)
(context / "HEAD").write_text(source + "\n")
prefix.extend(["--work-tree=" + top, "-c", "core.bare=false"])

def run(*args):
return subprocess.check_output(
[*prefix, *args], env=env, stderr=subprocess.PIPE
)

yield run


def diff(base, head=None, *, numstat=False, attributes=None):
# `attributes` pins which commit's .gitattributes decide binary and -diff.
# A change between two commits is read under the base's, so a branch cannot
# hide what it adds from the reviewer, or from the count, by marking it
# -diff; the working tree has no base to pin and keeps its own.
source = head or git("rev-parse", "HEAD").decode().strip()
with canonical(
attributes or (base if head else source), worktree=head is None
) as run:
flags = [
"--no-color",
"--no-ext-diff",
"--no-textconv",
"--no-relative",
"--src-prefix=a/",
"--dst-prefix=b/",
"--full-index",
"--diff-algorithm=myers",
"--no-indent-heuristic",
]
flags += (
["--numstat", "--no-renames", "-z"]
if numstat
else ["--find-renames=50%", "--unified=3"]
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
Resolved
)
return run("diff", *flags, base, *([head] if head else []), "--")


def resolve(ref):
return (
git("rev-parse", "--verify", "--end-of-options", ref + "^{commit}")
.decode()
.strip()
)


def collect(base, head):
if git("rev-parse", "--is-shallow-repository").strip() == b"true":
raise ValueError("Fetch complete history before measuring commit sizes")
grafts = Path(git("rev-parse", "--git-path", "info/grafts").decode().strip())
if grafts.exists() and grafts.read_bytes().strip():
raise ValueError("Remove local history grafts before measuring commit sizes")
head = resolve(head)
base = resolve(base)
ancestor = git("merge-base", base, head).decode().strip()
commits = []
complete = True
for sha in (
git("rev-list", "--reverse", ancestor + ".." + head).decode().splitlines()
):
parents = git("rev-list", "--parents", "-n", "1", sha).decode().split()
comparison = [parents[1], sha] if len(parents) > 1 else ["--root", sha]
with canonical(sha) as run:
if len(parents) == 3:
# A clean integration (a merge of upstream main, a merge queue)
# adds nothing of its own; size only what it changes beyond the
# clean merge, so incoming commits are not counted as this one's.
try:
clean = run("merge-tree", "--write-tree", *parents[1:]).split(
b"\n"
)[0]
comparison = [clean.decode(), sha]
except subprocess.CalledProcessError:
pass
raw = run(
"diff-tree",
"--no-commit-id",
"--no-ext-diff",
"--no-textconv",
"-r",
"--numstat",
"--no-renames",
"-z",
*comparison,
)
files = []
for record in raw.split(b"\0"):
if not record:
continue
added, deleted, path = record.split(b"\t", 2)
files.append(
{
"path": os.fsdecode(path),
"additions": int(added) if added != b"-" else None,
"deletions": int(deleted) if deleted != b"-" else None,
}
)
known = all(f[k] is not None for f in files for k in ("additions", "deletions"))
complete = complete and known
commits.append(
{
"sha": sha,
"files": files,
"known_lines": sum(
(f["additions"] or 0) + (f["deletions"] or 0) for f in files
),
"complete": known,
}
)
return {"head": head, "base": ancestor, "commits": commits, "complete": complete}


def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--base")
parser.add_argument("--diff", help="Local bundle comparison: base or base...HEAD")
parser.add_argument("--numstat", action="store_true")
parser.add_argument("--head", default="HEAD")
args = parser.parse_args()
try:
if args.diff:
if args.diff.endswith("...HEAD"):
head = resolve("HEAD")
base = git("merge-base", args.diff[:-7], head).decode().strip()
raw = diff(base, head, numstat=args.numstat)
else:
base = (
git(
"rev-parse",
"--verify",
"--end-of-options",
args.diff + "^{commit}",
)
.decode()
.strip()
)
raw = diff(base, numstat=args.numstat)

sys.stdout.buffer.write(raw)
return 0
if not args.base:
parser.error("--base or --diff is required")
result = collect(args.base, args.head)
except (subprocess.CalledProcessError, ValueError, OSError):
print(
json.dumps(
{
"commits": [],
"complete": False,
"error": "Cannot inspect the full local commit range; use a checkout with its base and head",
}
)
)
return 1
print(json.dumps(result))
return 0


if __name__ == "__main__":
raise SystemExit(main())
Loading