Skip to content

docs: add an AI policy and agent instructions - #5377

Draft
marcoscaceres wants to merge 23 commits into
mainfrom
docs/ai-policy
Draft

docs: add an AI policy and agent instructions#5377
marcoscaceres wants to merge 23 commits into
mainfrom
docs/ai-policy

Conversation

@marcoscaceres

@marcoscaceres marcoscaceres commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Adds an AI policy, and the two files GitHub documents for telling a coding agent how this project works.

The policy asks for disclosure, one concern per pull request, an honest accounting of what an issue asked for, and that a test offered as proof of a fix can actually fail without it. It also says what the maintainers hold themselves to when we generate work ourselves, which is stricter than what we ask of anyone else, and it asks nothing of contributors beyond one model and their own judgment. CONTRIBUTING.md points at it, since that is the file an agent reads first.

.github/copilot-instructions.md carries the things an agent cannot infer: that karma loads bundles from builds/ and there is one per profile, so a source change does nothing until it is rebuilt, and that BROWSERS must be set or the suite hangs after printing START: with nothing explaining why. .github/workflows/copilot-setup-steps.yml pre-installs dependencies and keeps puppeteer's browser inside the workspace, since the default cache location is not guaranteed to survive into the agent's session.

Reviewed by Gemini and Copilot as well as by hand. Between them they found a contradiction between two sections of the policy, a labeling rule that required triage access most contributors do not have, a missing privacy rule, and the hang above, which was the worst of them.

Written with AI: Claude drafted the policy and the two configuration files, Gemini and Copilot reviewed them, and I read and ran everything before pushing. Per AI_POLICY.md.

Copilot AI lite review requested due to automatic review settings August 4, 2026 06:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds contributor-facing guidance and GitHub Copilot cloud-agent configuration so AI-assisted contributions follow project expectations and Copilot sessions start with a working, reproducible ReSpec toolchain.

Changes:

  • Add an AI_POLICY.md and link to it from README.md and CONTRIBUTING.md.
  • Add Copilot agent instructions in .github/copilot-instructions.md describing build/test and project conventions.
  • Add a Copilot setup workflow to preinstall dependencies/builds and validate puppeteer availability, plus ignore the local puppeteer cache directory.

Reviewed changes

Copilot reviewed 5 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
README.md Adds a pointer to the new AI policy for prospective AI-assisted contributors.
CONTRIBUTING.md Updates contributor guidance to include AI policy and adds an illustrative image.
AI_POLICY.md Introduces the project’s AI contribution policy and maintainer standards.
.gitignore Ignores the repo-local puppeteer cache directory used by Copilot setup.
.github/workflows/copilot-setup-steps.yml Adds Copilot setup steps to install deps, build bundles, and validate puppeteer browser availability.
.github/copilot-instructions.md Documents how Copilot should build/test and follow house style in this repo.
Suppressed comments (1)

.github/workflows/copilot-setup-steps.yml:49

  • The puppeteer browser check should run with the same PUPPETEER_CACHE_DIR as the install step; otherwise it may validate a different cache location than the one the agent will rely on (and if Copilot ignores job-level env, this step won't see it either). Set the cache dir explicitly on this step.
      # Fails the setup loudly if puppeteer has no usable browser, rather than
      # leaving the agent to discover it mid-task.
      - name: Check puppeteer has a browser
        run: |
          node -e "

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/copilot-setup-steps.yml
Comment thread .github/copilot-instructions.md
Copilot AI review requested due to automatic review settings August 4, 2026 07:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 7 changed files in this pull request and generated no new comments.

Suppressed comments (1)

AI_POLICY.md:32

  • This sentence is missing a verb (“If AI generated logic…”), which makes it ungrammatical and harder to understand. Consider rephrasing to include the missing verb and (optionally) hyphenating “AI-generated”.
If AI generated logic in your contribution, say so in the pull request
description. Generated logic means an agent, a function, an algorithm, a test, a
feature: something you would otherwise have had to work out.

Copilot AI review requested due to automatic review settings August 4, 2026 11:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 8 changed files in this pull request and generated 1 comment.

Suppressed comments (2)

AI_POLICY.md:6

  • The image alt text is extremely detailed and includes non-essential scene description. For accessibility, alt text should focus on what matters to a reader (the "AI is welcome here" message) and stay concise.
<img src="assets/ai-is-welcome-here.png" alt="A cheerful pixel-art robot smiling straight at the camera while a forest burns orange behind it. The caption reads &quot;AI is welcome here.&quot;" width="320">

CONTRIBUTING.md:1

  • The image alt text is very long and includes incidental background detail. For accessibility, consider using concise alt text that conveys the purpose/message of the image.
<img src="assets/ai-is-welcome-here.png" alt="A cheerful pixel-art robot smiling straight at the camera while a forest burns orange behind it. The caption reads &quot;AI is welcome here.&quot;" width="220" align="right">

Comment thread .github/workflows/copilot-setup-steps.yml
Copilot AI review requested due to automatic review settings August 4, 2026 11:09
@marcoscaceres

Copy link
Copy Markdown
Contributor Author

Also acted on the two suppressed comments, since both were right.

The puppeteer check step now sets PUPPETEER_CACHE_DIR explicitly rather than inheriting it. Same root cause as the thread above: job-level env is not honored, so the check would have validated a location nothing had populated.

And AI_POLICY.md now reads "If AI generated any of the logic in your contribution", which fixes the ambiguity.

Two things running it afterwards found that the review could not have predicted. Putting the browser inside the workspace means eslint walks it, producing 12 errors from Chrome's own bundled scripts, including 'chrome' is not defined and eval can be harmful. .gitignore was not enough on its own, since eslint does not consult it, so .cache/** is now in eslint's ignores alongside builds/** and js/**.

The other is that the workspace cache is about 340 MB and is not shared between checkouts. That is the right trade for an ephemeral agent environment, but wrong for a human with several worktrees, so the instructions now say to skip the export if you already have a browser in puppeteer's default cache.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 8 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.github/workflows/copilot-setup-steps.yml:64

  • puppeteer is ESM-only in v25+, so require('puppeteer') in the node -e check will throw and make the setup fail even when the browser is installed. Use a dynamic import('puppeteer') instead.
          Promise.resolve(require('puppeteer').executablePath())

Copilot AI review requested due to automatic review settings August 4, 2026 11:12
Copilot AI review requested due to automatic review settings August 4, 2026 11:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 8 changed files in this pull request and generated no new comments.

@marcoscaceres marcoscaceres added the AI Contains AI-generated logic; see AI_POLICY.md label Aug 4, 2026
Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 5, 2026 04:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 8 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.github/copilot-instructions.md:96

  • This bullet summary contradicts AI_POLICY.md: AI_POLICY says a Co-Authored-By trailer naming the model is welcome, but these instructions say to never add one. Since this section explicitly points to AI_POLICY as the source of truth, the summary here should match it (or drop the conflicting guidance).
- Commit messages are one imperative subject line, lowercase after any prefix, no
  trailing period, and no body unless the diff cannot be understood without one.
  Never add `Co-Authored-By` for a model.

Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 5, 2026 05:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 9 changed files in this pull request and generated no new comments.

Suppressed comments (2)

AI_POLICY.md:78

  • The commit-message guidance here says a Co-Authored-By trailer naming the model is welcome, but .github/copilot-instructions.md later says "Never add Co-Authored-By for a model." Having both rules in-repo is contradictory and will confuse contributors/agents about the expected house style. Consider aligning the policy with the Copilot instructions (or vice-versa) so there is a single rule.
**The commit message** is one imperative subject line, lowercase after any
prefix, no trailing period. A body only when the diff genuinely cannot be
understood without one, and then a sentence or two. No bullet lists, no
"Summary", no recap of what the diff already shows. A `Co-Authored-By` trailer
naming the model that wrote it is welcome, and consistent with what already
appears in this project's history.

.github/workflows/copilot-setup-steps.yml:70

  • The catch handler assumes the rejection reason has a .message property. If the promise rejects with a non-Error (string, null, etc.), this will throw inside the catch, making the failure output less reliable. Coerce the error to a string (or narrow to Error) before logging so this step always fails loudly with a useful message.
          import('puppeteer')
            .then(p => p.default.executablePath())
            .then(p => { fs.accessSync(p); console.log('browser ok:', p); })
            .catch(e => { console.error('no usable browser:', e.message); process.exit(1); })

Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 5, 2026 05:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 9 changed files in this pull request and generated no new comments.

Suppressed comments (1)

AI_POLICY.md:78

  • AI_POLICY.md says a Co-Authored-By trailer naming the model is welcome, but .github/copilot-instructions.md explicitly says to never add Co-Authored-By for a model. These docs should be consistent so contributors (and agents) don’t get conflicting guidance.
"Summary", no recap of what the diff already shows. A `Co-Authored-By` trailer
naming the model that wrote it is welcome, and consistent with what already
appears in this project's history.

Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 5, 2026 06:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 9 changed files in this pull request and generated no new comments.

Suppressed comments (3)

CLAUDE.md:33

  • The Key modules table row includes an inline example ` | var | ` containing pipe characters. In GitHub Flavored Markdown tables, | inside a cell can be parsed as a column separator, which can break the table rendering. Use escaping or raw HTML to render the pipes literally inside the cell.
| `src/core/inlines.js`               | All inline shorthand syntax: `[= =]`, `{{ }}`, `[[SPEC]]`, `[[[...]]]`, ` | var | `   |

.github/workflows/copilot-setup-steps.yml:69

  • The puppeteer probe logs e.message, but the caught value is not guaranteed to be an Error. If it’s a string or another value, e.message will be undefined and the workflow will emit an unhelpful error. Normalize the message before logging so failures are always actionable.
            .catch(e => { console.error('no usable browser:', e.message); process.exit(1); })

.github/copilot-instructions.md:96

  • This guidance on commit messages contradicts AI_POLICY.md: the policy says a Co-Authored-By trailer naming the model is welcome, but these agent instructions say to never add it. Since this file points readers to AI_POLICY.md as the source of house style, these should be consistent to avoid confusing contributors/agents.
- Commit messages are one imperative subject line, lowercase after any prefix, no
  trailing period, and no body unless the diff cannot be understood without one.
  Never add `Co-Authored-By` for a model.

Comment thread .github/copilot-instructions.md Outdated
need that line, and you should skip it: the workspace copy is around 340 MB and does
not get shared between checkouts.

Karma reads the bundles in `builds/`, not `src/`. A source change has no effect on

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: Except for unit tests - which use src directly. Faster flow.

spelling alone when it is load-bearing, such as an established identifier or a
verbatim quotation.

Run `npx prettier --write` on changed files. CI fails on unformatted code.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pnpm format --write

Comment thread .github/copilot-instructions.md Outdated
the first line.
- `Closes #N` only when every ask in the issue is delivered, otherwise `Refs #N`.
Confirm the number is an issue and not a pull request before citing it.
- Commit messages are one imperative subject line, lowercase after any prefix, no

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also mention the pull request title format.

Comment thread .github/copilot-instructions.md Outdated

## Adding a module

A new module under `src/core/` exports `name` and a `run(conf)`. `run` may be

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a module is meant for only certain profile, add to profile folder instead of core.

Comment thread .github/copilot-instructions.md
Comment thread AI_POLICY.md
Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 5, 2026 07:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 9 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.github/workflows/copilot-setup-steps.yml:69

  • The puppeteer probe logs e.message without ensuring the rejection value is an Error. If a non-Error is thrown/rejected, this will print undefined (or itself throw if e is nullish), making the setup failure harder to diagnose. Coerce to a string (or narrow with instanceof Error) before logging.
            .catch(e => { console.error('no usable browser:', e.message); process.exit(1); })

Comment thread CLAUDE.md

Build, test, code style, and the house style for pull requests, commits and comments live in the file below, shared with other agents rather than duplicated:

@.github/copilot-instructions.md

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be too much in context. Claude might not even read the full file.

Comment thread CONTRIBUTING.md
@@ -1,3 +1,7 @@
Everyone is welcome to contribute.
<img src="assets/ai-is-welcome-here.png" alt="Pixel-art robot grinning in front of a burning forest, captioned &quot;AI is welcome here.&quot;" width="220" align="right">

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's not add this image here. It's enough in AI Policy.

Comment thread CLAUDE.md
errorFilters,
warningFilters,
} from "../SpecHelper.js";
const errors = errorFilters.filter("module/name");

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const pluginErrors = errorFilters.filter("module/name");
// ...
const errors = pluginErrors(doc);

Comment thread CLAUDE.md

Almost always caused by the branch being behind main. Fix:

1. `git rebase main` (resolve conflicts — for `builds/` files, always take `--ours`)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe suggest reject any changes to builds/ and build again, instead of trying to rebase (consumes less tokens, is faster).

Comment thread CLAUDE.md
Almost always caused by the branch being behind main. Fix:

1. `git rebase main` (resolve conflicts — for `builds/` files, always take `--ours`)
2. `git diff main --name-only | xargs npx prettier --write`

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
2. `git diff main --name-only | xargs npx prettier --write`
2. `git diff main --name-only | xargs pnpm format --write`

Comment thread CLAUDE.md

Do NOT commit `builds/` files — CI rebuilds them. On rebase conflicts in build artifacts, take main's version.

## Pre-PR checklist (learned from Copilot reviews)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Pre-PR checklist (learned from Copilot reviews)
## Pre-PR checklist

Comment thread CLAUDE.md

Do NOT commit `builds/` files — CI rebuilds them. On rebase conflicts in build artifacts, take main's version.

## Pre-PR checklist (learned from Copilot reviews)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps don't recommend automatic pushes (keep in CLAUDE.local.md). This will encourage will be send PRs manually, which reduces spam/low quality chances.

Comment thread CLAUDE.md
Comment on lines +122 to +124
## Cross-spec headings API

`POST /xref/headings` on respec.org looks up section heading text by `{spec, id}`, sourced from w3c/webref `ed/headings/`. This is what lets `[[[SPEC#id]]]` render the actual heading rather than just the spec title.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is more of spec writing, then ReSpec.

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Copilot <noreply@github.com>
Copilot AI review requested due to automatic review settings August 5, 2026 23:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 10 changed files in this pull request and generated no new comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI Contains AI-generated logic; see AI_POLICY.md

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants