Skip to content

[TESTERS NEEDED] Rewrite AGENTS.md with community-first agent guidelines#8604

Open
SableRaf wants to merge 5 commits intoprocessing:mainfrom
SableRaf:agents-md-update
Open

[TESTERS NEEDED] Rewrite AGENTS.md with community-first agent guidelines#8604
SableRaf wants to merge 5 commits intoprocessing:mainfrom
SableRaf:agents-md-update

Conversation

@SableRaf
Copy link
Contributor

@SableRaf SableRaf commented Mar 5, 2026

Note

See more context in the similar PR in processing/processing4#1453

This PR is a rewrite of the AGENTS.md first introduced in processing/p5.js#8194.

What's new?

  • Agents should now point contributors toward Discord, CONTRIBUTING.md, CODE_OF_CONDUCT.md, and the contributor guidelines before doing anything else.
  • Agents should now try to understand where their interlocutor is coming from (e.g.: Are they a first-time contributor? Have they used p5.js before? Do they have JavaScript experience?) and adapt their responses accordingly.
  • Responses should be shorter and more conversational.
  • Agents should redirect contributors who are looking for a different project (Processing Java, p5.js Web Editor, p5.js Website) to the right repo.
  • Agents should now be aware of the 1.x/2.0 branch split and should point contributors to the 2.x project board.
  • Agents should now check that an issue is unassigned and approved before encouraging someone to work on it.
  • GSoC candidates should be directed to the Processing Foundation's GSoC guidelines.
  • Agents should now suggest non-code contributions like translation, documentation, and testing.

How to test this PR

The best way to test this is to check out this PR an agentic coding environment (Claude Code, Copilot, Cursor, etc.) and try a few contributor scenarios:

  • A first-time contributor with no specific issue in mind (should be onboarded and helped to find an open issue)
  • Someone asking about Processing Java (should be redirected to the Processing repo)
  • A GSoC candidate wanting to contribute a PR (should be redirected to GSoC Guidelines)
  • An inexperienced contributor who wants to jump straight to writing code (should be redirected)
  • An experienced contributor wanting to skip the onboarding (should be allowed)

Test the same input multiple times and check whether the responses are consistently helpful for the particular scenario.

Note any misaligned behavior and share it in the comments below. Please specify which model, version, and environment was used.

Note that this is largely subjective so use your judgment when you evaluate agent behavior. Discussion in the comments is encouraged.

Common failure-cases for agent behavior

  • Not reading the AGENTS.md file
  • Recommending issues that are already assigned or not ready for contribution
  • Being suspicious of or interrogating the contributor's motivations
  • Jumping straight to code or codebase analysis without first orienting the contributor
  • Responding to a p5.js question with Processing contribution guidelines
  • Doing the thinking for the contributor, instead of redirecting to the community
  • Asking multiple questions at once in a way that feels like an interrogation
  • Being preachy or reciting the guidelines back to the contributor
  • Replying with an overwhelming wall-of-text

Other changes

  • Added copilot-instructions.md to direct copilot to read AGENTS.md, since it tended to skip it.

SableRaf added 3 commits March 5, 2026 13:46
Revise AGENTS.md to target p5.js: replace Processing-specific references with p5.js equivalents.
@ksen0
Copy link
Member

ksen0 commented Mar 5, 2026

Thanks @SableRaf ! This update seems like a positive improvement.


## How to help

### 0. Welcome them and find out where they are
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not super important but both this and the next point are labelled as 0, is that intentional?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah good catch! I fixed it

@SableRaf
Copy link
Contributor Author

SableRaf commented Mar 5, 2026

Note that Claude CLI tends to ignore AGENTS.md if the initial user request is specific enough (tested with: "I want to fix issue #8551")

I have managed to correct this behavior locally by adding a CLAUDE.md file that includes the line:

BEFORE you do ANYTHING ELSE, ALWAYS refer to the AGENTS.md document for complete guidelines.

However, I have resisted including vendor-specific prompt files in the root of the project so far, as it feels like free advertising or even endorsement.

Would it be worth it in this case? Curious what others think.

Update .github/copilot-instructions.md to explicitely require consulting AGENTS.md BEFORE any other actions.
@SableRaf
Copy link
Contributor Author

SableRaf commented Mar 5, 2026

Failure cases when asking to explore the codebase

Claude acts suspicious of the contributor:

image

Claude skips reading AGENTS.md:

image

@SableRaf
Copy link
Contributor Author

SableRaf commented Mar 5, 2026

I got better results with GPT

image image image

@davepagurek
Copy link
Contributor

I'm also a little hesitant, at least at first, to add instructions for specific tools, for the reasons you describe and also because it seems like we might be playing whack-a-mole as new tools come up. But it's also true that new tools, and even commonly used ones like Claude, may just ignore our instructions. To a certain extent, I feel like maybe we do what we can via methods like this, but then beyond a reasonable amount of effort on our part, fall back on our AI usage policy to close PRs that ignore them?

@SableRaf SableRaf changed the title Rewrite AGENTS.md with community-first agent guidelines [TESTERS NEEDED] Rewrite AGENTS.md with community-first agent guidelines Mar 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants