Skip to content
2 changes: 1 addition & 1 deletion .github/workflows/contributor-pr-reply.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ jobs:
const script = require('./scripts/contributor-pr-reply.js');
return await script({ github, context, core });
- name: Send Slack notification
if: ${{ steps.script.outputs.slack_notification != ''}}
if: ${{ !cancelled() && steps.script.outputs.slack_notification != '' }}
uses: slackapi/slack-github-action@v4.0.0
with:
webhook-type: incoming-webhook
Expand Down
16 changes: 14 additions & 2 deletions docs/community-automations.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,10 +61,22 @@ In `scripts/constants.js` set:

# `contributor-pr-reply`

Sends reply to a community pull requests.
Sends reply to a community pull request, and acts on the issue that pull request closes.

In `scripts/contants.js` set:
What happens depends on who the linked issue is assigned to:

- assigned to someone else: posts a closing message instead of the reply, closes the pull request, and notifies Slack
- assigned to the author: sends the reply, then requests `rtibblesbot` for review and adds the `community-review` label
- not assigned, or no issue linked: sends the reply

An issue counts as linked only when the description closes it, for example `Fixes #123`. Assignments on any other issue are ignored. A pull request with no linked issue is asked to add one under `## References`.

If the author is assigned to any linked issue, the pull request is not closed.

In `scripts/constants.js` set:
- `BOT_MESSAGE_PULL_REQUEST`: Message text
- `BOT_MESSAGE_LINK_ISSUE`: Text added when no issue is linked
- `BOT_MESSAGE_PULL_REQUEST_CLOSED`: Closing message text

# `holiday-message`

Expand Down
9 changes: 8 additions & 1 deletion scripts/constants.js
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,12 @@ const BOT_MESSAGE_KEYWORD_GOOD_FIRST_ISSUE =
` to learn about the contributing process. 😊${GSOC_NOTE}`;

const BOT_MESSAGE_PULL_REQUEST = author =>
`👋 Hi @${author}, thanks for contributing! \n\n **For the review process to begin, please verify that the following is satisfied:**\n\n- [ ] **Contribution is aligned with our [contributing guidelines](https://learningequality.org/contributing-to-our-open-code-base)**\n- [ ] **Pull request description has correctly filled _AI usage_ section & follows our AI guidance:**\n\n <details>\n <summary><b><i>AI guidance</i></b></summary>\n\n <br>\n\n **State explicitly whether you didn't use or used AI & how.**\n\n If you used it, ensure that the PR is aligned with [Using AI](https://learningequality.org/contributing-to-our-open-code-base/#using-generative-ai) as well as our DEEP framework. DEEP asks you:\n\n - **Disclose** — Be open about when you've used AI for support.\n - **Engage critically** — Question what is generated. Review code for correctness and unnecessary complexity.\n - **Edit** — Review and refine AI output. Remove unnecessary code and verify it still works after your edits.\n - **Process sharing** — Explain how you used the AI so others can learn.\n\n <br>\n\n Examples of good disclosures:\n\n > "I used Claude Code to implement the component, prompting it to follow the pattern in ComponentX. I reviewed the generated code, removed unnecessary error handling, and verified the tests pass."\n\n > "I brainstormed the approach with Gemini, then had it write failing tests for the feature. After reviewing the tests, I used Claude Code to generate the implementation. I refactored the output to reduce verbosity and ran the full test suite."\n\n </details>\n\nAlso check that issue requirements are satisfied & you ran \`pre-commit\` locally. \n\n**Pull requests that don't follow the guidelines will be closed.**\n\n**Reviewer assignment can take up to 2 weeks.**`;
`👋 Hi @${author}, thanks for contributing! \n\n **For the review process to begin, please verify that the following is satisfied:**\n\n- [ ] **Contribution is aligned with our [contributing guidelines](https://learningequality.org/contributing-to-our-open-code-base)**\n- [ ] **Pull request description has correctly filled _AI usage_ section & follows our AI guidance:**\n\n <details>\n <summary><b><i>AI guidance</i></b></summary>\n\n <br>\n\n **State explicitly whether you didn't use or used AI & how.**\n\n If you used it, ensure that the PR is aligned with [Using AI](https://learningequality.org/contributing-to-our-open-code-base/#using-generative-ai) as well as our DEEP framework. DEEP asks you:\n\n - **Disclose** — Be open about when you've used AI for support.\n - **Engage critically** — Question what is generated. Review code for correctness and unnecessary complexity.\n - **Edit** — Review and refine AI output. Remove unnecessary code and verify it still works after your edits.\n - **Process sharing** — Explain how you used the AI so others can learn.\n\n <br>\n\n Examples of good disclosures:\n\n > "I used Claude Code to implement the component, prompting it to follow the pattern in ComponentX. I reviewed the generated code, removed unnecessary error handling, and verified the tests pass."\n\n > "I brainstormed the approach with Gemini, then had it write failing tests for the feature. After reviewing the tests, I used Claude Code to generate the implementation. I refactored the output to reduce verbosity and ran the full test suite."\n\n </details>\n\n**📢✨ Before we assign a reviewer:**\n\n- **\`@rtibblesbot\` will pre-review this pull request.** Its comments are generated by an LLM, and should be evaluated accordingly.\n- **We'll also invite community pre-review.** See the [community review guidance](https://github.com/learningequality/.github/blob/main/docs/community-review.md) for both authors and reviewers.\n\nAlso check that issue requirements are satisfied & you ran \`pre-commit\` locally.\n\n**Pull requests that don't follow the guidelines will be closed.**\n\n**Reviewer assignment can take up to 2 weeks.**`;

const BOT_MESSAGE_LINK_ISSUE = `**📎 We couldn't find an issue linked to this pull request.** Please edit the description and link one under **References**, for example \`Fixes #123\` or \`Closes #123\`. A plain \`#123\` mention doesn't link the PR to the issue.`;
Comment thread
akolson marked this conversation as resolved.

const BOT_MESSAGE_PULL_REQUEST_CLOSED = (author, issueUrl) =>
`👋 Hi @${author}, thanks for your interest! \n\n **We're closing this pull request because ${issueUrl} is assigned to someone else.** Visit [Contributing guidelines](https://learningequality.org/contributing-to-our-open-code-base) to learn about the contributing process and how to find suitable issues. If there are no unassigned 'help wanted' issues available, please wait until new ones are added. \n\n We really appreciate your willingness to help. 😊${GSOC_NOTE}`;

const HOLIDAY_MESSAGE = `Season's greetings! 👋 \n\n We'd like to thank everyone for another year of fruitful collaborations, engaging discussions, and for the continued support of our work. **Learning Equality will be on holidays from December 22 to January 5.** We look forward to much more in the new year and wish you a very happy holiday season!${GSOC_NOTE}`;

Expand Down Expand Up @@ -180,6 +185,8 @@ module.exports = {
BOT_MESSAGE_ASSIGN_NOT_GOOD_FIRST_ISSUE,
BOT_MESSAGE_KEYWORD_GOOD_FIRST_ISSUE,
BOT_MESSAGE_PULL_REQUEST,
BOT_MESSAGE_LINK_ISSUE,
BOT_MESSAGE_PULL_REQUEST_CLOSED,
BOT_MESSAGE_RTIBBLESBOT_REVIEW,
RTIBBLESBOT_USERNAME,
LABEL_COMMUNITY_REVIEW,
Expand Down
70 changes: 67 additions & 3 deletions scripts/contributor-pr-reply.js
Original file line number Diff line number Diff line change
@@ -1,15 +1,60 @@
const { BOT_MESSAGE_PULL_REQUEST } = require('./constants');
const { sendBotMessage } = require('./utils');
const {
BOT_MESSAGE_PULL_REQUEST,
BOT_MESSAGE_LINK_ISSUE,
BOT_MESSAGE_PULL_REQUEST_CLOSED,
LABEL_COMMUNITY_REVIEW,
RTIBBLESBOT_USERNAME,
} = require('./constants');
const { sendBotMessage, getLinkedIssues } = require('./utils');

module.exports = async ({ github, context, core }) => {
try {
const owner = context.repo.owner;
const repo = context.repo.repo;
const number = context.payload.pull_request.number;
const url = context.payload.pull_request.html_url;
const title = context.payload.pull_request.title;
const author = context.payload.pull_request.user.login;

const botMessageUrl = await sendBotMessage(number, BOT_MESSAGE_PULL_REQUEST(author), {
const lookup = await getLinkedIssues(number, { github, context, core });
const linkedIssues = lookup || [];
const authorAssigned = linkedIssues.some(issue => issue.assignees.includes(author));
const assignedElsewhere = linkedIssues.find(
issue => issue.assignees.length > 0 && !issue.assignees.includes(author),
);

if (!authorAssigned && assignedElsewhere) {
const botMessageUrl = await sendBotMessage(
number,
BOT_MESSAGE_PULL_REQUEST_CLOSED(author, assignedElsewhere.url),
{ github, context, core },
);
try {
await github.rest.pulls.update({
owner,
repo,
pull_number: number,
state: 'closed',
});
core.setOutput(
'slack_notification',
`*[${repo}] <${botMessageUrl}|Closed> pull request on assigned issue <${assignedElsewhere.url}|#${assignedElsewhere.number}>: <${url}|${title}>*`,
);
} catch (error) {
core.setOutput(
'slack_notification',
`*[${repo}] Failed to close pull request <${url}|${title}>, but the closing message was posted*`,
);
core.setFailed(`Failed to close the pull request: ${error.message}`);
Comment thread
rtibblesbot marked this conversation as resolved.
}
return;
}

const reply =
lookup !== null && lookup.length === 0
? `${BOT_MESSAGE_PULL_REQUEST(author)}\n\n${BOT_MESSAGE_LINK_ISSUE}`
: BOT_MESSAGE_PULL_REQUEST(author);
const botMessageUrl = await sendBotMessage(number, reply, {
github,
context,
core,
Expand All @@ -21,6 +66,25 @@ module.exports = async ({ github, context, core }) => {
} else {
core.setOutput('slack_notification', '');
}

if (authorAssigned) {
try {
await github.rest.pulls.requestReviewers({
owner,
repo,
pull_number: number,
reviewers: [RTIBBLESBOT_USERNAME],
});
await github.rest.issues.addLabels({
owner,
repo,
issue_number: number,
labels: [LABEL_COMMUNITY_REVIEW],
});
} catch (error) {
core.setFailed(`Failed to request review or add the label: ${error.message}`);
}
}
} catch (error) {
core.setOutput('slack_notification', '');
core.setFailed(`Action failed with error: ${error.message}`);
Expand Down
239 changes: 239 additions & 0 deletions scripts/contributor-pr-reply.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,239 @@
const test = require('node:test');
const assert = require('node:assert/strict');

const contributorPrReply = require('./contributor-pr-reply');
const { LABEL_COMMUNITY_REVIEW, RTIBBLESBOT_USERNAME } = require('./constants');

const AUTHOR = 'outsider';

function fakeContext(author = AUTHOR) {
return {
repo: { owner: 'learningequality', repo: 'kolibri' },
payload: {
pull_request: {
number: 42,
html_url: 'https://github.com/learningequality/kolibri/pull/42',
title: 'Fix it',
user: { login: author },
},
},
};
}

function fakeCore() {
const outputs = {};
const warnings = [];
const failures = [];
return {
outputs,
warnings,
failures,
info() {},
warning(message) {
warnings.push(message);
},
setFailed(message) {
failures.push(message);
},
setOutput(name, value) {
outputs[name] = value;
},
};
}

function fakeGithub(linkedIssues = [], { graphqlError } = {}) {
const calls = { comments: [], updates: [], reviewers: [], labels: [], order: [] };
return {
calls,
graphql: async () => {
if (graphqlError) {
throw new Error(graphqlError);
}
return {
repository: {
pullRequest: {
closingIssuesReferences: {
nodes: linkedIssues.map(issue => ({
number: issue.number,
title: `Issue ${issue.number}`,
url: `https://github.com/learningequality/kolibri/issues/${issue.number}`,
assignees: { nodes: issue.assignees.map(login => ({ login })) },
})),
},
},
},
};
},
rest: {
issues: {
createComment: async ({ issue_number, body }) => {
calls.comments.push({ issue_number, body });
calls.order.push('comment');
return { data: { html_url: 'https://github.com/learningequality/kolibri/pull/42#c1' } };
},
addLabels: async ({ labels }) => {
calls.labels.push(...labels);
calls.order.push('label');
},
},
pulls: {
update: async ({ pull_number, state }) => {
calls.updates.push({ pull_number, state });
calls.order.push('close');
},
requestReviewers: async ({ reviewers }) => {
calls.reviewers.push(...reviewers);
calls.order.push('review');
},
},
},
};
}

async function run(github, context = fakeContext()) {
const core = fakeCore();
await contributorPrReply({ github, context, core });
return core;
}

test('a linked issue assigned to someone else closes the pull request', async () => {
const github = fakeGithub([{ number: 7, assignees: ['someone-else'] }]);
const core = await run(github);

assert.deepEqual(github.calls.updates, [{ pull_number: 42, state: 'closed' }]);
assert.equal(github.calls.comments.length, 1);
assert.match(github.calls.comments[0].body, /issues\/7 is assigned to someone else/);
assert.match(core.outputs.slack_notification, /Closed/);
assert.deepEqual(github.calls.reviewers, []);
assert.deepEqual(github.calls.labels, []);
assert.deepEqual(core.failures, []);
});

test('the closing message replaces the standard reply', async () => {
const github = fakeGithub([{ number: 7, assignees: ['someone-else'] }]);
await run(github);

assert.doesNotMatch(github.calls.comments[0].body, /For the review process to begin/);
});

test('a failed close still reports to Slack', async () => {
const github = fakeGithub([{ number: 7, assignees: ['someone-else'] }]);
github.rest.pulls.update = async () => {
throw new Error('Resource not accessible');
};
const core = await run(github);

assert.equal(github.calls.comments.length, 1);
assert.match(core.outputs.slack_notification, /Failed to close/);
Comment thread
rtibblesbot marked this conversation as resolved.
assert.equal(core.failures.length, 1);
});

test('a cross-repo linked issue is named by its url', async () => {
const github = fakeGithub([{ number: 7, assignees: ['someone-else'] }]);
const core = await run(github);

assert.match(github.calls.comments[0].body, /https:\/\/github\.com\/.+\/issues\/7/);
assert.match(core.outputs.slack_notification, /issues\/7/);
});

test('a linked issue assigned to the author requests review and adds the label', async () => {
const github = fakeGithub([{ number: 7, assignees: [AUTHOR] }]);
const core = await run(github);

assert.deepEqual(github.calls.reviewers, [RTIBBLESBOT_USERNAME]);
assert.deepEqual(github.calls.labels, [LABEL_COMMUNITY_REVIEW]);
assert.deepEqual(github.calls.updates, []);
assert.match(github.calls.comments[0].body, /For the review process to begin/);
assert.match(core.outputs.slack_notification, /Reply sent/);
});

test('the reply is sent before the review request and the label', async () => {
const github = fakeGithub([{ number: 7, assignees: [AUTHOR] }]);
await run(github);

assert.deepEqual(github.calls.order, ['comment', 'review', 'label']);
});

test('a failed review request still leaves the reply in place', async () => {
const github = fakeGithub([{ number: 7, assignees: [AUTHOR] }]);
github.rest.pulls.requestReviewers = async () => {
throw new Error('Reviewer cannot be requested');
};
const core = await run(github);

assert.equal(github.calls.comments.length, 1);
assert.equal(core.failures.length, 1);
assert.match(core.outputs.slack_notification, /Reply sent/);
});

test('the author counts as assigned alongside other assignees', async () => {
const github = fakeGithub([{ number: 7, assignees: ['someone-else', AUTHOR] }]);
await run(github);

assert.deepEqual(github.calls.updates, []);
assert.deepEqual(github.calls.reviewers, [RTIBBLESBOT_USERNAME]);
});

test('an issue assigned to the author outweighs another assigned elsewhere', async () => {
const github = fakeGithub([
{ number: 7, assignees: ['someone-else'] },
{ number: 8, assignees: [AUTHOR] },
]);
await run(github);

assert.deepEqual(github.calls.updates, []);
assert.deepEqual(github.calls.reviewers, [RTIBBLESBOT_USERNAME]);
});

test('no linked issue leaves the pull request alone', async () => {
const github = fakeGithub([]);
const core = await run(github);

assert.deepEqual(github.calls.updates, []);
assert.deepEqual(github.calls.reviewers, []);
assert.deepEqual(github.calls.labels, []);
assert.equal(github.calls.comments.length, 1);
assert.match(core.outputs.slack_notification, /Reply sent/);
});

test('a linked issue with no assignee leaves the pull request alone', async () => {
const github = fakeGithub([{ number: 7, assignees: [] }]);
await run(github);

assert.deepEqual(github.calls.updates, []);
assert.deepEqual(github.calls.reviewers, []);
assert.deepEqual(github.calls.labels, []);
});

test('an unlinked pull request is asked to link an issue', async () => {
const github = fakeGithub([]);
await run(github);

assert.match(github.calls.comments[0].body, /link one under \*\*References\*\*/);
});

test('a linked pull request is not asked to link an issue', async () => {
const github = fakeGithub([{ number: 7, assignees: [] }]);
await run(github);

assert.doesNotMatch(github.calls.comments[0].body, /link one under/);
});

test('a failed lookup falls back to the standard reply', async () => {
const github = fakeGithub([], { graphqlError: 'API down' });
const core = await run(github);

assert.deepEqual(github.calls.updates, []);
assert.deepEqual(github.calls.reviewers, []);
assert.equal(github.calls.comments.length, 1);
assert.equal(core.warnings.length, 1);
assert.match(core.warnings[0], /API down/);
assert.deepEqual(core.failures, []);
});

test('a failed lookup does not ask the author to link an issue', async () => {
const github = fakeGithub([], { graphqlError: 'API down' });
await run(github);

assert.doesNotMatch(github.calls.comments[0].body, /link one under/);
});
Loading
Loading