Feature/contest discussion automation by river0525 · Pull Request #30 · AtCoder-NoviSteps/NoviSteps-Writeups

river0525 · 2026-07-29T14:05:24Z

概要

ABC終了後、自動でDiscussionを作成する機能を実装。

Architecture

NoviSteps-Writeups リポジトリ単独で完結する GitHub Actions (cron) から Python スクリプトを実行する。AtCoder の問題ページを直接スクレイピングして問題一覧を取得し、GitHub GraphQL API (createDiscussion) で Discussion を作成する。

Add pyproject.toml with requests/selectolax runtime deps and pytest/responses dev deps, plus the generated uv.lock. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Verifies (manually, via workflow_dispatch) that the default GITHUB_TOKEN combined with permissions.discussions:write is sufficient to create Discussions via the GraphQL API, so no separate PAT/repo secret is needed for the contest-discussions automation. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…cument exception surface)

…inished_abc_ids

… GraphQL

Ties together atcoder.py and github_client.py: for each recently finished ABC, fetch tasks, skip titles that already have a Discussion, and create the rest. Per-contest and per-task failures are caught and logged so one bad contest/task doesn't stop the run. --contest-id lets main() bypass auto-detection for manual backfill.

Runs weekly via cron (Sat 13:50 UTC, 10 min after the ABC 100-min window ends at 22:40 JST) and supports manual backfill via workflow_dispatch with an optional contest_id input. Uses the ambient GITHUB_TOKEN with discussions:write permission, no PAT required (verified in the Phase 1 smoke test). astral-sh/setup-uv@v3 has no python-version input (confirmed via its action.yml), so Python 3.13 is installed explicitly with actions/setup-python@v5 beforehand.

Superseded by post_discussions.yml, which now covers the real discussion-posting job; the smoke test already confirmed that the default GITHUB_TOKEN can create Discussions via GraphQL.

…ilure

Copilot

Pull request overview

This PR adds an automated pipeline (Python script + scheduled GitHub Actions workflow) that detects recently finished AtCoder Beginner Contests (ABCs), scrapes the tasks list from AtCoder, and creates per-problem GitHub Discussions in this repository when missing.

Changes:

Introduces a new contest_discussions Python package to fetch contest/task data and post Discussions via the GitHub GraphQL API.
Adds a scheduled + manual-dispatch GitHub Actions workflow to run the posting script using uv.
Adds pytest coverage (with fixtures + HTTP mocking) for the AtCoder scraping, GitHub client, and orchestration logic.

Reviewed changes

Copilot reviewed 19 out of 22 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
uv.lock	Locks Python dependencies for the new automation tooling.
pyproject.toml	Defines the new Python project, dependencies, and pytest config.
.gitignore	Ignores uv virtualenv/cache and Python bytecode artifacts.
.github/workflows/post_discussions.yml	Schedules and dispatches the script execution with proper permissions.
src/contest_discussions/init.py	Introduces the new Python package namespace.
src/contest_discussions/constants.py	Stores repository/category IDs and shared constants for posting.
src/contest_discussions/atcoder.py	Implements contest discovery and task scraping from AtCoder endpoints/pages.
src/contest_discussions/github_client.py	Implements GitHub GraphQL calls to read/create Discussions.
src/contest_discussions/main.py	Orchestrates contest selection, de-duping, and discussion creation with CLI entrypoint.
tests/test_atcoder.py	Tests scraping + contest selection logic with mocked HTTP responses.
tests/test_github_client.py	Tests GraphQL client behavior using mocked responses.
tests/test_main.py	Tests orchestration behavior (skips, partial failures, backfill mode).
tests/fixtures/contests_sample.json	Fixture for contests.json filtering tests.
tests/fixtures/abc_tasks_sample.html	Fixture for AtCoder tasks-page parsing tests.
docs/dev-notes/2026-07-21/contest-discussion-automation/plan.md	Design/implementation plan documentation.
docs/dev-notes/2026-07-21/contest-discussion-automation/phase-1.md	Implementation phase notes for the feature.
docs/dev-notes/2026-07-21/contest-discussion-automation/phase-2.md	Implementation phase notes for the feature.
docs/dev-notes/2026-07-21/contest-discussion-automation/phase-3.md	Implementation phase notes for the feature.
docs/dev-notes/2026-07-21/contest-discussion-automation/phase-4.md	Implementation phase notes for the feature.
docs/dev-notes/2026-07-21/contest-discussion-automation/phase-5.md	Implementation phase notes for the feature.
docs/dev-notes/2026-07-21/contest-discussion-automation/phase-6.md	Implementation phase notes for the feature.
docs/dev-notes/2026-07-21/contest-discussion-automation/phase-7.md	Implementation phase notes for the feature.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

river0525 · 2026-07-29T14:27:00Z

+def existing_discussion_titles(token: str) -> set[str]:
+    """Returns the titles of the most recent Discussions in the General category.
+
+    Only the first 100 (no pagination) — accepted limitation given weekly ABC
+    volume; revisit if the General category ever accumulates enough non-contest
+    discussions to push older unposted contest titles out of this window.
+    Network/HTTP failures propagate as requests.RequestException (e.g.
+    requests.HTTPError, requests.Timeout) — callers should catch that.
+    """
+    data = _post_graphql(
+        token,
+        _EXISTING_TITLES_QUERY,
+        {"repositoryId": REPOSITORY_ID, "categoryId": GENERAL_CATEGORY_ID},
+    )
+    nodes = data["node"]["discussions"]["nodes"]
+
+    return {node["title"] for node in nodes}


Fixed in 0903188. existing_discussion_titles() now requests 100 discussions per page and follows pageInfo.endCursor until hasNextPage is false, so older Discussions are included during normal runs and backfills. Added a pagination regression test.

river0525 · 2026-07-29T14:27:04Z

+    if now is None:
+        now = datetime.now(timezone.utc)
+
+    response = requests.get(CONTESTS_JSON_URL, timeout=REQUEST_TIMEOUT_SECONDS)


Fixed in 0903188. find_recently_finished_abc_ids() now rejects a supplied naive datetime with ValueError("now must be timezone-aware"), preventing the cutoff from depending on the runner's local timezone. Added a regression test.

Copilot

Pull request overview

Copilot reviewed 19 out of 22 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

src/contest_discussions/main.py:16

build_title() assumes contest_id is always in the form abcNNN. When --contest-id is provided via CLI/workflow and is malformed (e.g. whitespace, uppercase, wrong prefix), int(contest_id[3:]) raises a bare ValueError that is hard to diagnose. Add explicit validation with a clear error message before parsing.

def build_title(contest_id: str, task: dict) -> str:
    number = int(contest_id[3:])
    return f"ABC {number} {task['problem_index']} - {task['name']}"

src/contest_discussions/github_client.py:106

_post_graphql() unconditionally returns payload["data"]. If GitHub returns a non-standard/partial payload without a data field (while still being HTTP 200 and without an errors key), this becomes a KeyError that obscures the real response. Prefer an explicit data is None check and raise a clearer RuntimeError including the payload.

    if "errors" in payload:
        raise RuntimeError(f"GitHub GraphQL API returned errors: {payload['errors']}")

    return payload["data"]

.github/workflows/post_discussions.yml:36

CI should fail fast if uv.lock and pyproject.toml are out of sync. Using uv sync without --frozen can allow implicit resolution drift in automation. Use uv sync --frozen to guarantee the workflow installs exactly what is locked.

      - name: Install dependencies
        run: uv sync

KATO-Hiro

PR 作成ありがとうございます
とても助かります
遅くなりましたが、拝見いたしました

[high]

.github/workflows/post_discussions.yml で cron の実行時間の設定で、現状だとコンテスト情報が Problems API へ反映される前に実行する可能性があるかもしれません
動作確認済みという理解でいいでしょうか?

[low] 残りのコメントは安定した運用ためにあるいいと思われる項目です。これらのうち、該当コードを再読して必要だと判断・納得されたものに関してはご対応いただけると幸いです

私の理解不足や勘違いなどもあるかもしれませんので、対処不要と判断された項目に関してはご放念ください

KATO-Hiro · 2026-07-31T22:26:14Z

+            print(f"Created: {url}")
+            existing_titles.add(title)
+
+    if fetch_failures == len(contest_ids):


create_failures を数えた方がいいかもしれません

理由: token 権限不足・secondary rate limit・API 障害などで、すべての作成に失敗してもワークフローが正常に実行されたと表示される可能性があるため

KATO-Hiro · 2026-07-31T22:35:36Z

+"""
+
+
+def existing_discussion_titles(token: str) -> set[str]:


orderBy: CREATED_AT DESC なので、自動モード（直近 3 コンテスト＝最大 21 件）は 1 ページで十分そうです

KATO-Hiro · 2026-07-31T22:40:28Z

+logger = logging.getLogger(__name__)
+
+
+def build_title(contest_id: str, task: dict) -> str:


例えば、main()もしくはこのメソッドで、re.fullmatch(r"abc\d+", contest_id) のように検証を入れてはいかがでしょうか?

理由: 現時点ではABC のみの想定なので、ARC / AGC などの誤爆を回避するため

KATO-Hiro · 2026-07-31T22:44:04Z

+_TASK_HREF_PATTERN = re.compile(r"^/contests/([^/]+)/tasks/([^/]+)$")
+
+
+def fetch_tasks(contest_id: str) -> list[dict[str, str]]:


# kenkoooo's problems.json lags behind by hours after a contest ends, # so task data is scraped directly instead.

のようなコメントがあると、Problems の API を意図して使っていないことが明確になると思います

KATO-Hiro · 2026-07-31T22:47:34Z

+
+---
+
+### Task 2 (ユーザー確認必須・手動): 実際に1回動かして確認


動作確認済み　という理解でいいのでしょうか?

KATO-Hiro · 2026-07-31T22:49:48Z

+
+---
+
+### Task 2: phase-N.md の破棄


コードから読み取るのが難しい内容（意思決定・注意事項など）を集約されてはいかがでしょうか?

理由: planのコードは陳腐化する可能性が高いため

KATO-Hiro · 2026-07-31T22:53:59Z

+          python-version: "3.13"
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v3


astral-sh/setup-uv@c771a70 # v9.0.0

ようなコミットハッシュを使われてはいかがでしょうか?
（actions などの他の部分も同様です）

理由: セキュリティ対策（改ざん防止、サプライチェーン攻撃対策）のため

KATO-Hiro · 2026-07-31T23:33:34Z

+
+on:
+  schedule:
+    - cron: '50 13 * * 6'


Problems API からコンテスト情報を取得する場合、コンテスト終了後10分では反映されていない可能性が高いかと思います

けんちょんさん、ウルズニャーさんとの議論から、日曜日に投稿できる状態であれば十分そうです
riverさんのお考えはいかがでしょうか?

- cron: '50 8 * * 0' timezone: 'Asia/Tokyo'

river0525 and others added 21 commits July 21, 2026 17:23

chore: add .gitignore (worktrees, Python artifacts)

51511e1

docs: add implementation plan for contest discussion automation

8645e22

feat: initialize uv project for contest-discussions automation

46e4ecc

Add pyproject.toml with requests/selectolax runtime deps and pytest/responses dev deps, plus the generated uv.lock. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

feat: add contest_discussions package skeleton

6ec76fb

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

feat: add fixed repository/category IDs for NoviSteps-Writeups

95b6fc9

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

fix: translate constants.py comment to English for consistency

f5c095a

test: add AtCoder tasks page HTML fixture

62a255c

feat: fetch task list from AtCoder tasks page

87c6b38

fix: harden fetch_tasks (byte-decode HTML, cross-check contest_id, do…

1e31a95

…cument exception surface)

test: add AtCoder Problems contests.json fixture

1f2a819

feat: find recently finished ABC contest ids

8b1fd52

fix: guard limit<=0 edge case, clarify docstrings for find_recently_f…

6f6a33a

…inished_abc_ids

test: add failing tests for github_client discussion query/create

fb0672d

feat: query existing discussion titles and create new discussions via…

5991aef

… GraphQL

test: add failing tests for main orchestration/CLI

446c28b

fix: log tracebacks on failure, test CLI entrypoint and same-run dedup

9318580

chore(ci): remove throwaway smoke-test workflow

50d20bd

Superseded by post_discussions.yml, which now covers the real discussion-posting job; the smoke test already confirmed that the default GITHUB_TOKEN can create Discussions via GraphQL.

fix: add job timeout/concurrency guard, fail run on systemic fetch fa…

34b12ba

…ilure

river0525 requested review from KATO-Hiro and Copilot July 29, 2026 14:10

Copilot started reviewing on behalf of river0525 July 29, 2026 14:11 View session

Copilot AI reviewed Jul 29, 2026

View reviewed changes

fix: address Copilot review findings

0903188

river0525 requested a review from Copilot July 29, 2026 16:31

Copilot started reviewing on behalf of river0525 July 29, 2026 16:32 View session

Copilot AI reviewed Jul 29, 2026

View reviewed changes

KATO-Hiro requested changes Jul 31, 2026

View reviewed changes

		logger = logging.getLogger(__name__)


		def build_title(contest_id: str, task: dict) -> str:

		_TASK_HREF_PATTERN = re.compile(r"^/contests/([^/]+)/tasks/([^/]+)$")


		def fetch_tasks(contest_id: str) -> list[dict[str, str]]:


		---

		### Task 2 (ユーザー確認必須・手動): 実際に1回動かして確認

Uh oh!

Conversation

river0525 commented Jul 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

概要

Architecture

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

river0525 Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

river0525 Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

KATO-Hiro left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KATO-Hiro Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KATO-Hiro Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

KATO-Hiro Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

KATO-Hiro Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KATO-Hiro Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KATO-Hiro Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

KATO-Hiro Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

KATO-Hiro Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

river0525 commented Jul 29, 2026 •

edited

Loading

KATO-Hiro left a comment •

edited

Loading

KATO-Hiro Jul 31, 2026 •

edited

Loading

KATO-Hiro Jul 31, 2026 •

edited

Loading

KATO-Hiro Jul 31, 2026 •

edited

Loading