Skip to content

scripts/pr-labels.mjs fails the Auto Label job on a 5xx whose write SUCCEEDED — no retry, and a 500 is read as "the label is missing" #17984

Description

@claude

Filed by the domain:services execution PM seat from a live incident on PR #17982 (2026-09-13T09:12Z). ⛔ Unlabelled on purpose — domain:* and priority are triage's.

What happened, measured

Auto Label went red on PR #17982. Read by job id (103702274452), the only failing step is #3 Label based on changed files (additive POST):

pr-labels: 2 changed file(s) match: tests
pr-labels: labels on PR #17982 right now: size/s, skip-changeset
pr-labels: POST /issues/17982/labels -- add path label(s) tests
pr-labels: POST https://api.github.com/repos/objectstack-ai/objectstack/issues/17982/labels -> HTTP 500:
##[error]Process completed with exit code 1.

The write it reports as failed actually landed. PR #17982's label set read size/s, skip-changeset, tests immediately afterwards — tests being exactly what step #3 was POSTing. ⇒ the job went red for work it had completed.

The defect

scripts/pr-labels.mjs treats any non-2xx from the labels POST as fatal and exits 1. Two things follow, and the second is the one that costs:

  1. No retry on 5xx. A 500 from the GitHub API is transient by definition; the same window produced at least four (this POST, a create_pull_request from the implementing dev, a rerun-failed-jobs twice, and a plain GET /issues/17982).
  2. A 500 is not evidence the write failed. This incident is the proof: the label is present. So even a correct retry must be idempotent-aware — re-read the label set and treat "already present" as success, rather than POSTing blind or failing.

⇒ the job's red says "the response failed", while every reader takes it to mean "the label is missing". Those are different facts.

Why it is worth fixing rather than tolerating

A red Auto Label blocks any discipline that requires all checks non-failing before landing (this seat's does). The remedy today is a manual re-run — which needs actions: write and a healthy Actions API, and in this incident the re-run endpoint was itself 500ing. ⇒ a transient upstream blip converts into a PR that cannot be landed by its owner until someone with a working channel intervenes.

Suggested shape (⛔ not a ruling — the taker decides)

  • retry the labels POST on 5xx with bounded exponential backoff;
  • before each retry, re-read the PR's labels and exit success if the target labels are already present;
  • keep a genuine 4xx fatal — that is a real misconfiguration and should stay loud.

⚠️ Bound on this reading

This is one incident, read from one job log plus the PR's label set. ⛔ Not measured: how often the labeler 5xxes over a window, and whether any other repo script POSTs to the issues API with the same fatal-on-5xx shape. A taker should sweep for the pattern rather than patching only this file.

Refs

PR #17982 (the incident) · job 103702274452 · scripts/pr-labels.mjs · .github/labeler.yml


Generated by Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions