Skip to content

gh simulator (1/3): the world file, the call log and the read paths #86

Closed

Description

@glennfaison

Category: feature
Area: Evaluation (tests/)
Impact: Everything else in #60 builds on a simulator that can be seeded from a scenario and read from. Without the read paths nothing can look at a world through gh.
Confidence: traced

Part 1 of 3 of #60. Next: #87, then #88.

Problem (the use case)

A maintainer who wants to see whether a model follows shepherd-github-pr has to point it at a real repository. A gh that answers from a world file makes the run repeatable and safe, once it can at least be read from.

Proposed solution

Add the simulator's core and its read paths:

  • An executable gh (tests/sim/bin/gh) that a runner puts first on PATH. GH_SIM_WORLD names the world file and GH_SIM_LOG the call log.
  • A world file is the scenarios' format: a bare world, or a scenario file (its world: is used, with the label catalog expanded). A helper turns a scenario into a world file a run can write to.
  • Every call is appended to the log (argv, exit code) whether it is supported or not.
  • Read paths, with --json, --jq, --label, --state, --limit and --search where the skills use them: issue list|view, pr list|view|checks, label list, repo view, auth status, run view.
  • Anything else exits non-zero with unsupported: <command>, and an unknown --json field fails as gh does.
  • Seeded fields the world keeps as state (lease, readySha, openFindings, shepherdAttempts) are shown as the comments a real repository would hold, so the marker readers in lib.sh find them.
  • A manifest lists every gh command and its status. A test greps skills/ for gh calls and fails for one that is not in the manifest.

Acceptance criteria

  • Each read path above returns what the world holds, in the shape gh uses (OPEN/CLOSED/MERGED, [{name}] labels, check buckets).
  • An unsupported command, flag or --json field fails loudly, and the call is in the log.
  • Every golden scenario's world can be loaded and read.
  • The manifest test knows every gh command in the skills. Commands the write paths and api will add are marked planned.

Non-goals

Writes, api and fault injection (#87 and #88). Rate limits, pagination, webhooks.

Blocked by: none (#59 is closed)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions