Skip to content

feat(api): publish an alias index so clients resolve a locator without UUIDv5 (#84) - #88

Merged
maehr merged 3 commits into
textrefs:stagingfrom
maehr:feat/84-alias-index
Aug 24, 2026
Merged

feat(api): publish an alias index so clients resolve a locator without UUIDv5 (#84)#88
maehr merged 3 commits into
textrefs:stagingfrom
maehr:feat/84-alias-index

Conversation

@maehr

@maehr maehr commented Aug 24, 2026

Copy link
Copy Markdown
Member

Summary

Publish the reference lookup as data, so a client that knows {work, citation system, locator} can reach the record without a UUIDv5 implementation of its own.

Closes #84.

Commit type

  • fix — bug fix
  • feat — new feature
  • docs — documentation only
  • refactor — code restructure without behaviour change
  • chore / build / ci — tooling, dependencies, CI

Area

  • site / docs
  • standard
  • registry data
  • release / CI / tooling

Related issues

Closes #84. Companion to #83 (collection endpoints), which this builds on. Complementary to #42 (client-side UUID generator): #42 lets an author mint an identifier for a reference that is not yet in the registry; this lets a client look up one that is. Relevant to #52 (WebMCP) and #74 (datapackage.json descriptors).

What lands

1. /reg/work/{key}/aliases.json — a per-work locator index.

{
  "work_key": "plato.republic",
  "preferred_citation_system_key": "stephanus",
  "refs": { "stephanus": { "514a": "dc799d4b-9b17-5d76-85aa-dfd001c5321d" } }
}

Two fetches resolve a passage: read refs[system][locator], then GET /id/ref/{uuid}.json. For a bare locator with no system, use preferred_citation_system_key.

2. /dump/aliases.json — the complete alias table, registered in datapackage.json with a byte count and a sha256: hash. data/README.md and data/.gitignore have named this artifact all along; no code wrote it.

Decisions worth reviewing

  • Bare UUIDs, not IRIs. Halves the largest file (1.15 MB rather than 2.1 MB for tanakh), and /id/ref/{uuid}.json needs the UUID anyway. The per-work index is plain JSON with no @context — it is a lookup table, not JSON-LD, and the OpenAPI and the docs say so.
  • Built from the reference records, not from the flat alias table. That table mixes /cite/ aliases targeting reference IRIs with external mapping identifiers targeting work IRIs. /dump/aliases.json keeps full IRIs for exactly that reason, so a consumer can tell the two kinds apart.
  • No pagination. Paginating would force a client to fetch every page to find one locator, which defeats the purpose. The largest file is 1.15 MB, under the 2 MB cap. src/pages/reg/work/[key]/refs/[page].astro stays the documented fallback.
  • Alias keys are sorted in the dump, so the body — and therefore its hash — depends on registry content alone, not on the order the compiler visited the work files.
  • .github/workflows/release.yml is in this PR. aliases.json matches neither existing glob and fail_on_unmatched_files: true is set, so a tagged release would otherwise ship a descriptor listing a resource that is not in the release. Splitting the workflow line into its own main PR would open exactly that window. CONTRIBUTING.md routes infra to main; the release workflow only fires on tags cut from main, so this reaches production through the next staging → main squash with no gap.

Acceptance criteria

# Criterion Result
1 An index for every work 12 of 12
2 Every reference reachable by {system, locator} 67,959 of 67,959
3 Each mapped UUID is the id of its CanonicalReference 67,959 mapped entries, no collisions
4 dist/dump/aliases.json in datapackage.json with bytes + sha256 13,518,919 B, sha256:dbaba8b5…, verified against the file on disk
5 Largest emitted index measured below
6 No IRI, UUID, or alias target changes the four JSONL bodies are byte-identical to staging
7 npm run verify and npm run validate:data both pass

Criterion 5 — measured sizes

Built against the pinned submodule. 204,361 pages, 67,959 references, 135,942 alias entries.

Work Refs aliases.json gzip
tanakh 23,213 1,178,980 B (1.12 MB) 579,748 B
homer.iliad 15,693 745,086 B 388,318 B
homer.odyssey 12,110 574,250 B 299,624 B
new-testament 7,954 406,006 B 198,661 B
aristotle.nicomachean-ethics 5,220 254,318 B 129,682 B
plato.republic 1,475 67,946 B 36,540 B
the other six ≤ 693 ≤ 31,212 B ≤ 17,335 B

Largest index: tanakh at 1.12 MB raw, 566 KB gzipped — under the 2 MB cap, so no pagination. All 12 files together are 3.33 MB.

/dump/aliases.json is 12.89 MB raw, 1.97 MB gzipped. For proportion, references.jsonl is already 53.6 MB, so the dump grows by about a quarter. GitHub Pages compresses application/json, so the gzip column is the wire cost. The table is derivable from references.jsonl plus works.jsonl; it is published for convenience, which is what criterion 4 and data/README.md ask for.

Tests

  • src/lib/alias-index.test.ts — 12 tests against registry.fixture.ts, which is the hard case: one work, two citation systems, the same locator 1 under both, one active reference and one draft. Covers reachability, UUID-not-IRI values, ADR-0005 two-system separation, empty-refs works, draft inclusion, order independence, no caller mutation, and a locator spelled __proto__ (a locator is author-controlled, so plain assignment would set a prototype and silently drop the entry — hence Object.fromEntries).
  • scripts/compile.test.ts — 7 new tests over the extracted dumpResources() / describeResource(), including an independent recomputation of every bytes and sha256, and a guard that the four JSONL bodies did not move.

writeDump() stays private: it derives its path from the module-level distRoot, and scripts/compile.ts is imported by every Astro page through src/lib/registry.ts, so an exported filesystem writer would be reachable from page render code.

Note for review

All 12 works currently use exactly one citation system, so the nested refs[system] shape is exercised only by the fixture. That is deliberate — the fixture is the only place the ADR-0005 collision case exists today.

Testing

  • npm run verify:fast passes locally for routine docs, styling, or route work
  • npm run verify passes locally for registry-data, standard, release, production-build, or CI changes
  • npm run validate:data passes locally for registry-data and standard PRs

Checklist

  • Commit message follows Conventional Commits
  • I have read the Code of Conduct and the Contributing Guide
  • This PR introduces no copyrighted full text, critical apparatus, or protected translations
  • I agree my contribution is released under the licence applicable to the changed files (AGPL-3.0-or-later for code, CC BY-SA 4.0 for docs/standard, CC0 1.0 for registry data)

maehr and others added 3 commits August 24, 2026 21:21
…extrefs#84)

The only public handle on a reference is its UUIDv5, seeded from the work
key, the citation system key, and the locator (ADR-0002). A client that
knows a passage but ships no UUIDv5 implementation could reach the record
only through the /cite/ redirect, which is HTML.

Publish the same lookup as data. Each work gets one JSON map of locator to
reference UUID, grouped by citation system key, so a client resolves a
passage in two fetches and parses JSON alone.

The index comes from the reference records, not from the flat alias table:
that table mixes /cite/ aliases targeting reference IRIs with external
mapping identifiers targeting work IRIs, and the records already carry the
three fields the index groups by.

Values are bare UUIDs, not IRIs. That halves the largest file (1.15 MB
rather than 2.1 MB for tanakh), and /id/ref/{uuid}.json needs the UUID
anyway. The body is minified and carries no @context: it is a lookup
table, not JSON-LD.

Every work gets a file, including a work with no references. A client that
holds a work key must be able to tell "no references yet" from "wrong key",
and only a 200 with an empty refs object does that.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MoyDrm1kdM6tuisCHarWk2
)

data/README.md and data/.gitignore both name a compiled aliases.json. No
code wrote it. Close that gap.

writeDump() hard-coded JSONL serialisation and its media type into the
resource map, so a plain JSON resource did not fit. Split the pure halves
apart. dumpResources() builds every body, and describeResource() derives
the Frictionless bytes and sha256. writeDump() keeps the I/O and stays
private, because it derives its path from the module-level distRoot, and
compile.ts is imported by every Astro page through src/lib/registry.ts.

Sort the alias keys, so the body — and therefore its hash — depends on the
registry content alone, never on the order the compiler visited the work
files in.

Add dist/dump/aliases.json to the release workflow. It matches neither
existing glob, and fail_on_unmatched_files is set, so a tagged release
would otherwise ship a descriptor listing a resource that is not there.

The four JSONL bodies are byte-identical to before the refactor.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MoyDrm1kdM6tuisCHarWk2
Add both artifacts to the OpenAPI contract, with the WorkAliasIndex and
AliasTable schemas. Both responses declare application/json, the media type
the static host actually sends.

Correct the statements that the two new files falsify. The Collections tag
described JSON-LD collections alone. The Bulk tag claimed every /dump/
artifact arrives as application/octet-stream. The datapackage description
counted four resources. The media-type paragraph asserted that every .json
body is JSON-LD by content.

Add a "Find a reference by its locator" section to the URL layout page,
with the two-fetch procedure worked through Republic 514a.

Bring the parity statements in the standard, the authoring guide, and the
README up to date with the sixth dump artifact.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MoyDrm1kdM6tuisCHarWk2
Copilot AI lite review requested due to automatic review settings August 24, 2026 19:23

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a published, static alias lookup surface so clients can resolve {work_key, citation_system_key, locator} to a reference UUID without needing to implement UUIDv5 themselves, and it formalizes the alias table as a first-class dump resource alongside the existing JSONL exports.

Changes:

  • Add a per-work locator index endpoint at /reg/work/{key}/aliases.json built from compiled reference records.
  • Emit /dump/aliases.json and include it in the dump resource set (with bytes + sha256 in datapackage.json), with corresponding tests.
  • Document and advertise the new artifacts across the OpenAPI spec, site docs, and release workflow packaging.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/pages/reg/work/[key]/aliases.json.ts New per-work JSON locator index endpoint generated at build time.
src/pages/id/work/[key]/index.astro Advertises the locator index as an alternate link and adds a UI link to it.
src/lib/alias-index.ts Implements deterministic per-work alias index construction from reference records.
src/lib/alias-index.test.ts Unit tests covering reachability, sorting determinism, collision cases, and __proto__ locator safety.
scripts/compile.ts Adds dumpResources() / describeResource() and emits /dump/aliases.json as a described dump resource.
scripts/compile.test.ts Tests dump resource set shape, alias dump body properties, and descriptor bytes/sha256 correctness.
api/openapi.yaml Documents /reg/work/{key}/aliases.json, /dump/aliases.json, and updates bulk/collection semantics.
src/content/docs/get-started/url-layout.md Documents how clients resolve locators via the per-work index and the new bulk alias dump.
src/content/docs/get-started/authoring.md Updates data build documentation to include aliases.json in dump output.
src/content/docs/standard/versioning.md Updates dump layout and alias permanence section to reflect the published alias artifacts.
README.md Updates command table to reflect that compile:data emits aliases.json too.
AGENTS.md Updates repo layout notes to include the per-work locator index under /reg/.
.github/workflows/release.yml Ensures dist/dump/aliases.json is included in release uploads (with fail-on-unmatched intact).

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@maehr
maehr merged commit 8bcfdca into textrefs:staging Aug 24, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants