diff --git a/.github/workflows/docker.yml b/.github/workflows/docker.yml new file mode 100644 index 00000000..b1db1a45 --- /dev/null +++ b/.github/workflows/docker.yml @@ -0,0 +1,61 @@ +name: Docker + +on: + push: + tags: + - '@lde/search-api-server@*' + workflow_dispatch: + inputs: + version: + description: 'Published package version to build the image from' + required: true + +permissions: + contents: read + packages: write + +jobs: + publish-image: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + + - name: Determine version + id: version + run: | + if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then + echo "version=${{ inputs.version }}" >> "$GITHUB_OUTPUT" + else + echo "version=${GITHUB_REF_NAME##*@}" >> "$GITHUB_OUTPUT" + fi + + # The release run pushes the tag before it publishes to npm, so the + # version this workflow was triggered for may not be installable yet. + - name: Wait for the npm publish + run: | + for attempt in $(seq 1 30); do + if npm view "@lde/search-api-server@${{ steps.version.outputs.version }}" version > /dev/null 2>&1; then + exit 0 + fi + sleep 10 + done + echo "@lde/search-api-server@${{ steps.version.outputs.version }} never appeared on npm" >&2 + exit 1 + + - uses: docker/setup-buildx-action@v3 + + - uses: docker/login-action@v3 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - uses: docker/build-push-action@v6 + with: + context: packages/search-api-server + build-args: | + VERSION=${{ steps.version.outputs.version }} + push: true + tags: | + ghcr.io/ldelements/search-api-server:${{ steps.version.outputs.version }} + ghcr.io/ldelements/search-api-server:latest diff --git a/README.md b/README.md index 943258d4..262e9ffb 100644 --- a/README.md +++ b/README.md @@ -148,6 +148,11 @@ await pipeline.run(); npm Engine- and domain-agnostic GraphQL surface for search: builds an executable GraphQL schema from a SearchSchema at runtime and serves it as a framework-agnostic fetch handler with a self-contained playground + + @lde/search-api-server + npm + The served search API as a bootable process and prebuilt Docker image: mounts a schema-declaration module, binds the GraphQL handler to a Typesense engine, and serves /graphql plus /health from environment config + @lde/search-pipeline npm @@ -241,6 +246,8 @@ graph TD docgen search --> text-normalization search-api-graphql --> search + search-api-server --> search-api-graphql + search-api-server --> search-typesense search-typesense --> search search-typesense --> text-normalization search-typesense --> pipeline diff --git a/docs/decisions/0015-ship-the-served-search-api-as-a-docker-image-installed-from-npm.md b/docs/decisions/0015-ship-the-served-search-api-as-a-docker-image-installed-from-npm.md new file mode 100644 index 00000000..27a41717 --- /dev/null +++ b/docs/decisions/0015-ship-the-served-search-api-as-a-docker-image-installed-from-npm.md @@ -0,0 +1,66 @@ +# 15. Ship the served search API as a Docker image installed from npm + +Date: 2026-07-23 + +## Status + +Accepted + +Extends [ADR 14 (Serve the search GraphQL API with graphql-yoga)](./0014-serve-the-search-graphql-api-with-graphql-yoga.md). +Implements the image layer (layer 3) of +[#600](https://github.com/ldelements/lde/issues/600). + +## Context + +The `createSearchGraphQLHandler()` fetch handler (ADR 14) still requires a JS +host to mount it. Turnkey, non-JS and ops-driven deployments – and the +dedicated search-API pod topology #600 recommends – need a prebuilt image that +boots from configuration alone. + +Three constraints shaped the design: + +1. `@lde/search-api-graphql` deliberately names neither the domain nor the + engine, so a bootable server that binds Typesense cannot live there. +2. A mounted ES module cannot use bare imports (`import '@lde/search'` does not + resolve from a mounted path), so the schema mount cannot be authored against + the library. The SHACL + `search:` source #600 assumed does not exist yet – + schema generation is [#495](https://github.com/ldelements/lde/issues/495)’s + still-open scope. +3. Docker-building the Nx workspace inside the image duplicates the release + pipeline and produces images that do not correspond to published, versioned + artifacts. + +## Decision + +- **A separate composition package, `@lde/search-api-server`**: environment + config, schema-module loading, and a `node:http` server around the ADR 14 + handler bound to `createTypesenseSearchEngine`. The engine-agnostic handler + package stays engine-agnostic. +- **The schema mounts as a plain-data declaration module**: a `.mjs` file + default-exporting `SearchType` declarations, validated at boot by + `searchSchema()` – the exact “declarations built outside TypeScript” + path the runtime validation exists for. Optional functions (`derive`, + `transform`) remain expressible; a serving process never calls them. When + #495 delivers the SHACL + `search:` generator, mounted SHACL becomes a + second source for the same schema. +- **The image installs the published package from npm** (`npm install +@lde/search-api-server@` in a `node:alpine` base) instead of + building the workspace. Each image corresponds one-to-one to a published, + provenance-attested npm version, and the Dockerfile stays a few lines. +- **Publishing is tag-triggered**: the release run’s + `@lde/search-api-server@` tag triggers the Docker workflow, which + waits for the version to appear on npm (the release tags before it + publishes), then pushes `ghcr.io/ldelements/search-api-server:` and + `:latest`. + +## Consequences + +- Ops deploys get a turnkey `/graphql` + playground from a schema mount and + environment variables; JS hosts keep mounting the handler directly. +- The image cannot serve custom-code GraphQL fields – by design (#600): such + consumers use the handler, or build `FROM` this image. +- Image availability trails the npm publish by up to the workflow’s wait; a + missed publish (e.g. the manual first-version bootstrap) surfaces as a failed + Docker run to re-trigger via `workflow_dispatch`. +- The first image requires the package’s manual npm bootstrap + (see [Releasing a new package](../../CLAUDE.md#releasing-a-new-package)). diff --git a/package-lock.json b/package-lock.json index d2389389..31da4c2b 100644 --- a/package-lock.json +++ b/package-lock.json @@ -20436,6 +20436,10 @@ "resolved": "packages/search-api-graphql", "link": true }, + "node_modules/@lde/search-api-server": { + "resolved": "packages/search-api-server", + "link": true + }, "node_modules/@lde/search-pipeline": { "resolved": "packages/search-pipeline", "link": true @@ -35793,7 +35797,7 @@ }, "packages/dataset-registry-client": { "name": "@lde/dataset-registry-client", - "version": "0.8.6", + "version": "0.9.0", "license": "MIT", "dependencies": { "@lde/dataset": "^0.7.8", @@ -37240,7 +37244,7 @@ }, "packages/fastify-rdf": { "name": "@lde/fastify-rdf", - "version": "0.4.8", + "version": "0.5.0", "license": "MIT", "dependencies": { "@fastify/accepts": "^5.0.0", @@ -37959,11 +37963,11 @@ }, "packages/pipeline": { "name": "@lde/pipeline", - "version": "0.34.4", + "version": "0.35.0", "license": "MIT", "dependencies": { "@lde/dataset": "^0.7.8", - "@lde/dataset-registry-client": "^0.8.6", + "@lde/dataset-registry-client": "^0.9.0", "@lde/distribution-health": "^0.2.7", "@lde/distribution-probe": "^0.2.6", "@lde/sparql-importer": "^0.6.7", @@ -37985,7 +37989,7 @@ }, "packages/pipeline-console-reporter": { "name": "@lde/pipeline-console-reporter", - "version": "0.25.4", + "version": "0.26.0", "license": "MIT", "dependencies": { "chalk": "^5.4.1", @@ -37996,7 +38000,7 @@ }, "peerDependencies": { "@lde/dataset": "^0.7.8", - "@lde/pipeline": "^0.34.4" + "@lde/pipeline": "^0.35.0" } }, "packages/pipeline-console-reporter/node_modules/ansi-regex": { @@ -38176,7 +38180,7 @@ }, "packages/pipeline-shacl-sampler": { "name": "@lde/pipeline-shacl-sampler", - "version": "0.8.4", + "version": "0.9.0", "license": "MIT", "dependencies": { "@rdfjs/types": "^2.0.1", @@ -38187,7 +38191,7 @@ }, "peerDependencies": { "@lde/dataset": "^0.7.8", - "@lde/pipeline": "^0.34.4" + "@lde/pipeline": "^0.35.0" } }, "packages/pipeline-shacl-sampler/node_modules/n3": { @@ -38205,7 +38209,7 @@ }, "packages/pipeline-shacl-validator": { "name": "@lde/pipeline-shacl-validator", - "version": "0.17.4", + "version": "0.18.0", "license": "MIT", "dependencies": { "@rdfjs/types": "^2.0.1", @@ -38219,7 +38223,7 @@ }, "peerDependencies": { "@lde/dataset": "^0.7.8", - "@lde/pipeline": "^0.34.4" + "@lde/pipeline": "^0.35.0" } }, "packages/pipeline-shacl-validator/node_modules/n3": { @@ -38238,7 +38242,7 @@ }, "packages/pipeline-void": { "name": "@lde/pipeline-void", - "version": "0.32.4", + "version": "0.33.0", "license": "MIT", "dependencies": { "@rdfjs/types": "^2.0.1", @@ -38249,7 +38253,7 @@ }, "peerDependencies": { "@lde/dataset": "^0.7.8", - "@lde/pipeline": "^0.34.4" + "@lde/pipeline": "^0.35.0" } }, "packages/pipeline-void/node_modules/n3": { @@ -38305,7 +38309,7 @@ }, "packages/search-api-graphql": { "name": "@lde/search-api-graphql", - "version": "0.11.0", + "version": "0.12.0", "license": "MIT", "dependencies": { "@escape.tech/graphql-armor-cost-limit": "^2.4.3", @@ -38319,9 +38323,25 @@ "tslib": "^2.3.0" } }, + "packages/search-api-server": { + "name": "@lde/search-api-server", + "version": "0.0.0", + "license": "MIT", + "dependencies": { + "@lde/search": "^0.12.0", + "@lde/search-api-graphql": "^0.12.0", + "@lde/search-typesense": "^0.15.0", + "@whatwg-node/server": "^0.11.0", + "tslib": "^2.3.0", + "typesense": "^3.0.6" + }, + "bin": { + "search-api-server": "dist/cli.js" + } + }, "packages/search-pipeline": { "name": "@lde/search-pipeline", - "version": "0.11.2", + "version": "0.12.0", "license": "MIT", "dependencies": { "@lde/search": "^0.12.0", @@ -38331,7 +38351,7 @@ "tslib": "^2.3.0" }, "devDependencies": { - "@lde/search-typesense": "^0.14.2", + "@lde/search-typesense": "^0.15.0", "@rdfjs/types": "^2.0.1", "n3": "^2.1.1", "testcontainers": "^12.0.3", @@ -38339,7 +38359,7 @@ }, "peerDependencies": { "@lde/dataset": "^0.7.8", - "@lde/pipeline": "^0.34.4" + "@lde/pipeline": "^0.35.0" } }, "packages/search-pipeline/node_modules/n3": { @@ -38358,7 +38378,7 @@ }, "packages/search-typesense": { "name": "@lde/search-typesense", - "version": "0.14.2", + "version": "0.15.0", "license": "MIT", "dependencies": { "@lde/search": "^0.12.0", @@ -38371,7 +38391,7 @@ }, "peerDependencies": { "@lde/dataset": "^0.7.8", - "@lde/pipeline": "^0.34.4" + "@lde/pipeline": "^0.35.0" } }, "packages/search/node_modules/n3": { diff --git a/packages/search-api-server/Dockerfile b/packages/search-api-server/Dockerfile new file mode 100644 index 00000000..83dc494c --- /dev/null +++ b/packages/search-api-server/Dockerfile @@ -0,0 +1,16 @@ +# The prebuilt served search API (https://github.com/ldelements/lde/issues/600, +# layer 3): installs the published @lde/search-api-server from npm rather than +# building the workspace, so the image is small and reproducible from public, +# provenance-attested artifacts. Built by .github/workflows/docker.yml on each +# release tag of the package. +FROM node:24-alpine + +ARG VERSION=latest +ENV NODE_ENV=production +RUN npm install --global @lde/search-api-server@${VERSION} + +USER node +EXPOSE 4000 +HEALTHCHECK --interval=30s --timeout=3s \ + CMD wget --quiet --output-document=- http://127.0.0.1:${PORT:-4000}/health || exit 1 +CMD ["search-api-server"] diff --git a/packages/search-api-server/README.md b/packages/search-api-server/README.md new file mode 100644 index 00000000..de4d3da9 --- /dev/null +++ b/packages/search-api-server/README.md @@ -0,0 +1,118 @@ +# @lde/search-api-server + +The served [@lde/search](../search) API as a bootable process and prebuilt +Docker image: mount a schema-declaration module, point it at Typesense, and it +serves `/graphql` – POST execution, the self-contained playground, the SDL – +plus `/health`, with CORS and depth/cost limits on by default. + +This is the composition layer +([#600](https://github.com/ldelements/lde/issues/600), layer 3) that binds the +engine-agnostic [`@lde/search-api-graphql`](../search-api-graphql) handler +(layer 2) to the [`@lde/search-typesense`](../search-typesense) engine. Use it +for turnkey, non-JS and ops-driven deployments; a JS host that wants custom +GraphQL fields mounts the handler itself instead (or builds `FROM` this image). + +## Run + +```sh +docker run --publish 4000:4000 \ + --volume ./search-schema.mjs:/config/search-schema.mjs:ro \ + --env TYPESENSE_HOST=typesense.internal \ + --env TYPESENSE_API_KEY=search-only-key \ + ghcr.io/ldelements/search-api-server +``` + +Or without Docker (the same environment variables apply): + +```sh +npx @lde/search-api-server +``` + +## The schema module + +The mounted module default-exports the deployment’s search type declarations as +**plain data** – it must not import `@lde/search` (bare specifiers do not +resolve from a mounted file), and does not need to: the server validates the +declarations at boot, exactly as it would a SHACL generator’s output. Optional +functions (`derive`, `transform`) are allowed; they are projection-time +declarations a serving process never calls. If you must import, bundle the +module (e.g. esbuild) before mounting it. + +```js +// search-schema.mjs +export default [ + { + name: 'Dataset', + class: 'http://www.w3.org/ns/dcat#Dataset', + fields: [ + { + name: 'title', + kind: 'text', + locales: ['nl', 'en'], + output: true, + searchable: { weight: 5 }, + }, + { + name: 'keyword', + kind: 'keyword', + array: true, + facetable: true, + output: true, + }, + ], + }, +]; + +// Optional: forwarded to buildGraphQLSchema (per-type options, maxPerPage, …). +export const schemaOptions = { maxPerPage: 50 }; + +// Optional: forwarded to createTypesenseSearchEngine (collection overrides, …). +export const engineOptions = {}; +``` + +Once the SHACL + `search:` generator lands +([#495](https://github.com/ldelements/lde/issues/495)), mounted SHACL becomes +an additional source for the same schema. + +## Configuration + +| Variable | Default | Meaning | +| -------------------- | --------------------------- | -------------------------------------------------- | +| `SCHEMA_MODULE` | `/config/search-schema.mjs` | Path of the mounted schema-declaration module | +| `PORT` | `4000` | TCP port the server binds | +| `GRAPHQL_ENDPOINT` | `/graphql` | Path serving GraphQL, the playground and the SDL | +| `PLAYGROUND` | `true` | Serve the playground on GET (`false`/`0` disables) | +| `MAX_DEPTH` | handler default (15) | Query depth cap | +| `MAX_COST` | handler default (5000) | Query cost cap | +| `TYPESENSE_HOST` | **required** | Typesense host | +| `TYPESENSE_PORT` | `8108` | Typesense port | +| `TYPESENSE_PROTOCOL` | `http` | `http` or `https` | +| `TYPESENSE_API_KEY` | **required** | Use a search-only key: the server only ever reads | + +A misconfigured boot reports **all** problems in one error, not one per crash +loop. + +## Endpoints + +- `POST /graphql` – GraphQL execution. +- `GET /graphql` – the self-contained playground (no external CDN, no framing + headers, so a docs site can `