#1045: ✨ Complete MCP Server Scaffold by mistryrn · Pull Request #1065 · overture-stack/arranger

mistryrn · 2026-05-20T21:43:16Z

Summary

Implements a working MCP server for the Arranger introspection API, built on @modelcontextprotocol/sdk v1.x with the Streamable HTTP transport. Adds MCP Resources and Tools for the Arranger introspection endpoints, and supporting unit tests and integration tests. Also adds Docker build targets for the MCP Server to both the local and Jenkins Dockerfiles.

Issues

Description of Changes

MCP Server

Adds @modelcontextprotocol/sdk, zod (v3 for consistency with repo), and pino as dependencies
Adds configuration via Zod-validated environment variables
Adds Arranger connection validation at startup
Adds Pino logger
Adds an Express server with a configurable MCP endpoint for Streamable HTTP transport
Adds MCP Resources for the Arranger introspection endpoints:
- arranger-server-introspection → arranger://introspection/server
- arranger-sqon-schema → arranger://introspection/sqon
- arranger-catalog-fields → arranger://introspection/catalog/{catalogId}
Adds MCP Tools for the Arranger introspection endpoints:
- list-catalogs
- get-sqon-schema
- get-catalog-fields
Adds unit tests for the configuration validation and connection validation functions
Adds integration tests for the MCP Server
Adds Dockerfile build targets for the MCP Server

Special Instructions

Before running these changes, you will need to:

Install the new dependencies:

# from project root
npm ci

(Optional) Re-build the modules:

# from project root
npm run modules:build

Configure environment variables for the mcp-server app:

# from project root
cd apps/mcp-server
cp .env.schema .env

Testing Instructions

Local Arranger

To test the MCP Server against a local instance of Arranger Server:

Confirm your apps/mcp-server/.env config aligns with your local Arranger server
Ensure ES and Arranger server are running:

# from project root
# start ES (note: you may need to seed ES with `make seed-es` after if this is your first time)
make start-es
# start arranger server (config may vary)
ES_INDEX=file_centric DOCUMENT_TYPE=file CONFIGS_PATH=$(pwd)/docker/server npm run dev:server

Start the MCP Server:

# from project root
npm run mcp-server:dev

Start the MCP Inspector:

# from project root
npm run mcp-server:inspect

You can then open the MCP Inspector URL in your web browser (http://localhost:6274/?MCP_PROXY_AUTH_TOKEN={AUTH_TOKEN}), connect to the MCP Server via Streamable HTTP, and test the Resources and Tools.

Remote Arranger

To test against a remote instance of Arranger Server:

Update the ARRANGER_BASE_URL and ARRANGER_CATALOGUES in your MCP Server .env file to point to and reflect the state of your remote Arranger.
Follow steps 3-5 of the local testing instructions.

LM Studio

To test with LM Studio instead of MCP Inspector:

Follow the LM Studio instructions to add an MCP server configuration: https://lmstudio.ai/docs/app/mcp
- Provide the config JSON in apps/mcp-server/mcp-inspector.json

Docker

To test local Docker builds:

Build a local image of the MCP Server image:

# from project root
docker build --target mcp-server -f docker/Dockerfile.local -t arranger/mcp-server:local .

Run the image:

# from project root
docker run --env-file apps/mcp-server/.env -p 3100:3100 arranger/mcp-server:local

The server will be running at http://localhost:3100 and you can test it with MCP Inspector or LM Studio as per the previous instructions.

Note

To test the Docker image with a local Arranger Server, update your .env to use host.docker.internal instead of localhost as part of the ARRANGER_BASE_URL.

Readiness Checklist

Self Review
- I have performed a self review of code
- I have run the application locally and manually tested the feature
- I have checked all updates to correct typos and misspellings
Formatting
- Code follows the project style guide
- Autmated code formatters (ie. Prettier) have been run
Local Testing
- Successfully built all packages locally
- Successfully ran all test suites, all unit and integration tests pass
Updated Tests
- Unit and integration tests have been added that describe the bug that was fixed or the features that were added
Documentation
- All new environment variables added to .env.schema file and documented in the README
- All changes to server HTTP endpoints have open-api documentation
- All new functions exported from their module have TSDoc comment documentation

Implements a working MCP server for the Arranger introspection API, built on `@modelcontextprotocol/sdk` v1.x with the Streamable HTTP transport. * Adds configuration via Zod-validated environment variables * Adds Arranger connection validation at startup * Adds Pino logging * Adds an Express server with the configured MCP endpoint for Streamable HTTP transport * Adds MCP Resources for the Arranger introspection endpoints: * `arranger://introspection/server` * `arranger://introspection/sqon` * `arranger://introspection/catalog/{catalogId}` * Adds MCP Tools for the Arranger introspection endpoints: * `list-catalogs` * `get-sqon-schema` * `get-catalog-fields`

* Update `fieldShape` Zod schema in MCP tools to reflect new Arranger introspection response types * Update MCP Server validation unit tests to reflect new Arranger introspection response types * Update MCP Server integration tests to reflect new Arranger introspection response types

* Remove unnecessary `pnpm-lock.yaml`, this project exclusively uses `npm` not `pnpm`

* Moved `tsx` from `devDependencies` to `dependencies` in `apps/mcp-server/package.json` for the Jenkins Docker build (which runs `npm ci --omit=dev`)

* Updates `Dockerfile.local` and `Dockerfile.jenkins` (and their corresponding `.dockerignore` files) to include build targets for `mcp-server`

mistryrn

Leaving some comments calling out some of the decisions and TODOs. One other TODO for me is to give this PR a "catalogue"-pass (i.e. there are still catalog vs catalogue inconsistencies in some of the new code I've added, I need to clean that up)

mistryrn · 2026-05-28T18:34:37Z

+// This code was adapted from the official MCP Server "Streamable HTTP" example:
+// https://github.com/modelcontextprotocol/typescript-sdk/blob/v1.x/src/examples/server/simpleStreamableHttp.ts
+export const createHttpApp = (config: ArrangerMcpConfig, serverFactory: () => McpServer): McpHttpApp => {


As the comment states, this code is based off of the MCP "Streamable HTTP" example: https://github.com/modelcontextprotocol/typescript-sdk/blob/v1.x/src/examples/server/simpleStreamableHttp.ts

I haven't scrutinized it much myself yet as I've been focused on building the required functionality so far, but now that everything is up and running I'll take a closer look. Open to suggestions/improvements 🙏

mistryrn · 2026-05-28T18:36:06Z

-export const buildCatalogResources = (
-	serverIntrospection: ArrangerServerIntrospection,
-): McpResourceDefinition[] =>
-	Object.keys(serverIntrospection.catalogs).map((catalogId) => ({
-		description: `Field-level introspection for the "${catalogId}" catalog.`,
-		name: `arranger_catalog_${catalogId}`,
-		uri: `arranger://introspection/catalog/${catalogId}`,
-	}));


Replaced the custom resource definitions in here with usages of the MCP SDK

mistryrn · 2026-05-28T18:36:57Z

+import { type McpServer } from '@modelcontextprotocol/sdk/server/mcp';
+import { z } from 'zod';

-export const buildFoundationTools = (): McpToolDefinition[] => [


Similar to previous comment on resources.ts, replaced the custom tool definitions in here with usages of the MCP SDK

mistryrn · 2026-05-28T18:44:50Z

+const fieldShape = z.object({
+	displayName: z.string(),
+	type: z.string(),
+	unit: z.string().nullable().optional(),
+});


Couldn't use the CatalogFieldIntrospection type exported from search-server here, as the MCP API wants Zod schemas (not types) for its inputSchema/outputSchema params.

Possible future enhancement/suggestion for the introspection types (https://github.com/overture-stack/arranger/blob/main/apps/search-server/src/introspection/types.ts#L38-L42):

define as Zod schemas, then infer TS types from those schemas, giving us both options

e.g.,

export const CatalogFieldIntrospectionSchema = z.object({ displayName: z.string(), type: z.string(), unit: z.string().nullable().optional(); }); export type CatalogFieldIntrospection = z.infer<typeof CatalogFieldIntrospectionSchema>;

noted! adding a tech-debt item to be addressed soon

mistryrn · 2026-05-28T19:03:14Z

+// In-memory event store implementation from the MCP TypeScript SDK examples:
+// https://github.com/modelcontextprotocol/typescript-sdk/blob/v1.x/src/examples/shared/inMemoryEventStore.ts
+// TODO: Replace with a persistent storage solution for production use


Calling this out as well 🙏

adding it to the tech-debt doc on my local (so that it's visible outside this file), to be committed later.

Severity: medium (state is lost on restart, no resumability for clients in production).

justincorrigible · 2026-05-29T13:19:25Z

+| -------------------------- | ----------------------------------------------------------------------- | -------- | ------------ | ----------------------- |
+| `ARRANGER_BASE_URL`        | URL for the Arranger Server                                             | `string` | **Required** | `http://localhost:5050` |
+| `ARRANGER_CATALOGUES`      | Comma-separated list of Arranger catalogues to expose to the MCP Server | `string` | **Required** | `server`                |
+| `ARRANGER_REQUEST_TIMEOUT` | Timeout for requests to Arranger                                        | `number` | Optional     | `10_0000`               |


justincorrigible · 2026-05-29T13:20:57Z

+| `ARRANGER_REQUEST_TIMEOUT` | Timeout for requests to Arranger                                        | `number` | Optional     | `10_0000`               |
+| `MCP_HOST`                 | Host URL for the MCP server                                             | `string` | Optional     | `0.0.0.0`               |
+| `MCP_PORT`                 | Port the MCP Server will listen for requests on                         | `number` | Optional     | `3100`                  |
+| `MCP_PATH`                 | Endpoint for the MCP Stremable HTTP transport                           | `string` | Optional     | `/mcp`                  |


justincorrigible · 2026-05-29T13:29:37Z

+		logger.info(`MCP server running at http://${host}:${port}${path}`);
+	});
+
+	process.on('SIGINT', async () => {


docker stop sends SIGTERM first; and containers that don't handle it will sit until the 10s SIGKILL timeout. SIGTERM should run the same graceful shutdown.
something along the lines of this is what needs to happen here

const shutdown = async () => { logger.info('Shutting down server...'); await shutdownTransports(); logger.info('Server shutdown complete.'); process.exit(0); }; process.on('SIGINT', shutdown); process.on('SIGTERM', shutdown);

justincorrigible · 2026-05-29T13:32:39Z

+		"@modelcontextprotocol/sdk": "^1.29.0",
+		"cors": "^2.8.6",
+		"dotenv": "^16.6.1",
+		"express": "^4.21.2",


we may need to bump this to ^5 across the repo, because of @mcp/sdk's internal express dep version... importing the types for 4 could lead to maintenance complications wherever the API changes drift apart

TODO?

justincorrigible · 2026-05-29T14:36:07Z

+		"prettier": "^3.4.2",
+		"tsx": "^4.19.3"
+	},
+	"private": "true",


while npm takes both/either, switching to bool for consistency with all other modules/packages in this repo

Suggested change

"private": "true",

"private": true,

justincorrigible · 2026-05-29T14:42:21Z

+			const structured = { catalogId: data.catalogId, fields: data.fields };
+			return {
+				content: [{ type: 'text', text: JSON.stringify(structured, null, 2) }],
+				structuredContent: structured,


not wrong, strictly speaking, but worth noting for future implementation:
structuredContent here makes this tool machine-readable for MCP clients... the other two tools return plain text; this inconsistency will matter when clients start consuming the these programmatically rather than via MCP Inspector

justincorrigible · 2026-05-29T15:02:19Z

+					if (sid && transports[sid]) {
+						logger.info(`Transport closed for session ${sid}, removing from transports map`);
+						// eslint-disable-next-line @typescript-eslint/no-dynamic-delete
+						delete transports[sid];


while I see you nicely cleaning transports on delete and shutdown, the entry will persist if the connection is closed poorly (e.g. drop, crash).

to prevent PR scope creep, I'm adding a relevant item in the tech-debt doc, to do timestap tracking and expiration.
e.g.

const transports: Record<string, { transport: StreamableHTTPServerTransport; lastSeenAt: number }> = {}; // update lastSeenAt on every request that hits an existing session // setInterval every 5 min, evict anything idle > 30 min

justincorrigible · 2026-05-29T16:26:02Z

@@ -1,27 +1,58 @@
-import type { McpToolDefinition } from './types.js';
+import { type McpServer } from '@modelcontextprotocol/sdk/server/mcp';
+import { z } from 'zod';


let's try to keep consistent use of z as zod imports, to avoid single letter variable names, please

mistryrn added 12 commits May 20, 2026 16:22

📝 Update MCP Server README

04a2e9f

✅ Add Unit Tests for validateArrangerConnection

6ebb16c

✅ Add Unit Tests for createArrangerMcpConfig

38b9205

✅ Add Integration Tests for MCP Server

a3a24d0

💡 Update comments in MCP Server integration tests for clarity

9f26b7f

Merge branch 'main' into feat/1045-complete-mcp-server-scaffold

0b5f1ac

Merge branch 'main' into feat/1045-complete-mcp-server-scaffold

c1de31e

🔥 Remove pnpm-lock.yaml

1bd48fd

* Remove unnecessary `pnpm-lock.yaml`, this project exclusively uses `npm` not `pnpm`

💚 Move MCP Server tsx from devDependencies to dependencies

3d1503a

* Moved `tsx` from `devDependencies` to `dependencies` in `apps/mcp-server/package.json` for the Jenkins Docker build (which runs `npm ci --omit=dev`)

👷 Add Dockerfile target for mcp-server

7d22617

* Updates `Dockerfile.local` and `Dockerfile.jenkins` (and their corresponding `.dockerignore` files) to include build targets for `mcp-server`

mistryrn commented May 28, 2026

View reviewed changes

mistryrn marked this pull request as ready for review May 28, 2026 19:06

mistryrn requested a review from justincorrigible May 28, 2026 20:13

justincorrigible reviewed May 29, 2026

View reviewed changes

justincorrigible force-pushed the main branch 3 times, most recently from 879e561 to 6de99b3 Compare May 30, 2026 01:47

	\| `ARRANGER_REQUEST_TIMEOUT` \| Timeout for requests to Arranger \| `number` \| Optional \| `10_0000` \|
	\| `ARRANGER_REQUEST_TIMEOUT_MS` \| Timeout for requests to Arranger \| `number` \| Optional \| `10_000` \|

	\| `MCP_PATH` \| Endpoint for the MCP Stremable HTTP transport \| `string` \| Optional \| `/mcp` \|
	\| `MCP_PATH` \| Endpoint for the MCP Streamable HTTP transport \| `string` \| Optional \| `/mcp` \|

Conversation

mistryrn commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Issues

Description of Changes

MCP Server

Special Instructions

Testing Instructions

Local Arranger

Remote Arranger

LM Studio

Docker

Readiness Checklist

Uh oh!

mistryrn left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

justincorrigible May 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

justincorrigible May 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mistryrn commented May 20, 2026 •

edited

Loading

justincorrigible May 29, 2026 •

edited

Loading

justincorrigible May 29, 2026 •

edited

Loading