Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/generate-toolkit-docs.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ This workflow regenerates toolkit JSON and opens a PR with the changes. It can b

Required secrets:

- `ENGINE_API_URL`
- `ENGINE_API_URL` (public catalog API host)
- `ANTHROPIC_API_KEY`

Optional secrets:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/generate-toolkit-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ jobs:
--preserve-last-known-good \
--verbose \
--api-source public-catalog \
--tool-metadata-url "$ENGINE_API_URL" \
--api-url "$ARCADE_API_URL" \
--llm-provider anthropic \
--llm-model "$ANTHROPIC_MODEL" \
--llm-api-key "$ANTHROPIC_API_KEY" \
Expand All @@ -84,7 +84,7 @@ jobs:
echo "succeeded=true" >>"$GITHUB_OUTPUT"
working-directory: toolkit-docs-generator
env:
ENGINE_API_URL: ${{ secrets.ENGINE_API_URL }}
ARCADE_API_URL: ${{ secrets.ENGINE_API_URL }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
ANTHROPIC_MODEL: ${{ secrets.ANTHROPIC_MODEL || 'claude-sonnet-4-6' }}
# Stronger model for the secret-coherence editor. Keeps
Expand Down
3 changes: 0 additions & 3 deletions toolkit-docs-generator/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,6 @@ it. The sidebar sync writes navigation only, and never touches toolkit JSON.
### Data sources

- `PublicCatalogApiSource` fetches tool metadata from the Engine public catalog API.
- `EngineApiSource` fetches tool metadata from the authenticated Engine API (deprecated).
- `ArcadeApiSource` fetches tool metadata from the Arcade API (deprecated).
- `DesignSystemMetadataSource` loads toolkit metadata from `@arcadeai/design-system`.
- `MarkdownCurationSource` compiles documentation chunks, import declarations,
and subpages from the configured curation directory. When configured, that
Expand Down Expand Up @@ -140,7 +138,6 @@ public, read-only values configured through these Vercel environment variables:
## Key files

- `src/sources/public-catalog-api.ts` — tool metadata from Engine public catalog
- `src/sources/engine-api.ts` — tool metadata from authenticated Engine API (deprecated)
- `src/sources/markdown-curation.ts` — Markdown and MDX curation compiler
([format reference](CURATION.md))
- `src/sources/toolkit-data-source.ts` — unified data source
Expand Down
3 changes: 1 addition & 2 deletions toolkit-docs-generator/CURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -251,8 +251,7 @@ generation PR, or generate that one toolkit with Engine credentials and run
```bash
../node_modules/.bin/tsx src/cli/index.ts generate \
--providers "GoogleFlights" \
--tool-metadata-url "$ENGINE_API_URL" \
--tool-metadata-key "$ENGINE_API_KEY" \
--api-url "$ARCADE_API_URL" \
--custom-sections ./curation \
--skip-examples --skip-summary --skip-secret-coherence \
--output data/toolkits
Expand Down
17 changes: 6 additions & 11 deletions toolkit-docs-generator/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ It runs these steps:

Required secrets:

- `ENGINE_API_URL`
- `ENGINE_API_URL` (API host for the public catalog; rename to `ARCADE_API_URL` in a follow-up)
- `ANTHROPIC_API_KEY` for examples, summaries, and secret-coherence edits

Optional secrets:
Expand Down Expand Up @@ -107,8 +107,7 @@ The summary generator is configured to **never list OAuth scopes** in the genera
```bash
pnpm dlx tsx src/cli/index.ts generate \
--providers "Github" \
--tool-metadata-url "$ENGINE_API_URL" \
--tool-metadata-key "$ENGINE_API_KEY" \
--api-url "$ARCADE_API_URL" \
--llm-provider openai \
--llm-model gpt-4.1-mini \
--llm-api-key "$OPENAI_API_KEY" \
Expand Down Expand Up @@ -138,8 +137,7 @@ Generate a single toolkit:
```bash
pnpm dlx tsx src/cli/index.ts generate \
--providers "Github:1.0.0" \
--tool-metadata-url "$ENGINE_API_URL" \
--tool-metadata-key "$ENGINE_API_KEY" \
--api-url "$ARCADE_API_URL" \
--llm-provider openai \
--llm-model gpt-4.1-mini \
--llm-api-key "$OPENAI_API_KEY" \
Expand All @@ -152,8 +150,7 @@ Generate all toolkits:
pnpm dlx tsx src/cli/index.ts generate \
--all \
--skip-unchanged \
--tool-metadata-url "$ENGINE_API_URL" \
--tool-metadata-key "$ENGINE_API_KEY" \
--api-url "$ARCADE_API_URL" \
--llm-provider openai \
--llm-model gpt-4.1-mini \
--llm-api-key "$OPENAI_API_KEY" \
Expand All @@ -165,8 +162,7 @@ Generate without LLM output:
```bash
pnpm dlx tsx src/cli/index.ts generate \
--providers "Asana:0.1.3" \
--tool-metadata-url "$ENGINE_API_URL" \
--tool-metadata-key "$ENGINE_API_KEY" \
--api-url "$ARCADE_API_URL" \
--skip-examples \
--skip-summary \
--output data/toolkits
Expand Down Expand Up @@ -231,8 +227,7 @@ deletes it and rebuilds `index.json`.
- `--all` generate all toolkits
- `--providers` generate a subset of toolkits
- `--skip-unchanged` only write changed toolkits
- `--api-source` select `public-catalog` (default with `ENGINE_API_URL`), `tool-metadata`
(deprecated; requires `ENGINE_API_KEY`), `list-tools` (deprecated), or `mock`
- `--api-source` select `public-catalog` (default with `ARCADE_API_URL` or `ENGINE_API_URL`) or `mock`
- `--previous-output` compare against a previous output directory
- `--custom-sections` load an authoritative Markdown/MDX curation directory
- `--skip-examples`, `--skip-summary` disable LLM steps
Expand Down
33 changes: 8 additions & 25 deletions toolkit-docs-generator/src/cli/api-source.ts
Original file line number Diff line number Diff line change
@@ -1,21 +1,13 @@
export type ApiSource =
| "public-catalog"
| "list-tools"
| "tool-metadata"
| "mock";
export type ApiSource = "public-catalog" | "mock";

type ApiSourceOptions = {
apiSource?: string;
toolMetadataUrl?: string;
toolMetadataKey?: string;
apiUrl?: string;
};

const EXPLICIT_API_SOURCES: Record<string, ApiSource> = {
"public-catalog": "public-catalog",
public: "public-catalog",
"list-tools": "list-tools",
engine: "tool-metadata",
"tool-metadata": "tool-metadata",
mock: "mock",
};

Expand All @@ -26,36 +18,27 @@ const resolveExplicitApiSource = (apiSource: string): ApiSource => {
}

throw new Error(
`Invalid --api-source "${apiSource}". Use "public-catalog", "list-tools", "tool-metadata", or "mock".`
`Invalid --api-source "${apiSource}". Use "public-catalog" or "mock".`
);
};

const resolveAutoDetectedApiSource = (options: ApiSourceOptions): ApiSource => {
const hasToolMetadataKey = !!(
options.toolMetadataKey ?? process.env.ENGINE_API_KEY
);
const hasToolMetadataUrl = !!(
options.toolMetadataUrl ?? process.env.ENGINE_API_URL
);

if (hasToolMetadataKey && hasToolMetadataUrl) {
return "tool-metadata";
}
const hasApiUrl = !!(options.apiUrl ?? resolveApiBaseUrlFromEnv());

if (hasToolMetadataUrl) {
if (hasApiUrl) {
return "public-catalog";
}

return "mock";
};

export const resolveApiBaseUrlFromEnv = (): string | undefined =>
process.env.ARCADE_API_URL ?? process.env.ENGINE_API_URL;

export const resolveApiSource = (options: ApiSourceOptions): ApiSource => {
if (options.apiSource) {
return resolveExplicitApiSource(options.apiSource);
}

return resolveAutoDetectedApiSource(options);
};

export const isDeprecatedApiSource = (apiSource: ApiSource): boolean =>
apiSource === "tool-metadata" || apiSource === "list-tools";
Loading
Loading