From 84e54837131642d01fdbc6721a350e997f27f927 Mon Sep 17 00:00:00 2001 From: MK Date: Sat, 18 Apr 2026 17:09:11 -0400 Subject: [PATCH 1/2] Restructure docs for useforge.ai sync pipeline MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reorganize flat docs/ directory into site-matching subdirectories: - getting-started/, core-concepts/, security/, skills/, deployment/, reference/, plus faq.md Add YAML frontmatter (title, description, order) to every doc. Strip H1 headers, breadcrumbs, and navigation lines. Split monolithic docs: - skills.md → 6 files (skill-md-format, embedded-skills, writing-custom-skills, skills-cli, contributing-a-skill, your-first-skill) - deployment.md → 4 files (docker, kubernetes, production-checklist, monitoring) - configuration.md → 3 files (configuration, forge-yaml-schema, environment-variables) - security/overview.md → 3 files (overview, trust-model, audit-logging) Create standalone pages: agent-skills-compatibility.md, faq.md --- docs/{ => core-concepts}/channels.md | 11 +- docs/{ => core-concepts}/hooks.md | 11 +- .../how-forge-works.md} | 11 +- .../memory-system.md} | 11 +- .../runtime-engine.md} | 11 +- docs/{ => core-concepts}/scheduling.md | 11 +- docs/core-concepts/skill-md-format.md | 165 +++++++++++ .../tools-and-builtins.md} | 11 +- docs/deployment.md | 114 -------- docs/deployment/docker.md | 59 ++++ docs/deployment/kubernetes.md | 36 +++ docs/deployment/monitoring.md | 59 ++++ docs/deployment/production-checklist.md | 28 ++ docs/faq.md | 102 +++++++ docs/getting-started/configuration.md | 42 +++ docs/{ => getting-started}/contributing.md | 6 +- docs/{ => getting-started}/installation.md | 11 +- .../quick-start.md} | 11 +- docs/getting-started/your-first-skill.md | 70 +++++ docs/reference/agent-skills-compatibility.md | 41 +++ .../cli-reference.md} | 11 +- docs/{ => reference}/command-integration.md | 9 +- docs/reference/environment-variables.md | 32 +++ .../forge-yaml-schema.md} | 38 +-- .../framework-plugins.md} | 9 +- .../web-dashboard.md} | 11 +- docs/security/audit-logging.md | 33 +++ .../security/{signing.md => build-signing.md} | 11 +- .../security/{egress.md => egress-control.md} | 9 +- docs/security/guardrails.md | 11 +- docs/security/overview.md | 9 +- .../{secrets.md => secret-management.md} | 11 +- docs/security/trust-model.md | 54 ++++ docs/skills/contributing-a-skill.md | 67 +++++ docs/{skills.md => skills/embedded-skills.md} | 269 +----------------- docs/skills/skills-cli.md | 23 ++ docs/skills/writing-custom-skills.md | 143 ++++++++++ 37 files changed, 1057 insertions(+), 514 deletions(-) rename docs/{ => core-concepts}/channels.md (98%) rename docs/{ => core-concepts}/hooks.md (97%) rename docs/{architecture.md => core-concepts/how-forge-works.md} (99%) rename docs/{memory.md => core-concepts/memory-system.md} (96%) rename docs/{runtime.md => core-concepts/runtime-engine.md} (98%) rename docs/{ => core-concepts}/scheduling.md (92%) create mode 100644 docs/core-concepts/skill-md-format.md rename docs/{tools.md => core-concepts/tools-and-builtins.md} (98%) delete mode 100644 docs/deployment.md create mode 100644 docs/deployment/docker.md create mode 100644 docs/deployment/kubernetes.md create mode 100644 docs/deployment/monitoring.md create mode 100644 docs/deployment/production-checklist.md create mode 100644 docs/faq.md create mode 100644 docs/getting-started/configuration.md rename docs/{ => getting-started}/contributing.md (97%) rename docs/{ => getting-started}/installation.md (75%) rename docs/{quickstart.md => getting-started/quick-start.md} (91%) create mode 100644 docs/getting-started/your-first-skill.md create mode 100644 docs/reference/agent-skills-compatibility.md rename docs/{commands.md => reference/cli-reference.md} (98%) rename docs/{ => reference}/command-integration.md (97%) create mode 100644 docs/reference/environment-variables.md rename docs/{configuration.md => reference/forge-yaml-schema.md} (66%) rename docs/{plugins.md => reference/framework-plugins.md} (97%) rename docs/{dashboard.md => reference/web-dashboard.md} (98%) create mode 100644 docs/security/audit-logging.md rename docs/security/{signing.md => build-signing.md} (88%) rename docs/security/{egress.md => egress-control.md} (99%) rename docs/security/{secrets.md => secret-management.md} (93%) create mode 100644 docs/security/trust-model.md create mode 100644 docs/skills/contributing-a-skill.md rename docs/{skills.md => skills/embedded-skills.md} (52%) create mode 100644 docs/skills/skills-cli.md create mode 100644 docs/skills/writing-custom-skills.md diff --git a/docs/channels.md b/docs/core-concepts/channels.md similarity index 98% rename from docs/channels.md rename to docs/core-concepts/channels.md index 0147484..218bc75 100644 --- a/docs/channels.md +++ b/docs/core-concepts/channels.md @@ -1,6 +1,8 @@ -# Channel Adapters - -> Part of [Forge Documentation](../README.md) +--- +title: "Channels" +description: "Bridge messaging platforms like Slack and Telegram to your AI agent." +order: 4 +--- Channel adapters bridge messaging platforms (Slack, Telegram) to your A2A-compliant agent. Each adapter normalizes platform-specific events into a common `ChannelEvent` format, forwards them to the agent's A2A server, and delivers responses back to the originating platform. @@ -227,6 +229,3 @@ type ChannelPlugin interface { 2. Implement `ChannelPlugin`. 3. Register the plugin in the channel registry. 4. Add config generation in `generateChannelConfig()` and env vars in `generateEnvVars()`. - ---- -← [Memory](memory.md) | [Back to README](../README.md) | [Security Overview](security/overview.md) → diff --git a/docs/hooks.md b/docs/core-concepts/hooks.md similarity index 97% rename from docs/hooks.md rename to docs/core-concepts/hooks.md index 3efb880..44d9318 100644 --- a/docs/hooks.md +++ b/docs/core-concepts/hooks.md @@ -1,6 +1,8 @@ -# Hooks - -> Part of [Forge Documentation](../README.md) +--- +title: "Hooks" +description: "Hook into the agent loop for logging, enforcement, and auditing." +order: 7 +--- The hook system allows custom logic to run at key points in the LLM agent loop. Hooks can observe, modify context, or block execution. @@ -142,6 +144,3 @@ exec := engine.NewLLMExecutor(engine.LLMExecutorConfig{ ``` If no `HookRegistry` is provided, an empty one is created automatically. - ---- -← [Scheduling](scheduling.md) | [Back to README](../README.md) | [Commands](commands.md) → diff --git a/docs/architecture.md b/docs/core-concepts/how-forge-works.md similarity index 99% rename from docs/architecture.md rename to docs/core-concepts/how-forge-works.md index dccc437..792090c 100644 --- a/docs/architecture.md +++ b/docs/core-concepts/how-forge-works.md @@ -1,6 +1,8 @@ -# Architecture - -> Part of [Forge Documentation](../README.md) +--- +title: "How Forge Works" +description: "Understand Forge's architecture, module system, and data flows." +order: 1 +--- Forge is a portable runtime for building and running secure AI agents from simple skill definitions. @@ -359,6 +361,3 @@ The A2A server adds: - **Request size limits** — `MaxHeaderBytes` (1 MiB) and `http.MaxBytesReader` (2 MiB) on request bodies; returns 413 on excess See [Egress Security](security/egress.md) for details. - ---- -← [Installation](installation.md) | [Back to README](../README.md) | [Skills](skills.md) → diff --git a/docs/memory.md b/docs/core-concepts/memory-system.md similarity index 96% rename from docs/memory.md rename to docs/core-concepts/memory-system.md index d3326bd..c41c6d9 100644 --- a/docs/memory.md +++ b/docs/core-concepts/memory-system.md @@ -1,6 +1,8 @@ -# Memory - -> Part of [Forge Documentation](../README.md) +--- +title: "Memory System" +description: "Session persistence, context management, and long-term memory." +order: 5 +--- Forge provides two layers of memory management: session persistence for multi-turn conversations and long-term memory for cross-session knowledge. @@ -122,6 +124,3 @@ Environment variables: | `FORGE_SESSION_MAX_AGE` | Session idle timeout, e.g. `30m`, `1h` (default: `30m`) | | `FORGE_MEMORY_LONG_TERM` | Set `true` to enable long-term memory | | `FORGE_EMBEDDING_PROVIDER` | Override embedding provider | - ---- -← [Runtime](runtime.md) | [Back to README](../README.md) | [Channels](channels.md) → diff --git a/docs/runtime.md b/docs/core-concepts/runtime-engine.md similarity index 98% rename from docs/runtime.md rename to docs/core-concepts/runtime-engine.md index 7c0d4dd..f0b5c0c 100644 --- a/docs/runtime.md +++ b/docs/core-concepts/runtime-engine.md @@ -1,6 +1,8 @@ -# LLM Runtime Engine - -> Part of [Forge Documentation](../README.md) +--- +title: "Runtime Engine" +description: "The LLM runtime engine powering tool calling, memory, and hooks." +order: 6 +--- The runtime engine powers `forge run` — executing agent tasks via LLM providers with tool calling, conversation memory, and lifecycle hooks. @@ -254,6 +256,3 @@ The runner registers five hook groups: logging, audit, progress, global guardrai ## Streaming The LLM tool-calling loop runs non-streaming internally. `ExecuteStream` calls `Execute` and emits the final response on a channel. However, the **UI chat proxy** (`forge-ui/chat.go`) streams A2A SSE events to the browser in real-time — `status` events carry incremental text, `progress` events carry tool execution updates, and `result` events carry the final response. The frontend renders text and tool progress as each event arrives. - ---- -← [Tools](tools.md) | [Back to README](../README.md) | [Memory](memory.md) → diff --git a/docs/scheduling.md b/docs/core-concepts/scheduling.md similarity index 92% rename from docs/scheduling.md rename to docs/core-concepts/scheduling.md index 23bae69..a51b607 100644 --- a/docs/scheduling.md +++ b/docs/core-concepts/scheduling.md @@ -1,6 +1,8 @@ -# Scheduling (Cron) - -> Part of [Forge Documentation](../README.md) +--- +title: "Scheduling" +description: "Built-in cron scheduler for recurring agent tasks." +order: 8 +--- Forge includes a built-in cron scheduler for recurring tasks, configurable in `forge.yaml` or created dynamically by the agent at runtime. @@ -53,6 +55,3 @@ When a schedule includes `channel` and `channel_target`, the agent's response is - **Persistence**: Schedules are stored in `.forge/memory/SCHEDULES.md` and survive restarts - **History**: The last 50 executions are recorded with status, duration, and correlation IDs - **Audit events**: `schedule_fire`, `schedule_complete`, `schedule_skip`, `schedule_modify` - ---- -← [Guardrails](security/guardrails.md) | [Back to README](../README.md) | [Hooks](hooks.md) → diff --git a/docs/core-concepts/skill-md-format.md b/docs/core-concepts/skill-md-format.md new file mode 100644 index 0000000..89d1086 --- /dev/null +++ b/docs/core-concepts/skill-md-format.md @@ -0,0 +1,165 @@ +--- +title: "SKILL.md Format" +description: "Define agent skills using the SKILL.md format with YAML frontmatter and tool definitions." +order: 2 +--- + +## SKILL.md Format + +Skills are defined in Markdown files inside `skills//SKILL.md`. Each file supports optional YAML frontmatter and two body formats. + +```markdown +--- +name: weather +icon: 🌤️ +category: utilities +tags: + - weather + - forecast + - api +description: Weather data skill +metadata: + forge: + requires: + bins: + - curl + env: + required: [] + one_of: [] + optional: [] +--- +## Tool: weather_current + +Get current weather for a location. + +**Input:** location (string) - City name or coordinates +**Output:** Current temperature, conditions, humidity, and wind speed + +## Tool: weather_forecast + +Get weather forecast for a location. + +**Input:** location (string), days (integer: 1-7) +**Output:** Daily forecast with high/low temperatures and conditions +``` + +Each `## Tool:` heading defines a tool the agent can call. The frontmatter declares binary dependencies and environment variable requirements. Skills compile into JSON artifacts and prompt text during `forge build`. + +### YAML Frontmatter + +Top-level fields: + +| Field | Required | Description | +|-------|----------|-------------| +| `name` | yes | Skill identifier (kebab-case) | +| `icon` | yes | Emoji displayed in the TUI skill picker | +| `category` | yes | Grouping for `forge skills list --category` (e.g., `sre`, `developer`, `research`, `utilities`) | +| `tags` | yes | Discovery keywords for `forge skills list --tags` (kebab-case) | +| `description` | yes | One-line summary | + +The `metadata.forge.requires` block declares runtime dependencies: + +- **`bins`** — Binary dependencies that must be in `$PATH` at runtime +- **`env.required`** — Environment variables that must be set +- **`env.one_of`** — At least one of these environment variables must be set +- **`env.optional`** — Optional environment variables for extended functionality + +Frontmatter is parsed by `ParseWithMetadata()` in `forge-skills/parser/parser.go` and feeds into the compilation pipeline. + +### Legacy List Format + +```markdown +# Agent Skills + +- translate +- summarize +- classify +``` + +Single-word list items (no spaces, max 64 characters) create name-only skill entries. This format is simpler but provides less metadata. + +## Skill Registry + +Forge ships with a built-in skill registry. Add skills to your project with a single command: + +```bash +# Add a skill from the registry +forge skills add tavily-research + +# Validate skill requirements +forge skills validate + +# Audit skill security +forge skills audit --embedded +``` + +`forge skills add` copies the skill's SKILL.md and any associated scripts into your project's `skills/` directory. It validates binary and environment requirements, checks for existing values in your environment, `.env` file, and encrypted secrets, and prompts only for truly missing values with a suggestion to use `forge secrets set` for sensitive keys. If the skill declares `egress_domains`, they are automatically merged into the `forge.yaml` `egress.allowed_domains` list (deduplicated and sorted). + +## Skills as First-Class Tools + +Script-backed skills are automatically registered as **first-class LLM tools** at runtime. When a skill has scripts in `skills/scripts/`, Forge: + +1. Parses the skill's SKILL.md for tool definitions, descriptions, and input schemas +2. Creates a named tool for each `## Tool:` entry (e.g., `tavily_research` becomes a tool the LLM can call directly) +3. Executes the skill's shell script with JSON input when the LLM invokes it + +This means the LLM sees skill tools alongside builtins like `web_search` and `http_request` — no generic `cli_execute` indirection needed. + +For skills **without** scripts (binary-backed skills like `k8s-incident-triage`), Forge injects the full skill instructions into the system prompt. The complete SKILL.md body — including triage steps, detection heuristics, output structure, and safety constraints — is included inline so the LLM follows the skill protocol without needing an extra tool call. Skills are invoked via `cli_execute` with the declared binary dependencies. + +``` +┌─────────────────────────────────────────────────┐ +│ LLM Tool Registry │ +├─────────────────┬───────────────────────────────┤ +│ Builtins │ web_search, http_request │ +│ Skill Tools │ tavily_research, codegen_* │ ← auto-registered from scripts +│ read_skill │ load any SKILL.md on demand │ +│ cli_execute │ run approved binaries │ +├─────────────────┴───────────────────────────────┤ +│ System Prompt: full skill instructions inline │ ← binary-backed skills +└─────────────────────────────────────────────────┘ +``` + +## Skill Execution Security + +Skill scripts run in a restricted environment via `SkillCommandExecutor`: + +- **Isolated environment**: Only `PATH`, `HOME`, and explicitly declared env vars are passed through +- **OAuth token resolution**: When `OPENAI_API_KEY` is set to `__oauth__`, the executor resolves OAuth credentials and injects the access token, `OPENAI_BASE_URL`, and the configured model as `REVIEW_MODEL` +- **Configurable timeout**: Each skill declares a `timeout_hint` in its YAML frontmatter (e.g., 300s for research) +- **No shell execution**: Scripts run via `bash