diff --git a/public/screenshot/product/falcon/1.png b/public/screenshot/product/falcon/1.png new file mode 100644 index 00000000..537d4046 Binary files /dev/null and b/public/screenshot/product/falcon/1.png differ diff --git a/public/screenshot/product/falcon/2.png b/public/screenshot/product/falcon/2.png new file mode 100644 index 00000000..df397f6e Binary files /dev/null and b/public/screenshot/product/falcon/2.png differ diff --git a/public/screenshot/product/falcon/3.png b/public/screenshot/product/falcon/3.png new file mode 100644 index 00000000..b0c8192d Binary files /dev/null and b/public/screenshot/product/falcon/3.png differ diff --git a/public/screenshot/product/falcon/4.png b/public/screenshot/product/falcon/4.png new file mode 100644 index 00000000..ad69cb5e Binary files /dev/null and b/public/screenshot/product/falcon/4.png differ diff --git a/public/screenshot/product/falcon/5.png b/public/screenshot/product/falcon/5.png new file mode 100644 index 00000000..314027fc Binary files /dev/null and b/public/screenshot/product/falcon/5.png differ diff --git a/public/screenshot/product/falcon/6.png b/public/screenshot/product/falcon/6.png new file mode 100644 index 00000000..9e5e7489 Binary files /dev/null and b/public/screenshot/product/falcon/6.png differ diff --git a/public/screenshot/product/falcon/7.png b/public/screenshot/product/falcon/7.png new file mode 100644 index 00000000..8bea59e1 Binary files /dev/null and b/public/screenshot/product/falcon/7.png differ diff --git a/public/screenshot/product/falcon/8.png b/public/screenshot/product/falcon/8.png new file mode 100644 index 00000000..3fc3b208 Binary files /dev/null and b/public/screenshot/product/falcon/8.png differ diff --git a/public/screenshot/product/falcon/9.png b/public/screenshot/product/falcon/9.png new file mode 100644 index 00000000..535d43bc Binary files /dev/null and b/public/screenshot/product/falcon/9.png differ diff --git a/src/lib/navigation.ts b/src/lib/navigation.ts index 72d093f9..4a025482 100644 --- a/src/lib/navigation.ts +++ b/src/lib/navigation.ts @@ -136,6 +136,21 @@ export const tabNavigation: NavTab[] = [ }, ] }, + { + group: 'Falcon AI', + icon: 'rocket', + items: [ + { title: 'Overview', href: '/docs/falcon-ai' }, + { + title: 'Features', + items: [ + { title: 'Using Falcon AI', href: '/docs/falcon-ai/features/chat' }, + { title: 'Skill Builder', href: '/docs/falcon-ai/features/skills' }, + { title: 'MCP Connectors', href: '/docs/falcon-ai/features/mcp-connectors' }, + ] + }, + ] + }, { group: 'Evaluation', icon: 'chart', diff --git a/src/pages/docs/falcon-ai/features/chat.mdx b/src/pages/docs/falcon-ai/features/chat.mdx new file mode 100644 index 00000000..06d7d4ef --- /dev/null +++ b/src/pages/docs/falcon-ai/features/chat.mdx @@ -0,0 +1,150 @@ +--- +title: "Using Falcon AI" +description: "Open Falcon AI from any page, ask questions, upload files, and get streaming responses with tool calls and completion cards." +--- + +## About + +Falcon AI runs as a chat interface inside the Future AGI dashboard. It can be opened as a sidebar from any page or as a full-page view for longer conversations. The sidebar stays open while you navigate between pages, so context is never lost. Conversations save automatically and can be resumed later. + +Falcon AI automatically detects what page you are on and uses it as context. Ask "why is this score low?" while viewing an evaluation, and it knows which evaluation you mean. It can also fetch content from URLs you paste, extract text from uploaded files, and stream responses with real-time tool execution. + +--- + +## Opening Falcon AI + + + + Press `Cmd+K` (Mac) or `Ctrl+K` (Windows/Linux) to open a sidebar overlay on the right side of the dashboard. It stays open as you navigate between pages. + + ![Open Falcon AI sidebar](/screenshot/product/falcon/1.png) + + + Click **Falcon AI** in the navigation sidebar to open the full-page view at `/dashboard/falcon-ai`. A conversation history panel on the left lets you search, rename, and delete past conversations. + + ![Open Falcon AI full page](/screenshot/product/falcon/2.png) + + + +--- + +## Asking questions + +Type a question in the input area and press Enter. Falcon AI detects the domain of your request and loads the right tools automatically. + +![Ask a question](/screenshot/product/falcon/3.png) + +To reference a different page than the one you are on, either navigate there first or specify it in your message: + +> "On the evaluations page, which model had the highest faithfulness score?" + +--- + +## Adding context + +Falcon AI detects page context automatically based on the current dashboard page. You can also attach entities manually by clicking **+ Add context** in the input area. Up to 5 entities can be attached at a time. Context chips appear above the input with an X to remove them. + +--- + +## Quick actions and slash commands + +On a new conversation, quick action buttons appear above the input: **Analyze with compass**, **Create custom views**, **Build a dataset**, **Create an evaluation**, **Run simulation for my agent**. They disappear after the first message. + +![Quick action buttons](/screenshot/product/falcon/4.png) + +Type `/` at the start of a message to open the command picker. All active skills, both built-in and custom, appear as slash commands. Select one to run its workflow in the current conversation. + +--- + +## File uploads + +Click the attachment button or drag files into the input area. Falcon AI extracts text content and uses it as context for your question. + +| File type | What happens | +|-----------|-------------| +| **PDF** | Text is extracted from all pages | +| **Excel / CSV** | Spreadsheet data is converted to text | +| **Word (.docx)** | Document text is extracted | +| **Images (PNG, JPG)** | Image is encoded and sent to the model for visual understanding | +| **Text / Markdown / JSON** | Content is included directly | + + + Maximum file size is 10 MB per upload. + + +--- + +## URL fetching + +Paste a URL in your message and Falcon AI automatically fetches its content. This works with: + +- **Web pages**: HTML is cleaned and converted to text +- **JSON APIs**: Response is formatted as a code block +- **GitHub raw files**: Content is included as a code block +- **Jupyter notebooks**: Code and markdown cells are extracted + +Up to 3 URLs are fetched in parallel, with a maximum of 50 KB of content per URL. + +--- + +## Following responses + +Responses stream token by token with Markdown formatting. When Falcon AI calls platform tools, collapsible cards show each step: + +- A **spinner** while the tool is running +- A **checkmark** when it completes +- A **warning icon** if it errors + +![Response with tool calls](/screenshot/product/falcon/5.png) + +When a tool creates or modifies a platform entity, a **completion card** appears with a direct link to the result. + +Falcon AI can call multiple tools in parallel when they are independent, and chains them sequentially when one depends on another. A single turn can run up to 50 tool-call iterations. + +--- + +## Stopping a response + +Click the **Stop** button in the input area while Falcon AI is streaming. The current tool execution is cancelled and the response ends at whatever has been generated so far. + +--- + +## Conversation history + + + + Click the **history** (clock) button at the top of the sidebar to see past conversations. + + + The left panel shows all conversations with search. Right-click a conversation to rename or delete it. + + + +![Conversation history](/screenshot/product/falcon/6.png) + +Conversations persist across sessions. If you close the browser and come back, your full history is available. + +--- + +## Reconnection + +If your connection drops mid-response, Falcon AI automatically replays missed events when you reconnect so you see the complete response. + +--- + +## Rate limits + +Falcon AI allows 10 messages per 60 seconds per user. If you hit the limit, wait briefly before sending the next message. + +--- + +## Next Steps + + + + Use built-in workflows or create custom slash commands. + + + Connect external tools like Linear, Slack, and GitHub. + + diff --git a/src/pages/docs/falcon-ai/features/mcp-connectors.mdx b/src/pages/docs/falcon-ai/features/mcp-connectors.mdx new file mode 100644 index 00000000..35ece8b3 --- /dev/null +++ b/src/pages/docs/falcon-ai/features/mcp-connectors.mdx @@ -0,0 +1,121 @@ +--- +title: "MCP Connectors" +description: "Connect external MCP servers to Falcon AI to use tools from services like Linear, Slack, GitHub, Sentry, and custom APIs." +--- + +## About + +Falcon AI comes with built-in tools for the Future AGI platform, but many workflows involve external services: project trackers, communication tools, monitoring systems, and internal APIs. MCP Connectors extend Falcon AI by connecting it to any server that implements the [Model Context Protocol](https://modelcontextprotocol.io). Once connected, Falcon AI discovers the server's tools and can call them during conversations alongside built-in platform tools. + +This means tasks like "create a Linear ticket for this failing evaluation" or "post this cost report to Slack" happen inside Falcon AI without switching tools. + +--- + +## Examples + +- **Project management**: Connect Linear, Jira, or Asana to create and update issues from evaluation or trace analysis. +- **Communication**: Connect Slack or email to share reports and alerts directly. +- **Monitoring**: Connect Sentry or PagerDuty to pull error context into debugging conversations. +- **Internal APIs**: Connect custom MCP servers that expose your organization's tools. + +--- + +## Adding a connector + + + + Open Falcon AI settings and go to the **Connectors** section. Click **Add Connector**. + + ![Add connector form](/screenshot/product/falcon/9.png) + + + + Fill in the connector fields: + + | Field | Required | Description | + |-------|----------|-------------| + | **Name** | Yes | Display name for the connector (e.g., "Linear", "Sentry") | + | **Server URL** | Yes | The MCP server endpoint URL | + | **Transport** | Yes | `streamable_http` (default, recommended) or `sse` (Server-Sent Events) | + | **Auth type** | Yes | How to authenticate with the server (see below) | + + + + Choose the authentication method that your MCP server requires: + + | Auth type | Fields | Description | + |-----------|--------|-------------| + | **None** | -- | No authentication required | + | **API Key** | Header name, Header value | Sends a custom header with each request (e.g., `X-API-Key: your-key`) | + | **Bearer Token** | Token | Sends `Authorization: Bearer ` with each request | + | **OAuth 2.1** | Client ID, Client secret, Auth URL, Token URL, Scopes | Full OAuth flow with automatic token refresh | + + + For OAuth connectors, Falcon AI handles the entire authorization flow. After saving the connector, click **Authenticate** to open the OAuth consent screen. Tokens are stored securely and refreshed automatically when they expire. + + + + + Click **Test Connection** to verify that Falcon AI can reach the MCP server and authenticate successfully. If the test fails, the error message is displayed so you can debug the configuration. + + + + Click **Discover Tools** to query the MCP server for its available tools. Falcon AI reads the server's tool schema and displays the list with names, descriptions, and parameter definitions. + + The discovery result is cached. Re-run discovery if the server adds new tools. + + + + Not all discovered tools need to be active. Select which tools Falcon AI should have access to from the discovered list. Only enabled tools appear in conversations. + + This is useful when a server exposes many tools but you only need a subset, keeping Falcon AI's tool set focused and reducing context window usage. + + + +--- + +## Using connector tools in chat + +Once enabled, connector tools appear in Falcon AI conversations alongside built-in platform tools. Falcon AI decides when to use them based on your request. Tool names from connectors are prefixed with the connector name to avoid collisions (e.g., `linear_create_issue`). + +**Examples:** + +> "Create a Linear ticket for the faithfulness regression we found in the last evaluation run." + +> "Post a summary of today's error spikes to the #ml-alerts Slack channel." + +> "Check Sentry for any new issues related to the summarization service." + +--- + +## Transport options + +MCP Connectors support two transport protocols: + +| Transport | How it works | When to use | +|-----------|-------------|-------------| +| **Streamable HTTP** | Standard HTTP POST requests with JSON-RPC 2.0 payloads | Default. Works with most MCP servers. | +| **SSE** (Server-Sent Events) | Long-lived HTTP connection with server-pushed events | Use when the server requires SSE transport or for streaming tool results. | + +Falcon AI automatically tries multiple endpoint paths (with and without `/mcp` suffix) to find the correct one for your server. + +--- + +## Managing connectors + +- **Edit**: Update any connector field from the Connectors settings page. Re-test and re-discover after changes. +- **Delete**: Remove a connector and all its cached tool schemas. Tools from deleted connectors are immediately unavailable in conversations. +- **Re-authenticate**: For OAuth connectors, click **Authenticate** again if the authorization has been revoked or if scopes need to change. + +--- + +## Next Steps + + + + Learn the basics of the chat interface. + + + Create custom workflows that can use connector tools. + + diff --git a/src/pages/docs/falcon-ai/features/skills.mdx b/src/pages/docs/falcon-ai/features/skills.mdx new file mode 100644 index 00000000..5fd8279b --- /dev/null +++ b/src/pages/docs/falcon-ai/features/skills.mdx @@ -0,0 +1,133 @@ +--- +title: "Skill Builder" +description: "Use built-in skills for common workflows or create custom slash commands that package multi-step instructions for your team." +--- + +## About + +The same analysis gets repeated across conversations and team members: checking regressions, generating cost reports, investigating error spikes. Skills package these workflows into reusable slash commands. Type `/` in the chat input, select a skill, and Falcon AI follows the packaged instructions with the right tools loaded. + +Falcon AI ships with six built-in skills for common workflows. You can also create custom skills scoped to your workspace. + +--- + +## Built-in skills + +These skills are available in every workspace and cannot be edited or deleted. + +### Build a Dataset + +Guides you through creating a dataset step by step. Falcon AI asks for a name, helps define columns, and walks you through adding rows, whether manually, from a file, or with synthetic generation. + +**Example**: `/build-a-dataset` → "I need a dataset of customer support tickets with columns for query, response, and sentiment." + +--- + +### Debug Traces + +Investigates traces with quantified analysis rather than vague summaries. Falcon AI reports specific error counts, latency percentile distributions, and recurring patterns across spans. + +**Example**: `/debug-traces` → "Why are we seeing timeout errors on the summarization endpoint?" + +--- + +### Compare Models + +Runs tradeoff analysis across multiple model variants. Falcon AI evaluates cost, quality, and latency side by side and highlights which model wins on each dimension. + +**Example**: `/compare-models` → "Compare GPT-4o and Claude Sonnet on our QA dataset for faithfulness and cost." + +--- + +### Run Evaluations + +Helps select the right evaluation template for your use case and explains results in context. Falcon AI picks templates based on your data type and walks through the scores. + +**Example**: `/run-evaluations` → "Evaluate the customer-support dataset for hallucination and toxicity." + +--- + +### Optimize Prompts + +Analyzes prompt versions and produces specific, actionable suggestions. Instead of generic advice, Falcon AI compares outputs across versions and points to what changed and why. + +**Example**: `/optimize-prompts` → "My summarization prompt is producing outputs that are too long. Help me tighten it." + +--- + +### Analyze Costs + +Produces cost breakdowns with exact dollar amounts and percentage savings opportunities. Falcon AI segments by model, project, and time period. + +**Example**: `/analyze-costs` → "Show me a cost breakdown for the last 30 days by model." + +--- + +## Custom skills + +Create skills specific to your team's workflows. Custom skills are scoped to the workspace and available to all workspace members. + +### Creating a skill + + + + In the Falcon AI chat input, click the **customize** button to open the skill picker. Click **Create Skill** to open the editor. + + ![Open skill editor](/screenshot/product/falcon/7.png) + + + + Fill in the skill fields: + + ![Skill editor fields](/screenshot/product/falcon/8.png) + + | Field | Required | Description | + |-------|----------|-------------| + | **Name** | Yes | Display name shown in the command picker (e.g., "Weekly Cost Review") | + | **Description** | Yes | Short description shown below the name in the command picker | + | **Icon** | No | Icon displayed next to the skill name | + | **Instructions** | Yes | The prompt that Falcon AI follows when the skill is triggered. Write these as direct instructions for the AI. | + | **Trigger phrases** | No | Phrases that activate the skill automatically when typed in a message. Press Enter after each phrase. | + + + + Skill instructions work best when they are specific and structured. Include: + + - **What to do first**: Which tools to call and in what order + - **How to present results**: Tables, comparisons, summaries + - **What to ask the user**: If the skill needs input, tell Falcon AI to ask for it + + **Example instruction for a weekly review skill:** + + ``` + 1. Get evaluation scores for all datasets in this workspace from the last 7 days. + 2. Compare each dataset's scores to the previous 7-day period. + 3. Flag any metric that dropped by more than 5%. + 4. Present results as a table with columns: Dataset, Metric, This Week, Last Week, Change. + 5. If any regressions are found, suggest which traces to investigate. + ``` + + + + Type `/` in the chat input to open the command picker. Select your skill to run it. You can also type a message after selecting the skill to provide additional context. + + Skills also trigger automatically when a message matches one of the configured trigger phrases. + + + +### Editing and deleting skills + +Open the skill picker, click an existing custom skill to open the editor. Update any field and save, or click **Delete** to remove it. Built-in skills cannot be edited or deleted. + +--- + +## Next Steps + + + + Learn the basics of the chat interface. + + + Extend Falcon AI with tools from external services. + + diff --git a/src/pages/docs/falcon-ai/index.mdx b/src/pages/docs/falcon-ai/index.mdx new file mode 100644 index 00000000..54debfb3 --- /dev/null +++ b/src/pages/docs/falcon-ai/index.mdx @@ -0,0 +1,81 @@ +--- +title: "Falcon AI" +description: "An AI copilot embedded in the Future AGI dashboard that handles platform tasks, runs analysis, and answers questions through natural language." +--- + +## About + +Falcon AI is a copilot built into the Future AGI dashboard. It has access to over 300 platform tools and can work across datasets, evaluations, traces, experiments, prompts, and admin settings through natural language. It knows what page you are on, what entity you are looking at, and acts on that context directly. + +{/* TODO: Add hero screenshot showing Falcon AI sidebar with a multi-step conversation */} + +You describe a task, Falcon AI executes it. You ask a follow-up, it goes deeper. A single conversation can span multiple features: start from an evaluation regression, drill into the failing traces, inspect the dataset behind them, and compare against a different model. + +--- + +## What Falcon AI can do + +**Analyze.** Ask questions about your data and get quantified answers, not summaries. + +> "Which eval metrics dropped this week compared to last week?" +> "What's the p95 latency for the summarization endpoint?" +> "Show me a cost breakdown by model for the last 30 days." + +**Create.** Build platform entities without leaving the chat. + +> "Create a dataset called qa-golden with columns for query, expected_answer, and context." +> "Run faithfulness and hallucination evals on the customer-support dataset." +> "Set up an A/B experiment comparing GPT-4o and Claude Sonnet on the QA dataset." + +**Debug.** Search traces, drill into spans, correlate across features. + +> "Show me traces with timeout errors from the last 24 hours." +> "Find traces where the model hallucinated and show me what context was retrieved." + +**Chain.** Work across features in a single conversation. Each follow-up builds on the previous result. + +> "The faithfulness score on run 12 dropped. Show me the failing traces, then compare the prompts used in run 11 vs run 12." + +--- + +## Key capabilities + +| Capability | Details | +|------------|---------| +| **Page-aware context** | Automatically detects the current dashboard page and entity. Ask "why is this score low?" and it knows which evaluation you mean. | +| **300+ tools** | Covers datasets, evaluations, traces, experiments, prompts, agents, simulations, cost analytics, and admin settings. | +| **Multi-step execution** | Chains up to 50 tool calls per turn. Runs independent calls in parallel, sequential calls in order. | +| **Skills** | Pre-built and custom slash commands that package multi-step workflows. Type `/` to access them. | +| **File and URL input** | Upload PDFs, CSVs, images, or paste URLs. Falcon AI extracts content and uses it as context. | +| **MCP Connectors** | Connect external services (Linear, Slack, GitHub, Sentry) so actions like "create a ticket for this regression" work in chat. | + +--- + +## Falcon AI vs MCP Server + +Future AGI has two AI interfaces for different contexts: + +| | Falcon AI | MCP Server | +|--|-----------|------------| +| **Where** | Inside the dashboard (browser) | Inside your IDE (Cursor, Claude Code, VS Code) | +| **Who** | Platform users browsing the dashboard | Developers writing code | +| **Context** | Knows what page is open, what entity is being viewed | Knows the codebase and files being edited | +| **Output** | Rich rendering: charts, tables, completion cards | Text-only responses | + +Both share the same tool layer. + +--- + +## Next Steps + + + + Open the chat, ask questions, upload files, and follow responses. + + + Use built-in workflows or create custom slash commands. + + + Connect external tools like Linear, Slack, and GitHub. + +