diff --git a/README.md b/README.md index 4aaf0ae..58d9a16 100644 --- a/README.md +++ b/README.md @@ -21,22 +21,32 @@ policy enforcement, bounded recovery, and production-friendly delivery interface --- -QueryForge is a local-first AI data analytics platform built around one principle: +QueryForge is a local-first, domain-first AI data analytics platform built around +one principle: **generated SQL should be governed like application code, not trusted like prose**. -It combines natural-language-to-SQL with semantic contracts, AST-level security, -read-only execution, multi-candidate selection, repair budgets, and complete run -artifacts. The result is a reference implementation for building AI data systems -that are useful, inspectable, and difficult to misuse. +Users create or select a data domain first—such as retail, finance, product, or +the bundled Anime Streaming sample—then onboard that domain's data, review its +semantic contract, and ask questions inside the same governance boundary. +QueryForge combines that workflow with natural-language-to-SQL, AST-level +security, read-only execution, multi-candidate selection, repair budgets, and +complete run artifacts. > QueryForge currently targets SQLite and controlled environments. It is a > portfolio-grade reference architecture, not a multi-tenant analytics service. ## Product Tour +
+ QueryForge Data Domain Center for creating and selecting isolated business contexts + Data Domain Center — create or select a governed context before adding data or semantics. +
+ +
+
QueryForge Studio overview with semantic health, business metrics, and engagement trends - Workspace overview — live semantic health, governed metrics, and anime-platform activity. + Domain overview — the Anime Streaming dataset is shown as one selected sample, not the platform identity.

@@ -57,6 +67,7 @@ delivery loop: | --- | --- | | Trust the generated SQL | Parse with SQLGlot and enforce a named policy before execution | | Keep business meaning consistent | Define metrics, dimensions, grain, and join paths in YAML | +| Prevent context from leaking | Scope sources, semantic contracts, policies, and run history to a selected data domain | | Recover from imperfect output | Reflect, repair, and retry within explicit budgets | | Handle harder questions | Use bounded schema discovery and parallel SQL candidates | | Trace what happened | Persist run state, policy decisions, quality evidence, and artifacts | @@ -127,9 +138,10 @@ queryforge \ ### Explore QueryForge Studio -The repository now includes a complete visual workspace for onboarding data, -reviewing the required semantic layer, asking governed questions, inspecting SQL -and Trust Trace evidence, and auditing run history. +The repository includes a complete visual workspace for creating and switching +data domains, onboarding domain-owned data, reviewing the required semantic +layer, asking governed questions, inspecting SQL and Trust Trace evidence, and +auditing domain-scoped run history. ```bash # Terminal 1: QueryForge API @@ -141,9 +153,10 @@ make web-install make web-dev ``` -Open . If the Python API is offline, the interface -automatically uses a deterministic demo response so every screen remains -explorable. See the [Studio guide](docs/studio.md). +Open . Start in **Data Domains**, select the bundled Anime +Streaming example or create a clean domain, then add data inside that domain. If +the Python API is offline, the sample-domain analysis remains explorable with a +deterministic demo response. See the [Studio guide](docs/studio.md). ## How It Works @@ -217,6 +230,23 @@ QueryForge requires a validated model by default; it auto-discovers a model besi the database and rejects schema-only analysis unless the caller explicitly selects the diagnostic escape hatch. +In Studio, semantic construction is domain-first and gated: + +```text +Create/select domain + → upload domain-owned sources + → profile physical schema + → confirm entity identity and grain + → define dimensions, measures, metrics, and time + → review relationships, cardinality, and Join Paths + → classify sensitivity, ownership, policy, and quality + → validate 100% of blocking checks + → publish data + semantics atomically +``` + +Technical names are treated as evidence, not business truth. A new domain starts +empty and never inherits the Anime sample's entities or metrics. + Build or incrementally refresh one: ```bash @@ -235,9 +265,10 @@ The repository also includes a Monday-morning metrics, relationships, Join Paths, and data-quality contracts against a reviewed baseline. -### Anime platform showcase +### Bundled sample domain: Anime Streaming -The bundled dataset is purpose-built for QueryForge and fully synthetic: **370,762 +Anime Streaming is one ready-to-run example data domain, not a product-wide +schema. The dataset is purpose-built for QueryForge and fully synthetic: **370,762 rows**, **15 tables**, **30 declared relationships**, **7 governed Join Paths**, and **11 business metrics** across content, engagement, subscriptions, advertising, community, and merchandise. @@ -340,7 +371,7 @@ python -m queryforge.interfaces.mcp.server --transport stdio | Surface | Entry point | Best for | | --- | --- | --- | -| Studio | `make web-dev` | Visual onboarding, semantic authoring, and governed analysis | +| Studio | `make web-dev` | Data-domain management, visual onboarding, semantic authoring, and governed analysis | | CLI | `queryforge --question "..."` | Local exploration and engineering workflows | | REST | `POST /ask` and `POST /plan` | Application integration | | SSE | `POST /ask/stream` | Progress-aware clients | diff --git a/README.zh-CN.md b/README.zh-CN.md index fae89cd..893001f 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -21,20 +21,28 @@ --- -QueryForge 是一个本地优先的 AI 数据分析平台，它围绕一个核心原则设计： +QueryForge 是一个本地优先、以数据域为第一入口的 AI 数据分析平台，它围绕一个核心原则设计： **模型生成的 SQL 应该像应用代码一样被治理，而不是像自然语言一样被直接信任。** -项目将 NL2SQL、语义契约、AST 级安全策略、只读执行、多候选选择、有界修复和完整 -运行产物串成一个闭环，适合作为可信 AI 数据系统的参考实现。 +用户先创建或选择数据域（如零售、金融、产品分析，或仓库内置的 Anime Streaming +示例域），再在该域内上传数据、评审语义契约和发起分析。项目把这套域级工作流与 +NL2SQL、AST 级安全策略、只读执行、多候选选择、有界修复和完整运行产物串成闭环。 > QueryForge 当前专注 SQLite 和受控环境，是面向作品展示与架构验证的参考项目， > 不是可直接公网部署的多租户分析服务。 ## 产品界面 +
+ QueryForge 数据域中心：创建和选择相互隔离的业务上下文 + 数据域中心——先创建或选择治理边界，再接入该域的数据与语义。 +
+ +
+
QueryForge Studio 总览：语义契约健康度、业务指标与互动趋势 - 工作台总览——实时展示语义健康度、受治理指标和动漫平台业务动态。 + 数据域总览——Anime Streaming 只是当前选中的示例域，不再是平台身份。

@@ -54,6 +62,7 @@ QueryForge 是一个本地优先的 AI 数据分析平台，它围绕一个核 | --- | --- | | 如何信任生成的 SQL | 使用 SQLGlot 解析 AST，执行前应用具名安全策略 | | 如何保证业务口径一致 | 用 YAML 定义指标、维度、粒度和 Join Path | +| 如何防止上下文串域 | 数据源、语义契约、策略和运行历史全部绑定当前数据域 | | 模型输出不完美怎么办 | 在明确预算内反思、修复和重试 | | 复杂问题如何处理 | 启用有界 Schema 探索和并发 SQL 候选 | | 如何追踪运行过程 | 保存状态、策略决策、质量证据和交付产物 | @@ -119,8 +128,9 @@ queryforge \ ### 体验 QueryForge Studio -仓库现在包含一个完整的可视化工作台：可以接入数据、评审强制语义层、提出受治理的 -自然语言问题、检查 SQL 与 Trust Trace 证据，并审计历史运行。 +仓库现在包含一个完整的可视化工作台：可以创建和切换数据域、在域内接入数据、评审 +强制语义层、提出受治理的自然语言问题、检查 SQL 与 Trust Trace 证据，并审计域内 +历史运行。 ```bash # 终端 1：QueryForge API @@ -132,8 +142,9 @@ make web-install make web-dev ``` -打开 。Python API 离线时，界面会自动使用确定性的演示结果， -所有页面仍然可以完整体验。详见 [Studio 使用指南](docs/studio.md)。 +打开 ，从「Data Domains」进入：可选择内置 Anime Streaming +示例域，也可创建一个完全空白的新域，然后只在当前域内上传数据。Python API 离线时， +示例域分析会使用确定性的演示结果。详见 [Studio 使用指南](docs/studio.md)。 ## 系统架构 @@ -203,6 +214,23 @@ metrics: 物理质量规则。QueryForge 现在默认强制使用经过校验的语义模型，并自动发现数据库旁边 的模型；除非显式启用诊断逃生口，否则不允许只依赖裸 Schema 查询。 +Studio 中的语义层构建是「域优先 + 强门禁」流程： + +```text +创建/选择数据域 + → 上传该域数据 + → Profiling 物理结构 + → 确认实体身份和行粒度 + → 定义维度、度量、指标和时间口径 + → 评审关系、基数和 Join Path + → 标注敏感级别、Owner、策略和质量规则 + → 通过全部阻断性检查 + → 数据 + 语义原子发布 +``` + +技术字段名只作为推断证据，不能替代业务定义。新建域从空白契约开始，绝不会继承 +Anime 示例域的实体或指标。 + 创建或增量更新语义层： ```bash @@ -219,9 +247,10 @@ python scripts/build_semantic_model.py \ [语义漂移检查](.github/workflows/semantic-weekly.yml)，会基于审核后的基线检查 Schema、指标、关系、Join Path 和数据质量契约。 -### 动漫平台数据集 +### 内置示例数据域：Anime Streaming -内置样例是专为 QueryForge 构造的全合成数据：**370,762 行**、**15 张表**、 +Anime Streaming 是一个开箱即用的示例数据域，不是全平台 Schema。其数据是专为 +QueryForge 构造的全合成数据：**370,762 行**、**15 张表**、 **30 条声明关系**、**7 条受治理 Join Path**、**11 个业务指标**，覆盖内容、 观看、订阅、广告、社区和动漫周边。 @@ -321,7 +350,7 @@ python -m queryforge.interfaces.mcp.server --transport stdio | 方式 | 入口 | 适合场景 | | --- | --- | --- | -| Studio | `make web-dev` | 可视化数据接入、语义编写与受治理分析 | +| Studio | `make web-dev` | 数据域管理、可视化接入、语义编写与受治理分析 | | CLI | `queryforge --question "..."` | 本地探索和工程工作流 | | REST | `POST /ask`、`POST /plan` | 应用集成 | | SSE | `POST /ask/stream` | 需要进度事件的客户端 | diff --git a/docs/assets/queryforge-studio-analysis.png b/docs/assets/queryforge-studio-analysis.png index 735899a..8e96fb5 100644 Binary files a/docs/assets/queryforge-studio-analysis.png and b/docs/assets/queryforge-studio-analysis.png differ diff --git a/docs/assets/queryforge-studio-domains.png b/docs/assets/queryforge-studio-domains.png new file mode 100644 index 0000000..cb9a541 Binary files /dev/null and b/docs/assets/queryforge-studio-domains.png differ diff --git a/docs/assets/queryforge-studio-overview.png b/docs/assets/queryforge-studio-overview.png index 60cfa9e..65afe25 100644 Binary files a/docs/assets/queryforge-studio-overview.png and b/docs/assets/queryforge-studio-overview.png differ diff --git a/docs/assets/queryforge-studio-semantic.png b/docs/assets/queryforge-studio-semantic.png index 4ec16f1..8b8c616 100644 Binary files a/docs/assets/queryforge-studio-semantic.png and b/docs/assets/queryforge-studio-semantic.png differ diff --git a/docs/studio.md b/docs/studio.md index 1380055..67e57ef 100644 --- a/docs/studio.md +++ b/docs/studio.md @@ -1,8 +1,9 @@ # QueryForge Studio QueryForge Studio is the browser interface for the governed analytics runtime. -It is designed to make the semantic layer visible and editable instead of -hiding it behind an opaque chat box. +It is designed around a domain-first rule: business context must be selected +before data, semantics, or analysis can exist. The semantic layer stays visible +and editable instead of hiding behind an opaque chat box. ## Start the interface @@ -26,30 +27,51 @@ The Studio sends requests through its same-origin `/api/queryforge/*` proxy. Set `QUERYFORGE_API_URL` in `web/.env.local` if the API is not available at `http://127.0.0.1:8000`. +## Domain-first workflow + +1. Open **Data Domains**. +2. Select the bundled Anime Streaming sample or create a clean business domain. +3. Add SQLite, CSV, or Parquet files inside the active domain. +4. Profile physical schema and candidate keys. +5. Review identity, grain, dimensions, measures, time fields, and ownership. +6. Define metrics, relationships, cardinality, and safe Join Paths. +7. Classify sensitivity, apply SQL policy, and pass quality checks. +8. Publish data and semantics atomically. +9. Ask questions and inspect Trust Trace evidence within that same domain. + +The top-bar domain switcher changes the context for Overview, Data Sources, +Semantic Studio, Ask & Analyze, and Run History. Sources and runs carry a +`domain_id`; uploaded objects use +`domains/{domainId}/sources/{sourceId}/...`. A new domain never reuses semantic +definitions from the sample. + ## Mandatory semantic onboarding -The Data Sources workflow profiles an uploaded SQLite, CSV, or Parquet asset, -generates a semantic draft, and then stops at a human review gate. Publication -remains disabled until: +The Data Sources workflow profiles an uploaded asset, generates a conservative +semantic draft, and then stops at a human review gate. Physical column names are +evidence—not accepted business truth. Publication remains disabled until: -1. entities, dimensions, metrics, relationships, and join paths have been +1. business entity identity and row grain are explicit; +2. primary keys, dimensions, measures, time semantics, and metric units are reviewed; -2. contract validation succeeds; and -3. the reviewer explicitly confirms the semantic contract. +3. relationship cardinality and every multi-hop Join Path are approved; +4. owner, sensitivity, policy, and data-quality contracts are complete; +5. blocking validation succeeds; and +6. the reviewer explicitly confirms the contract. -The server enforces the same rule. Calling the upload endpoint without -`reviewed=true` returns an error, so the requirement cannot be bypassed through -the UI. +The server enforces both boundaries. Calling the upload endpoint without a valid +`domain_id` or without `reviewed=true` returns an error, so neither domain +selection nor semantic review can be bypassed through the UI. ## Runtime persistence -Hosted deployments use D1 for source metadata and run history and R2 for uploaded -files. The Python runtime still owns SQL planning, policy enforcement, read-only -execution, and analytical artifacts. +Hosted deployments use D1 for domains, source metadata, and run history and R2 +for uploaded files. The Python runtime still owns SQL planning, policy +enforcement, read-only execution, and analytical artifacts. The bundled anime dataset under `sample_data/anime_streaming/` is read-only -showcase data for the Studio. Upload tests and hosted storage never modify or -delete it. +showcase data attached to the Anime Streaming sample domain. It is not the +platform schema. Upload tests and hosted storage never modify or delete it. ## Validate the frontend diff --git a/web/README.md b/web/README.md index c00b52d..edfe00f 100644 --- a/web/README.md +++ b/web/README.md @@ -1,8 +1,9 @@ # QueryForge Studio -The visual workspace for QueryForge. It brings data onboarding, mandatory -semantic review, governed natural-language analysis, SQL inspection, trust -evidence, and run history into one responsive interface. +The domain-first visual workspace for QueryForge. Users create or select a data +domain, then onboard that domain's sources, build its mandatory semantic +contract, run governed analysis, inspect SQL and trust evidence, and audit +domain-scoped history in one responsive interface. ## Run locally @@ -13,8 +14,9 @@ npm ci npm run dev ``` -Open . The Studio starts in demo mode when the Python API -is unavailable, so the full product tour remains interactive. +Open . The bundled Anime Streaming domain remains +interactive in demo mode when the Python API is unavailable. Newly created +domains start empty and never inherit sample entities or metrics. For live answers, start the QueryForge API from the repository root in a second terminal: @@ -28,18 +30,19 @@ If the API uses another address, copy `.env.example` to `.env.local` and change ## Product areas -- **Overview** — platform metrics, engagement trends, semantic contract health, - and recent activity. -- **Data Sources** — anime dataset inventory plus SQLite, CSV, and Parquet - onboarding. -- **Semantic Studio** — entities, metrics, relationships, join paths, and - contract status in one graph-and-editor workspace. +- **Data Domains** — create, select, and manage isolated business contexts. +- **Overview** — active-domain readiness, governed metrics, contract health, and + recent activity. +- **Data Sources** — domain-scoped SQLite, CSV, and Parquet onboarding. +- **Semantic Studio** — a required contract builder for identity, grain, + dimensions, measures, metrics, relationships, Join Paths, policy, and quality. - **Ask & Analyze** — natural language to governed SQL with progress, results, export, and a complete Trust Trace. -- **Run History** — filterable, persistent history for audit and replay. +- **Run History** — filterable, domain-scoped history for audit and replay. Data uploads are intentionally atomic: the upload endpoint refuses publication -until a semantic contract has been reviewed and validated. +until a data domain is selected and its semantic contract has been reviewed and +validated. ## Quality checks @@ -54,8 +57,9 @@ tests. The hosted build uses Cloudflare-compatible bindings: -- `DB` (D1) stores source metadata and Studio run history. -- `UPLOADS` (R2) stores uploaded source files. +- `DB` (D1) stores data domains, source metadata, and domain-scoped run history. +- `UPLOADS` (R2) stores files under + `domains/{domainId}/sources/{sourceId}/...`. Database migrations live in `drizzle/`. The bundled anime source data remains in the repository-level `sample_data/` directory and is not copied or modified by diff --git a/web/app/api/studio/domains/route.ts b/web/app/api/studio/domains/route.ts new file mode 100644 index 0000000..3614975 --- /dev/null +++ b/web/app/api/studio/domains/route.ts @@ -0,0 +1,109 @@ +import { ensureStudioSchema, getStudioBindings } from "@/db/runtime"; + +function slugify(value: string) { + return value + .trim() + .toLowerCase() + .replace(/[^a-z0-9]+/g, "-") + .replace(/^-+|-+$/g, "") + .slice(0, 48); +} + +export async function GET() { + try { + const { db } = getStudioBindings(); + await ensureStudioSchema(db); + const result = await db + .prepare( + `SELECT + d.id, + d.name, + d.slug, + d.description, + d.owner, + d.status, + d.is_sample, + d.created_at, + d.updated_at, + COUNT(DISTINCT s.id) AS source_count, + COUNT(DISTINCT r.id) AS run_count + FROM studio_domains d + LEFT JOIN studio_sources s ON s.domain_id = d.id + LEFT JOIN studio_runs r ON r.domain_id = d.id + GROUP BY d.id + ORDER BY d.is_sample DESC, d.created_at ASC`, + ) + .all(); + return Response.json({ domains: result.results }); + } catch (error) { + return Response.json( + { + domains: [], + detail: + error instanceof Error ? error.message : "Domain storage unavailable.", + }, + { status: 503 }, + ); + } +} + +export async function POST(request: Request) { + try { + const payload = (await request.json()) as { + name?: string; + description?: string; + owner?: string; + }; + const name = payload.name?.trim() ?? ""; + if (name.length < 2 || name.length > 80) { + return Response.json( + { detail: "Domain name must contain 2–80 characters." }, + { status: 400 }, + ); + } + + const { db } = getStudioBindings(); + await ensureStudioSchema(db); + const id = `domain_${crypto.randomUUID().replaceAll("-", "")}`; + const slugBase = slugify(name) || "data-domain"; + const slug = `${slugBase}-${id.slice(-6)}`; + const description = + payload.description?.trim() || + "A governed data domain for source ingestion, semantic modeling, and trusted analysis."; + const owner = payload.owner?.trim() || "Workspace admin"; + + await db + .prepare( + `INSERT INTO studio_domains ( + id, name, slug, description, owner, status, is_sample + ) VALUES (?, ?, ?, ?, ?, 'draft', 0)`, + ) + .bind(id, name, slug, description, owner) + .run(); + + return Response.json( + { + domain: { + id, + name, + slug, + description, + owner, + status: "draft", + isSample: false, + sourceCount: 0, + runCount: 0, + }, + }, + { status: 201 }, + ); + } catch (error) { + return Response.json( + { + detail: + error instanceof Error ? error.message : "Domain creation failed.", + }, + { status: 503 }, + ); + } +} diff --git a/web/app/api/studio/runs/route.ts b/web/app/api/studio/runs/route.ts index 00fbdb4..4da2d40 100644 --- a/web/app/api/studio/runs/route.ts +++ b/web/app/api/studio/runs/route.ts @@ -1,17 +1,21 @@ import { ensureStudioSchema, getStudioBindings } from "@/db/runtime"; -export async function GET() { +export async function GET(request: Request) { try { + const domainId = new URL(request.url).searchParams.get("domain_id"); const { db } = getStudioBindings(); await ensureStudioSchema(db); - const result = await db - .prepare( - `SELECT id, question, status, model, row_count, duration, created_at - FROM studio_runs - ORDER BY created_at DESC - LIMIT 50`, - ) - .all(); + const statement = db.prepare( + `SELECT id, domain_id, question, status, model, row_count, duration, + created_at + FROM studio_runs + ${domainId ? "WHERE domain_id = ?" : ""} + ORDER BY created_at DESC + LIMIT 50`, + ); + const result = domainId + ? await statement.bind(domainId).all() + : await statement.all(); return Response.json({ runs: result.results }); } catch (error) { return Response.json( @@ -29,15 +33,16 @@ export async function POST(request: Request) { try { const payload = (await request.json()) as { id?: string; + domainId?: string; question?: string; status?: string; model?: string; rowCount?: number; duration?: string; }; - if (!payload.id || !payload.question) { + if (!payload.id || !payload.domainId || !payload.question) { return Response.json( - { detail: "Run id and question are required." }, + { detail: "Run id, data domain, and question are required." }, { status: 400 }, ); } @@ -46,11 +51,12 @@ export async function POST(request: Request) { await db .prepare( `INSERT OR REPLACE INTO studio_runs ( - id, question, status, model, row_count, duration - ) VALUES (?, ?, ?, ?, ?, ?)`, + id, domain_id, question, status, model, row_count, duration + ) VALUES (?, ?, ?, ?, ?, ?, ?)`, ) .bind( payload.id, + payload.domainId, payload.question, payload.status ?? "Passed", payload.model ?? "configured model", diff --git a/web/app/api/studio/upload/route.ts b/web/app/api/studio/upload/route.ts index 52329dc..3e1e9b5 100644 --- a/web/app/api/studio/upload/route.ts +++ b/web/app/api/studio/upload/route.ts @@ -3,6 +3,23 @@ import { ensureStudioSchema, getStudioBindings } from "@/db/runtime"; const MAX_FILE_BYTES = 25 * 1024 * 1024; const ALLOWED_EXTENSIONS = new Set(["sqlite", "db", "csv", "parquet"]); +type UploadedSemanticContract = { + version: number; + entity: string; + description: string; + owner: string; + grain: string; + primaryKey: string; + sensitivity: "public" | "internal" | "restricted"; + dimensions: string[]; + metrics: Array<{ + name: string; + description: string; + aggregation: string; + expression: string; + }>; +}; + function safeFileName(name: string) { return name.replace(/[^a-zA-Z0-9._-]+/g, "-").replace(/^-+|-+$/g, ""); } @@ -11,19 +28,74 @@ function extension(name: string) { return name.split(".").pop()?.toLowerCase() ?? ""; } -export async function GET() { +function parseSemanticContract(value: FormDataEntryValue | null) { + if (typeof value !== "string") return null; try { + const contract = JSON.parse(value) as Partial; + const required = [ + contract.entity, + contract.description, + contract.owner, + contract.grain, + contract.primaryKey, + ]; + const sensitivityValid = ["public", "internal", "restricted"].includes( + String(contract.sensitivity), + ); + const dimensionsValid = + Array.isArray(contract.dimensions) && + contract.dimensions.length > 0 && + contract.dimensions.every( + (dimension) => + typeof dimension === "string" && dimension.trim().length > 0, + ); + const metricsValid = + Array.isArray(contract.metrics) && + contract.metrics.length > 0 && + contract.metrics.every( + (metric) => + metric && + typeof metric.name === "string" && + metric.name.trim() && + typeof metric.description === "string" && + metric.description.trim() && + typeof metric.aggregation === "string" && + metric.aggregation.trim() && + typeof metric.expression === "string" && + metric.expression.trim(), + ); + if ( + required.some( + (field) => typeof field !== "string" || !field.trim(), + ) || + !sensitivityValid || + !dimensionsValid || + !metricsValid + ) { + return null; + } + return contract as UploadedSemanticContract; + } catch { + return null; + } +} + +export async function GET(request: Request) { + try { + const domainId = new URL(request.url).searchParams.get("domain_id"); const { db } = getStudioBindings(); await ensureStudioSchema(db); - const result = await db - .prepare( - `SELECT id, name, source_type, size_bytes, table_count, status, - contract_status, created_at - FROM studio_sources - ORDER BY created_at DESC - LIMIT 50`, - ) - .all(); + const statement = db.prepare( + `SELECT id, domain_id, name, source_type, size_bytes, table_count, status, + contract_status, created_at + FROM studio_sources + ${domainId ? "WHERE domain_id = ?" : ""} + ORDER BY created_at DESC + LIMIT 50`, + ); + const result = domainId + ? await statement.bind(domainId).all() + : await statement.all(); return Response.json({ sources: result.results }); } catch (error) { return Response.json( @@ -40,12 +112,31 @@ export async function GET() { export async function POST(request: Request) { try { const form = await request.formData(); + const domainId = String(form.get("domain_id") ?? "").trim(); + if (!domainId) { + return Response.json( + { detail: "Select a data domain before uploading data." }, + { status: 400 }, + ); + } if (form.get("reviewed") !== "true") { return Response.json( { detail: "A reviewed semantic contract is required." }, { status: 400 }, ); } + const semanticContract = parseSemanticContract( + form.get("semantic_contract"), + ); + if (!semanticContract) { + return Response.json( + { + detail: + "A complete semantic contract with entity, grain, owner, dimensions, and metrics is required.", + }, + { status: 400 }, + ); + } const files = form .getAll("files") @@ -74,15 +165,27 @@ export async function POST(request: Request) { const { db, uploads } = getStudioBindings(); await ensureStudioSchema(db); + const domain = await db + .prepare("SELECT id, name FROM studio_domains WHERE id = ? LIMIT 1") + .bind(domainId) + .first<{ id: string; name: string }>(); + if (!domain) { + return Response.json( + { detail: "The selected data domain does not exist." }, + { status: 404 }, + ); + } const sourceId = crypto.randomUUID(); const stored: Array<{ name: string; objectKey: string; size: number }> = []; for (const file of files) { - const objectKey = `sources/${sourceId}/${safeFileName(file.name)}`; + const objectKey = `domains/${domainId}/sources/${sourceId}/${safeFileName(file.name)}`; await uploads.put(objectKey, await file.arrayBuffer(), { httpMetadata: { contentType: file.type || "application/octet-stream" }, customMetadata: { sourceId, + domainId, + semanticEntity: semanticContract.entity, semanticReviewed: "true", }, }); @@ -98,25 +201,26 @@ export async function POST(request: Request) { new Set(stored.map((file) => extension(file.name).toUpperCase())), ).join(" · "); const semanticModel = { + ...semanticContract, reviewed: true, - entity: "uploaded_watch_event", - grain: ["event_id"], - owner: "engagement-analytics", - sensitivity: "internal", - dimensions: 7, - metrics: ["uploaded_watch_hours", "uploaded_completion_rate"], + domain: { id: domainId, name: domain.name }, + inference: { + status: "human_reviewed", + note: "Physical inference was confirmed by the accountable domain owner.", + }, contract: { status: "passed", blocking_failures: 0 }, }; await db .prepare( `INSERT INTO studio_sources ( - id, name, source_type, object_key, size_bytes, table_count, + id, domain_id, name, source_type, object_key, size_bytes, table_count, status, semantic_json, reviewed, contract_status - ) VALUES (?, ?, ?, ?, ?, ?, 'ready', ?, 1, 'passed')`, + ) VALUES (?, ?, ?, ?, ?, ?, ?, 'ready', ?, 1, 'passed')`, ) .bind( sourceId, + domainId, sourceName, sourceType, stored[0].objectKey, @@ -130,6 +234,7 @@ export async function POST(request: Request) { { source: { id: sourceId, + domainId, name: sourceName, sourceType, sizeBytes: totalBytes, diff --git a/web/app/globals.css b/web/app/globals.css index 053c447..40ebd12 100644 --- a/web/app/globals.css +++ b/web/app/globals.css @@ -4691,3 +4691,865 @@ kbd { transition-duration: 0.01ms !important; } } + +/* Data-domain workspace */ +.topbar-actions { + position: relative; +} + +.domain-switcher-menu { + position: absolute; + z-index: 80; + top: 48px; + right: 44px; + width: 300px; + overflow: hidden; + border: 1px solid #343d45; + border-radius: 12px; + background: #0e1318; + box-shadow: 0 22px 60px rgba(0, 0, 0, 0.45); + padding: 7px; +} + +.domain-switcher-heading { + display: flex; + align-items: center; + justify-content: space-between; + color: var(--muted); + font-size: 8px; + font-weight: 750; + letter-spacing: 0.08em; + padding: 7px 8px 9px; + text-transform: uppercase; +} + +.domain-switcher-heading button { + background: transparent; + color: var(--lime); + font-size: 8px; + font-weight: 750; +} + +.domain-switcher-option { + display: grid; + width: 100%; + min-height: 56px; + grid-template-columns: 34px 1fr 20px; + align-items: center; + gap: 9px; + border-radius: 8px; + background: transparent; + color: var(--text-soft); + padding: 7px 8px; + text-align: left; +} + +.domain-switcher-option:hover, +.domain-switcher-option.active { + background: rgba(255, 255, 255, 0.035); + color: var(--text); +} + +.domain-option-icon, +.domain-card-icon { + display: grid; + width: 34px; + height: 34px; + place-items: center; + border: 1px solid rgba(82, 216, 209, 0.25); + border-radius: 9px; + background: rgba(82, 216, 209, 0.07); + color: var(--cyan); + font-family: "SFMono-Regular", Consolas, monospace; + font-size: 10px; + font-style: normal; + font-weight: 800; +} + +.domain-switcher-option > span:nth-child(2) { + display: flex; + min-width: 0; + flex-direction: column; + gap: 4px; +} + +.domain-switcher-option strong { + overflow: hidden; + font-size: 9px; + text-overflow: ellipsis; + white-space: nowrap; +} + +.domain-switcher-option small { + color: var(--muted); + font-size: 7px; +} + +.domain-switcher-option em { + color: var(--lime); + font-style: normal; +} + +.domain-switcher-all { + width: 100%; + border-top: 1px solid var(--border-soft); + background: transparent; + color: var(--text-soft); + font-size: 8px; + padding: 11px 8px 6px; + text-align: left; +} + +.domain-switcher-all:hover { + color: var(--lime); +} + +.domain-hero { + display: grid; + grid-template-columns: minmax(0, 0.75fr) minmax(580px, 1.25fr); + align-items: center; + gap: 34px; + overflow: hidden; + margin-bottom: 14px; + background: + radial-gradient(circle at 90% 10%, rgba(82, 216, 209, 0.07), transparent 35%), + linear-gradient(120deg, rgba(183, 244, 61, 0.05), transparent 45%), + var(--surface); + padding: 28px; +} + +.domain-hero h2 { + max-width: 570px; + margin: 9px 0 8px; + font-size: clamp(23px, 2.5vw, 34px); + letter-spacing: -0.045em; +} + +.domain-hero p { + max-width: 620px; + margin: 0; + color: var(--text-soft); + font-size: 10px; + line-height: 1.65; +} + +.domain-flow { + display: grid; + grid-template-columns: repeat(4, 1fr); + gap: 0; + border: 1px solid var(--border); + border-radius: 12px; + background: rgba(6, 9, 12, 0.7); + padding: 15px; +} + +.domain-flow > span { + position: relative; + display: flex; + min-height: 82px; + align-items: center; + justify-content: center; + flex-direction: column; + gap: 8px; + border-right: 1px solid var(--border-soft); + text-align: center; +} + +.domain-flow > span:last-child { + border-right: 0; +} + +.domain-flow i { + color: var(--lime); + font-family: "SFMono-Regular", Consolas, monospace; + font-size: 9px; + font-style: normal; +} + +.domain-flow strong { + font-size: 9px; +} + +.domain-flow b { + position: absolute; + z-index: 2; + right: -7px; + display: grid; + width: 14px; + height: 20px; + place-items: center; + background: #0c1115; + color: var(--muted); + font-size: 10px; +} + +.domain-grid { + display: grid; + grid-template-columns: repeat(2, minmax(0, 1fr)); + gap: 13px; +} + +.domain-card, +.domain-create-card { + min-height: 306px; + border: 1px solid var(--border); + border-radius: var(--radius); + background: var(--surface); +} + +.domain-card { + display: flex; + flex-direction: column; + padding: 22px; +} + +.domain-card.active { + border-color: rgba(183, 244, 61, 0.32); + background: + linear-gradient(125deg, rgba(183, 244, 61, 0.045), transparent 42%), + var(--surface); +} + +.domain-card-top { + display: flex; + align-items: flex-start; + gap: 13px; +} + +.domain-card-icon { + width: 44px; + height: 44px; + flex: 0 0 44px; +} + +.domain-card-icon.sample { + border-color: rgba(157, 140, 255, 0.27); + background: rgba(157, 140, 255, 0.08); + color: #c8beff; +} + +.domain-card-top > div { + min-width: 0; +} + +.domain-card-labels { + display: flex; + gap: 5px; + margin-bottom: 7px; +} + +.domain-card h2 { + margin: 0 0 4px; + font-size: 18px; + letter-spacing: -0.025em; +} + +.domain-card code { + color: var(--muted); + font-size: 8px; +} + +.domain-card > p { + min-height: 48px; + margin: 18px 0 16px; + color: var(--text-soft); + font-size: 9px; + line-height: 1.65; +} + +.domain-card-stats { + display: grid; + grid-template-columns: repeat(4, 1fr); + border-top: 1px solid var(--border-soft); + border-bottom: 1px solid var(--border-soft); +} + +.domain-card-stats span { + display: flex; + flex-direction: column; + gap: 4px; + border-right: 1px solid var(--border-soft); + padding: 12px 8px; +} + +.domain-card-stats span:first-child { + padding-left: 0; +} + +.domain-card-stats span:last-child { + border-right: 0; +} + +.domain-card-stats strong { + font-family: "SFMono-Regular", Consolas, monospace; + font-size: 12px; + font-weight: 600; +} + +.domain-card-stats small { + color: var(--muted); + font-size: 7px; +} + +.domain-card footer { + display: flex; + align-items: center; + justify-content: space-between; + gap: 12px; + margin-top: auto; + padding-top: 16px; +} + +.domain-card footer > span { + color: var(--muted); + font-size: 8px; +} + +.domain-card footer > span strong { + color: var(--text-soft); +} + +.domain-create-card { + display: grid; + min-height: 230px; + place-content: center; + border-style: dashed; + background: + radial-gradient(circle at 50% 30%, rgba(183, 244, 61, 0.04), transparent 34%), + #0b0f13; + color: var(--text-soft); + padding: 30px; + text-align: center; +} + +.domain-create-card > span { + display: grid; + width: 48px; + height: 48px; + place-items: center; + margin: 0 auto 13px; + border: 1px solid rgba(183, 244, 61, 0.25); + border-radius: 50%; + background: var(--lime-soft); + color: var(--lime); + font-size: 20px; +} + +.domain-create-card strong { + color: var(--text); + font-size: 12px; +} + +.domain-create-card p { + margin: 7px 0 0; + color: var(--muted); + font-size: 8px; +} + +.domain-create-card:hover { + border-color: rgba(183, 244, 61, 0.4); +} + +.onboarding-actions { + display: flex; + gap: 9px; + margin-top: 24px; +} + +.domain-readiness { + position: relative; + z-index: 2; + align-self: center; + margin: 28px; + border: 1px solid var(--border); + border-radius: 13px; + background: rgba(7, 10, 13, 0.75); + padding: 18px; +} + +.domain-readiness > div { + display: grid; + min-height: 45px; + grid-template-columns: 25px 1fr; + align-items: center; + gap: 9px; + border-bottom: 1px solid var(--border-soft); + color: var(--muted); +} + +.domain-readiness > div:last-child { + border-bottom: 0; +} + +.domain-readiness > div > span { + display: grid; + width: 21px; + height: 21px; + place-items: center; + border: 1px solid #303943; + border-radius: 50%; + font-family: "SFMono-Regular", Consolas, monospace; + font-size: 7px; +} + +.domain-readiness > div.complete { + color: var(--text-soft); +} + +.domain-readiness > div.complete > span { + border-color: rgba(183, 244, 61, 0.3); + background: var(--lime-soft); + color: var(--lime); +} + +.domain-readiness strong { + font-size: 9px; +} + +.domain-empty-panel, +.domain-source-empty, +.semantic-empty-state, +.domain-analysis-lock { + display: flex; + align-items: center; + gap: 18px; + min-height: 150px; + padding: 24px; +} + +.domain-empty-panel > div, +.domain-source-empty > div, +.semantic-empty-state > div { + flex: 1; +} + +.domain-empty-panel h2, +.domain-source-empty h2, +.semantic-empty-state h2 { + margin: 7px 0 6px; + font-size: 18px; +} + +.domain-empty-panel p, +.domain-source-empty p, +.semantic-empty-state p { + max-width: 720px; + margin: 0; + color: var(--muted); + font-size: 9px; + line-height: 1.6; +} + +.source-summary-grid.single { + grid-template-columns: 1fr; +} + +.source-summary-grid.single .add-source-card { + min-height: 160px; +} + +.sources-layout.domain-only { + grid-template-columns: 1fr; +} + +.semantic-health-strip.draft { + border-color: rgba(255, 185, 103, 0.2); +} + +.semantic-health-strip.draft .health-score-mini strong { + color: var(--orange); +} + +.semantic-empty-icon { + display: grid; + width: 52px; + height: 52px; + flex: 0 0 52px; + place-items: center; + border: 1px solid rgba(157, 140, 255, 0.25); + border-radius: 13px; + background: rgba(157, 140, 255, 0.08); + color: var(--purple); + font-size: 20px; +} + +.semantic-builder { + display: grid; + grid-template-columns: 210px minmax(0, 1fr) 250px; + gap: 12px; +} + +.semantic-builder-steps { + display: flex; + flex-direction: column; + gap: 5px; + border: 1px solid var(--border); + border-radius: var(--radius); + background: #0d1116; + padding: 16px 10px; +} + +.semantic-builder-steps > .panel-kicker { + padding: 2px 8px 8px; +} + +.semantic-builder-steps button { + display: grid; + min-height: 45px; + grid-template-columns: 28px 1fr; + align-items: center; + gap: 7px; + border-radius: 8px; + background: transparent; + color: var(--muted); + padding: 6px 8px; + text-align: left; +} + +.semantic-builder-steps button.active { + background: rgba(183, 244, 61, 0.07); + color: var(--text); +} + +.semantic-builder-steps button span { + color: var(--lime); + font-family: "SFMono-Regular", Consolas, monospace; + font-size: 7px; +} + +.semantic-builder-steps button strong { + font-size: 8px; +} + +.semantic-contract-editor { + padding: 22px; +} + +.contract-editor-intro { + margin: -4px 0 18px; + color: var(--muted); + font-size: 9px; + line-height: 1.6; +} + +.semantic-contract-fields { + display: grid; + grid-template-columns: 1fr 1fr; + gap: 13px; +} + +.semantic-contract-fields label, +.create-domain-body label { + display: flex; + flex-direction: column; + gap: 6px; +} + +.semantic-contract-fields label.wide { + grid-column: 1 / -1; +} + +.semantic-contract-fields label > span, +.create-domain-body label > span { + color: var(--text-soft); + font-size: 8px; + font-weight: 700; +} + +.semantic-contract-fields small, +.create-domain-body label > small { + color: var(--muted); + font-size: 7px; +} + +.contract-guardrail, +.domain-boundary-note { + display: flex; + gap: 9px; + margin-top: 14px; + border: 1px solid rgba(255, 185, 103, 0.18); + border-radius: 8px; + background: rgba(255, 185, 103, 0.045); + color: #a99b88; + padding: 11px; +} + +.contract-guardrail > span, +.domain-boundary-note > span { + color: var(--orange); + font-weight: 800; +} + +.contract-guardrail p, +.domain-boundary-note p { + margin: 0; + font-size: 8px; + line-height: 1.55; +} + +.semantic-contract-editor > footer { + display: flex; + align-items: center; + justify-content: space-between; + gap: 12px; + margin-top: 16px; + border-top: 1px solid var(--border-soft); + color: var(--muted); + font-size: 8px; + padding-top: 16px; +} + +.semantic-checklist { + padding: 18px; +} + +.semantic-checklist h3 { + margin: 8px 0 13px; + font-size: 14px; +} + +.semantic-checklist > div { + display: grid; + min-height: 39px; + grid-template-columns: 22px 1fr; + align-items: center; + gap: 5px; + border-bottom: 1px solid var(--border-soft); + color: var(--muted); +} + +.semantic-checklist > div.passed { + color: var(--text-soft); +} + +.semantic-checklist > div > span { + color: var(--muted); +} + +.semantic-checklist > div.passed > span { + color: var(--lime); +} + +.semantic-checklist strong { + font-size: 8px; +} + +.semantic-checklist > p { + margin: 14px 0 0; + color: var(--orange); + font-size: 7px; + line-height: 1.5; +} + +.domain-analysis-lock { + min-height: 430px; + justify-content: center; + background: + radial-gradient(circle at 28% 50%, rgba(157, 140, 255, 0.07), transparent 28%), + var(--surface); +} + +.analysis-lock-visual { + display: flex; + min-width: 390px; + align-items: center; + justify-content: center; +} + +.analysis-lock-visual > span, +.analysis-lock-visual > b, +.analysis-lock-visual > em { + display: grid; + width: 70px; + height: 70px; + place-items: center; + border: 1px solid var(--border); + border-radius: 18px; + background: #0b0f13; + color: var(--purple); + font-size: 20px; + font-style: normal; +} + +.analysis-lock-visual > b { + color: var(--lime); +} + +.analysis-lock-visual > em { + color: var(--cyan); + font-family: "SFMono-Regular", Consolas, monospace; + font-size: 12px; +} + +.analysis-lock-visual > i { + width: 55px; + height: 1px; + background: repeating-linear-gradient( + 90deg, + var(--border) 0, + var(--border) 6px, + transparent 6px, + transparent 10px + ); +} + +.domain-analysis-lock > div:last-child { + max-width: 540px; +} + +.domain-analysis-lock h2 { + margin: 9px 0 8px; + font-size: 25px; + letter-spacing: -0.04em; +} + +.domain-analysis-lock p { + margin: 0; + color: var(--text-soft); + font-size: 10px; + line-height: 1.65; +} + +.analysis-lock-checks { + display: flex; + flex-wrap: wrap; + gap: 7px; + margin-top: 18px; +} + +.analysis-lock-checks span { + border: 1px solid var(--border); + border-radius: 999px; + color: var(--muted); + font-size: 8px; + padding: 7px 9px; +} + +.analysis-lock-checks span.ready { + border-color: rgba(183, 244, 61, 0.18); + background: var(--lime-soft); + color: var(--lime); +} + +.create-domain-modal { + width: min(620px, 96vw); +} + +.create-domain-body { + display: flex; + flex-direction: column; + gap: 14px; + padding: 22px 24px; +} + +.domain-name-preview { + display: flex; + align-items: center; + gap: 12px; + border: 1px solid var(--border); + border-radius: 10px; + background: #0a0e12; + padding: 12px; +} + +.domain-name-preview > span { + display: grid; + width: 42px; + height: 42px; + place-items: center; + border: 1px solid rgba(82, 216, 209, 0.25); + border-radius: 10px; + background: rgba(82, 216, 209, 0.07); + color: var(--cyan); + font-family: "SFMono-Regular", Consolas, monospace; + font-size: 10px; + font-weight: 800; +} + +.domain-name-preview > div { + display: flex; + flex-direction: column; + gap: 5px; +} + +.domain-name-preview strong { + font-size: 10px; +} + +.domain-name-preview code { + color: var(--muted); + font-size: 7px; +} + +@media (max-width: 1100px) { + .domain-hero, + .semantic-builder { + grid-template-columns: 1fr; + } + + .domain-flow { + min-width: 0; + } + + .semantic-builder-steps { + display: grid; + grid-template-columns: repeat(3, 1fr); + } + + .semantic-builder-steps > .panel-kicker { + grid-column: 1 / -1; + } + + .domain-analysis-lock { + flex-direction: column; + } +} + +@media (max-width: 760px) { + .domain-switcher-menu { + position: fixed; + top: var(--topbar-height); + right: 10px; + left: 10px; + width: auto; + } + + .domain-grid, + .semantic-contract-fields { + grid-template-columns: 1fr; + } + + .semantic-contract-fields label.wide { + grid-column: auto; + } + + .domain-flow { + grid-template-columns: 1fr 1fr; + } + + .domain-flow > span:nth-child(2) { + border-right: 0; + } + + .semantic-builder-steps { + grid-template-columns: 1fr 1fr; + } + + .domain-empty-panel, + .domain-source-empty, + .semantic-empty-state { + align-items: flex-start; + flex-direction: column; + } + + .domain-card footer, + .semantic-contract-editor > footer { + align-items: flex-start; + flex-direction: column; + } + + .analysis-lock-visual { + min-width: 0; + transform: scale(0.8); + } +} diff --git a/web/app/page.tsx b/web/app/page.tsx index aee69fd..892efb9 100644 --- a/web/app/page.tsx +++ b/web/app/page.tsx @@ -9,7 +9,7 @@ import { useState, } from "react"; -type View = "overview" | "sources" | "semantic" | "ask" | "runs"; +type View = "domains" | "overview" | "sources" | "semantic" | "ask" | "runs"; type ConnectionState = "checking" | "live" | "demo"; type SemanticTab = "graph" | "metrics" | "paths"; @@ -47,6 +47,7 @@ type QueryResult = { type RunRecord = { id: string; + domainId: string; question: string; status: "Passed" | "Blocked" | "Running"; model: string; @@ -57,6 +58,7 @@ type RunRecord = { type UploadedSource = { id: string; + domainId: string; name: string; type: string; size: string; @@ -65,12 +67,67 @@ type UploadedSource = { status: "Ready" | "Draft"; }; +type DataDomain = { + id: string; + name: string; + slug: string; + description: string; + owner: string; + status: "ready" | "draft"; + isSample: boolean; + sourceCount: number; + runCount: number; +}; + +type SemanticDraft = { + entity: string; + description: string; + owner: string; + grain: string; + primaryKey: string; + sensitivity: "public" | "internal" | "restricted"; + dimensions: string[]; + metrics: Array<{ + name: string; + description: string; + aggregation: "count" | "sum"; + expression: string; + }>; +}; + +const SAMPLE_DOMAIN_ID = "domain_anime_streaming"; + +const INITIAL_SEMANTIC_DRAFT: SemanticDraft = { + entity: "uploaded_record", + description: "One governed business record represented by the uploaded source.", + owner: "workspace-admin", + grain: "record_id", + primaryKey: "record_id", + sensitivity: "internal", + dimensions: ["record_id", "entity_id", "recorded_at"], + metrics: [ + { + name: "uploaded_record_count", + description: "Count of reviewed records at the declared grain.", + aggregation: "count", + expression: "COUNT(*)", + }, + { + name: "uploaded_amount_sum", + description: "Sum of the profiled amount field in source-native units.", + aggregation: "sum", + expression: "SUM(amount)", + }, + ], +}; + const NAV_ITEMS: Array<{ id: View; label: string; caption: string; icon: string; }> = [ + { id: "domains", label: "Data Domains", caption: "Create & switch context", icon: "◎" }, { id: "overview", label: "Overview", caption: "Workspace health", icon: "◇" }, { id: "sources", label: "Data Sources", caption: "Ingest & profile", icon: "▦" }, { id: "semantic", label: "Semantic Studio", caption: "Model & govern", icon: "⌘" }, @@ -78,6 +135,21 @@ const NAV_ITEMS: Array<{ { id: "runs", label: "Run History", caption: "Trace every decision", icon: "↺" }, ]; +const INITIAL_DOMAINS: DataDomain[] = [ + { + id: SAMPLE_DOMAIN_ID, + name: "Anime Streaming", + slug: "anime-streaming", + description: + "Synthetic streaming analytics showcase spanning content, engagement, subscriptions, advertising, and merchandise.", + owner: "Content analytics", + status: "ready", + isSample: true, + sourceCount: 1, + runCount: 5, + }, +]; + const ENTITIES: Entity[] = [ { name: "studio", @@ -459,6 +531,7 @@ LIMIT 6;`, const INITIAL_RUNS: RunRecord[] = [ { id: "qf_7a3e2c91", + domainId: SAMPLE_DOMAIN_ID, question: "Compare watch hours and completion rate by genre", status: "Passed", model: "qwen-plus", @@ -468,6 +541,7 @@ const INITIAL_RUNS: RunRecord[] = [ }, { id: "qf_4ce8b1a2", + domainId: SAMPLE_DOMAIN_ID, question: "Top anime by merchandise GMV this quarter", status: "Passed", model: "qwen-plus", @@ -477,6 +551,7 @@ const INITIAL_RUNS: RunRecord[] = [ }, { id: "qf_e832bb77", + domainId: SAMPLE_DOMAIN_ID, question: "Show every user email with subscription revenue", status: "Blocked", model: "qwen-plus", @@ -486,6 +561,7 @@ const INITIAL_RUNS: RunRecord[] = [ }, { id: "qf_729af843", + domainId: SAMPLE_DOMAIN_ID, question: "Monthly active subscribers by plan tier", status: "Passed", model: "gpt-4.1-mini", @@ -495,6 +571,7 @@ const INITIAL_RUNS: RunRecord[] = [ }, { id: "qf_c239a101", + domainId: SAMPLE_DOMAIN_ID, question: "Which studios have the highest average rating?", status: "Passed", model: "qwen-plus", @@ -562,7 +639,7 @@ function normalizeQueryResult(payload: Record): QueryResult { } export default function Home() { - const [activeView, setActiveView] = useState("overview"); + const [activeView, setActiveView] = useState("domains"); const [mobileMenuOpen, setMobileMenuOpen] = useState(false); const [connection, setConnection] = useState("checking"); @@ -575,6 +652,10 @@ export default function Home() { const [semanticTab, setSemanticTab] = useState("graph"); const [entityFilter, setEntityFilter] = useState(""); const [toast, setToast] = useState(""); + const [domains, setDomains] = useState(INITIAL_DOMAINS); + const [activeDomainId, setActiveDomainId] = useState(SAMPLE_DOMAIN_ID); + const [domainMenuOpen, setDomainMenuOpen] = useState(false); + const [createDomainOpen, setCreateDomainOpen] = useState(false); const [uploadOpen, setUploadOpen] = useState(false); const [settingsOpen, setSettingsOpen] = useState(false); const [uploadStep, setUploadStep] = useState(1); @@ -582,12 +663,19 @@ export default function Home() { const [isProfiling, setIsProfiling] = useState(false); const [semanticReviewed, setSemanticReviewed] = useState(false); const [semanticValidated, setSemanticValidated] = useState(false); + const [semanticDraft, setSemanticDraft] = useState( + INITIAL_SEMANTIC_DRAFT, + ); const [uploadedSources, setUploadedSources] = useState([]); const [runFilter, setRunFilter] = useState<"All" | "Passed" | "Blocked">( "All", ); const fileInputRef = useRef(null); + const activeDomain = + domains.find((domain) => domain.id === activeDomainId) ?? + INITIAL_DOMAINS[0]; + useEffect(() => { const controller = new AbortController(); const timer = window.setTimeout(() => controller.abort(), 2500); @@ -608,6 +696,39 @@ export default function Home() { }, []); useEffect(() => { + fetch("/api/studio/domains") + .then((response) => { + if (!response.ok) throw new Error("Persistence unavailable"); + return response.json() as Promise<{ + domains?: Array>; + }>; + }) + .then((payload) => { + const persisted = (payload.domains ?? []).map((domain) => ({ + id: String(domain.id), + name: String(domain.name), + slug: String(domain.slug), + description: String(domain.description ?? ""), + owner: String(domain.owner ?? "Workspace admin"), + status: + String(domain.status) === "ready" + ? ("ready" as const) + : ("draft" as const), + isSample: Boolean(domain.is_sample), + sourceCount: Number(domain.source_count ?? 0), + runCount: Number(domain.run_count ?? 0), + })); + if (persisted.length) { + setDomains(persisted); + setActiveDomainId((current) => + persisted.some((domain) => domain.id === current) + ? current + : persisted[0].id, + ); + } + }) + .catch(() => undefined); + fetch("/api/studio/upload") .then((response) => { if (!response.ok) throw new Error("Persistence unavailable"); @@ -618,6 +739,7 @@ export default function Home() { .then((payload) => { const persisted = (payload.sources ?? []).map((source) => ({ id: String(source.id), + domainId: String(source.domain_id ?? SAMPLE_DOMAIN_ID), name: String(source.name), type: String(source.source_type ?? "DATA"), size: formatBytes(Number(source.size_bytes ?? 0)), @@ -639,6 +761,7 @@ export default function Home() { .then((payload) => { const persisted = (payload.runs ?? []).map((run) => ({ id: String(run.id), + domainId: String(run.domain_id ?? SAMPLE_DOMAIN_ID), question: String(run.question), status: String(run.status) === "Blocked" @@ -681,12 +804,29 @@ export default function Home() { ENTITIES.find((entity) => entity.name === selectedEntity) ?? ENTITIES[0]; const filteredRuns = runs.filter( - (run) => runFilter === "All" || run.status === runFilter, + (run) => + run.domainId === activeDomain.id && + (runFilter === "All" || run.status === runFilter), + ); + const activeDomainSources = uploadedSources.filter( + (source) => source.domainId === activeDomain.id, + ); + const activeDomainRuns = runs.filter( + (run) => run.domainId === activeDomain.id, ); async function runQuery(nextQuestion?: string) { const submitted = (nextQuestion ?? query).trim(); if (!submitted || isRunning) return; + if (!activeDomain.isSample) { + setActiveView("ask"); + setToast( + activeDomainSources.length + ? "Publish an executable connector for this domain before running analysis." + : "Upload data and publish its semantic contract before running analysis.", + ); + return; + } setQuery(submitted); setActiveView("ask"); @@ -734,6 +874,7 @@ export default function Home() { setRuns((current) => [ { id: nextResult.runId, + domainId: activeDomain.id, question: submitted, status: "Passed", model: connection === "live" ? "configured model" : "demo-model", @@ -748,6 +889,7 @@ export default function Home() { headers: { "content-type": "application/json" }, body: JSON.stringify({ id: nextResult.runId, + domainId: activeDomain.id, question: submitted, status: "Passed", model: connection === "live" ? "configured model" : "demo-model", @@ -798,6 +940,11 @@ export default function Home() { setUploadFiles(files); setSemanticReviewed(false); setSemanticValidated(false); + setSemanticDraft({ + ...INITIAL_SEMANTIC_DRAFT, + description: `One governed business record represented by data uploaded to ${activeDomain.name}.`, + owner: activeDomain.owner, + }); } async function profileFiles() { @@ -810,6 +957,24 @@ export default function Home() { async function validateUploadSemantic() { if (!semanticReviewed) return; + const requiredFields = [ + semanticDraft.entity, + semanticDraft.description, + semanticDraft.owner, + semanticDraft.grain, + semanticDraft.primaryKey, + ]; + const metricsValid = + semanticDraft.metrics.length > 0 && + semanticDraft.metrics.every( + (metric) => + metric.name && metric.description && metric.expression, + ); + if (requiredFields.some((value) => !value.trim()) || !metricsValid) { + setSemanticValidated(false); + setToast("Complete the entity, grain, owner, and metric contract first."); + return; + } await sleep(500); setSemanticValidated(true); setToast("Semantic contract passed all blocking checks."); @@ -820,7 +985,16 @@ export default function Home() { const totalBytes = uploadFiles.reduce((sum, file) => sum + file.size, 0); const payload = new FormData(); uploadFiles.forEach((file) => payload.append("files", file)); + payload.append("domain_id", activeDomain.id); payload.append("reviewed", "true"); + payload.append( + "semantic_contract", + JSON.stringify({ + ...semanticDraft, + reviewed: true, + version: 1, + }), + ); let persisted = false; try { @@ -836,6 +1010,7 @@ export default function Home() { setUploadedSources((current) => [ { id: crypto.randomUUID(), + domainId: activeDomain.id, name: uploadFiles.length === 1 ? uploadFiles[0].name @@ -851,6 +1026,17 @@ export default function Home() { }, ...current, ]); + setDomains((current) => + current.map((domain) => + domain.id === activeDomain.id + ? { + ...domain, + sourceCount: domain.sourceCount + 1, + status: "ready", + } + : domain, + ), + ); setUploadOpen(false); setUploadStep(1); setUploadFiles([]); @@ -864,10 +1050,62 @@ export default function Home() { } function selectRun(run: RunRecord) { + setActiveDomainId(run.domainId); setQuery(run.question); setActiveView("ask"); } + function selectDomain(domain: DataDomain, view: View = "overview") { + setActiveDomainId(domain.id); + setDomainMenuOpen(false); + setActiveView(view); + setQuery( + domain.isSample + ? EXAMPLE_QUESTIONS[0] + : "Ask a governed question about this data domain", + ); + setToast(`${domain.name} is now the active data domain.`); + } + + async function createDomain(input: { + name: string; + description: string; + owner: string; + }) { + let created: DataDomain = { + id: `domain_${crypto.randomUUID().replaceAll("-", "")}`, + name: input.name, + slug: input.name + .toLowerCase() + .replace(/[^a-z0-9]+/g, "-") + .replace(/^-+|-+$/g, ""), + description: input.description, + owner: input.owner || "Workspace admin", + status: "draft", + isSample: false, + sourceCount: 0, + runCount: 0, + }; + try { + const response = await fetch("/api/studio/domains", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify(input), + }); + if (response.ok) { + const payload = (await response.json()) as { domain: DataDomain }; + created = payload.domain; + } + } catch { + // Keep the complete workflow available in local demo mode. + } + setDomains((current) => [...current, created]); + setActiveDomainId(created.id); + setCreateDomainOpen(false); + setActiveView("sources"); + setToast(`${created.name} created. Add its first governed data source.`); + } + return (