From 07d3579f7c4b30c05f9b3e52560d792fc98b53c8 Mon Sep 17 00:00:00 2001 From: NoyaOffer Date: Thu, 22 Oct 2026 18:16:02 +0300 Subject: [PATCH 1/2] docs: document AI agent features as capabilities, remove context engine page - Rename agent pages to AI Governance, AI Triage & Resolution, AI Test Coverage, AI Catalog, and AI Performance & Cost, with redirects from the old paths - Rename the code review page to AI Code Review and the Slack page sidebar to Slack agent - Remove the context engine page and badges, redirect it to the agent overview - Update cross-references across feature, guide, FAQ, and home pages Co-Authored-By: Claude Opus 5.5 --- .../{catalog-agent.mdx => ai-catalog.mdx} | 14 +++--- docs/cloud/ai-agents/ai-credits.mdx | 8 +-- ...governance-agent.mdx => ai-governance.mdx} | 14 +++--- ...cost-agent.mdx => ai-performance-cost.mdx} | 12 ++--- ...ndation-agent.mdx => ai-test-coverage.mdx} | 10 ++-- ...ion-agent.mdx => ai-triage-resolution.mdx} | 12 ++--- docs/cloud/ai-agents/context-engine.mdx | 49 ------------------- docs/cloud/ai-agents/overview.mdx | 42 ++++++++++------ docs/cloud/ai-agents/slack-agent.mdx | 4 +- .../best-practices/triage-and-response.mdx | 2 +- docs/cloud/cloud-vs-oss.mdx | 12 ++--- .../alerts-and-incidents/incident-merging.mdx | 2 +- .../alerts-and-incidents/incidents.mdx | 2 +- docs/cloud/features/ci.mdx | 4 +- .../catalog.mdx | 17 +++---- .../data-health.mdx | 1 - docs/cloud/features/config-as-code.mdx | 4 +- .../data-lineage/column-level-lineage.mdx | 1 - .../data-lineage/exposures-lineage.mdx | 1 - docs/cloud/features/data-lineage/lineage.mdx | 3 +- .../data-tests/cloud-tests-overview.mdx | 2 +- .../data-tests/test-coverage-screen.mdx | 6 +-- .../performance-monitoring.mdx | 5 +- docs/cloud/features/table-usage.mdx | 7 ++- docs/cloud/general/ai-privacy-policy.mdx | 2 +- docs/cloud/general/security-and-privacy.mdx | 2 +- .../guides/set-up-elementary-checklist.md | 6 +-- docs/cloud/guides/set-up-elementary.mdx | 6 +-- .../start-using-elementary-checklist.md | 10 ++-- docs/cloud/guides/start-using-elementary.mdx | 10 ++-- .../cloud/integrations/bi/connect-bi-tool.mdx | 1 - docs/cloud/integrations/bi/explo.mdx | 1 - docs/cloud/integrations/bi/hex.mdx | 1 - docs/cloud/integrations/bi/lightdash.mdx | 1 - docs/cloud/integrations/bi/looker.mdx | 1 - docs/cloud/integrations/bi/metabase.mdx | 1 - docs/cloud/integrations/bi/mode.mdx | 1 - docs/cloud/integrations/bi/power-bi.mdx | 1 - docs/cloud/integrations/bi/sigma.mdx | 1 - docs/cloud/integrations/bi/tableau.mdx | 1 - docs/cloud/integrations/bi/thoughtspot.mdx | 1 - .../integrations/code-repo/azure-devops.mdx | 1 - .../integrations/code-repo/bitbucket.mdx | 1 - .../code-repo/connect-code-repo.mdx | 1 - docs/cloud/integrations/code-repo/github.mdx | 1 - docs/cloud/integrations/code-repo/gitlab.mdx | 1 - docs/cloud/integrations/dwh/athena.mdx | 1 - docs/cloud/integrations/dwh/bigquery.mdx | 1 - docs/cloud/integrations/dwh/clickhouse.mdx | 1 - docs/cloud/integrations/dwh/databricks.mdx | 1 - docs/cloud/integrations/dwh/dremio.mdx | 1 - docs/cloud/integrations/dwh/duckdb.mdx | 1 - docs/cloud/integrations/dwh/fabric.mdx | 1 - docs/cloud/integrations/dwh/postgres.mdx | 1 - docs/cloud/integrations/dwh/redshift.mdx | 1 - docs/cloud/integrations/dwh/snowflake.mdx | 1 - docs/cloud/integrations/dwh/spark.mdx | 1 - docs/cloud/integrations/dwh/sqlserver.mdx | 1 - docs/cloud/integrations/dwh/trino.mdx | 1 - docs/cloud/integrations/dwh/vertica.mdx | 1 - .../integrations/elementary-integrations.mdx | 1 - docs/cloud/integrations/governance/atlan.mdx | 1 - .../integrations/metadata-layer/biglake.mdx | 1 - .../integrations/metadata-layer/glue.mdx | 1 - docs/cloud/integrations/pipeline/fivetran.mdx | 1 - .../integrations/semantic-layer/snowflake.mdx | 1 - .../airflow.mdx | 1 - .../dbt-cloud.mdx | 1 - .../dbt-core.mdx | 1 - .../dbt-fusion.mdx | 1 - .../orchestration-tools.mdx | 1 - docs/cloud/introduction.mdx | 6 +-- docs/cloud/python-sdk/overview.mdx | 4 +- docs/cloud/quickstart.mdx | 2 +- docs/cloud/resources/faq.mdx | 14 +++--- docs/cloud/what-is-elementary.mdx | 4 +- docs/docs.json | 39 ++++++++++++--- docs/home.mdx | 18 +++---- docs/skills.md | 4 +- docs/snippets/cloud/features.mdx | 20 ++++---- 80 files changed, 182 insertions(+), 240 deletions(-) rename docs/cloud/ai-agents/{catalog-agent.mdx => ai-catalog.mdx} (56%) rename docs/cloud/ai-agents/{governance-agent.mdx => ai-governance.mdx} (68%) rename docs/cloud/ai-agents/{performance-cost-agent.mdx => ai-performance-cost.mdx} (77%) rename docs/cloud/ai-agents/{test-recommendation-agent.mdx => ai-test-coverage.mdx} (82%) rename docs/cloud/ai-agents/{triage-resolution-agent.mdx => ai-triage-resolution.mdx} (72%) delete mode 100644 docs/cloud/ai-agents/context-engine.mdx diff --git a/docs/cloud/ai-agents/catalog-agent.mdx b/docs/cloud/ai-agents/ai-catalog.mdx similarity index 56% rename from docs/cloud/ai-agents/catalog-agent.mdx rename to docs/cloud/ai-agents/ai-catalog.mdx index 493ae4bc4..11bf67605 100644 --- a/docs/cloud/ai-agents/catalog-agent.mdx +++ b/docs/cloud/ai-agents/ai-catalog.mdx @@ -1,18 +1,18 @@ --- -title: "Catalog Agent" -sidebarTitle: "Catalog agent" +title: "AI Catalog" +sidebarTitle: "AI Catalog" icon: "book-open" badge: "Elementary Cloud" --- - + -**The Catalog Agent helps data analysts, business users, and AI tools find the right data on their own so engineers don't have to be the bottleneck.** By interpreting natural language questions and surfacing the right context—including data health, lineage, and metric definitions—it enables faster, more confident access to the data people need. The Catalog Agent lays the groundwork for more accurate and effective AI-powered workflows across your organization. +**AI Catalog helps data analysts, business users, and AI tools find the right data on their own so engineers don't have to be the bottleneck.** It's the Elementary AI Agent's capability for data discovery. By interpreting natural language questions and surfacing the right context—including data health, lineage, and metric definitions—it enables faster, more confident access to the data people need. AI Catalog lays the groundwork for more accurate and effective AI-powered workflows across your organization. -### Using the Catalog Agent +### Using AI Catalog -Open the Catalog Agent to start a conversation. Just ask a question like **“Which table should I use to analyze user retention?”** or **“Can I trust the orders table?”** and the agent will guide you to the right assets or insights. +Open the agent to start a conversation. Just ask a question like **“Which table should I use to analyze user retention?”** or **“Can I trust the orders table?”** and the agent will guide you to the right assets or insights. The agent will interpret your request using metadata, lineage, and semantic context, and suggest relevant datasets, tables, or metrics. It will also surface freshness, incident status, and test results, and explain how the data is used and whether it can be trusted. @@ -50,7 +50,7 @@ For every asset, the agent provides real-time indicators of reliability: ### Who It’s For -The Catalog Agent is built for **data analysts**, **business users**, **AI copilots**, and **intelligent applications** that need to find and understand the right data without making the data team a bottleneck. +AI Catalog is built for **data analysts**, **business users**, **AI copilots**, and **intelligent applications** that need to find and understand the right data without making the data team a bottleneck. We take data privacy seriously. Read our [**AI Privacy Policy**](/cloud/general/ai-privacy-policy) to understand how AI features are secured. diff --git a/docs/cloud/ai-agents/ai-credits.mdx b/docs/cloud/ai-agents/ai-credits.mdx index a233fc0dd..9bdae9847 100644 --- a/docs/cloud/ai-agents/ai-credits.mdx +++ b/docs/cloud/ai-agents/ai-credits.mdx @@ -14,10 +14,10 @@ With [MCP](/cloud/mcp/overview), Elementary returns data to the connected agent. | Action | What the AI does | |---|---| -| [**Incident investigation**](/cloud/ai-agents/triage-resolution-agent) | Diagnoses a data issue: root cause, impacted tables, suggested fix | -| [**Data discovery**](/cloud/ai-agents/catalog-agent) | Identifies which tables or columns to use for a specific metric or use case | -| [**Test coverage**](/cloud/ai-agents/test-recommendation-agent) | Generates dbt test PRs based on configured data policies and opens them for review | -| [**CI PR review**](/cloud/features/ci) | Lineage-aware review on every dbt PR, surfacing downstream impact before merge | +| [**Incident investigation**](/cloud/ai-agents/ai-triage-resolution) | Diagnoses a data issue: root cause, impacted tables, suggested fix | +| [**Data discovery**](/cloud/ai-agents/ai-catalog) | Identifies which tables or columns to use for a specific metric or use case | +| [**Test coverage**](/cloud/ai-agents/ai-test-coverage) | Generates dbt test PRs based on configured data policies and opens them for review | +| [**AI Code Review**](/cloud/features/ci) | Lineage-aware review on every dbt PR, surfacing downstream impact before merge | --- diff --git a/docs/cloud/ai-agents/governance-agent.mdx b/docs/cloud/ai-agents/ai-governance.mdx similarity index 68% rename from docs/cloud/ai-agents/governance-agent.mdx rename to docs/cloud/ai-agents/ai-governance.mdx index 3b490bbaf..441905d88 100644 --- a/docs/cloud/ai-agents/governance-agent.mdx +++ b/docs/cloud/ai-agents/ai-governance.mdx @@ -1,22 +1,22 @@ --- -title: "Governance Agent" -sidebarTitle: "Governance agent" +title: "AI Governance" +sidebarTitle: "AI Governance" icon: "tags" badge: "Elementary Cloud" --- - + **Governing your data isn't just about internal organization, it's the key to making your data reliably usable by both humans and AI.** -The Governance Agent helps you scale and maintain trusted, well-documented data assets without manual effort. It evaluates your metadata, identifies gaps, and suggests improvements, all based on how your organization already tags, documents, and assigns ownership. +AI Governance is the Elementary AI Agent's capability for keeping data assets documented, owned, and classified without manual effort. The agent evaluates your metadata, identifies gaps, and suggests improvements, all based on how your organization already tags, documents, and assigns ownership. -By learning from your environment’s existing conventions, the agent brings consistency, policy enforcement, and AI-readiness across your pipelines. +By learning from your environment’s existing conventions, it brings consistency, policy enforcement, and AI-readiness across your pipelines. -### **Using the Governance Agent** +### **Using AI Governance** -You can trigger the Governance Agent from the Catalog screen, and ask it a question like **"Which tables in the marketing domain are missing ownership?"**. It will analyze the asset’s metadata and context, validate it against both best practices and your custom policies, and generate a list of actionable suggestions. When you’re ready, the agent can open a pull request with the proposed metadata changes. +Open the agent from the Catalog screen and ask it a question like **"Which tables in the marketing domain are missing ownership?"**. It will analyze the asset’s metadata and context, validate it against both best practices and your custom policies, and generate a list of actionable suggestions. When you’re ready, the agent can open a pull request with the proposed metadata changes.
diff --git a/docs/cloud/ai-agents/performance-cost-agent.mdx b/docs/cloud/ai-agents/ai-performance-cost.mdx similarity index 77% rename from docs/cloud/ai-agents/performance-cost-agent.mdx rename to docs/cloud/ai-agents/ai-performance-cost.mdx index b7c659fdc..af5247e04 100644 --- a/docs/cloud/ai-agents/performance-cost-agent.mdx +++ b/docs/cloud/ai-agents/ai-performance-cost.mdx @@ -1,19 +1,19 @@ --- -title: "Performance & Cost Agent" -sidebarTitle: "Performance & cost agent" +title: "AI Performance & Cost" +sidebarTitle: "AI Performance & Cost" icon: "comments-dollar" badge: "Elementary Cloud" --- - + -**Optimize long-running queries, eliminate bottlenecks, and keep compute costs under control without digging through logs or rewriting SQL by hand.** The Performance and Cost Agent helps you identify inefficiencies in your pipelines and resolve them quickly, so your data stays fresh, fast, and within budget. +**Optimize long-running queries, eliminate bottlenecks, and keep compute costs under control without digging through logs or rewriting SQL by hand.** AI Performance & Cost is the Elementary AI Agent's capability for finding inefficiencies in your pipelines and resolving them quickly, so your data stays fresh, fast, and within budget. It improves performance by analyzing query logic, historical execution patterns, and resource usage. This makes it easier to meet data SLAs and ensure products, dashboards, and AI models get the data they need, when they need it. -### Using the Performance and Cost Agent -Trigger the agent from any asset flagged as underperforming or from a query known to be slow or expensive. It reviews the SQL, surrounding metadata, and historical performance, then recommends code changes and resource configuration tweaks to improve efficiency. +### Using AI Performance & Cost +Open the agent from any asset flagged as underperforming or from a query known to be slow or expensive. It reviews the SQL, surrounding metadata, and historical performance, then recommends code changes and resource configuration tweaks to improve efficiency. You’ll see an estimate of time and cost savings before applying changes. Once approved, the agent opens a pull request with its suggested improvements and can continue iterating until performance targets are met. diff --git a/docs/cloud/ai-agents/test-recommendation-agent.mdx b/docs/cloud/ai-agents/ai-test-coverage.mdx similarity index 82% rename from docs/cloud/ai-agents/test-recommendation-agent.mdx rename to docs/cloud/ai-agents/ai-test-coverage.mdx index d390b1b9e..a32fbb6d9 100644 --- a/docs/cloud/ai-agents/test-recommendation-agent.mdx +++ b/docs/cloud/ai-agents/ai-test-coverage.mdx @@ -1,18 +1,18 @@ --- -title: "Test Recommendation Agent" -sidebarTitle: "Test recommendation agent" +title: "AI Test Coverage" +sidebarTitle: "AI Test Coverage" icon: "vial-circle-check" badge: "Elementary Cloud" --- - + -**Make sure you’re covered—not by how many tests you write, but by how many problems you catch.** The Test Recommendation Agent analyzes your data asset, metadata, lineage, and existing test suite to suggest high-impact tests that match your team’s style and priorities. +**Make sure you’re covered—not by how many tests you write, but by how many problems you catch.** AI Test Coverage is the Elementary AI Agent's capability for finding coverage gaps and closing them. The agent analyzes your data assets, metadata, lineage, and existing test suite to suggest high-impact tests that match your team’s style and priorities. It helps you expand meaningful coverage, reduce alert fatigue, and maintain a reliable test strategy as your pipelines evolve. -### Using the Test Recommendation Agent +### Using AI Test Coverage Open the agent from the test configuration screen and ask a question like **"What tests can ensure all of my customer data is up to date?"** or **"Which upstream tables need tests to protect my revenue dashboard?"**. The agent generates recommendations based on asset type, criticality, lineage, and your existing test configurations. You can approve, adjust, or skip suggestions as needed. Once approved, the agent opens a pull request with changes that follow your team’s current test structure. diff --git a/docs/cloud/ai-agents/triage-resolution-agent.mdx b/docs/cloud/ai-agents/ai-triage-resolution.mdx similarity index 72% rename from docs/cloud/ai-agents/triage-resolution-agent.mdx rename to docs/cloud/ai-agents/ai-triage-resolution.mdx index 993eb57b2..9dc1d6fb1 100644 --- a/docs/cloud/ai-agents/triage-resolution-agent.mdx +++ b/docs/cloud/ai-agents/ai-triage-resolution.mdx @@ -1,20 +1,20 @@ --- -title: "Triage & Resolution Agent" -sidebarTitle: "Triage & resolution agent" +title: "AI Triage & Resolution" +sidebarTitle: "AI Triage & Resolution" icon: "toolbox" badge: "Elementary Cloud" --- - + **Automates root cause analysis, clarifies the impact, and suggests actionable fixes so you spend less time triaging and more time solving what matters.** The result is a shift from manual investigation to a faster, more focused workflow, allowing you to focus on delivering business value rather than fighting fires. -Whether it's a failed test, a model issue, or an unexpected anomaly, it inspects recent code changes, execution history, lineage, and, when allowed, the data itself. It prioritizes the incidents that matter, proposes a fix, and can open a pull request for you. That way, incidents stop slowing you down, and pipelines begin to fix themselves. +AI Triage & Resolution is the Elementary AI Agent's capability for investigating and fixing incidents. Whether it's a failed test, a model issue, or an unexpected anomaly, the agent inspects recent code changes, execution history, lineage, and, when allowed, the data itself. It prioritizes the incidents that matter, proposes a fix, and can open a pull request for you. That way, incidents stop slowing you down, and pipelines begin to fix themselves. -## Using the Triage & Resolution agent +## Using AI Triage & Resolution -From the Incident Management screen click **“Investigate with AI”** to trigger the agent. It will begin analyzing the incident and surface insights. You can follow along, ask clarifying questions, and when you're ready, allow the agent to open a pull request with its suggested fix - all within your review flow. +From the Incident Management screen click **“Investigate with AI”** to start an investigation. The agent will begin analyzing the incident and surface insights. You can follow along, ask clarifying questions, and when you're ready, allow the agent to open a pull request with its suggested fix - all within your review flow.
diff --git a/docs/cloud/ai-agents/context-engine.mdx b/docs/cloud/ai-agents/context-engine.mdx deleted file mode 100644 index 258a673c4..000000000 --- a/docs/cloud/ai-agents/context-engine.mdx +++ /dev/null @@ -1,49 +0,0 @@ ---- -title: "Context Engine" -sidebarTitle: "Context Engine" -icon: "brain-circuit" ---- - -The Context Engine is the knowledge layer that powers the Elementary AI Agent and the MCP Server. - -It connects lineage across your entire stack — from your data warehouse and dbt models through to BI dashboards and code — and layers in everything Elementary knows about how your data is actually performing. The result is a unified, always-current picture of your stack: what exists, how it's connected, how it's running, and its health status. - -## What it holds - -**From your connected integrations:** -- **Data warehouse** — table metadata, schema definitions, query history, usage stats -- **dbt** — model definitions, test configurations, run results, documentation -- **BI tools** — dashboards, reports, and their upstream dependencies -- **Code repository** — recent commits, pull requests, change history -- **Orchestration** — job runs, execution history - -**From Elementary's own monitoring:** -- **Test results** — pass/fail history across dbt, Python, and Cloud tests, coverage gaps -- **Test performance** — execution times and cost per test run -- **Task performance** — execution times, cost, trends over time -- **Health scores** — data quality dimensions across all assets - -Lineage connects these layers. A table in your warehouse maps to a dbt model, which feeds a dashboard, which is owned by a team. When something breaks, the Context Engine already knows the blast radius. - -## What it enables - -Because the Context Engine holds a complete, connected view of your stack, the Elementary AI Agent can: - -- Trace the root cause of an incident across models, code changes, and upstream dependencies -- Understand the downstream impact before escalating or fixing -- Identify coverage gaps and recommend tests for the right assets -- Understand which tests are redundant and remove them to speed up the pipeline -- Enrich metadata with descriptions that reflect how assets are actually used - -The MCP Server exposes this same context to external AI tools — Claude, Cursor, or any MCP-compatible client — so they can query your stack with full awareness. - -The more integrations you connect, the more complete the context. Lineage between dbt and BI tools, for example, only works when both are connected. - - - - Add data sources to expand what the Context Engine can see. - - - Expose the Context Engine to external AI tools. - - diff --git a/docs/cloud/ai-agents/overview.mdx b/docs/cloud/ai-agents/overview.mdx index 2638cec8c..b41468cc6 100644 --- a/docs/cloud/ai-agents/overview.mdx +++ b/docs/cloud/ai-agents/overview.mdx @@ -7,31 +7,29 @@ badge: "Elementary Cloud" The Elementary AI Agent automates the workflows that slow data teams down most: finding coverage gaps, triaging incidents, fixing broken pipelines, enriching metadata, and optimizing query performance. -It's a single agent backed by the [Context Engine](/cloud/ai-agents/context-engine) — a continuously updated knowledge layer built from every integration you connect. Because it has full context across your stack, it can act on incidents, recommend tests, and surface insights without requiring you to pull together the picture manually. +It's one agent with a set of capabilities, each built for a specific workflow. You work with the same agent whether you open it from an incident, the catalog, the test coverage screen, or a Slack thread. Every action the agent takes goes through a pull request or requires explicit confirmation. Your team stays in control of what gets merged. -## Specialized agents - -The Elementary AI Agent orchestrates a set of specialized agents, each focused on a specific workflow: +## Capabilities - + Enriches metadata and enforces documentation standards across your data assets. - + Investigates incidents, traces root causes, and opens pull requests with proposed fixes. - + Identifies coverage gaps and recommends or generates tests for the right assets. - - Surfaces data assets, their relationships, and usage context in natural language. + + Finds the right data assets in natural language, with their relationships, usage, and health. - + Detects slow or expensive models, flags redundant tests that slow down the pipeline, and suggests optimizations. - + Reviews every pull request that touches your dbt models, with a summary comment and findings on the changed lines. @@ -39,7 +37,7 @@ The Elementary AI Agent orchestrates a set of specialized agents, each focused o ## Where you can use it - + Tag `@Elementary` in an alert thread to investigate without leaving Slack. @@ -47,13 +45,27 @@ The Elementary AI Agent orchestrates a set of specialized agents, each focused o +## What the agent works from + +The agent reasons over what Elementary already collects from your connected integrations, tied together through [lineage](/cloud/features/data-lineage/lineage): + +- **Lineage** across your warehouse, dbt models, BI dashboards, and code, down to the column level +- **Test results and incidents** from dbt, Python, and cloud tests, including history and coverage gaps +- **Run and performance history** for models, tests, and orchestration jobs, including execution time and cost +- **Code changes** from your connected repository, such as recent commits and pull requests +- **Metadata** such as descriptions, tags, owners, and table usage + +This is what lets the agent trace a failure back to the commit that caused it, or tell you which dashboards an incident affects, without you assembling the picture first. + +The more integrations you connect, the more the agent can see. Lineage between dbt and BI tools, for example, only works when both are connected. See [integrations](/cloud/integrations/elementary-integrations). + ## How it works -The agent runs on self-hosted models through Amazon Bedrock. No data leaves your environment, and all AI features are opt-in by default. Agent behavior is governed by AI policies you define — reviewable, adjustable, and aligned to your governance standards. +The agent runs on self-hosted models through Amazon Bedrock. No data leaves your environment, and all AI features are opt-in by default. Agent behavior is governed by AI policies you define: reviewable, adjustable, and aligned to your governance standards. - - Understand what the agent knows and how it knows it. + + What uses AI credits, and how to get more from them. How AI features are secured and what data is accessed. diff --git a/docs/cloud/ai-agents/slack-agent.mdx b/docs/cloud/ai-agents/slack-agent.mdx index 7ae117bcb..1d600f21f 100644 --- a/docs/cloud/ai-agents/slack-agent.mdx +++ b/docs/cloud/ai-agents/slack-agent.mdx @@ -1,13 +1,13 @@ --- title: "Elementary Agent in Slack" -sidebarTitle: "Slack" +sidebarTitle: "Slack agent" icon: "slack" badge: "Elementary Cloud" --- Tag `@Elementary` in a Slack thread to ask the agent a question, and it answers in that thread. When you ask in the thread of an Elementary alert, the agent already knows which incident the alert refers to, so you can go straight to "why did this fail?" without repeating any context. -It is the same agent as in the web app, backed by the same [Context Engine](/cloud/ai-agents/context-engine), so it can investigate incidents, trace root causes, explain lineage, and check test coverage. Actions that change anything still go through a pull request or an explicit confirmation. +It is the same agent as in the web app, with the same [capabilities](/cloud/ai-agents/overview#capabilities), so it can investigate incidents, trace root causes, explain lineage, and check test coverage. Actions that change anything still go through a pull request or an explicit confirmation. ## How it works diff --git a/docs/cloud/best-practices/triage-and-response.mdx b/docs/cloud/best-practices/triage-and-response.mdx index bacce06dd..34ecacbdd 100644 --- a/docs/cloud/best-practices/triage-and-response.mdx +++ b/docs/cloud/best-practices/triage-and-response.mdx @@ -109,7 +109,7 @@ When triaging incidents, there are 4 steps to go through: 4. Post mortem - Quality learnings from incidents is how you improve over time, and reduce the time to resolution and frequency of future incidents. -Use the **Investigate with AI** action on any incident to trigger the [Triage & Resolution Agent](/cloud/ai-agents/triage-resolution-agent). It can perform impact analysis, root cause analysis, and even suggest [merging related incidents](/cloud/features/alerts-and-incidents/incident-merging#ai-agent-assisted-merge) by shared root cause. +Use the **Investigate with AI** action on any incident to start [AI Triage & Resolution](/cloud/ai-agents/ai-triage-resolution). It can perform impact analysis, root cause analysis, and even suggest [merging related incidents](/cloud/features/alerts-and-incidents/incident-merging#ai-agent-assisted-merge) by shared root cause. ### Impact analysis diff --git a/docs/cloud/cloud-vs-oss.mdx b/docs/cloud/cloud-vs-oss.mdx index fa8e2211f..e2a5b8123 100644 --- a/docs/cloud/cloud-vs-oss.mdx +++ b/docs/cloud/cloud-vs-oss.mdx @@ -15,9 +15,9 @@ A self-maintained, open-source CLI for teams running dbt. It requires a dbt proj A fully managed platform for data observability, quality, and governance, with no infrastructure to maintain. It connects directly to your data warehouse and does not require dbt, and covers your whole stack end to end - from ingestion through the warehouse and dbt to BI and AI consumption - across multiple projects, with column-level lineage, ML-powered detection, governance, and enterprise security built in. -All of these signals feed the context engine that powers the Elementary AI agents and MCP. The agents work in the background while you supervise, and help you work at scale: they recommend tests, triage and resolve incidents, enrich governance metadata, and optimize costly models and tests. +On top of that, the Elementary AI Agent works in the background while you supervise, and helps you work at scale: it recommends tests, triages and resolves incidents, enriches governance metadata, and optimizes costly models and tests. -Through the MCP, the Elementary context is provided to other agents, like analytics and code agents, so they return more accurate results on trustworthy data. +Through the MCP server, other agents, like analytics and code agents, get the same lineage, test results, and incidents, so they return more accurate results on trustworthy data. No lock-in. Any dbt tests and configuration stay in your dbt project and code repository, and Cloud lets you manage tests and monitors from the UI or as code. OSS and Cloud share the same foundation, so you can move between them freely. @@ -32,7 +32,7 @@ Below is a detailed comparison between the OSS and Cloud features: | Feature | **OSS** | **Cloud** | |------------------------|------------------------------------------|---------------------------------------------------------------------------------------------------| | **Requirements** |
  • A dbt project, with the [Elementary dbt package](/data-tests/dbt/quickstart-package) installed
  • Your own environment to host the CLI and the report
|
  • A [data warehouse connection](/cloud/integrations/elementary-integrations)
  • dbt is optional — connect a dbt project to also collect its models, tests, and run results
| -| **AI agents & MCP** |
  • No AI
|
  • A [team of AI agents](/cloud/ai-agents/overview) you put to work at scale while you supervise (governance, triage, test recommendation, catalog, performance & cost)
  • [Context engine](/cloud/ai-agents/context-engine) across your stack
  • [MCP server](/cloud/mcp/overview) for your own AI tools
| +| **AI agent & MCP** |
  • No AI
|
  • The [Elementary AI Agent](/cloud/ai-agents/overview), which you put to work at scale while you supervise (AI Governance, AI Triage & Resolution, AI Test Coverage, AI Catalog, AI Performance & Cost)
  • [MCP server](/cloud/mcp/overview) for your own AI tools
| | **Coverage** |
  • Single dbt project and its warehouse
|
  • Multiple projects, connected directly to your warehouse
  • End to end, from [ingestion](/cloud/integrations/pipeline/fivetran) (Fivetran, SAP, and more) through the warehouse and dbt to [BI](/cloud/features/data-lineage/exposures-lineage) and [AI consumption](/cloud/mcp/overview)
| | **Data lineage** |
  • Table-level lineage
|
  • [Column-level lineage up to BI](/cloud/features/data-lineage/column-level-lineage)
  • [Downstream exposures](/cloud/features/data-lineage/exposures-lineage)
| | **Detection** |
  • dbt tests and anomaly detection tests (freshness, volume)
|
  • [ML-powered anomaly detection](/cloud/features/anomaly-detection/monitors-overview)
  • [No-code cloud tests](/cloud/features/data-tests/cloud-tests-overview), including [data contracts](/cloud/features/data-tests/data-contract-test) and [custom SQL tests](/cloud/features/data-tests/custom-sql-tests)
  • [Schema validation](/cloud/features/data-tests/schema-validation-test)
  • Bulk add/edit for tests
| @@ -47,7 +47,7 @@ Below is a detailed comparison between the OSS and Cloud features: - Elementary OSS is an open-source CLI and dbt package that you host yourself. It covers a single dbt project with dbt test results, table-level lineage, test coverage, and basic alerting. Elementary Cloud is a fully managed platform that adds ML-powered anomaly detection, column-level lineage up to BI, incident management, governance, enterprise security, and a team of AI agents you put to work at scale, across your whole stack, with no infrastructure to maintain. + Elementary OSS is an open-source CLI and dbt package that you host yourself. It covers a single dbt project with dbt test results, table-level lineage, test coverage, and basic alerting. Elementary Cloud is a fully managed platform that adds ML-powered anomaly detection, column-level lineage up to BI, incident management, governance, enterprise security, and an AI agent you put to work at scale, across your whole stack, with no infrastructure to maintain. For Elementary Cloud, no. You connect your data warehouse to get started, and Elementary monitors the databases and schemas you select. dbt is one popular integration, not a prerequisite — if you do use it, the Elementary dbt package adds automatic collection of your dbt test and run results on top. @@ -61,7 +61,7 @@ Below is a detailed comparison between the OSS and Cloud features: [Talk to the team](https://meetings-eu1.hubspot.com/joost-boonzajer-flaes/intro-call-docs) and they'll help set up Elementary and open an account for you. If you're not sure whether OSS or Cloud fits your setup, they can help you decide too. - Yes. Cloud builds on the same open-source foundation and includes the OSS capabilities, then adds automated detection, lineage, governance, security, and AI agents on top. + Yes. Cloud builds on the same open-source foundation and includes the OSS capabilities, then adds automated detection, lineage, governance, security, and an AI agent on top. Yes. If you use dbt, Cloud uses the same Elementary dbt package, so moving from OSS to Cloud means connecting your existing dbt project rather than starting over. You can also connect your data warehouse directly. See the [dbt package quickstart](/data-tests/dbt/quickstart-package). @@ -73,7 +73,7 @@ Below is a detailed comparison between the OSS and Cloud features: Elementary Cloud is SOC 2 Type II certified and HIPAA compliant. It collects metadata, logs, and aggregated metrics only, with no read access to raw data in your warehouse. It supports [RBAC](/cloud/features/roles-and-permissions), SSO, audit logs, and [PrivateLink](/cloud/integrations/security-and-connectivity/aws-privatelink-integration). See [security and privacy](/cloud/general/security-and-privacy) for details. - Choose OSS for a free, self-hosted package on a single dbt project. Choose Cloud to cover multiple projects end to end with automated detection, governance, enterprise security, and AI agents you supervise at scale, without managing infrastructure. Still unsure? [Talk to the team](https://meetings-eu1.hubspot.com/joost-boonzajer-flaes/intro-call-docs) and we'll help you decide. + Choose OSS for a free, self-hosted package on a single dbt project. Choose Cloud to cover multiple projects end to end with automated detection, governance, enterprise security, and an AI agent you supervise at scale, without managing infrastructure. Still unsure? [Talk to the team](https://meetings-eu1.hubspot.com/joost-boonzajer-flaes/intro-call-docs) and we'll help you decide. diff --git a/docs/cloud/features/alerts-and-incidents/incident-merging.mdx b/docs/cloud/features/alerts-and-incidents/incident-merging.mdx index ca7e70a9b..b40497717 100644 --- a/docs/cloud/features/alerts-and-incidents/incident-merging.mdx +++ b/docs/cloud/features/alerts-and-incidents/incident-merging.mdx @@ -53,7 +53,7 @@ The rule requires at least two freshness failures from the same source within a ## AI agent-assisted merge -The [Triage & Resolution Agent](/cloud/ai-agents/triage-resolution-agent) can investigate multiple open incidents, identify shared root causes, and propose merging them. When the agent suggests a merge: +[AI Triage & Resolution](/cloud/ai-agents/ai-triage-resolution) can investigate multiple open incidents, identify shared root causes, and propose merging them. When the agent suggests a merge: 1. A merge plan appears in the chat with the selected incidents and a proposed primary. 2. Review the plan and click **Confirm** to proceed. diff --git a/docs/cloud/features/alerts-and-incidents/incidents.mdx b/docs/cloud/features/alerts-and-incidents/incidents.mdx index 44f347cb7..2df3f1b37 100644 --- a/docs/cloud/features/alerts-and-incidents/incidents.mdx +++ b/docs/cloud/features/alerts-and-incidents/incidents.mdx @@ -48,4 +48,4 @@ The detail drawer also shows the incident's metadata (assignee, severity, status ## Triage with the AI Agent -From any incident, click **Investigate with AI** to trigger the [Triage & Resolution Agent](/cloud/ai-agents/triage-resolution-agent). It inspects code changes, execution history, and lineage to surface the root cause and propose a fix. The agent can also identify related incidents and [suggest merging them](/cloud/features/alerts-and-incidents/incident-merging#ai-agent-assisted-merge). +From any incident, click **Investigate with AI** to start [AI Triage & Resolution](/cloud/ai-agents/ai-triage-resolution). It inspects code changes, execution history, and lineage to surface the root cause and propose a fix. The agent can also identify related incidents and [suggest merging them](/cloud/features/alerts-and-incidents/incident-merging#ai-agent-assisted-merge). diff --git a/docs/cloud/features/ci.mdx b/docs/cloud/features/ci.mdx index e320d2dc2..32fa457ea 100644 --- a/docs/cloud/features/ci.mdx +++ b/docs/cloud/features/ci.mdx @@ -1,6 +1,6 @@ --- -title: "Data Quality Code Review" -sidebarTitle: "Code Review" +title: "AI Code Review" +sidebarTitle: "AI Code Review" icon: "comment-code" badge: "Elementary Cloud" --- diff --git a/docs/cloud/features/collaboration-and-communication/catalog.mdx b/docs/cloud/features/collaboration-and-communication/catalog.mdx index 22a0f76c7..750ececc3 100644 --- a/docs/cloud/features/collaboration-and-communication/catalog.mdx +++ b/docs/cloud/features/collaboration-and-communication/catalog.mdx @@ -1,6 +1,5 @@ --- title: "Data Catalog" -badge: "Context Engine" --- The Elementary Cloud catalog is a dynamic workspace for managing data assets. It combines your dbt models, sources, columns, tests, and metrics with metadata, test results, and ownership in one place. @@ -40,9 +39,9 @@ You can view the data assets by their location in the DWH or in the dbt project, You can edit metadata directly in the catalog — model and column descriptions, owners, tags, and critical asset status — individually or in bulk. For example, assign an owner to multiple models at once or apply a shared tag to a group of assets. All edits are synced back to your dbt project as a pull request, so documentation stays version-controlled. -You can also use the [Governance Agent](/cloud/ai-agents/governance-agent) to identify missing documentation, unowned assets, or inconsistent tags and generate suggested changes automatically. +You can also use [AI Governance](/cloud/ai-agents/ai-governance) to identify missing documentation, unowned assets, or inconsistent tags and generate suggested changes automatically. - + Identify metadata gaps and generate descriptions, tags, and ownership assignments automatically. @@ -70,10 +69,10 @@ View upstream and downstream dependencies for each dataset, and export the full Lineage export ### Table usage -See how often each table is queried in production over a rolling 30-day window, compare usage at a glance from the **Metadata** tab, and hover for percentile rank, query volume, and unique users. Usage is also available as context to Elementary’s AI agents. Read more in [Table usage](/cloud/features/table-usage). +See how often each table is queried in production over a rolling 30-day window, compare usage at a glance from the **Metadata** tab, and hover for percentile rank, query volume, and unique users. Usage is also available to the Elementary AI Agent. Read more in [Table usage](/cloud/features/table-usage). -### AI Agents for Discovery and Governance -Elementary includes two agents that use the catalog to support your team: -- The [Catalog agent](/cloud/ai-agents/catalog-agent) helps users find relevant models, columns, or metrics using natural language. This is especially helpful for new team members or business users who are less familiar with your dbt structure. -- The [Governance agent](/cloud/ai-agents/governance-agent) identifies missing documentation, unowned assets, or inconsistent tags. It surfaces these issues automatically and can suggest actions to improve coverage. -These agents run continuously and help keep the catalog useful and complete without requiring constant manual work. +### AI Catalog and AI Governance +Two capabilities of the Elementary AI Agent use the catalog to support your team: +- [AI Catalog](/cloud/ai-agents/ai-catalog) helps users find relevant models, columns, or metrics using natural language. This is especially helpful for new team members or business users who are less familiar with your dbt structure. +- [AI Governance](/cloud/ai-agents/ai-governance) identifies missing documentation, unowned assets, or inconsistent tags. It surfaces these issues automatically and can suggest actions to improve coverage. +Both run continuously and help keep the catalog useful and complete without requiring constant manual work. diff --git a/docs/cloud/features/collaboration-and-communication/data-health.mdx b/docs/cloud/features/collaboration-and-communication/data-health.mdx index 95b60fb26..30b044304 100644 --- a/docs/cloud/features/collaboration-and-communication/data-health.mdx +++ b/docs/cloud/features/collaboration-and-communication/data-health.mdx @@ -1,7 +1,6 @@ --- title: Data Health Dashboard sidebarTitle: Data Health Dashboard -badge: "Context Engine" --- import DataHealthIntro from '/snippets/cloud/features/data-health/data-health-intro.mdx'; diff --git a/docs/cloud/features/config-as-code.mdx b/docs/cloud/features/config-as-code.mdx index aec6020c1..549e4c999 100644 --- a/docs/cloud/features/config-as-code.mdx +++ b/docs/cloud/features/config-as-code.mdx @@ -38,8 +38,8 @@ From there, your team can review, approve, and merge using the exact same proces ### AI-Powered Configurations -Elementary’s [AI Agents](/cloud/ai-agents/overview) work the same way: -- You can ask them to recommend tests, optimize queries, or update metadata. +The [Elementary AI Agent](/cloud/ai-agents/overview) works the same way: +- You can ask it to recommend tests, optimize queries, or update metadata. - Once you’re satisfied with the iteration, the agent will automatically open a PR on your behalf—integrating seamlessly into your workflow. ### Works with MCP diff --git a/docs/cloud/features/data-lineage/column-level-lineage.mdx b/docs/cloud/features/data-lineage/column-level-lineage.mdx index ffc8f559d..3b6479e6e 100644 --- a/docs/cloud/features/data-lineage/column-level-lineage.mdx +++ b/docs/cloud/features/data-lineage/column-level-lineage.mdx @@ -1,7 +1,6 @@ --- title: Column-Level Lineage sidebarTitle: Column level lineage -badge: "Context Engine" --- The table nodes in Elementary lineage can be expanded to show the columns. When you diff --git a/docs/cloud/features/data-lineage/exposures-lineage.mdx b/docs/cloud/features/data-lineage/exposures-lineage.mdx index b630388f4..74c887b3a 100644 --- a/docs/cloud/features/data-lineage/exposures-lineage.mdx +++ b/docs/cloud/features/data-lineage/exposures-lineage.mdx @@ -1,7 +1,6 @@ --- title: Lineage to Downstream Dashboards and Tools sidebarTitle: Lineage to BI -badge: "Context Engine" --- import BiCards from '/snippets/cloud/integrations/cards-groups/bi-cards.mdx'; diff --git a/docs/cloud/features/data-lineage/lineage.mdx b/docs/cloud/features/data-lineage/lineage.mdx index a7546ed6f..4e504f029 100644 --- a/docs/cloud/features/data-lineage/lineage.mdx +++ b/docs/cloud/features/data-lineage/lineage.mdx @@ -1,10 +1,9 @@ --- title: End-to-End Data Lineage sidebarTitle: Lineage overview -badge: "Context Engine" --- -Lineage is the foundation of the [Context Engine](/cloud/ai-agents/context-engine) — it's how Elementary maps your entire data stack, from ingestion sources through transformations to BI and downstream consumers. Every asset, test result, and health signal is connected through this graph, which is what allows the AI Agent and MCP to reason across your stack. +Lineage is how Elementary maps your entire data stack, from ingestion sources through transformations to BI and downstream consumers. Every asset, test result, and health signal is connected through this graph, which is what allows the [Elementary AI Agent](/cloud/ai-agents/overview) and the [MCP server](/cloud/mcp/overview) to reason across your stack. Elementary builds column-level lineage automatically from your data warehouse metadata and integrations with dbt, BI tools, Python pipelines, and other sources. The graph is updated frequently so it reflects your current state. diff --git a/docs/cloud/features/data-tests/cloud-tests-overview.mdx b/docs/cloud/features/data-tests/cloud-tests-overview.mdx index f9e3b0e30..9542b377e 100644 --- a/docs/cloud/features/data-tests/cloud-tests-overview.mdx +++ b/docs/cloud/features/data-tests/cloud-tests-overview.mdx @@ -26,7 +26,7 @@ Freshness, volume, and data contract tests use warehouse metadata collected by E ## Adding cloud tests -Cloud freshness and volume tests are activated automatically on all your sources. You can add additional tests from the UI or via the [Test Recommendation Agent](/cloud/ai-agents/test-recommendation-agent): +Cloud freshness and volume tests are activated automatically on all your sources. You can add additional tests from the UI or via [AI Test Coverage](/cloud/ai-agents/ai-test-coverage): 1. Navigate to the **Test Configuration** page and click **Add Test**. For asset-based tests, you can also start by selecting assets in the Catalog. 2. Filter on **Elementary Cloud** and choose a test. diff --git a/docs/cloud/features/data-tests/test-coverage-screen.mdx b/docs/cloud/features/data-tests/test-coverage-screen.mdx index acd9fa8a6..76b0b12ce 100644 --- a/docs/cloud/features/data-tests/test-coverage-screen.mdx +++ b/docs/cloud/features/data-tests/test-coverage-screen.mdx @@ -6,7 +6,7 @@ sidebarTitle: Test coverage management There are two ways to manage test coverage in Elementary: - **Test Coverage screen** — a visual overview of coverage across all your assets and dimensions, with the ability to add missing tests directly from the UI -- **[Test Recommendation Agent](/cloud/ai-agents/test-recommendation-agent)** — analyzes your pipeline, lineage, and existing tests to suggest what’s missing and where, then adds tests on your behalf +- **[AI Test Coverage](/cloud/ai-agents/ai-test-coverage)** — analyzes your pipeline, lineage, and existing tests to suggest what’s missing and where, then adds tests on your behalf Use the screen to get a high-level picture and act on gaps in bulk. Use the agent when you want coverage recommendations driven by context — patterns, lineage, and asset criticality. @@ -76,8 +76,8 @@ The first **6 core dimensions** each contribute **15%** to the total score. The -## Test Recommendation AI Agent +## AI Test Coverage -Alongside the Test Coverage screen, you can also use our [**Test Recommendation Agent**](https://docs.elementary-data.com/cloud/ai-agents/test-recommendation-agent) to help you improve test coverage. +Alongside the Test Coverage screen, you can also use [**AI Test Coverage**](/cloud/ai-agents/ai-test-coverage) to help you improve test coverage. Together, the coverage screen and the agent give you visibility and guidance to focus your efforts where they’ll have the most impact. diff --git a/docs/cloud/features/performance-monitoring/performance-monitoring.mdx b/docs/cloud/features/performance-monitoring/performance-monitoring.mdx index d4a74cdd4..4b51a7366 100644 --- a/docs/cloud/features/performance-monitoring/performance-monitoring.mdx +++ b/docs/cloud/features/performance-monitoring/performance-monitoring.mdx @@ -1,7 +1,6 @@ --- title: Performance Monitoring sidebarTitle: Performance monitoring -badge: "Context Engine" --- Monitoring the performance of your data pipeline is critical for maintaining data quality, reliability, and operational efficiency. @@ -10,7 +9,7 @@ Proactively monitoring performance issues enables to detect bottlenecks and oppo There are two ways to identify and act on performance issues in Elementary: - **Performance pages** — explore execution times, trends, and fail rates for models and tests across your pipeline -- **[Performance & Cost Agent](/cloud/ai-agents/performance-cost-agent)** — analyzes patterns across your pipeline, surfaces inefficiencies, suggests query improvements, and flags redundant tests that are slowing things down +- **[AI Performance & Cost](/cloud/ai-agents/ai-performance-cost)** — analyzes patterns across your pipeline, surfaces inefficiencies, suggests query improvements, and flags redundant tests that are slowing things down Use the pages to monitor and spot outliers. Use the agent to investigate and get actionable recommendations. @@ -46,4 +45,4 @@ It is also useful to use the navigation bar to filter the results, and see run t ## Optimize with the AI Agent -Once you've identified slow or expensive models and tests, use the [Performance & Cost Agent](/cloud/ai-agents/performance-cost-agent) to go deeper. It analyzes query patterns, surfaces inefficiencies, suggests improved SQL, and identifies redundant tests that slow down your pipeline — so you can act on what the performance data is showing. \ No newline at end of file +Once you've identified slow or expensive models and tests, use [AI Performance & Cost](/cloud/ai-agents/ai-performance-cost) to go deeper. It analyzes query patterns, surfaces inefficiencies, suggests improved SQL, and identifies redundant tests that slow down your pipeline — so you can act on what the performance data is showing. \ No newline at end of file diff --git a/docs/cloud/features/table-usage.mdx b/docs/cloud/features/table-usage.mdx index 506d623e4..3b4c49177 100644 --- a/docs/cloud/features/table-usage.mdx +++ b/docs/cloud/features/table-usage.mdx @@ -1,7 +1,6 @@ --- title: "Table usage" sidebarTitle: "Table usage" -badge: "Context Engine" --- Table usage shows how your data assets are **actually consumed** in the warehouse—how often they are queried and by who—based on **query activity** over a **rolling 30-day window**. That visibility helps you **prioritize incidents and data quality** on the assets that drive the most load and downstream impact, and get a **clearer end-to-end view** of what is happening in production. @@ -24,9 +23,9 @@ Hover the usage indicator for a specific asset to open a tooltip with: - **Query count** — Total queries against the table in that period. - **Unique users** — Distinct users who ran queries against the table in that period. -## AI agents +## Elementary AI Agent -The same usage signals are available as **context** to Elementary **AI agents**, so you can ask natural-language questions that depend on real consumption—for example, **“Show me the top 5 most used tables.”** +The same usage signals are available to the **Elementary AI Agent**, so you can ask natural-language questions that depend on real consumption—for example, **“Show me the top 5 most used tables.”** - For how data is handled by Elementary's AI features and AI agents, see the [AI Privacy Policy](/cloud/general/ai-privacy-policy). + For how data is handled by Elementary's AI features, including the AI agent, see the [AI Privacy Policy](/cloud/general/ai-privacy-policy). ## Security highlights diff --git a/docs/cloud/guides/set-up-elementary-checklist.md b/docs/cloud/guides/set-up-elementary-checklist.md index 3b44efe2a..fd321ece1 100644 --- a/docs/cloud/guides/set-up-elementary-checklist.md +++ b/docs/cloud/guides/set-up-elementary-checklist.md @@ -16,7 +16,7 @@ Let's get you set up. ## Elementary's Core Principles * **Code as the source of truth** [ ] Elementary keeps your code as the system of record. Any change made in the app opens a pull request in your repository, fitting naturally into your CI/CD process and keeping engineers in control. You can work from code, from the UI, or through MCP and everything stays aligned. -* **Scaling reliability with AI** [ ] Elementary uses AI to help you scale your reliability practices. Ella, our network of AI agents, assists with test creation, metadata enrichment, troubleshooting, and optimization so teams can stay proactive and focus on higher-value work. +* **Scaling reliability with AI** [ ] Elementary uses AI to help you scale your reliability practices. The Elementary AI Agent assists with test creation, metadata enrichment, troubleshooting, and optimization so teams can stay proactive and focus on higher-value work. * **Enabling all users** [ ] Elementary is built for the entire organization. Business users get a clear catalog, ownership information, health scores, and AI assistance so everyone can find, understand, and trust the data without depending on engineering. ## Technical Setup @@ -32,7 +32,7 @@ Let's get you set up. Only if you use dbt. Collects dbt artifacts and enables Elementary's built-in tests (anomaly detection, schema change detection). If you already have the package deployed in the relevant environment you want to monitor, make sure it's up to date to the latest version, and upgrade it if not. [ ] [Invite team members](/cloud/features/roles-and-permissions) [ ] [Code repository connection (optional)](/cloud/integrations/code-repo/connect-code-repo) - Allow users and AI agents to open PRs and manage changes through your CI/CD process. + Allow users and the AI agent to open PRs and manage changes through your CI/CD process. [ ] [Integrate with your BI tool (optional)](/cloud/integrations/bi/connect-bi-tool) Enable column-level lineage and full context for BI assets, including the health of upstream sources and models. [ ] Connect to your external Catalog (Optional) @@ -49,7 +49,7 @@ Let's get you set up. [ ] Connect [ticketing system](/cloud/integrations/alerts) (optional) We support [Jira](/cloud/integrations/alerts/jira), [ServiceNow](/cloud/integrations/alerts/servicenow) and [Linear](/cloud/integrations/alerts/linear). -### AI Agents +### AI Agent [ ] Sign [AI features consent form](https://2cwlc1.share-eu1.hsforms.com/2toUDbhpMRTK0WPmv2az_2g) Elementary AI uses only metadata, not raw or personal data. LLMs run via Amazon Bedrock with no data sharing to third parties. Learn more [here](/cloud/general/ai-privacy-policy). diff --git a/docs/cloud/guides/set-up-elementary.mdx b/docs/cloud/guides/set-up-elementary.mdx index e54bd5cfd..8e7e19c2e 100644 --- a/docs/cloud/guides/set-up-elementary.mdx +++ b/docs/cloud/guides/set-up-elementary.mdx @@ -19,7 +19,7 @@ Let's get you set up. ## Elementary's Core Principles * **Code as the source of truth** - Elementary keeps your code as the system of record. Any change made in the app opens a pull request in your repository, fitting naturally into your CI/CD process and keeping engineers in control. You can work from code, from the UI, or through MCP and everything stays aligned. -* **Scaling reliability with AI** - Elementary uses AI to help you scale your reliability practices. Ella, our network of AI agents, assists with test creation, metadata enrichment, troubleshooting, and optimization so teams can stay proactive and focus on higher-value work. +* **Scaling reliability with AI** - Elementary uses AI to help you scale your reliability practices. The Elementary AI Agent assists with test creation, metadata enrichment, troubleshooting, and optimization so teams can stay proactive and focus on higher-value work. * **Enabling all users** - Elementary is built for the entire organization. Business users get a clear catalog, ownership information, health scores, and AI assistance so everyone can find, understand, and trust the data without depending on engineering. ## Technical Setup @@ -39,7 +39,7 @@ Here's a link to a checkbox version of this guide that you can download and use Only if you use dbt. Collects dbt artifacts and enables Elementary's built-in tests (anomaly detection, schema change detection). If you already have the package deployed in the relevant environment you want to monitor, make sure it's up to date to the latest version, and upgrade it if not. - [Invite team members](/cloud/features/roles-and-permissions) - [Code repository connection (optional)](/cloud/integrations/code-repo/connect-code-repo) - Allow users and AI agents to open PRs and manage changes through your CI/CD process. + Allow users and the AI agent to open PRs and manage changes through your CI/CD process. - [Integrate with your BI tool (optional)](/cloud/integrations/bi/connect-bi-tool) Enable column-level lineage and full context for BI assets, including the health of upstream sources and models. - Connect to your external Catalog (Optional) @@ -56,7 +56,7 @@ Here's a link to a checkbox version of this guide that you can download and use - Connect [ticketing system](/cloud/integrations/alerts) (optional) We support [Jira](/cloud/integrations/alerts/jira), [ServiceNow](/cloud/integrations/alerts/servicenow) and [Linear](/cloud/integrations/alerts/linear). -### AI Agents +### AI Agent - Sign [AI features consent form](https://2cwlc1.share-eu1.hsforms.com/2toUDbhpMRTK0WPmv2az_2g) Elementary AI uses only metadata, not raw or personal data. LLMs run via Amazon Bedrock with no data sharing to third parties. Learn more [here](/cloud/general/ai-privacy-policy). diff --git a/docs/cloud/guides/start-using-elementary-checklist.md b/docs/cloud/guides/start-using-elementary-checklist.md index 30ca4370c..84a9cb0b8 100644 --- a/docs/cloud/guides/start-using-elementary-checklist.md +++ b/docs/cloud/guides/start-using-elementary-checklist.md @@ -10,7 +10,7 @@ We'll take a few of your key assets and run them through the core reliability st [ ] In the [Catalog](/cloud/features/collaboration-and-communication/catalog), identify and mark three [critical assets](/cloud/features/data-governance/critical_assets) [ ] Add or update [descriptions, tags, and owners](/cloud/features/data-governance/manage-metadata) for these assets -[ ] Leverage the [Governance AI agent](/cloud/ai-agents/governance-agent) - Complete missing metadata based on project context and according to your instructions. +[ ] Use [AI Governance](/cloud/ai-agents/ai-governance) to complete missing metadata based on project context and according to your instructions. [ ] View [end-to-end, column-level lineage](/cloud/features/data-lineage/lineage) for each critical asset ## Setup monitoring on critical assets @@ -19,7 +19,7 @@ We'll take a few of your key assets and run them through the core reliability st [ ] Open the [Test Coverage](/cloud/features/data-tests/test-coverage-screen) view to understand existing coverage by dimension [ ] [Add missing tests](/cloud/features/data-tests/data-tests-overview#test-configuration) such as freshness, volume, uniqueness, [anomaly detection](/data-tests/how-anomaly-detection-works), [schema changes](/data-tests/schema-tests/schema-changes) or custom logic -[ ] Leverage the [Test Recommendation AI agent](/cloud/ai-agents/test-recommendation-agent) for suggested tests based on patterns and lineage +[ ] Use [AI Test Coverage](/cloud/ai-agents/ai-test-coverage) for suggested tests based on patterns and lineage [ ] For source monitoring: Elementary automatically adds ML-based [freshness](/cloud/features/anomaly-detection/automated-freshness) and [volume](/cloud/features/anomaly-detection/automated-volume) tests to all sources to catch pipeline and ingestion issues early ## Setup Alerts @@ -35,7 +35,7 @@ We'll take a few of your key assets and run them through the core reliability st *Goal: Investigate and resolve issues quickly and confidently.* [ ] Triage: Review lineage and upstream failures alongside recent commits and dbt run history to understand what changed and what the issue is impacting -[ ] Use the [Triage & Resolution AI agent](/cloud/ai-agents/triage-resolution-agent) to run the investigation for you by analyzing lineage, failures, recent changes, and dependencies, and surfacing root causes +[ ] Use [AI Triage & Resolution](/cloud/ai-agents/ai-triage-resolution) to run the investigation for you by analyzing lineage, failures, recent changes, and dependencies, and surfacing root causes [ ] Use the [Incidents](/cloud/features/alerts-and-incidents/incident-management) screen to manage issues collaboratively and keep track of everything that's currently open ## Optimize Performance @@ -43,7 +43,7 @@ We'll take a few of your key assets and run them through the core reliability st *Goal: Meet your SLAs, improve run times, and ensure your pipelines operate efficiently.* [ ] Identify long-running tests and models using the [Performance pages](/cloud/features/performance-monitoring/performance-monitoring) -[ ] Use the [Optimization AI agent](/cloud/ai-agents/performance-cost-agent) to optimize queries by surfacing inefficiencies, suggesting improved SQL patterns, and identifying opportunities to reduce data scans or simplify logic +[ ] Use [AI Performance & Cost](/cloud/ai-agents/ai-performance-cost) to optimize queries by surfacing inefficiencies, suggesting improved SQL patterns, and identifying opportunities to reduce data scans or simplify logic ## Measure Progress and Data Health @@ -58,7 +58,7 @@ We'll take a few of your key assets and run them through the core reliability st [ ] Use the [Catalog](/cloud/features/collaboration-and-communication/catalog) to explore assets, their definitions, ownership, and current health status [ ] Review [Lineage](/cloud/features/data-lineage/lineage) to see how the asset is built, what depends on it, and whether any upstream issues or test failures affect it -[ ] Use the [Discovery AI agent](/cloud/ai-agents/catalog-agent) to get clear explanations of the asset, how it's calculated, and any current reliability concerns +[ ] Use [AI Catalog](/cloud/ai-agents/ai-catalog) to get clear explanations of the asset, how it's calculated, and any current reliability concerns ## Advanced: Use MCP to Extend Elementary Everywhere diff --git a/docs/cloud/guides/start-using-elementary.mdx b/docs/cloud/guides/start-using-elementary.mdx index 49e5e82d5..2bf8f6fda 100644 --- a/docs/cloud/guides/start-using-elementary.mdx +++ b/docs/cloud/guides/start-using-elementary.mdx @@ -17,7 +17,7 @@ We'll take a few of your key assets and run them through the core reliability st - In the [Catalog](/cloud/features/collaboration-and-communication/catalog), identify and mark three [critical assets](/cloud/features/data-governance/critical_assets) - Add or update [descriptions, tags, and owners](/cloud/features/collaboration-and-communication/catalog) for these assets -- Leverage the [Governance AI agent](/cloud/ai-agents/governance-agent) - Complete missing metadata based on project context and according to your instructions. +- Use [AI Governance](/cloud/ai-agents/ai-governance) to complete missing metadata based on project context and according to your instructions. - View [end-to-end, column-level lineage](/cloud/features/data-lineage/lineage) for each critical asset ## Setup monitoring on critical assets @@ -26,7 +26,7 @@ We'll take a few of your key assets and run them through the core reliability st - Open the [Test Coverage](/cloud/features/data-tests/test-coverage-screen) view to understand existing coverage by dimension - [Add missing tests](/cloud/features/data-tests/cloud-tests-overview) such as freshness, volume, uniqueness, [anomaly detection](/data-tests/how-anomaly-detection-works), [schema changes](/data-tests/schema-tests/schema-changes) or custom logic -- Leverage the [Test Recommendation AI agent](/cloud/ai-agents/test-recommendation-agent) for suggested tests based on patterns and lineage +- Use [AI Test Coverage](/cloud/ai-agents/ai-test-coverage) for suggested tests based on patterns and lineage - For source monitoring: Elementary automatically adds ML-based [freshness](/cloud/features/anomaly-detection/automated-freshness) and [volume](/cloud/features/anomaly-detection/automated-volume) tests to all sources to catch pipeline and ingestion issues early ## Setup Alerts @@ -42,7 +42,7 @@ We'll take a few of your key assets and run them through the core reliability st *Goal: Investigate and resolve issues quickly and confidently.* - Triage: Review lineage and upstream failures alongside recent commits and dbt run history to understand what changed and what the issue is impacting -- Use the [Triage & Resolution AI agent](/cloud/ai-agents/triage-resolution-agent) to run the investigation for you by analyzing lineage, failures, recent changes, and dependencies, and surfacing root causes +- Use [AI Triage & Resolution](/cloud/ai-agents/ai-triage-resolution) to run the investigation for you by analyzing lineage, failures, recent changes, and dependencies, and surfacing root causes - Use the [Incidents](/cloud/features/alerts-and-incidents/incidents) screen to manage issues collaboratively and keep track of everything that's currently open ## Optimize Performance @@ -50,7 +50,7 @@ We'll take a few of your key assets and run them through the core reliability st *Goal: Meet your SLAs, improve run times, and ensure your pipelines operate efficiently.* - Identify long-running tests and models using the [Performance pages](/cloud/features/performance-monitoring/performance-monitoring) -- Use the [Optimization AI agent](/cloud/ai-agents/performance-cost-agent) to optimize queries by surfacing inefficiencies, suggesting improved SQL patterns, and identifying opportunities to reduce data scans or simplify logic +- Use [AI Performance & Cost](/cloud/ai-agents/ai-performance-cost) to optimize queries by surfacing inefficiencies, suggesting improved SQL patterns, and identifying opportunities to reduce data scans or simplify logic ## Measure Progress and Data Health @@ -65,7 +65,7 @@ We'll take a few of your key assets and run them through the core reliability st - Use the [Catalog](/cloud/features/collaboration-and-communication/catalog) to explore assets, their definitions, ownership, and current health status - Review [Lineage](/cloud/features/data-lineage/lineage) to see how the asset is built, what depends on it, and whether any upstream issues or test failures affect it -- Use the [Discovery AI agent](/cloud/ai-agents/catalog-agent) to get clear explanations of the asset, how it's calculated, and any current reliability concerns +- Use [AI Catalog](/cloud/ai-agents/ai-catalog) to get clear explanations of the asset, how it's calculated, and any current reliability concerns ## Advanced: Use MCP to Extend Elementary Everywhere diff --git a/docs/cloud/integrations/bi/connect-bi-tool.mdx b/docs/cloud/integrations/bi/connect-bi-tool.mdx index a2856b97e..890b72dcc 100644 --- a/docs/cloud/integrations/bi/connect-bi-tool.mdx +++ b/docs/cloud/integrations/bi/connect-bi-tool.mdx @@ -1,7 +1,6 @@ --- title: "Automated lineage to Data Visualization layer" sidebarTitle: "Automated BI lineage" -badge: "Context Engine" --- import BiCards from '/snippets/cloud/integrations/cards-groups/bi-cards.mdx'; diff --git a/docs/cloud/integrations/bi/explo.mdx b/docs/cloud/integrations/bi/explo.mdx index e563f9085..7096444f6 100644 --- a/docs/cloud/integrations/bi/explo.mdx +++ b/docs/cloud/integrations/bi/explo.mdx @@ -1,6 +1,5 @@ --- title: "Explo" -badge: "Context Engine" --- Automates triage, test coverage, governance, and performance workflows. - - The knowledge layer that connects your stack and powers the agent. + + Column-level lineage from ingestion through your warehouse and dbt to BI. Expose your data context to Claude, Cursor, and any MCP-compatible tool. diff --git a/docs/cloud/python-sdk/overview.mdx b/docs/cloud/python-sdk/overview.mdx index f61f99124..44ac6ef15 100644 --- a/docs/cloud/python-sdk/overview.mdx +++ b/docs/cloud/python-sdk/overview.mdx @@ -55,9 +55,9 @@ When a Python pipeline reports assets, test results, and execution metadata, eve - **All test results appear together** — Python validations, dbt tests, cloud tests — in a single, consistent interface. - **Alerts fire through your existing channels** (Slack, PagerDuty, email), ensuring that pipeline-level issues trigger the same operational flow as warehouse-level ones. -- **Incidents are created automatically** for detected issues, including opening Jira tickets. Elementary's agentic tools then investigate root cause, assess downstream impact, and guide resolution. +- **Incidents are created automatically** for detected issues, including opening Jira tickets. The Elementary AI Agent then investigates root cause, assesses downstream impact, and guides resolution. - **Lineage becomes fully connected**, tying together Python assets, dbt models, warehouse tables, unstructured data, vectors, and ML outputs. -- **Every table, view, file, or vector store entity produced by Python becomes discoverable** through the Elementary catalog, data discovery agent, and MCP server — giving analysts, DS, and AI teams a shared understanding of the entire data ecosystem. +- **Every table, view, file, or vector store entity produced by Python becomes discoverable** through the Elementary catalog, AI Catalog, and MCP server — giving analysts, DS, and AI teams a shared understanding of the entire data ecosystem. This closes the gap between ingestion pipelines, warehouse transformations, ML prep code, and AI workloads — all observed in one place. diff --git a/docs/cloud/quickstart.mdx b/docs/cloud/quickstart.mdx index f8044af6a..4bf9af2be 100644 --- a/docs/cloud/quickstart.mdx +++ b/docs/cloud/quickstart.mdx @@ -9,7 +9,7 @@ import QuickstartSteps from '/snippets/cloud/quickstart-steps.mdx'; Welcome to Elementary! -Elementary is the platform for data observability, quality, and governance. Connect your stack and you get a team of AI agents that handle test coverage, incident triage, governance, and performance monitoring — continuously, without manual configuration. +Elementary is the platform for data observability, quality, and governance. Connect your stack and you get an AI agent that handles test coverage, incident triage, governance, and performance monitoring — continuously, without manual configuration. Setup starts with your data warehouse. You create an [environment](/cloud/environments), connect it to the warehouse, and choose which assets it monitors. dbt is optional — if you use it, connect your dbt project as one of the environment's asset sources to bring in your models, tests, and run results. diff --git a/docs/cloud/resources/faq.mdx b/docs/cloud/resources/faq.mdx index 84dee38ea..836a197cb 100644 --- a/docs/cloud/resources/faq.mdx +++ b/docs/cloud/resources/faq.mdx @@ -1,7 +1,7 @@ --- title: "Frequently asked questions" sidebarTitle: "FAQ" -description: "Common questions about Elementary: dbt, cloud tests, Cloud vs OSS, AI agents, credits, security, and supported data warehouses." +description: "Common questions about Elementary: dbt, cloud tests, Cloud vs OSS, the AI agent, credits, security, and supported data warehouses." icon: "circle-question" --- @@ -19,7 +19,7 @@ icon: "circle-question" - Short answer: choose **OSS** for free, self-hosted observability on a single dbt project; choose **Cloud** to cover your entire data stack with automation, governance, AI agents, and enterprise security - and Cloud works with your warehouse directly, so dbt isn't required to start. + Short answer: choose **OSS** for free, self-hosted observability on a single dbt project; choose **Cloud** to cover your entire data stack with automation, governance, an AI agent, and enterprise security - and Cloud works with your warehouse directly, so dbt isn't required to start. **Elementary OSS** is a free, open-source dbt package and CLI you host yourself. It covers one dbt project: dbt test and run results, table-level lineage, anomaly detection tests for freshness and volume, and basic Slack or Teams alerts, all in a self-hosted report. Your data and metadata stay entirely in your environment. @@ -29,22 +29,22 @@ icon: "circle-question" - **Incident management** with alert rules and routing - **Governance**: catalog, metadata management, AI-generated descriptions, critical assets - **Performance and cost monitoring** - - A team of **AI agents** and an **MCP server**, backed by a Context Engine that knows your whole stack + - The **Elementary AI Agent** and an **MCP server**, working from the lineage, test results, and incidents across your whole stack - **Enterprise security**: RBAC, SSO, audit logs, PrivateLink, SOC 2 Type II and HIPAA If you're not sure, [talk to the team](https://meetings-eu1.hubspot.com/joost-boonzajer-flaes/intro-call-docs) and they'll help you decide. See the full [Cloud vs OSS comparison](/cloud/cloud-vs-oss). - - A team of specialized agents - [governance](/cloud/ai-agents/governance-agent), [triage & resolution](/cloud/ai-agents/triage-resolution-agent), [test recommendation](/cloud/ai-agents/test-recommendation-agent), [catalog](/cloud/ai-agents/catalog-agent), [performance & cost](/cloud/ai-agents/performance-cost-agent), and [code review](/cloud/features/ci) - backed by a [Context Engine](/cloud/ai-agents/context-engine) built from every integration you connect. Every action goes through a pull request or explicit confirmation, so your team stays in control of what gets merged. Available on Elementary Cloud. + + One agent with a set of capabilities: [AI Governance](/cloud/ai-agents/ai-governance), [AI Triage & Resolution](/cloud/ai-agents/ai-triage-resolution), [AI Test Coverage](/cloud/ai-agents/ai-test-coverage), [AI Catalog](/cloud/ai-agents/ai-catalog), [AI Performance & Cost](/cloud/ai-agents/ai-performance-cost), and [AI Code Review](/cloud/features/ci). It works from the lineage, test results, incidents, and metadata Elementary collects from every integration you connect. Every action goes through a pull request or explicit confirmation, so your team stays in control of what gets merged. Available on Elementary Cloud. - Yes. Through the [MCP server](/cloud/mcp/overview) and the [Context Engine](/cloud/ai-agents/context-engine), Elementary exposes data health, quality dimensions, test results, and lineage to your AI agents and copilots - so an agent can check whether a table is trusted and fresh before it acts on it. This is how Elementary helps make your data ready for AI. + Yes. Through the [MCP server](/cloud/mcp/overview), Elementary exposes data health, quality dimensions, test results, and lineage to your AI agents and copilots - so an agent can check whether a table is trusted and fresh before it acts on it. This is how Elementary helps make your data ready for AI. - [AI credits](/cloud/ai-agents/ai-credits) are consumed only when Elementary's AI does the reasoning - incident investigation, data discovery, test generation, and CI PR review. Things that use no credits: all MCP tool calls (the connected LLM reasons and bills to its own provider), anomaly detection and monitors (statistical, not LLM), Slack alerts, and lineage or catalog browsing. You can also [bring your own LLM key (BYOK)](/cloud/ai-agents/byok) so reasoning bills to your Anthropic or OpenAI account instead of Elementary credits. + [AI credits](/cloud/ai-agents/ai-credits) are consumed only when Elementary's AI does the reasoning - incident investigation, data discovery, test generation, and AI Code Review. Things that use no credits: all MCP tool calls (the connected LLM reasons and bills to its own provider), anomaly detection and monitors (statistical, not LLM), Slack alerts, and lineage or catalog browsing. You can also [bring your own LLM key (BYOK)](/cloud/ai-agents/byok) so reasoning bills to your Anthropic or OpenAI account instead of Elementary credits. diff --git a/docs/cloud/what-is-elementary.mdx b/docs/cloud/what-is-elementary.mdx index ef0b5925a..416f62b69 100644 --- a/docs/cloud/what-is-elementary.mdx +++ b/docs/cloud/what-is-elementary.mdx @@ -22,7 +22,7 @@ From there, Elementary builds a connected model of your data and operates on it: - **Testing and monitoring.** Anomaly detection, no-code cloud tests, and tests defined in code, run in your pipeline or via [Elementary Runtime](/cloud/features/elementary-runtime) (optional). - **Column-level lineage across systems.** From source through the warehouse to BI dashboards and AI consumption. - **Incident management.** Related failures are grouped into one incident and routed by ownership, criticality, and impact, so one root problem does not turn into hundreds of alerts. -- **AI agents and an MCP server.** Backed by a context model of your whole stack, they handle triage, test recommendations, governance, and root cause analysis, and expose data health to your own AI tools. +- **An AI agent and an MCP server.** Working from the lineage, test results, and incidents across your whole stack, the agent handles triage, test coverage, governance, and root cause analysis, and the MCP server exposes data health to your own AI tools. Elementary does not require dbt. You connect your warehouse directly. If you do use dbt, the Elementary dbt package adds automatic collection of your dbt test and run results on top. @@ -40,7 +40,7 @@ Use it to: - Give engineers, analysts, and stakeholders a shared view of what is trusted and what is broken - Run checks in your pipeline as code, or via [Elementary Runtime](/cloud/features/elementary-runtime) (optional) - Reduce alert noise by grouping and routing alerts based on criticality, ownership, and impact -- Automate test recommendations, coverage, incident triage, and root cause analysis with AI agents +- Automate test recommendations, coverage, incident triage, and root cause analysis with the Elementary AI Agent - Expose data health, lineage, and quality to your AI agents through the MCP server - Enforce governance and support AI readiness, with enterprise security like RBAC, SSO, and audit logs diff --git a/docs/docs.json b/docs/docs.json index 40aac5442..045655cc2 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -40,12 +40,11 @@ "group": "Elementary AI Agent", "pages": [ "cloud/ai-agents/overview", - "cloud/ai-agents/context-engine", - "cloud/ai-agents/governance-agent", - "cloud/ai-agents/triage-resolution-agent", - "cloud/ai-agents/test-recommendation-agent", - "cloud/ai-agents/catalog-agent", - "cloud/ai-agents/performance-cost-agent", + "cloud/ai-agents/ai-governance", + "cloud/ai-agents/ai-triage-resolution", + "cloud/ai-agents/ai-test-coverage", + "cloud/ai-agents/ai-catalog", + "cloud/ai-agents/ai-performance-cost", "cloud/features/ci", "cloud/ai-agents/slack-agent", "cloud/ai-agents/mcp-connections", @@ -934,6 +933,34 @@ { "source": "/cloud/features/anomaly-detection/monitor-dwh-assets", "destination": "/cloud/environments" + }, + { + "source": "/cloud/ai-agents/context-engine", + "destination": "/cloud/ai-agents/overview" + }, + { + "source": "/cloud/ai-agents/governance-agent", + "destination": "/cloud/ai-agents/ai-governance" + }, + { + "source": "/cloud/ai-agents/triage-resolution-agent", + "destination": "/cloud/ai-agents/ai-triage-resolution" + }, + { + "source": "/cloud/ai-agents/test-recommendation-agent", + "destination": "/cloud/ai-agents/ai-test-coverage" + }, + { + "source": "/cloud/ai-agents/catalog-agent", + "destination": "/cloud/ai-agents/ai-catalog" + }, + { + "source": "/cloud/ai-agents/performance-cost-agent", + "destination": "/cloud/ai-agents/ai-performance-cost" + }, + { + "source": "/cloud/ai-agents", + "destination": "/cloud/ai-agents/overview" } ] } diff --git a/docs/home.mdx b/docs/home.mdx index 2fdabe0ee..8240956d3 100644 --- a/docs/home.mdx +++ b/docs/home.mdx @@ -168,11 +168,11 @@ icon: "house" link="/cloud/features/data-tests/test-coverage-screen" />
@@ -214,11 +214,11 @@ icon: "house" link="/cloud/features/alerts-and-incidents/incidents" /> @@ -263,7 +263,7 @@ icon: "house"
-

Elementary context
in your tools

+

Elementary
in your AI tools

The MCP Server brings models, lineage, and incidents into tools like Cursor and Claude—so you can act without leaving your workflow.

@@ -372,11 +372,11 @@ icon: "house" link="/cloud/features/data-governance/ai-descriptions" />
diff --git a/docs/skills.md b/docs/skills.md index 63d0a3f12..11ce2391b 100644 --- a/docs/skills.md +++ b/docs/skills.md @@ -103,8 +103,8 @@ The [MCP server](https://docs.elementary-data.com/cloud/mcp/intro) exposes Eleme - When documenting [MCP tools](https://docs.elementary-data.com/cloud/mcp/mcp-tools), describe each tool's purpose and expected output precisely. - If a feature is accessible via MCP, note it on the feature page. -### AI agents -Elementary's [AI agents](https://docs.elementary-data.com/cloud/ai-agents/overview) ([governance](https://docs.elementary-data.com/cloud/ai-agents/governance-agent), [triage](https://docs.elementary-data.com/cloud/ai-agents/triage-resolution-agent), [test recommendation](https://docs.elementary-data.com/cloud/ai-agents/test-recommendation-agent), etc.) are documented individually. When writing or updating these pages: +### AI agent capabilities +The [Elementary AI Agent](https://docs.elementary-data.com/cloud/ai-agents/overview) is one agent with a set of capabilities ([AI Governance](https://docs.elementary-data.com/cloud/ai-agents/ai-governance), [AI Triage & Resolution](https://docs.elementary-data.com/cloud/ai-agents/ai-triage-resolution), [AI Test Coverage](https://docs.elementary-data.com/cloud/ai-agents/ai-test-coverage), etc.), each documented on its own page. Refer to them as capabilities of the agent, not as separate agents. When writing or updating these pages: - Describe what the agent *decides* and *acts on*, not just what it shows. - Include the trigger / entry point (which screen, which action). - Mention what the agent needs to work well (e.g. metadata coverage, policies set up). diff --git a/docs/snippets/cloud/features.mdx b/docs/snippets/cloud/features.mdx index 09ec41cbf..73c7f0dfe 100644 --- a/docs/snippets/cloud/features.mdx +++ b/docs/snippets/cloud/features.mdx @@ -1,27 +1,27 @@ ### Elementary AI Agent -Elementary's AI agents run continuously on your data platform — triaging incidents, recommending tests, classifying assets, and surfacing performance issues. They're powered by the Context Engine, which maps your entire stack from ingestion through transformations to BI. +The Elementary AI Agent runs continuously on your data platform: triaging incidents, recommending tests, classifying assets, and surfacing performance issues. It works from Elementary's lineage of your entire stack, from ingestion through transformations to BI. - What the agents do, how they work, and how to get started. + What the agent does, how it works, and how to get started. - - The knowledge layer that powers all agents — lineage, metadata, test results, and execution history unified. - - + Investigates root cause, assesses blast radius, and suggests a fix — automatically when an incident opens. - + Analyzes your data and usage patterns to recommend the right tests for untested assets. - + Identifies undocumented assets, assigns owners, and generates descriptions — synced back to your dbt project. - + Surfaces slow models, redundant tests, and expensive queries with specific optimization recommendations. + + Finds the right data in natural language, with freshness, open incidents, and test results for each asset. + ### Data Tests @@ -40,7 +40,7 @@ Elementary's AI agents run continuously on your data platform — triaging incid Detect schema changes that break downstream consumers — tables, JSON fields, and BI dashboards. - Visual map of coverage gaps across your stack. Add tests from the UI or use the Test Recommendation Agent. + Visual map of coverage gaps across your stack. Add tests from the UI or use AI Test Coverage. From de02b425fd23e2c2981760b232f30e7637e2ea67 Mon Sep 17 00:00:00 2001 From: NoyaOffer Date: Thu, 22 Oct 2026 18:20:55 +0300 Subject: [PATCH 2/2] docs: replace versioned release notes with links to GitHub releases - Remove per-version release note pages for the dbt package and the CLI - Add a release notes page in the dbt package and OSS tabs linking to the GitHub releases of both repositories - Redirect old release note paths to the new pages Co-Authored-By: Claude Opus 5.5 --- docs/data-tests/dbt/release-notes.mdx | 8 +++ .../dbt/release-notes/releases/0.18.1.mdx | 13 ---- .../dbt/release-notes/releases/0.18.2.mdx | 13 ---- .../dbt/release-notes/releases/0.18.3.mdx | 12 ---- .../dbt/release-notes/releases/0.19.0.mdx | 12 ---- .../dbt/release-notes/releases/0.19.1.mdx | 12 ---- .../dbt/release-notes/releases/0.19.2.mdx | 18 ------ .../dbt/release-notes/releases/0.19.3.mdx | 12 ---- .../dbt/release-notes/releases/0.19.4.mdx | 11 ---- .../dbt/release-notes/releases/0.20.0.mdx | 17 ----- .../dbt/release-notes/releases/0.20.1.mdx | 11 ---- docs/docs.json | 62 ++++--------------- docs/oss/release-notes/release-notes.mdx | 8 +++ docs/oss/release-notes/releases/0.10.0.mdx | 38 ------------ docs/oss/release-notes/releases/0.11.2.mdx | 37 ----------- docs/oss/release-notes/releases/0.16.0.mdx | 9 --- docs/oss/release-notes/releases/0.16.1.mdx | 9 --- docs/oss/release-notes/releases/0.16.2.mdx | 20 ------ docs/oss/release-notes/releases/0.17.0.mdx | 12 ---- docs/oss/release-notes/releases/0.18.0.mdx | 14 ----- docs/oss/release-notes/releases/0.18.1.mdx | 13 ---- docs/oss/release-notes/releases/0.18.2.mdx | 17 ----- docs/oss/release-notes/releases/0.18.3.mdx | 11 ---- docs/oss/release-notes/releases/0.19.0.mdx | 16 ----- docs/oss/release-notes/releases/0.19.1.mdx | 12 ---- docs/oss/release-notes/releases/0.19.2.mdx | 11 ---- docs/oss/release-notes/releases/0.19.3.mdx | 11 ---- docs/oss/release-notes/releases/0.19.4.mdx | 12 ---- docs/oss/release-notes/releases/0.19.5.mdx | 15 ----- docs/oss/release-notes/releases/0.20.0.mdx | 15 ----- docs/oss/release-notes/releases/0.5.4.mdx | 36 ----------- docs/oss/release-notes/releases/0.6.1.mdx | 41 ------------ docs/oss/release-notes/releases/0.6.10.mdx | 40 ------------ docs/oss/release-notes/releases/0.6.3.mdx | 47 -------------- docs/oss/release-notes/releases/0.6.5.mdx | 41 ------------ docs/oss/release-notes/releases/0.6.7.mdx | 35 ----------- docs/oss/release-notes/releases/0.7.10.mdx | 41 ------------ docs/oss/release-notes/releases/0.7.2.mdx | 40 ------------ docs/oss/release-notes/releases/0.7.5.mdx | 41 ------------ docs/oss/release-notes/releases/0.7.6.mdx | 47 -------------- docs/oss/release-notes/releases/0.7.7.mdx | 50 --------------- docs/oss/release-notes/releases/0.8.0.mdx | 32 ---------- docs/oss/release-notes/releases/0.8.2.mdx | 32 ---------- docs/oss/release-notes/releases/0.9.1.mdx | 40 ------------ docs/snippets/oss/release-notes-links.mdx | 12 ++++ 45 files changed, 40 insertions(+), 1016 deletions(-) create mode 100644 docs/data-tests/dbt/release-notes.mdx delete mode 100644 docs/data-tests/dbt/release-notes/releases/0.18.1.mdx delete mode 100644 docs/data-tests/dbt/release-notes/releases/0.18.2.mdx delete mode 100644 docs/data-tests/dbt/release-notes/releases/0.18.3.mdx delete mode 100644 docs/data-tests/dbt/release-notes/releases/0.19.0.mdx delete mode 100644 docs/data-tests/dbt/release-notes/releases/0.19.1.mdx delete mode 100644 docs/data-tests/dbt/release-notes/releases/0.19.2.mdx delete mode 100644 docs/data-tests/dbt/release-notes/releases/0.19.3.mdx delete mode 100644 docs/data-tests/dbt/release-notes/releases/0.19.4.mdx delete mode 100644 docs/data-tests/dbt/release-notes/releases/0.20.0.mdx delete mode 100644 docs/data-tests/dbt/release-notes/releases/0.20.1.mdx create mode 100644 docs/oss/release-notes/release-notes.mdx delete mode 100644 docs/oss/release-notes/releases/0.10.0.mdx delete mode 100644 docs/oss/release-notes/releases/0.11.2.mdx delete mode 100644 docs/oss/release-notes/releases/0.16.0.mdx delete mode 100644 docs/oss/release-notes/releases/0.16.1.mdx delete mode 100644 docs/oss/release-notes/releases/0.16.2.mdx delete mode 100644 docs/oss/release-notes/releases/0.17.0.mdx delete mode 100644 docs/oss/release-notes/releases/0.18.0.mdx delete mode 100644 docs/oss/release-notes/releases/0.18.1.mdx delete mode 100644 docs/oss/release-notes/releases/0.18.2.mdx delete mode 100644 docs/oss/release-notes/releases/0.18.3.mdx delete mode 100644 docs/oss/release-notes/releases/0.19.0.mdx delete mode 100644 docs/oss/release-notes/releases/0.19.1.mdx delete mode 100644 docs/oss/release-notes/releases/0.19.2.mdx delete mode 100644 docs/oss/release-notes/releases/0.19.3.mdx delete mode 100644 docs/oss/release-notes/releases/0.19.4.mdx delete mode 100644 docs/oss/release-notes/releases/0.19.5.mdx delete mode 100644 docs/oss/release-notes/releases/0.20.0.mdx delete mode 100644 docs/oss/release-notes/releases/0.5.4.mdx delete mode 100644 docs/oss/release-notes/releases/0.6.1.mdx delete mode 100644 docs/oss/release-notes/releases/0.6.10.mdx delete mode 100644 docs/oss/release-notes/releases/0.6.3.mdx delete mode 100644 docs/oss/release-notes/releases/0.6.5.mdx delete mode 100644 docs/oss/release-notes/releases/0.6.7.mdx delete mode 100644 docs/oss/release-notes/releases/0.7.10.mdx delete mode 100644 docs/oss/release-notes/releases/0.7.2.mdx delete mode 100644 docs/oss/release-notes/releases/0.7.5.mdx delete mode 100644 docs/oss/release-notes/releases/0.7.6.mdx delete mode 100644 docs/oss/release-notes/releases/0.7.7.mdx delete mode 100644 docs/oss/release-notes/releases/0.8.0.mdx delete mode 100644 docs/oss/release-notes/releases/0.8.2.mdx delete mode 100644 docs/oss/release-notes/releases/0.9.1.mdx create mode 100644 docs/snippets/oss/release-notes-links.mdx diff --git a/docs/data-tests/dbt/release-notes.mdx b/docs/data-tests/dbt/release-notes.mdx new file mode 100644 index 000000000..486bd34ab --- /dev/null +++ b/docs/data-tests/dbt/release-notes.mdx @@ -0,0 +1,8 @@ +--- +title: "Release notes" +sidebarTitle: "Release notes" +--- + +import ReleaseNotesLinks from '/snippets/oss/release-notes-links.mdx'; + + diff --git a/docs/data-tests/dbt/release-notes/releases/0.18.1.mdx b/docs/data-tests/dbt/release-notes/releases/0.18.1.mdx deleted file mode 100644 index cd117b94f..000000000 --- a/docs/data-tests/dbt/release-notes/releases/0.18.1.mdx +++ /dev/null @@ -1,13 +0,0 @@ ---- -title: "dbt Package 0.18.1" -sidebarTitle: "0.18.1" ---- - -This update includes: - -- 📝 Updated README -- 🐛 Report the same test status as dbt -- 🔧 Added resource type option to generate_schema_baseline_test - -Check out the full details here: https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.18.1 - diff --git a/docs/data-tests/dbt/release-notes/releases/0.18.2.mdx b/docs/data-tests/dbt/release-notes/releases/0.18.2.mdx deleted file mode 100644 index b62a51317..000000000 --- a/docs/data-tests/dbt/release-notes/releases/0.18.2.mdx +++ /dev/null @@ -1,13 +0,0 @@ ---- -title: "dbt Package 0.18.2" -sidebarTitle: "0.18.2" ---- - -This update includes: - -- 🔧 Allow contributor PRs to run tests -- 🗄️ Temp tables macro improvements -- 🗄️ Support for Athena in integration tests - -Check out the full details here: https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.18.2 - diff --git a/docs/data-tests/dbt/release-notes/releases/0.18.3.mdx b/docs/data-tests/dbt/release-notes/releases/0.18.3.mdx deleted file mode 100644 index 43a42c8f2..000000000 --- a/docs/data-tests/dbt/release-notes/releases/0.18.3.mdx +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: "dbt Package 0.18.3" -sidebarTitle: "0.18.3" ---- - -This update includes: - -- 🗄️ ClickHouse integration - artifacts only -- 🐛 Fixed Databricks temp relations - -Check out the full details here: https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.18.3 - diff --git a/docs/data-tests/dbt/release-notes/releases/0.19.0.mdx b/docs/data-tests/dbt/release-notes/releases/0.19.0.mdx deleted file mode 100644 index 51bb79cb6..000000000 --- a/docs/data-tests/dbt/release-notes/releases/0.19.0.mdx +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: "dbt Package 0.19.0" -sidebarTitle: "0.19.0" ---- - -This update includes: - -- 📊 Added dbt group artifact and column support -- ⏰ Added seasonality parameter to test_event_freshness_anomalies - -Check out the full details here: https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.19.0 - diff --git a/docs/data-tests/dbt/release-notes/releases/0.19.1.mdx b/docs/data-tests/dbt/release-notes/releases/0.19.1.mdx deleted file mode 100644 index 82ac6e835..000000000 --- a/docs/data-tests/dbt/release-notes/releases/0.19.1.mdx +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: "dbt Package 0.19.1" -sidebarTitle: "0.19.1" ---- - -This update includes: - -- ☁️ BigQuery - save execution_project in adapter-specific fields -- 🐛 Fixed dbt metric collection - -Check out the full details here: https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.19.1 - diff --git a/docs/data-tests/dbt/release-notes/releases/0.19.2.mdx b/docs/data-tests/dbt/release-notes/releases/0.19.2.mdx deleted file mode 100644 index 190861002..000000000 --- a/docs/data-tests/dbt/release-notes/releases/0.19.2.mdx +++ /dev/null @@ -1,18 +0,0 @@ ---- -title: "dbt Package 0.19.2" -sidebarTitle: "0.19.2" ---- - -This update includes: - -- 📊 Updated tests quality dimension -- 🔧 Added metadata comment to all queries run with elementary.run.query -- 🔐 Snowflake user creation - use public key -- 📝 Added runtime log to dbt package -- 🐛 Fixed schema name generation for branch names with forward slashes -- 🔒 Added configuration to disable sample collection for PII tables -- 🛡️ Implemented column-level PII protection for sample collection -- 🗄️ Dremio improvements - -Check out the full details here: https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.19.2 - diff --git a/docs/data-tests/dbt/release-notes/releases/0.19.3.mdx b/docs/data-tests/dbt/release-notes/releases/0.19.3.mdx deleted file mode 100644 index ffa2720d3..000000000 --- a/docs/data-tests/dbt/release-notes/releases/0.19.3.mdx +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: "dbt Package 0.19.3" -sidebarTitle: "0.19.3" ---- - -This update includes: - -- 🗄️ Dremio types mapping improvements -- 🔧 dbt-fusion fix: meta under config in dbt_run_results - -Check out the full details here: https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.19.3 - diff --git a/docs/data-tests/dbt/release-notes/releases/0.19.4.mdx b/docs/data-tests/dbt/release-notes/releases/0.19.4.mdx deleted file mode 100644 index 9e9779c85..000000000 --- a/docs/data-tests/dbt/release-notes/releases/0.19.4.mdx +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: "dbt Package 0.19.4" -sidebarTitle: "0.19.4" ---- - -This update includes: - -- 📊 Dimension anomalies visualization - -Check out the full details here: https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.19.4 - diff --git a/docs/data-tests/dbt/release-notes/releases/0.20.0.mdx b/docs/data-tests/dbt/release-notes/releases/0.20.0.mdx deleted file mode 100644 index 40530dfa3..000000000 --- a/docs/data-tests/dbt/release-notes/releases/0.20.0.mdx +++ /dev/null @@ -1,17 +0,0 @@ ---- -title: "dbt Package 0.20.0" -sidebarTitle: "0.20.0" ---- - -This update includes: - -- 🗄️ dbt Fusion support -- ⚡ Performance improvements with indices for faster loading times -- 🐛 Fixed datetime casting macro for ClickHouse -- 🐛 Fixed Athena DROP TABLE parsing error with backtick quoting -- 🗄️ Dremio improvements: timestamp formatting, metadata permissions, and special character escaping -- 🐛 Fixed Databricks temp relations and Fusion bugfixes -- 📊 Index on test_result_rows.detected_at column for better performance - -Check out the full details here: https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.20.0 - diff --git a/docs/data-tests/dbt/release-notes/releases/0.20.1.mdx b/docs/data-tests/dbt/release-notes/releases/0.20.1.mdx deleted file mode 100644 index 1310adf56..000000000 --- a/docs/data-tests/dbt/release-notes/releases/0.20.1.mdx +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: "dbt Package 0.20.1" -sidebarTitle: "0.20.1" ---- - -This update includes: - -- 🔧 Fixed dbt Fusion version check to require major version 2 - -Check out the full details here: https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.20.1 - diff --git a/docs/docs.json b/docs/docs.json index 045655cc2..7b004fbbf 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -482,20 +482,9 @@ ] }, { - "group": "dbt Package Release Notes", - "collapsible": true, - "collapsed": true, + "group": "Release Notes", "pages": [ - "data-tests/dbt/release-notes/releases/0.20.1", - "data-tests/dbt/release-notes/releases/0.20.0", - "data-tests/dbt/release-notes/releases/0.19.4", - "data-tests/dbt/release-notes/releases/0.19.3", - "data-tests/dbt/release-notes/releases/0.19.2", - "data-tests/dbt/release-notes/releases/0.19.1", - "data-tests/dbt/release-notes/releases/0.19.0", - "data-tests/dbt/release-notes/releases/0.18.3", - "data-tests/dbt/release-notes/releases/0.18.2", - "data-tests/dbt/release-notes/releases/0.18.1" + "data-tests/dbt/release-notes" ] } ] @@ -694,43 +683,8 @@ { "group": "Release Notes", "pages": [ - "oss/release-notes/upgrading-elementary", - { - "group": "Releases", - "pages": [ - "oss/release-notes/releases/0.20.0", - "oss/release-notes/releases/0.19.5", - "oss/release-notes/releases/0.19.4", - "oss/release-notes/releases/0.19.3", - "oss/release-notes/releases/0.19.2", - "oss/release-notes/releases/0.19.1", - "oss/release-notes/releases/0.19.0", - "oss/release-notes/releases/0.18.3", - "oss/release-notes/releases/0.18.2", - "oss/release-notes/releases/0.18.1", - "oss/release-notes/releases/0.18.0", - "oss/release-notes/releases/0.17.0", - "oss/release-notes/releases/0.16.2", - "oss/release-notes/releases/0.16.1", - "oss/release-notes/releases/0.16.0", - "oss/release-notes/releases/0.11.2", - "oss/release-notes/releases/0.10.0", - "oss/release-notes/releases/0.9.1", - "oss/release-notes/releases/0.8.2", - "oss/release-notes/releases/0.8.0", - "oss/release-notes/releases/0.7.10", - "oss/release-notes/releases/0.7.7", - "oss/release-notes/releases/0.7.6", - "oss/release-notes/releases/0.7.5", - "oss/release-notes/releases/0.7.2", - "oss/release-notes/releases/0.6.10", - "oss/release-notes/releases/0.6.7", - "oss/release-notes/releases/0.6.5", - "oss/release-notes/releases/0.6.3", - "oss/release-notes/releases/0.6.1", - "oss/release-notes/releases/0.5.4" - ] - } + "oss/release-notes/release-notes", + "oss/release-notes/upgrading-elementary" ] } ] @@ -961,6 +915,14 @@ { "source": "/cloud/ai-agents", "destination": "/cloud/ai-agents/overview" + }, + { + "source": "/data-tests/dbt/release-notes/releases/:slug*", + "destination": "/data-tests/dbt/release-notes" + }, + { + "source": "/oss/release-notes/releases/:slug*", + "destination": "/oss/release-notes/release-notes" } ] } diff --git a/docs/oss/release-notes/release-notes.mdx b/docs/oss/release-notes/release-notes.mdx new file mode 100644 index 000000000..486bd34ab --- /dev/null +++ b/docs/oss/release-notes/release-notes.mdx @@ -0,0 +1,8 @@ +--- +title: "Release notes" +sidebarTitle: "Release notes" +--- + +import ReleaseNotesLinks from '/snippets/oss/release-notes-links.mdx'; + + diff --git a/docs/oss/release-notes/releases/0.10.0.mdx b/docs/oss/release-notes/releases/0.10.0.mdx deleted file mode 100644 index fdd2f3ce3..000000000 --- a/docs/oss/release-notes/releases/0.10.0.mdx +++ /dev/null @@ -1,38 +0,0 @@ ---- -title: "Elementary 0.10.0" -sidebarTitle: "0.10.0" ---- - -_August 24, 2023: [v0.10.0 Python](https://github.com/elementary-data/elementary/releases/tag/v0.10.0), [v0.10.0 dbt package](https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.10.0)_ - -### 🔥 What's New? - -- **We now officially support Spark and Databricks Unity Catalog!** - - Huge thanks to [Rom Shitrit](https://elementary-community.slack.com/team/U05G2NHMNRL) for working with us on Spark, it was a pleasure! - - Thanks so much [Chinnu](https://elementary-community.slack.com/team/U0571A75LP9) for making the Databricks Catalog support contribution, much appreciated! - - These integrations are very new so there may be some hiccups, please contact us in [#support](https://elementary-community.slack.com/archives/C02CTC89LAX) if you notice anything funny 🙏🏼 -- **Elementary reports can now be [uploaded to Azure Blob](https://docs.elementary-data.com/oss/guides/share-observability-reporthost-on-azure)!** ☁️ - - Thanks [Dharit Sura](https://elementary-community.slack.com/team/U047ZFZRDCH) for opening this issue and huge kudos to [Arun K C](https://elementary-community.slack.com/team/U05K8141BF1) for taking this on and making it happen! 💪🏼 -- **New selector filters for alerts** - - We added 2 new filters to `edr monitor`, `resource_type` and `status`! - - This means it is now possible to run `edr monitor` separately for warnings and failures, and send them to different slack channels! - - Thanks [Josh Elston-Green](https://elementary-community.slack.com/team/U04CSH84TBN) for requesting this, and also [Pascal Moreau](https://elementary-community.slack.com/team/U04GR7G5G0K) for opening the [issue on Github](https://github.com/elementary-data/elementary/issues/630) - - See the [docs](https://docs.elementary-data.com/guides/alerts-configuration#filter-alerts) for more details -- **Test your alerts in "dev" mode** - - You can now run alerts that ignore your custom channels and suppression settings. This is intended for testing alerts in dev that you don't want everyone to panic about. Just try `edr monitor --override-dbt-project-config` - - Thanks [Snir Israeli](https://elementary-community.slack.com/team/U04HZGN1AAY) for opening this feature request! - -### Other Changes ✨ - -- **As of this version we will no longer support dbt versions prior to 1.3** - - This means you shouldn't upgrade Elementary to 0.10.0 if you're using an older version of dbt 🙅🏽‍♀️ - - Older versions of Elementary are not affected by this - -### Bugfixes 🐛 - -- **Table name missing in some generic tests** - - Fixed by [Ronan OBERT](https://elementary-community.slack.com/team/U053CF0BX71)'s contribution 🙌 -- Fixed a bunch of typos in the docs - - Thanks to [Marcell Kovács](https://github.com/kovaacs) for noticing and fixing! We also added a typo checker, so you can except our docs to be typo free going forward ✅ -- Fixed some character encoding issues in the report - - Thanks [Assaf Haim](https://elementary-community.slack.com/team/U056LD1BJ74), [Christopher Ekfeldt](https://elementary-community.slack.com/team/U04BMS39WRL), [Ilya Michlin](https://elementary-community.slack.com/team/U05FS239ER1) and everyone else who raised this diff --git a/docs/oss/release-notes/releases/0.11.2.mdx b/docs/oss/release-notes/releases/0.11.2.mdx deleted file mode 100644 index a768d902f..000000000 --- a/docs/oss/release-notes/releases/0.11.2.mdx +++ /dev/null @@ -1,37 +0,0 @@ ---- -title: "Elementary 0.11.2" -sidebarTitle: "0.11.2" ---- - -_October 24, 2023: [v0.11.0 Python](https://github.com/elementary-data/elementary/releases/tag/v0.11.0), [v0.11.2 dbt package](https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.11.2)_ - -### 🔥 What's New? - -- **We added many cool new features to our anomaly detection tests!** - - - `fail_on_zero` - You can now configure your volume anomaly tests to fail if there are exactly 0 new records, in addition to the anomaly detection algorithm. Read more [here](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/fail_on_zero) - - `detection_delay` - If your data loads with an expected delay, you can now [delay the detection period](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/detection-delay). This will prevent false alerts on freshness and volume. Thanks [Francois Forster](https://github.com/francoisforster) for raising the issue and making this amazing contribution! - - `ignore_small_changes` - If your data set is very static, and even a small change can cause a false anomaly detection, you can now use percentiles to ignore small changes. Read more about this [here](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/ignore_small_changes) - -- ✨ **New test!** ✨ **[exposure_schema_validity](https://docs.elementary-data.com/guides/add-exposure-tests)** 👓 - - - This test helps to make sure your exposures are not broken by any changes in the upstream models and sources! If a column the exposure depends on changes or gets deleted, the test will fail and alert about the change to those who need to know. - -- Alerts linked to the report! ⚠️🔗📊 - - - If you are hosting your Elementary report on a server or a bucket, just add `--report-url link/to/report` to your `edr monitor` command and each alert will have a link to the results of the test in your hosted report - -- **Crazy performance improvements** ⌚️ - - You should see the impact in runtimes of - - - dbt tests hook - Reduced the duration of tests failed row sampling and failed rows count - - Schema changes test - Code was optimized, significantly improving performance - - Anomaly detection tests - Metrics backfill process was optimized, also reducing the time it takes them to run - -### Other Changes and Bug fixes 🐞 - -- We added a new contribution guide for both the [dbt package](https://github.com/elementary-data/dbt-data-reliability/blob/master/CONTRIBUTING.md) and [CLI](https://github.com/elementary-data/elementary/blob/master/CONTRIBUTING.md) -- Source Freshness Alerts now contain more accurate details of the dbt source freshness invocation -- Fix quote for temp table name by [@Trevor Osborne](https://github.com/trevdoz) -- Some code refactors by [@kovaacs](https://github.com/kovaacs) -- Fixing Dependencies in DbtRunner for Uninstalled Packages by [@Arun K C](https://github.com/Arun-kc) -- Further document custom slack channels by [@Javier López Tomás](https://github.com/JavierLopezT) diff --git a/docs/oss/release-notes/releases/0.16.0.mdx b/docs/oss/release-notes/releases/0.16.0.mdx deleted file mode 100644 index a31f54695..000000000 --- a/docs/oss/release-notes/releases/0.16.0.mdx +++ /dev/null @@ -1,9 +0,0 @@ ---- -title: "Elementary 0.16.0" -sidebarTitle: "0.16.0" ---- - -This update includes new features and improvements. - -Check out the full details here: https://github.com/elementary-data/elementary/releases/tag/v0.16.0 - diff --git a/docs/oss/release-notes/releases/0.16.1.mdx b/docs/oss/release-notes/releases/0.16.1.mdx deleted file mode 100644 index b920d99db..000000000 --- a/docs/oss/release-notes/releases/0.16.1.mdx +++ /dev/null @@ -1,9 +0,0 @@ ---- -title: "Elementary 0.16.1" -sidebarTitle: "0.16.1" ---- - -This update includes bug fixes and improvements. - -Check out the full details here: https://github.com/elementary-data/elementary/releases/tag/v0.16.1 - diff --git a/docs/oss/release-notes/releases/0.16.2.mdx b/docs/oss/release-notes/releases/0.16.2.mdx deleted file mode 100644 index d6c741f74..000000000 --- a/docs/oss/release-notes/releases/0.16.2.mdx +++ /dev/null @@ -1,20 +0,0 @@ ---- -title: "Elementary 0.16.2" -sidebarTitle: "0.16.2" ---- - -🚀 **Elementary v0.16.2 is Here!** - -We’re happy to share the latest update to our **open-source package**, bringing **smarter alerts, improved reporting, and enhanced infrastructure** to help **small data teams** - -monitor their data more effectively. - -**What’s New?** - -📊 **Report Enhancements** – Improved navigation, lineage visualization, and full test visibility. - -🚨**Smarter Alerts** – Grouped notifications, **Microsoft Teams Adaptive Cards**, and **seed alerts**. - -🛠️**General Improvements** – DBT package lock files, **better error logging**, and **Google Cloud Storage updates.** - -Check out the full details here: https://github.com/elementary-data/elementary/releases/tag/v0.16.2 . \ No newline at end of file diff --git a/docs/oss/release-notes/releases/0.17.0.mdx b/docs/oss/release-notes/releases/0.17.0.mdx deleted file mode 100644 index e0217c9f2..000000000 --- a/docs/oss/release-notes/releases/0.17.0.mdx +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: "Elementary 0.17.0" -sidebarTitle: "0.17.0" ---- - -This update includes: - -- 🔧 UI bug fixes in the CLI -- 🔔 Some improvements to alerts -- 🔄 Version alignment between the CLI and dbt package - -Nothing major—just some refinements to keep things running smoothly. Check out the full details here: https://github.com/elementary-data/elementary/releases/tag/v0.17.0 \ No newline at end of file diff --git a/docs/oss/release-notes/releases/0.18.0.mdx b/docs/oss/release-notes/releases/0.18.0.mdx deleted file mode 100644 index 9b65f3027..000000000 --- a/docs/oss/release-notes/releases/0.18.0.mdx +++ /dev/null @@ -1,14 +0,0 @@ ---- -title: "Python v0.18.0" -sidebarTitle: "0.18.0" ---- - -## What's Changed - -- Invocation Filter Fix: Invocation filters now apply to both reports and their summaries, ensuring consistent filtering.​ -- Python Version Support: Official support for Python 3.8 has been discontinued to align with dbt's supported versions.​ -- Test Description Bug: Fixed an issue where test descriptions were missing in alerts when using dbt version 1.9. - -Note: We've bumped the minor version to align with the recent minor version update in the dbt package. - -Full Changelog: [v0.17.0...v0.18.0](https://github.com/elementary-data/elementary/compare/v0.17.0...v0.18.0) \ No newline at end of file diff --git a/docs/oss/release-notes/releases/0.18.1.mdx b/docs/oss/release-notes/releases/0.18.1.mdx deleted file mode 100644 index 1b726bf7e..000000000 --- a/docs/oss/release-notes/releases/0.18.1.mdx +++ /dev/null @@ -1,13 +0,0 @@ ---- -title: "Elementary 0.18.1" -sidebarTitle: "0.18.1" ---- - -This update includes: - -- 🗄️ Athena now works in the CLI -- 🔍 Added NOT_CONTAINS filter type -- 🔧 Development workflow improvements - -Check out the full details here: https://github.com/elementary-data/elementary/releases/tag/v0.18.1 - diff --git a/docs/oss/release-notes/releases/0.18.2.mdx b/docs/oss/release-notes/releases/0.18.2.mdx deleted file mode 100644 index 9ebefcb81..000000000 --- a/docs/oss/release-notes/releases/0.18.2.mdx +++ /dev/null @@ -1,17 +0,0 @@ ---- -title: "Elementary 0.18.2" -sidebarTitle: "0.18.2" ---- - -This update includes: - -- 🔔 Subscribers in grouped alerts -- 🗄️ ClickHouse CLI integration -- 📁 FileSystemMessagingIntegration support -- ☁️ S3 report permissions control with --s3-acl option -- 🐛 Fixed setup of internal dbt project -- 🔧 Added function for `disable_elementary_logo_print` -- 📊 Updated report to 1.0.26 - -Check out the full details here: https://github.com/elementary-data/elementary/releases/tag/v0.18.2 - diff --git a/docs/oss/release-notes/releases/0.18.3.mdx b/docs/oss/release-notes/releases/0.18.3.mdx deleted file mode 100644 index ecf8bbc45..000000000 --- a/docs/oss/release-notes/releases/0.18.3.mdx +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: "Elementary 0.18.3" -sidebarTitle: "0.18.3" ---- - -This update includes: - -- 🐛 Fixed missing metrics in CLI - -Check out the full details here: https://github.com/elementary-data/elementary/releases/tag/v0.18.3 - diff --git a/docs/oss/release-notes/releases/0.19.0.mdx b/docs/oss/release-notes/releases/0.19.0.mdx deleted file mode 100644 index 6d410ff61..000000000 --- a/docs/oss/release-notes/releases/0.19.0.mdx +++ /dev/null @@ -1,16 +0,0 @@ ---- -title: "Elementary 0.19.0" -sidebarTitle: "0.19.0" ---- - -This update includes: - -- 🔔 Enhanced alert messages with support for multiple links and icons -- 🔄 Version alignment updates - -**Known Issues:** - -- dbt-databricks must be below 1.10.2 (See [issue 1931](https://github.com/elementary-data/elementary/issues/1931) for more details) - -Check out the full details here: https://github.com/elementary-data/elementary/releases/tag/v0.19.0 - diff --git a/docs/oss/release-notes/releases/0.19.1.mdx b/docs/oss/release-notes/releases/0.19.1.mdx deleted file mode 100644 index fe53b83ad..000000000 --- a/docs/oss/release-notes/releases/0.19.1.mdx +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: "Elementary 0.19.1" -sidebarTitle: "0.19.1" ---- - -This update includes: - -- 🔧 Added excludes option to edr monitor -- 📝 Text and markdown format support - -Check out the full details here: https://github.com/elementary-data/elementary/releases/tag/v0.19.1 - diff --git a/docs/oss/release-notes/releases/0.19.2.mdx b/docs/oss/release-notes/releases/0.19.2.mdx deleted file mode 100644 index e4b22961c..000000000 --- a/docs/oss/release-notes/releases/0.19.2.mdx +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: "Elementary 0.19.2" -sidebarTitle: "0.19.2" ---- - -This update includes: - -- 🐛 Fixed backwards compatibility issue with pydantic - -Check out the full details here: https://github.com/elementary-data/elementary/releases/tag/v0.19.2 - diff --git a/docs/oss/release-notes/releases/0.19.3.mdx b/docs/oss/release-notes/releases/0.19.3.mdx deleted file mode 100644 index 2b579fbb1..000000000 --- a/docs/oss/release-notes/releases/0.19.3.mdx +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: "Elementary 0.19.3" -sidebarTitle: "0.19.3" ---- - -This update includes: - -- 🔧 Fixed package version - -Check out the full details here: https://github.com/elementary-data/elementary/releases/tag/v0.19.3 - diff --git a/docs/oss/release-notes/releases/0.19.4.mdx b/docs/oss/release-notes/releases/0.19.4.mdx deleted file mode 100644 index 0bee7152f..000000000 --- a/docs/oss/release-notes/releases/0.19.4.mdx +++ /dev/null @@ -1,12 +0,0 @@ ---- -title: "Elementary 0.19.4" -sidebarTitle: "0.19.4" ---- - -This update includes: - -- 🔔 Improved freshness alerts with full source names -- 🔄 Updated elementary dbt package to v0.19.2 - -Check out the full details here: https://github.com/elementary-data/elementary/releases/tag/v0.19.4 - diff --git a/docs/oss/release-notes/releases/0.19.5.mdx b/docs/oss/release-notes/releases/0.19.5.mdx deleted file mode 100644 index 0d82d0ae4..000000000 --- a/docs/oss/release-notes/releases/0.19.5.mdx +++ /dev/null @@ -1,15 +0,0 @@ ---- -title: "Elementary 0.19.5" -sidebarTitle: "0.19.5" ---- - -This update includes: - -- 🔔 Slack integration improvements with timeout feature -- 🐛 Fixed Slack join channel recursion issue -- 📊 Dimension anomalies visualization -- ⚡ Performance improvements with alert filters using sets -- 🔄 Dremio types mapping updates - -Check out the full details here: https://github.com/elementary-data/elementary/releases/tag/v0.19.5 - diff --git a/docs/oss/release-notes/releases/0.20.0.mdx b/docs/oss/release-notes/releases/0.20.0.mdx deleted file mode 100644 index 03447ada2..000000000 --- a/docs/oss/release-notes/releases/0.20.0.mdx +++ /dev/null @@ -1,15 +0,0 @@ ---- -title: "Elementary 0.20.0" -sidebarTitle: "0.20.0" ---- - -This update includes: - -- 🔧 dbt Fusion support -- 🔔 Improvements to alert messages with attribution blocks -- 🐛 Bug fixes for ClickHouse, CI artifacts, and dbt profiles -- 📊 Teams alerts formatting improvements -- 🔄 Updated dbt package and removed deprecated flags - -Check out the full details here: https://github.com/elementary-data/elementary/releases/tag/v0.20.0 - diff --git a/docs/oss/release-notes/releases/0.5.4.mdx b/docs/oss/release-notes/releases/0.5.4.mdx deleted file mode 100644 index 81d573b7f..000000000 --- a/docs/oss/release-notes/releases/0.5.4.mdx +++ /dev/null @@ -1,36 +0,0 @@ ---- -title: "Elementary 0.5.4" -sidebarTitle: "0.5.4" ---- - -_November 3, 2022: [v0.5.4 Python](https://github.com/elementary-data/elementary/releases/tag/v0.5.4), [v0.5.4 dbt package](https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.5.4)_ - -### 🔥 What's new? - -- **Source freshness alerts** 🥬🔔 - - - Elementary now provides alerts for all dbt operations, including source freshness! - - Here are [the docs](/oss/guides/alerts/elementary-alerts#alert-on-source-freshness-failures). - -- **Artifacts are uploaded at on-run-end hooks** ⚡️ - - - **You no longer need to run** `dbt run --select elementary` **on merge!** - - This will keep metadata up-to-date and avoid inconsistency with alerts and report. - - We also made changes to the upload method to improve performance. - -- **New `dbt_invocations` table** 🆕⏱ - - - Tracks run times, command, vars, selected_resources, dbt and Elementary versions - -- **Improved report performance** 📊 - - - Based on your feedback, we invested a lot in performance improvements in the report to make it slick for big envs. - - We are still improving the lineage screen, but this is already an improvement. - -- **Improved lineage screen filters** 🔽 - - Thank you [@Björn van Dijkman](https://elementary-community.slack.com/team/U042EF5FP9S) for valuable product feedback! - -### ⭐️ Contributions & Acknowledgements - -- [@Valdar Tammik](https://elementary-community.slack.com/team/U04867979R8) added `disable_skipped_test_alerts` var, `true` by default. Thank you! 🤩 -- Docs additions and fixes by [@Alex Alves](https://elementary-community.slack.com/team/U03FN5JMVEY) and [@Rodrigo Loredo](https://elementary-community.slack.com/team/U03UZA2BYRZ), Thank you! 🤩 diff --git a/docs/oss/release-notes/releases/0.6.1.mdx b/docs/oss/release-notes/releases/0.6.1.mdx deleted file mode 100644 index ecc48dee0..000000000 --- a/docs/oss/release-notes/releases/0.6.1.mdx +++ /dev/null @@ -1,41 +0,0 @@ ---- -title: "Elementary 0.6.1" -sidebarTitle: "0.6.1" ---- - -_November 28, 2022: [v0.6.1 Python](https://github.com/elementary-data/elementary/releases/tag/v0.6.1), [v0.6.1 dbt package](https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.6.1)_ - -### 🔥 What's new? - -- **Elementary is 10x Faster!!!** ⚡️🏃‍ - - - **Report generation 10x Faster** 💪 - - We improved our mechanism of collecting test samples, greatly reducing the time it takes to run a report! - - **Lineage screen 10x Faster** 💪 - - Major improvements for large graphs! - - **On-run-end hook 10x Faster** 💪 - - We now cache collected dbt artifacts locally and only upload them if they changed from the last run. - -- **Support for showing test descriptions** in both reports and alerts! 📃 -- - -### 💫 More changes - -- **Test schema is no longer needed** (e.g. the `__tests` schema is now optional). -- AWS credentials for S3 automatically detected (via `boto3`) - Contributed by [Por Heng](https://github.com/pei0804)! 🎉 -- Added query ID to `dbt_run_results`, `model_run_results`, and `snapshot_run_results` (currently supported for Snowflake only) - Contributed by [Prabahar Kamatchirajan](https://github.com/kkprab)! 🎉 -- Added a flag of whether the invocation was the first or last of the day to `model_run_results` - Contributed by [Ekaterina Khrushch](https://github.com/eKatia)! 🎉 -- Added a `--quoting` flag to the CLI to allow overriding the default dbt quoting settings in the CLI's internal dbt package - Thanks [Daniel Gaimster](https://github.com/gtmehrer) for reporting! - -### 🐞 Bug fixes - -- Reduced the maximum query size on BigQuery to avoid "query too big" errors. -- Fixed a bug that caused uploading the dbt invocations to crash. -- Evaluate config args correctly by hierarchy even when they are False. - -### ⭐️ Additional Contributions & Acknowledgements - -- Thanks [Nabeel Zewail](https://github.com/nzewail) for making a first contribution, with the removal of `compiled_code` change. -- Thanks [Rochelle Smits](https://github.com/smitsrr) for making a first contribution, adding the documentation for `dbt_invocations`. -- Thanks [Wojciech Kopacki](https://github.com/kopackiw) for making a first contribution, with the documentation fix regarding running elementary's models. -- Thanks [Stéphane Burwash](https://github.com/SBurwash) for making a first contribution, and developing an **Elementary extension for Meltano!** diff --git a/docs/oss/release-notes/releases/0.6.10.mdx b/docs/oss/release-notes/releases/0.6.10.mdx deleted file mode 100644 index 6e8def21d..000000000 --- a/docs/oss/release-notes/releases/0.6.10.mdx +++ /dev/null @@ -1,40 +0,0 @@ ---- -title: "Elementary 0.6.10" -sidebarTitle: "0.6.10" ---- - -_February 8, 2023: [v0.6.10 Python](https://github.com/elementary-data/elementary/releases/tag/v0.6.10), [v0.6.13 dbt package](https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.6.13)_ - -### 🔥 What's new? - -- **Dashboard screen** 📊👀 - - - **We added a new dashboard screen to the UI!** - - See a breakdown of your test results, tables health, monitored tables. - - Understand the high level status and the trends over time. - - You can also filter by tag / owner and get a dashboard view only of the tables that are relevant to you! - - [You can play with it in the demo](https://storage.googleapis.com/elementary_static/elementary_demo.html#/dashboard). - - Already working on making the graphs clickable to enable drill down (in the next version) 💪 - -- **Major `on-run-end` performance improvement** ⚡️ - - - We invested in reducing even more the impact of artifacts uploading on your runs. - - The improvement is achieved with caching we implemented, to only upload artifacts if there is a change from the last run. - - _Note that this only works for dbt 1.4.0 and above (we made a contribution to dbt-core to support this)._ - -- **dbt artifacts additions** ➕ - - New table `dbt_seeds` - A contribution by [Franny Delaney](https://github.com/frannydelaney), thank you so much 🤩 - - `thread_id` was added to run results tables, and `dbt_user` env var was added to dbt_invocations - Thank you [Elias Gryp](https://github.com/eliasgeee) and [Ayal Alon](https://github.com/ayalalo) for suggesting these! - -### 🐞 Bug fixes - -- Fixed alerts duplications issue and email owners in Slack alerts - Thanks [Josh Elston-Green](https://elementary-community.slack.com/team/U04CSH84TBN) for reporting both! -- Fixed events tracking from the report are not disabled - Thanks [Bendik Nesbo](https://elementary-community.slack.com/team/U04LZ1T7DR6) for reporting! -- Fixed synonymous data types bug in schema changes from baseline test - Thanks [Manish Ramrakhiani](https://elementary-community.slack.com/team/U04GACNBRDM) for reporting! -- Fixed `insert_rows` failures on anomaly tests with small time buckets - Thanks [Noam Moskowitz](https://elementary-community.slack.com/team/U035FGRPXQX) for reporting! -- Fixed truncate param in CLI causing failures - Thank you [Chintan Patel](https://elementary-community.slack.com/team/U04B51NEJKC) for contributing! -- Fixes to our docs by [Ed Bizarro](https://elementary-community.slack.com/team/U04757V137A) and [Oskar Elvkull](https://elementary-community.slack.com/team/U04KFB4F7EJ) - Thank you for contributing! - - - Demo - diff --git a/docs/oss/release-notes/releases/0.6.3.mdx b/docs/oss/release-notes/releases/0.6.3.mdx deleted file mode 100644 index 562337293..000000000 --- a/docs/oss/release-notes/releases/0.6.3.mdx +++ /dev/null @@ -1,47 +0,0 @@ ---- -title: "Elementary 0.6.3" -sidebarTitle: "0.6.3" ---- - -_December 21, 2022: [v0.6.3 Python](https://github.com/elementary-data/elementary/releases/tag/v0.6.3), [v0.6.6 dbt package](https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.6.6)_ - -### 🔥 What's new? - -- **Slack alerts V2** 2️⃣ - - - **Elementary now supports customizing alerts, so you could notify data producers and consumers** 🧑‍🔧 - - Instead of sending detailed technical alerts, you can select fields. Learn more here [how to configure it](/oss/guides/alerts/elementary-alerts#customize-alerts-owners-test-descriptions-tags-subscribers-and-layout). - - Email owners will be translated to Slack handles in the alerts. Thanks [Josh Elston-Green](https://github.com/jelstongreen) for contributing. - - Alerts also underwent **layout improvements** 🎨 - -- **New data tests** 🩺 - - - [Advanced schema tests](/data-tests/schema-tests/schema-changes), **useful for source monitoring and enforcing data contracts:** ✅ - - `schema_changes_from_baseline` - Define a schema in your `models/` and ensure its validity. - - `json_schema` - Validate the schema of JSON columns. 😱 - - To ease configuration and implementation - both tests include operations to **auto-generate the schema.** - - New and [improved table anomalies tests](/data-tests/add-elementary-tests), including `where_expression`: 📉 - - `volume_anomalies` - Monitor your row count over time. - - `freshness_anomalies` - Monitor the freshness of you data over time. - -- **Report UI new features** 🆕 - - - Filter your report by **Tags & Owners sidebars** #️⃣🤵🏻 - - [dbt-expectations](https://github.com/calogica/dbt-expectations) integration in the UI, including test descriptions. - - Custom test descriptions were added to the UI. - -- **Monitor job results** ⚛️ - - - Added job-related columns to `dbt_invocations` and created `job_run_results` table. - - Keep track of your dbt jobs in dbt Cloud, GitHub Actions, Airflow, etc. - - The new columns [can be viewed here](https://github.com/elementary-data/dbt-data-reliability/blob/4695474ba10740a95822954c44adbc52f4f2af95/models/run_results.yml#L81). - -- **Sensitive data / PII** 🤫 - - Have sensitive data that contains PII? x - - Use the new flag (`-disable-samples`) to disable sampling of data in the report and alerts - -### 💫 More changes - -- Added support for `dbt-utils 1.0.0` - Thanks [Jumpei Chikamori](https://github.com/pei0804) for contributing. -- Elementary's tables are not full-refreshed by default - This can be changed using the `elementary_full_refresh` var. -- The report doesn't auto-open when using WSL - Thanks [Ivan Toriya](https://github.com/ivan-toriya) for contributing. diff --git a/docs/oss/release-notes/releases/0.6.5.mdx b/docs/oss/release-notes/releases/0.6.5.mdx deleted file mode 100644 index 8f6d19bdd..000000000 --- a/docs/oss/release-notes/releases/0.6.5.mdx +++ /dev/null @@ -1,41 +0,0 @@ ---- -title: "Elementary 0.6.5" -sidebarTitle: "0.6.5" ---- - -_January 10, 2023: [v0.6.5 Python](https://github.com/elementary-data/elementary/releases/tag/v0.6.5), [v0.6.10 dbt package](https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.6.10)_ - -### 🔥 What's new? - -- **Alerts suppression** 💤🔕 - - Don't want to get multiple alerts if the same test keeps failing? - - You can now configure an alert_suppression_interval, this is a "snooze" period for alerts on the same issue. - - [Docs here](/oss/guides/alerts/elementary-alerts#customize-alerts-owners-test-descriptions-tags-subscribers-layout-and-suppression) -- **Alerts filtering using selectors** 🔘 - - - You can now configure edr to send only selected alerts, using tags, owners or models selectors. - - This enables you to send only the alerts you actually care about. - - [Docs here](/oss/guides/alerts/elementary-alerts#execute-the-cli) - -- **Generate report for a single invocation** 1️⃣ - - Use selectors to generate a pre-filtered report with the results you want to see. - - Try it: `edr monitor report --select last_invocation` - - [Docs here]() -- **Granular time buckets in anomaly detection tests** ⏳ - - Now you can set time buckets that fit the updates cadence of your tables, to get better anomaly detection. - - The new config enables to bucket by `hour | day | week` and set the interval size as well. - - Example: 6 hours buckets for volume test - ```yml properties.yml - tests: - - elementary.volume_anomalies: - timestamp_column: updated_at - time_bucket: - period: hour - count: 6 - ``` - - This was a very popular request here on Slack, special thanks to [Rodrigo Loredo](https://github.com/rloredo) for your help with the implementation! - -### 💫 More changes - -- Following the breaking change we experienced in dbt 1.3.2, we added dbt release candidates versions to our CI. This will enable us to prevent such incidents in the future. -- We are evaluating a change in how we upload dbt artifacts to improve performance and unlock more use cases. For now you will see a warning message that will not impact your usage. diff --git a/docs/oss/release-notes/releases/0.6.7.mdx b/docs/oss/release-notes/releases/0.6.7.mdx deleted file mode 100644 index 96680de05..000000000 --- a/docs/oss/release-notes/releases/0.6.7.mdx +++ /dev/null @@ -1,35 +0,0 @@ ---- -title: "Elementary 0.6.7" -sidebarTitle: "0.6.7" ---- - -_January 19, 2023: [v0.6.7 Python](https://github.com/elementary-data/elementary/releases/tag/v0.6.7), [v0.6.11 dbt package](https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.6.11)_ - -### 🔥 What's new? - -- **Postgres integration** 🆕 - - Elementary now supports Postgres! Thank you [Tasos Kouridis](https://github.com/kouridis) for this meaningful contribution! - - [Docs here](/oss/integrations/postgres) -- **Freshness anomalies monitoring improvements** 🥬 - - We have split our freshness anomalies test into two different tests to better handle two common different use cases: - - `freshness_anomalies` - monitors anomalies in the frequency of updates. Imagine a table that is being updated hourly and then all of a sudden this table wasn't updated for 4 hours. Here you would probably use the `updated_at`/`inserted_at` column as an input for this test. - - `event_freshness_anomalies` - monitors that the underlying data is up to date. Imagine an events table that is being updated hourly correctly but the events themselves are coming with a delay of 2 days from your production databases. Here, you would probably use the `event_timestamp` column as an input for this test. - - [Docs here](/data-tests/anomaly-detection-tests/freshness-anomalies) - -### 💫 More changes - -- Added compatibility with dbt's upcoming release (1.4.0). -- Following the breaking change we experienced in dbt 1.3.2, we've re-added the source freshness test support. -- Added a few safety measures to make sure concurrent runs are always supported. -- Thank you [Emil Nilsson](https://github.com/esenilsson) for adding the ability to choose a custom S3 endpoint to host the report when using edr cli (`--s3-endpoint-url TEXT`). -- We updated the `schema_changes_from_baseline` test comparison to be case-insensitive. Thanks Manish Ramrakhiani for bringing this up! -- Thank you [Pratik Bothra](https://github.com/pratik60), [Clinton Adams](https://github.com/clad) and [Benoit Turbang](https://github.com/benoit-tropos-io) for helping us keep our documentation up to date and typos free 😊 - -### 🐞 Bug fixes - -- Resolved a race condition that sometimes caused a run failure (caused `TestRunSchema.test_runs: None is not an allowed value error`). -- Fixed a bug where a test that always failed has the value 0% for ‘Fail Rate’. Thanks Jon Simpson for bringing this up! -- Fixed a bug where `where_expression` was only implemented when a timestamp was defined. Thanks Ran Tibi for bringing this up! -- Fixed `UnicodeDecodeError` error in Windows due to their default encoding. - -As part of this version, we've invested in our testing infrastructure to ensure we're able to keep moving fast and providing valuable features! 💪⚡ diff --git a/docs/oss/release-notes/releases/0.7.10.mdx b/docs/oss/release-notes/releases/0.7.10.mdx deleted file mode 100644 index f2bf9a162..000000000 --- a/docs/oss/release-notes/releases/0.7.10.mdx +++ /dev/null @@ -1,41 +0,0 @@ ---- -title: "Elementary 0.7.10" -sidebarTitle: "0.7.10" ---- - -_May 17, 2023: [v0.7.10 Python](https://github.com/elementary-data/elementary/releases/tag/v0.7.10), [v0.7.8 dbt package](https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.7.8)_ - -### 🔥 What's new? - -- **New artifacts model - dbt_columns 🆕** - - Many users requested that we add this model with useful information about the columns in your project. - - We also plan to add features to the report and alerts based on this data. Stay tuned... :coming-soon: - - (This is not yet supported for Databricks, [working on it](https://github.com/elementary-data/elementary/issues/872)) -- **New lineage filters - Tags and owners #️⃣👥** - - You can now filter the lineage to see only the nodes that are relevant to a business department or a specific owner, and their upstream and downstream nodes. - - We also made some general improvements to the filters usability like adding search and clear all. -- **Test results sample size in the report can be changed 🔢** - - - The report is no longer limited to 5 results, you can change this by adding the var: - - `test_sample_row_count: 10` - -- New flag - `edr --version` 🏁 - - Thank you [@Manul Patel](https://elementary-community.slack.com/team/U054N92MU11) for contributing this  🤩 - -### 💫 More changes - -- Alerts suppression interval is no longer limited to 24 hours. -- Added indicative exceptions to Elementary tests. -- `time_bucket` can now be configured in model and var levels as well. -- Created workarounds to solve breaking changes in dbt 1.5.0 adapters: -- Not rely on Databricks adapter to create temp tables - Thanks [@Joseph Berni](https://elementary-community.slack.com/team/U03NWMS0Y93), [@fitz](https://elementary-community.slack.com/team/U03V5KGR3U3) and [@Dharit Sura](https://elementary-community.slack.com/team/U047ZFZRDCH) for reporting! -- Run queries from `run` instead of `run-operation` due to bug in Redshift adapter - Thank you [@Eugene Sobolev](https://elementary-community.slack.com/team/U054BV7MR0T) for reporting and investigating with us! - -### 🐞 Bug fixes - -- Support dbt run results compiled sql with % on Redshift new adapter - Thanks [@Fabien Traventhal](https://elementary-community.slack.com/team/U03G693L05R) for reporting! -- Fixed ignored backfill_days in no timestamp tests - Thanks [@leila](https://elementary-community.slack.com/team/U04HFCUM2G6) and [@Roland Baranovic](https://elementary-community.slack.com/team/U04K5SUJS8Z) for reporting! -- Fixed alerts `-group-by` empty value - Thank you [@Dimosthenis Schizas](https://elementary-community.slack.com/team/U054B4PNACE) for contributing 🤩 -- Owners accept dict format - Thank you [@Stephen Lloyd](https://elementary-community.slack.com/team/U03FQELBBV1) for reporting and [@Manul Patel](https://elementary-community.slack.com/team/U054N92MU11) for contributing 🤩 -- Paginate upload of source freshness data for large results - Thank you [@Fred](https://elementary-community.slack.com/team/U03QXQ3VCF8) for reporting and fixing 🤩 -- Thank you [@winzee](https://github.com/winzee) and [@vinooganesh](https://github.com/vinooganesh) for helping keep our docs accurate and typos free 🎉e Melhuish](https://elementary-community.slack.com/team/U04KWBDTP4J)! diff --git a/docs/oss/release-notes/releases/0.7.2.mdx b/docs/oss/release-notes/releases/0.7.2.mdx deleted file mode 100644 index 6d0a8ffed..000000000 --- a/docs/oss/release-notes/releases/0.7.2.mdx +++ /dev/null @@ -1,40 +0,0 @@ ---- -title: "Elementary 0.7.2" -sidebarTitle: "0.7.2" ---- - -_February 16, 2023: [v0.7.2 Python](https://github.com/elementary-data/elementary/releases/tag/v0.7.2), [v0.7.1 dbt package](https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.7.1)_ - -### 🔥 What's new? - -- **Report Summary Slack Message** 🆕🔔 - - - When you generate a new report and using `send-report`, you can also trigger a Slack notification summary of the report. - - This is useful for you and your team members to understand quickly if there is something in the new report you should address. - - [Docs here](/oss/guides/share-report-ui#slack-summary-configurations), screen shot attached 📸 - -- **Dashboard graphs are now clickable** #️⃣📊 - - - You can click on graphs and the dashboard, and you will be redirected to the relevant set of results. - - This means you can quickly navigate from the dashboard to a deep dive on the details. - -- **Config changes reset anomaly detection tests** 🔄 - - When you change the configuration of an Elementary test, the metrics for the tests will be automatically re-calculated according to the new configuration. - - This improves the accuracy of the tests, so changes you make will affect immediately and not only after days of accumulating new data. - -### 💫 More changes - -- Added a deployment guide for [Gitlab CI](/oss/deployment-and-configuration/gitlab-ci). Thank you [@Clinton Adams](https://elementary-community.slack.com/team/U04JFRG5N2J) for your help! 🤩 -- Added Spark time related macros, thanks [@Matias Stanislavsky](https://elementary-community.slack.com/team/U04MC30PLK0) for the help! 🤩 -- Changed lineage layout to reduce "spaghettiness" - Looking forward for your feedback on it [@Gautier De Peyronnet](https://elementary-community.slack.com/team/U044ZCCNQ6T)! 🍝 - -### 🐞 Bug fixes - -- Fix the `DBT_ORCHESTRATOR` env var not being used - Thanks [Andre Augusto](https://github.com/andreqaugusto) for contributing! 🤩 -- Fixed long Windows paths error which failed deployment for several users. -- Fixed Windows folder paths in the report appear empty - Thanks [@Tasos Kouridis](https://elementary-community.slack.com/team/U0486T1TD38) for reporting! -- Changed macro names to prevent collision with other dbt packages - Thanks [@Tiphaine Castel](https://elementary-community.slack.com/team/U04NWKLR8CE) for reporting! - - - Demo - diff --git a/docs/oss/release-notes/releases/0.7.5.mdx b/docs/oss/release-notes/releases/0.7.5.mdx deleted file mode 100644 index ee4412c2e..000000000 --- a/docs/oss/release-notes/releases/0.7.5.mdx +++ /dev/null @@ -1,41 +0,0 @@ ---- -title: "Elementary 0.7.5" -sidebarTitle: "0.7.5" ---- - -_March 6, 2023: [v0.7.5 Python](https://github.com/elementary-data/elementary/releases/tag/v0.7.5), [v0.7.3 dbt package](https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.7.3)_ - -### 🔥 What's new? - -- **Grouped alert for test results by table** 🤝🔔 - - - Feeling overwhelmed by alerts on each failed test? - - You can now add `-group-by table` to `edr monitor` and the table test results will be sent as one message. - - Docs are [here](/oss/guides/alerts/elementary-alerts#notification-settings-decide-how-you-want-to-get-your-alerts) ; Pic attached 📸 - - This was requested by [@Jake Beresford](https://elementary-community.slack.com/team/U03BN2E93SQ) and [@Ekaterina Khrushch](https://elementary-community.slack.com/team/U03PVN83DC2) - -- **Add `sum` to column anomaly tests** ➕ - - - You can now monitor anomalies on the sum of a numeric column. - - Thanks [tc-chrisbui](https://github.com/tc-chrisbui) for the contribution! 🤩 - -- **Add `where_expression` to Python tests** 🟰 - - You can now filter the data you are testing in a Python test. - - Thank you [Oskar Elvkull](https://github.com/Deathfireofdoom) for the contribution! 🤩 - -### 💫 More changes - -- Added dbt 1.5.0 support. 😎 - -### 🐞 Bug fixes - -- Fixed a bug in `dimension_anomalies` without a `timestamp_column` - Thanks [@Amihay Gonen](https://elementary-community.slack.com/team/U031P4UGEFK) and [@Alvin M](https://elementary-community.slack.com/team/U04HWDV7D4P) for reporting! -- Fixed a bug with anomaly monitors running on the wrong data type (only in 0.7.2) - Thanks again [@Amihay Gonen](https://elementary-community.slack.com/team/U031P4UGEFK)! -- Fixed problems in the numbers presented in the dashboard 🔢 - - Counting exposures as models - Thanks for reporting [@Willi Mueller](https://elementary-community.slack.com/team/U04NG6HNZC4) ! - - Classifying tests as the wrong kind of test - - Thanks for reporting [@Jan Santora](https://elementary-community.slack.com/team/U03TESH0JSD)! -- Fixed dashboard header sometimes having an empty timestamp - [@Mohamed Omran](https://elementary-community.slack.com/team/U04RBFR5VDK) thanks for reporting! - - - Demo - diff --git a/docs/oss/release-notes/releases/0.7.6.mdx b/docs/oss/release-notes/releases/0.7.6.mdx deleted file mode 100644 index 742e04839..000000000 --- a/docs/oss/release-notes/releases/0.7.6.mdx +++ /dev/null @@ -1,47 +0,0 @@ ---- -title: "Elementary 0.7.6" -sidebarTitle: "0.7.6" ---- - -_March 27, 2023: [v0.7.6 Python](https://github.com/elementary-data/elementary/releases/tag/v0.7.6), [v0.7.4 dbt package](https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.7.4)_ - -### 🔥 What's new? - -- **Custom test names are now supported!** 🖌🎨 - - For every dbt test, you can provide an indicative name and Elementary will show it! - - Here is an example: - -```yaml properties.yml -tests: - - accepted_values: - values: ["ios", "android", "windows"] - name: unexpected_operating_system -``` - -- You can read more about the use cases from [@Marta Massarotti](https://elementary-community.slack.com/team/U04JHEZRDUM) and [@Owen](https://elementary-community.slack.com/team/U04P81RELTS) in the [Github issue](https://github.com/elementary-data/elementary/issues/299). - -- **CLI improvements** 💻😎 - - Target path for all `edr` files - - We added a new `-target-path` flag for all the outputs to make it easier to maintain and deploy Elementary. - - Thanks [@Gemma Down](https://elementary-community.slack.com/team/U04LT9TNUE4) for the issue and use case! - - Added `edr monitor debug` command - - This would help you debug connection and profile issues. - -### 🐞 Bug fixes - -- -- Reduced high memory consumption of `edr` - Thanks [@Jake Beresford](https://elementary-community.slack.com/team/U03BN2E93SQ) for reporting and providing valuable context! -- Improved Redshift upload artifacts performance and prevent concurrency issues - Thanks [@Chintan Patel](https://elementary-community.slack.com/team/U04B51NEJKC), [@Hugo Vandernotte](https://elementary-community.slack.com/team/U04H0MT9YUS) and [@Fabien Traventhal](https://elementary-community.slack.com/team/U03G693L05R) for reporting! -- Fixed `upload source freshness` issue - Thanks [@Collin](https://elementary-community.slack.com/team/U03NBG5C3NK), [@Idris](https://elementary-community.slack.com/team/U04U0LQUBNG), [@Corentin Hembise](https://elementary-community.slack.com/team/U049BKJN4QZ) and [@Muthu](https://elementary-community.slack.com/team/U04TL5H856K) for reporting! -- Fixed bucket URL and html report are not sent in `send-report` - Thanks [@Jack Welty](https://elementary-community.slack.com/team/U04T7FELNJX) for reporting! -- Fix passed tests count as warnings in grouped alerts - Thanks [@John Walsh](https://elementary-community.slack.com/team/U04PB1PR57Z) for reporting! -- Fixed cases and types issues in schema changes from baseline - Thanks [@Mathias Reistad](https://elementary-community.slack.com/team/U04NFU1KJAY), [@Luis Ventura](https://elementary-community.slack.com/team/U04U21QJZFY), [@Yiming](https://elementary-community.slack.com/team/U04HDE0P4NA) and [@manish ramrakhiani](https://elementary-community.slack.com/team/U04GACNBRDM) for reporting, and thank you [Sadahiro Yoshikawa](https://github.com/sadahry) for contributing! 🤩 -- Special thanks to @Will Warner, @Vishal Kella and @Louis-Marius Gendreau for helping keep our docs correct and up to date! 🤩 - -### 📚 Reading list - -We want to share some blog posts by community members - - -- [@Alex Alves](https://elementary-community.slack.com/team/U03FN5JMVEY) wrote about [migrating to a new data warehouse](https://blog.multilayer.io/migrating-from-aws-redshift-to-a-new-data-warehouse-3e6b14d264bf) and how Elementary could help -- [@Jonathan Talmi](https://elementary-community.slack.com/team/U034B3V59LL) wrote about [observability at Super](https://medium.com/super/dbt-at-super-part-3-observability-c8755109901f), and specifically about how they utilize Elementary for alerts -- Also shout out to [@Stéphane Burwash](https://elementary-community.slack.com/team/U03R02KUP1V) for giving a talk about Elementary in a dbt meetup! diff --git a/docs/oss/release-notes/releases/0.7.7.mdx b/docs/oss/release-notes/releases/0.7.7.mdx deleted file mode 100644 index 08eb86330..000000000 --- a/docs/oss/release-notes/releases/0.7.7.mdx +++ /dev/null @@ -1,50 +0,0 @@ ---- -title: "Elementary 0.7.7" -sidebarTitle: "0.7.7" ---- - -_April 12, 2023: [v0.7.7 Python](https://github.com/elementary-data/elementary/releases/tag/v0.7.7), [v0.7.5 dbt package](https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.7.5)_ - -### 🔥 What's new? - -- **Seasonality and Anomaly direction** 📈 - - On this release we are introducing 2 new anomaly detection tests configurations. The goal is to improve the accuracy and unlock more detection use cases. 🎯💪 - - We prepared a [short video to explain](https://www.loom.com/share/b92d880b768d44a7bd2b0010a6223b62) 📽👇🏻 - - `seasonality: day_of_week` 🗓 - - If enabled, daily values will be compared only to the same week day previous values. - - This configuration would improve accuracy and detection of datasets that have different volume and metrics on weekends and different weekdays. - - `anomaly_direction: [ both | spike | drop ]` ↗️↘️ - - By default, elementary alerts on both spikes and drops of a metric. Using this configuration, you can limit anomaly tests to alert only on the wanted direction. - - For example - If you want volume anomalies to fail only on drop in row count, and not on a spike, just add `anomaly_direction: drop` to the test config. - - -
- -
- - -- **on-run-end performance improvements** ⏳ - - We fixed problems in artifacts caching, and implemented an incremental upload. This means that upon a change to models / tests / sources - elementary will now only update the changed resources. - - This should improve the performance of the hooks, and we have additional ideas and planned tasks around further improvements. - - Special thanks to [@panapost](https://elementary-community.slack.com/team/U050WNK4SQL) for helping with this effort 🤩 - -### 🐞 Bug fixes - -- Added support to unsupported time buckets on Bigquery (week, month), and made buckets full and consistent to calendar times - Thank you [@Kenneth Roos](https://elementary-community.slack.com/team/U04UDU686P7) and [@Julian Schulte](https://elementary-community.slack.com/team/U04U9DAPR5L) for reporting! -- Added log rotation to `edr.log` to prevent large log files that won't open - Thank you [@Stéphane Burwash](https://elementary-community.slack.com/team/U03R02KUP1V) and [@Grant Mehrer](https://elementary-community.slack.com/team/U04E1UTPEAE) for reporting! -- Fixed elementary tests custom names issue - Thank you [@Fabien Traventhal](https://elementary-community.slack.com/team/U03G693L05R) for reporting! -- Fixed subscribers parsing issue - Thank you [@Muthu](https://elementary-community.slack.com/team/U04TL5H856K) for reporting! -- Added `target-path` to upload_source_freshness - Requested by [@Ange Melhuish](https://elementary-community.slack.com/team/U04KWBDTP4J)! diff --git a/docs/oss/release-notes/releases/0.8.0.mdx b/docs/oss/release-notes/releases/0.8.0.mdx deleted file mode 100644 index 802f10d13..000000000 --- a/docs/oss/release-notes/releases/0.8.0.mdx +++ /dev/null @@ -1,32 +0,0 @@ ---- -title: "Elementary 0.8.0" -sidebarTitle: "0.8.0" ---- - -_June 1, 2023: [v0.8.0 Python](https://github.com/elementary-data/elementary/releases/tag/v0.8.0), [v0.8.0 dbt package](https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.8.0)_ -_As this is a minor version bump, you need to run `dbt run -s elementary`_ - -### 🔥 What's new? - -- **🆕 Jobs info from Orchestrator 🆕** - - - Elementary now supports collecting metadata about your jobs from your orchestration tool! - - The goal is to provide context that is useful to triage and resolve data issues: - - As a first step, you could filter the lineage by job in the Elementary report. - - More orchestrator related features are coming soon 😎 - - Here is the [guide for enabling jobs info collection.](https://docs.elementary-data.com/deployment-and-configuration/collect-job-data). - -- **You can now configure all test params in the project / model / test level 🤯** - - Why is it useful? - - It enables you to tailor the tests to the dataset and get higher level of accuracy! - - You can leverage inheritance, configure at a higher level (like folder of models) and save the need to configure by test. - - Some examples: - - You can set `days_back: 90` to tests with `time_bucket: period: week`, and `days_back: 7` to tests with `time_bucket: period: hour` . - - You can set `timestamp_column: updated_at` in your dbt_project.yml if this is your convention, and override it for models where it's different. - - You can set `seasonality`, `time_bucket` and `timestamp_column` at the source level, and it will apply for all the tests you add to tables of this source. - - We also upgraded our documentation of the [tests configuration](https://docs.elementary-data.com/data-tests/elementary-tests-configuration) and [how the tests work](https://docs.elementary-data.com/data-tests/how-anomaly-detection-works), to make it clearer 😇 - -### 💫 More changes - -- Added `materialization` field to models run results, thank you [@Aril Mavinkere](https://elementary-community.slack.com/team/U058SJFFTEU) for contributing! 🤩 -- Removed `env` from report summary. diff --git a/docs/oss/release-notes/releases/0.8.2.mdx b/docs/oss/release-notes/releases/0.8.2.mdx deleted file mode 100644 index c22d52f12..000000000 --- a/docs/oss/release-notes/releases/0.8.2.mdx +++ /dev/null @@ -1,32 +0,0 @@ ---- -title: "Elementary 0.8.2" -sidebarTitle: "0.8.2" ---- - -_June 28, 2023: [v0.8.2 Python](https://github.com/elementary-data/elementary/releases/tag/v0.8.2), [v0.8.2 dbt package](https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.8.2)_ - -### 🔥 What's new? - -- **Node info is now available on the lineage view** ℹ️🔗 - - - When clicking on a node on the lineage you are now able to see the node's metadata (owners, tags and description) and job info (job id and name)! - - See how it looks in the attached image 👀 - - Stay tuned - next version will also include exposure info 😱📺 - -- **We improved our table components for environments with long tables name** 🌀 - - We've added the ability to minimize the side bar and set a default width for some of the columns - - Thanks [@Jon Simpson](https://elementary-community.slack.com/team/U0492SDN3NY) for bringing this up! - -### 💫 More changes - -- Fixed a bug where alerts tag selectors did not work. Thanks [@panapost](https://elementary-community.slack.com/team/U050WNK4SQL) and [@Noam Moskowitz](https://elementary-community.slack.com/team/U035FGRPXQX) for bringing this up! -- Fixed a bug where the report was being sent via Slack although it was hosted. Thanks [@Javier López Tomás](https://elementary-community.slack.com/team/U059F4VHBPY) for noticing. -- Handled a situation where the report failed to generate if there are html tags in any of the fields. Thanks [@Pedro Gillet Soares](https://elementary-community.slack.com/team/U0417704XME) for letting us know. -- Added pg_catalog to redshift macros so they work with late binding views. Thank you [kochalex](https://github.com/kochalex) for contributing! 🤩 - - - Node info in lineage view - diff --git a/docs/oss/release-notes/releases/0.9.1.mdx b/docs/oss/release-notes/releases/0.9.1.mdx deleted file mode 100644 index 20e0271b7..000000000 --- a/docs/oss/release-notes/releases/0.9.1.mdx +++ /dev/null @@ -1,40 +0,0 @@ ---- -title: "Elementary 0.9.1" -sidebarTitle: "0.9.1" ---- - -_July 20, 2023: [v0.9.1 Python](https://github.com/elementary-data/elementary/releases/tag/v0.9.1), [v0.9.0 dbt package](https://github.com/elementary-data/dbt-data-reliability/releases/tag/0.9.0)_ - -### 🔥 What's new? - -- **New Seasonality Options: hour_of_day and hour_of_week** 🌞🌙 - - Thanks to the brilliant contribution from [Moss Pauly](https://elementary-community.slack.com/team/U04A3D7GKTQ), you now have two new seasonality options: hour_of_day and hour_of_week. - - These options will help in making the anomaly detection tests more accurate for datasets with strong seasonality impact. - - Check out the [docs](https://docs.elementary-data.com/data-tests/anomaly-detection-configuration/seasonality) -- **Data for calculating Quality Score** 📊 - - We made the following changes to help calculating data sets quality score (refer to [this great article](https://www.getdbt.com/blog/data-quality-dimensions/) for more info): - - We've added two new columns, elementary_test_results.failed_row_count and dbt_tests.quality_dimension. This data enables you to gain deeper insights into data quality and better track your data pipeline's health. - - We automatically fill these fields for common test (dbt, dbt_utils, and dbt_expectations), if you would like them to work for other test, or overwrite the default ones, you will need to fill the test's meta with: - - quality_dimension - Name of the quality dimension this test measures. - - failed_row_count_calc - An SQL expression to calculate the amount of failed rows from the test (count(\*), sum(n_records), etc). - - Thank you [Saurabh Jha](https://elementary-community.slack.com/team/U04BKHPH4HY) for working with us on this! - - This feature is still experimental, and is not reflected in the report nor the alerts. -- **Alert Suppression in CLI** 🚦 - - Now, you can suppress alerts in the Command Line Interface (CLI) using the --suppression-interval flag. - - This feature gives you more control over your alerts and helps you manage notifications effectively. - - Check out the [docs](https://docs.elementary-data.com) - -### 💫 More changes - -- **Order Slack messages by test run time** ⏰ - - We've added a new option to order Slack messages by test run time. This makes it easier to track test results and prioritize action items - Thanks [Willi Mueller](https://elementary-community.slack.com/team/U04NG6HNZC4) for suggesting! -- **Set SQL expression as timestamp_column** - - You can now set an SQL expression as the timestamp column, offering more flexibility in handling time-based data. Fantastic idea from [Sham Sreedharan](https://elementary-community.slack.com/team/U04RSS9U9TL), and solves use cases raised by [Shay Misgav](https://elementary-community.slack.com/team/U059UGE5HJ7), [Renee Cooper](https://elementary-community.slack.com/team/U057SD8CBEY), [Steve Luk](https://elementary-community.slack.com/team/U04E19YNSEB) and [FABIEN RYCKOORT](https://elementary-community.slack.com/team/U05BPE6DETX). -- **Support for dbt where Parameter** - - Thanks [Taha Bel Khayate](https://elementary-community.slack.com/team/U05EVCEM1EJ) for surfacing. - -### 🐛 Bugfix 🐛 - -- We've resolved a pesky bug that caused dbt_columns to malfunction in specific access configurations when using BigQuery. -- Table anomalies tests fix - Reported by [Scott Anderson](https://elementary-community.slack.com/team/U05654UL877) ! -- New dimension values will no linger cause dimension anomalies test to fail - Thanks [Jason Deng](https://elementary-community.slack.com/team/U055Q3Y3PEE) for reporting! diff --git a/docs/snippets/oss/release-notes-links.mdx b/docs/snippets/oss/release-notes-links.mdx new file mode 100644 index 000000000..a22a0e2f9 --- /dev/null +++ b/docs/snippets/oss/release-notes-links.mdx @@ -0,0 +1,12 @@ +Release notes for the Elementary dbt package and the Elementary CLI are published on GitHub. + + + + Versions of the `elementary-data/elementary` dbt package. + + + Versions of the `elementary-data` Python package, which includes the CLI. + + + +To upgrade, see [Upgrading Elementary](/oss/release-notes/upgrading-elementary).