Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
4bba66f
feat(skills): add Govern Python API guidance
pportal2 Sep 11, 2026
7953866
fix(skills): correct Govern guidance from live validation
pportal2 Sep 11, 2026
9bc058b
fix(skills): cover Govern 15 capabilities and governance design
pportal2 Sep 11, 2026
c2d5536
feat(govern): add a single govern tool over the Govern Python SDK
pportal2 Sep 15, 2026
03ae32f
fix(govern): match delegation and users-container contracts verified …
pportal2 Sep 15, 2026
4afd6ec
Merge origin/main into feat/govern-python-skill
pportal2 Oct 1, 2026
336c71c
Merge origin/main into feat/govern-tool
pportal2 Oct 1, 2026
8f53862
fix(govern): list the govern tool in capabilities and state why HTTP …
pportal2 Oct 1, 2026
a40c4d4
feat(govern): run HTTP-mode calls as the caller through a Govern admi…
pportal2 Oct 1, 2026
77c5d2a
revert(govern): keep the govern tool stdio-only for the proof of concept
pportal2 Oct 1, 2026
11b3066
fix(govern): return compact search hits, plain not-found errors, and …
pportal2 Oct 1, 2026
af892c5
fix(govern): add IDs only to metadata updates and name the HTML error…
pportal2 Oct 2, 2026
0aea975
fix(govern): read the Govern node only from the environment and fix r…
pportal2 Oct 2, 2026
c5b3960
Merge origin/main into feat/govern-tool
pportal2 Oct 5, 2026
6dc4b90
feat(govern): use the active govern instance instead of environment v…
pportal2 Oct 5, 2026
cb1fd87
feat(operations): add shared API tools and Deployer support
pmasiphelps Oct 30, 2026
e7ec4f4
feat(govern): integrate original Govern work with shared API operations
pmasiphelps Nov 3, 2026
b53d5e1
feat(agent-management): add estate operations and project support
pmasiphelps Oct 30, 2026
f6e22a8
fix(agent-management): correct bounds, risk joins and mutation guidance
crmapj Nov 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions CODING_STANDARDS_AND_STRUCTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,9 +110,13 @@ export DKU_API_KEY="your-api-key"
## Fixed Tool Surface

- `runtime/run_mcp.py` is the single launcher. It requires `--transport stdio` for the local plugin or `--transport http` for the authenticated Streamable HTTP deployment. Keep the transports explicitly selected and preserve the same registered tool catalog.
- Register one directly visible tool catalog. Do not add an MCP search mode.
- Classify explicit tools with `PROJECT_READ_POLICY`, `PROJECT_EXECUTION_POLICY`, `ADMIN_POLICY` or `PROJECT_EDIT_POLICY`; use `INSTANCE_MANAGEMENT_POLICY` for connection controls. Execution includes status/polling, administration includes instance inspection and project folders, and project edits include project Cobuild. Only connection management is exempt from requiring a selected profile.
- `SUPPORTED_DSS_INSTANCE_TYPES` defines the currently supported backend types independently of all valid `InstanceType` values. Agent Management, Govern and Deployer remain configurable profiles with instance controls only until their product support lands.
- Register one stable, directly visible MCP tool catalog. Do not change registration when instances switch. The bounded operation-registry pilot exposes fixed search/describe/read/write tools; operation discovery does not dynamically register MCP tools.
- Classify v1 tools with `PROJECT_READ_POLICY`, `PROJECT_EXECUTION_POLICY`, `ADMIN_POLICY` or `PROJECT_EDIT_POLICY`; use `API_OPERATIONS_POLICY` for shared operation tools and `INSTANCE_MANAGEMENT_POLICY` for connection controls. The categories describe purpose: execution includes status/polling, administration includes instance inspection and project folders, and project edits include project Cobuild.
- Only the `INSTANCE_MANAGEMENT_POLICY` group is exempt from requiring a selected profile. All v1 and v2 Dataiku tools, including operation search/describe, require one. Read/write execution handlers validate their selected operation directly.
- Every tool declares allowed_instance_types metadata. Middleware enforces it against the pinned request instance before backend token exchange. Operation handlers additionally enforce the selected operation’s instance policy, read/write effect, arguments and transport before invoking its SDK adapter; missing declarations fail closed.
- `SUPPORTED_DSS_INSTANCE_TYPES` identifies types served by the DSS client independently of project-tool policies; Govern uses its own SDK client.
- Product operations use public Python SDK methods only; do not call private transport helpers or add direct endpoint fallbacks. Govern execution is stdio-only; Agent Management uses the SDK’s native HTTP bearer support.
- Operation arguments use one Pydantic model for discovery and execution. Never accept caller-supplied HTTP methods or paths. Keep backend requirements and response projections explicit; catalog presence does not verify live server support. Preserve Cobuild routing for in-project asset construction.
- The registered set is the contract; `tests/test_tool_surface.py` pins the exact catalog. Any tool add/remove/rename updates that pinned set in the same change.

## Guardrails
Expand Down
26 changes: 20 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,19 @@ Dataiku Headless is an MCP server and agent skill library for operating Dataiku

Cobuild runs as a retained conversation through MCP tools. This repository intentionally keeps its own tool surface small: inspection tools, three deterministic executions of existing assets (`build_datasets`, `run_recipe`, and `run_scenario`), narrow configuration actions such as selecting which container an existing object runs in, and a few bootstrap actions that Cobuild cannot perform, such as creating a project or uploading a local file.

Project and API Deployer operations discover infrastructures, published versions and deployments,
create deployment configurations, change target versions, apply updates asynchronously,
and inspect health, governance codes and update history. They operate on already published
bundles and API services. See the [Deployer guide](skills/dataiku-headless/references/deployer.md)
for node selection, production effects and coverage limits.

Direct project reads, execution and administration support Design, Automation, Agent Management.
Project edits and Cobuild require Design and Agent Management. The four shared operation tools expose
145 public SDK actions for Deployer, Govern, Agent Management, restricted to their matching profiles.
Search and describe load schemas on demand; separate read/write tools execute them.
See the [operation guide](skills/dataiku-headless/references/operations.md).


## Capability reference

For a quick reference to what Headless can inspect, what Cobuild builds, and the
Expand All @@ -133,7 +146,7 @@ To build or update a Dataiku plugin, follow the

## MCP Server

`dataiku_mcp` is a FastMCP server that exposes Dataiku operations as typed, async MCP tools. Tools are organized by domain: projects, project folders, flow, connections, datasets, data quality, managed folders, recipes, machine learning, insights, dashboards, scenarios, WebApps, wikis, agents, LLMs and knowledge banks, instance plugins, job management, administrative tasks, and Cobuild conversations.
`dataiku_mcp` is a FastMCP server that exposes Dataiku operations as typed, async MCP tools. Tools are organized by domain: projects, project folders, flow, connections, datasets, data quality, managed folders, recipes, machine learning, insights, dashboards, scenarios, WebApps, wikis, agents, LLMs and knowledge banks, instance plugins, Project and API Deployer, job management, administrative tasks, and Cobuild conversations.

- Async execution for all Dataiku API calls
- Progress notifications for long-running operations
Expand Down Expand Up @@ -180,11 +193,8 @@ Each profile requires `instance_type`: `design`, `automation`, `deployer`, `gove
`agent-management`. Existing stdio profiles missing this field are automatically
assigned `design` and rewritten on disk before loading. This temporary migration
is scheduled for deprecation by 0.9.0; existing values are preserved and validated.
The configured type determines which tools can run; see the
[instance-type policy](docs/capabilities.md#instance-types). Agent Management, Govern and Deployer
currently support instance controls only. Their status is `unsupported`, without
a DSS connection check. The setup page allows these profiles to be saved without testing.
Design and Automation require a successful connection test.
The declared type selects tool policy; backend features and permissions are validated by Dataiku. A local Deployer on a Design or Automation node requires a separate `deployer` profile for that URL.
Design, Automation and Deployer and Agent Management use the DSS client. Govern uses its own client in stdio and reports unsupported_authentication over HTTP.

**Environment override.** To select an explicit target, including inside a Code
Studio, set these three variables in your environment or copy
Expand Down Expand Up @@ -258,6 +268,10 @@ The Dataiku Headless marketplace plugin uses stdio transport. For customer-manag
│ │ ├── data_quality.py # Dataset Data Quality rule inspection tools
│ │ ├── dashboards.py # Dashboard inspection tools
│ │ ├── datasets.py # Dataset inspection tools + local-file upload writes
│ │ ├── deployer.py # Internal Deployer operation adapters
│ │ ├── agent_management.py # Fixed Agent Management operation adapters
│ │ ├── govern.py # Govern SDK operation adapters
│ │ ├── operations.py # Shared discovery/read/write MCP tools
│ │ ├── evaluation_stores.py # Evaluation Store inspection tools
│ │ ├── flow.py # Flow inspection tools
│ │ ├── instances.py # Multi-instance switching tools
Expand Down
1 change: 1 addition & 0 deletions dataiku_mcp/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
licensing,
llms_and_knowledge_banks,
managed_folders,
operations,
plugins,
project_folders,
project_libraries,
Expand Down
25 changes: 25 additions & 0 deletions dataiku_mcp/auth.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,31 @@ def get_dss_client() -> dataikuapi.DSSClient:
return client


def get_govern_client() -> dataikuapi.GovernClient:
"""Construct a Govern client with the pinned profile’s request credentials."""
instance = request.get_pinned_instance()
if instance.instance_type != "govern":
raise ValueError(
"Select an instance of type govern before using Govern operations."
)
if request.is_http_request():
raise ValueError(
"Govern operations require stdio; HTTP authentication is unsupported."
)
client = dataikuapi.GovernClient(
instance.url,
api_key=instance.api_key,
internal_ticket=instance.api_ticket,
extra_headers={"X-DKU-Client-Application": "dataiku-headless"},
)
client._session.verify = (
False
if instance.no_check_certificate
else instance.encrypted_rpc_cert_path or True
)
return client


def get_dataiku_version(client: dataikuapi.DSSClient) -> str:
"""Return the instance's Dataiku version, or an empty string if unavailable."""
try:
Expand Down
22 changes: 15 additions & 7 deletions dataiku_mcp/instance_policy.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,17 +18,25 @@

from .config.models import InstanceType

# Product-node support is added separately from this Design/Automation foundation.
SUPPORTED_DSS_INSTANCE_TYPES = frozenset({"design", "automation"})
# Types served by DSSClient; Govern uses its own SDK client.
SUPPORTED_DSS_INSTANCE_TYPES = frozenset(
{"design", "automation", "agent-management", "deployer"}
)

# These are @mcp.tool metadata policies. V1 tools expose individual actions;
# project authoring still routes through Cobuild and its fixed direct exceptions.
PROJECT_READ_POLICY = {"allowed_instance_types": sorted(SUPPORTED_DSS_INSTANCE_TYPES)}
# v2 tools discover and execute a fixed SDK operation catalog. Project authoring
# still routes through Cobuild and its fixed direct exceptions.
PROJECT_READ_POLICY = {
"allowed_instance_types": ["design", "agent-management", "automation"]
}
PROJECT_EXECUTION_POLICY = {
"allowed_instance_types": sorted(SUPPORTED_DSS_INSTANCE_TYPES)
"allowed_instance_types": ["design", "agent-management", "automation"]
}
ADMIN_POLICY = {"allowed_instance_types": ["design", "agent-management", "automation"]}
PROJECT_EDIT_POLICY = {"allowed_instance_types": ["design", "agent-management"]}
API_OPERATIONS_POLICY = {
"allowed_instance_types": ["agent-management", "deployer", "govern"]
}
ADMIN_POLICY = {"allowed_instance_types": sorted(SUPPORTED_DSS_INSTANCE_TYPES)}
PROJECT_EDIT_POLICY = {"allowed_instance_types": ["design"]}
INSTANCE_MANAGEMENT_POLICY = {"allowed_instance_types": list(get_args(InstanceType))}


Expand Down
166 changes: 166 additions & 0 deletions dataiku_mcp/operation_registry.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
# Copyright 2026 Dataiku SAS
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""One schema and policy contract for discovery, middleware and dispatch."""

import inspect
from dataclasses import dataclass
from types import MappingProxyType
from typing import Awaitable, Callable, Literal, get_type_hints

from pydantic import BaseModel, ConfigDict, ValidationError, create_model

from .config.models import InstanceType
from .tools.utils.redaction import contains_redacted_values

Effect = Literal["read", "write"]


class OperationArguments(BaseModel):
model_config = ConfigDict(
extra="forbid", strict=True, hide_input_in_errors=True, allow_inf_nan=False
)


@dataclass(frozen=True)
class Operation:
operation_id: str
domain: str
description: str
effect: Effect
allowed_instance_types: tuple[InstanceType, ...]
arguments: type[BaseModel]
handler: Callable[..., Awaitable[str]]
requirements: str
response_contract: str
example: dict
destructive: bool = False
idempotent: bool = False
category: str = ""
backend: Literal["dss", "govern"] = "dss"
transports: tuple[str, ...] = ("stdio", "http")
local_file_fields: tuple[str, ...] = ()

def validate(self, arguments: dict) -> dict:
if self.effect == "write" and contains_redacted_values(arguments):
raise ValueError(
"Do not submit redacted result placeholders as write values. Use a scoped update that preserves hidden fields, or report the coverage gap."
)
try:
value = self.arguments.model_validate(arguments)
except ValidationError as exc:
# Pydantic errors otherwise echo potentially sensitive input payloads.
problems = [
".".join(map(str, item["loc"])) + ": " + item["type"]
for item in exc.errors(include_input=False)
]
raise ValueError(
"Invalid operation arguments: " + "; ".join(problems)
) from None
# Keep nested models intact for typed handlers and retain default values.
return {name: getattr(value, name) for name in value.model_fields_set}

def summary(self) -> dict:
return {
"operation_id": self.operation_id,
"domain": self.domain,
"category": self.category,
"description": self.description,
"effect": self.effect,
"allowed_instance_types": list(self.allowed_instance_types),
"destructive": self.destructive,
"idempotent": self.idempotent,
"transports": list(self.transports),
}

def describe(self) -> dict:
return {
**self.summary(),
"input_schema": self.arguments.model_json_schema(),
"example_arguments": self.example,
"requirements": self.requirements,
"server_support": "unknown",
"response_contract": self.response_contract,
"transports": list(self.transports),
"backend": self.backend,
"local_file_fields": list(self.local_file_fields),
}


def arguments_for(
handler: Callable, *, exclude: tuple[str, ...] = ()
) -> type[BaseModel]:
hints = get_type_hints(handler, include_extras=True)
return create_model(
handler.__name__ + "Arguments",
__base__=OperationArguments,
**{
name: (
hints[name],
parameter.default
if parameter.default is not inspect.Parameter.empty
else ...,
)
for name, parameter in inspect.signature(handler).parameters.items()
if name not in exclude
},
)


def get_operations():
# Lazy import keeps server construction independent of tool registration.
from .operations_catalog import OPERATIONS

return OPERATIONS


def get_operation(operation_id: str) -> Operation:
try:
return get_operations()[operation_id]
except (KeyError, TypeError):
raise ValueError("Unknown operation_id. Use search_operations.") from None


def require_operation(
operation_id: str, effect: Effect, instance_type: InstanceType
) -> Operation:
operation = get_operation(operation_id)
if operation.effect != effect:
raise ValueError(
f"Operation '{operation_id}' requires the {operation.effect} dispatcher."
)
if instance_type not in operation.allowed_instance_types:
raise ValueError(
f"Operation '{operation_id}' is unavailable for instance type '{instance_type}'. Use switch_instance."
)
return operation


def make_catalog(operations: list[Operation]):
result = {operation.operation_id: operation for operation in operations}
if len(result) != len(operations):
raise ValueError("Duplicate operation_id")
return MappingProxyType(result)


def require_transport(operation: Operation, arguments: dict, is_http: bool) -> None:
transport = "http" if is_http else "stdio"
if transport not in operation.transports:
raise ValueError(
f"Operation '{operation.operation_id}' requires local stdio authentication; HTTP delegated tokens are not supported."
)
if is_http and any(arguments.get(field) for field in operation.local_file_fields):
raise ValueError(
"Local-file upload/download is unavailable in HTTP mode because it accesses the MCP host filesystem. Use stdio."
)
Loading
Loading