diff --git a/CODING_STANDARDS_AND_STRUCTURE.md b/CODING_STANDARDS_AND_STRUCTURE.md index e7c79a69..68e71b60 100644 --- a/CODING_STANDARDS_AND_STRUCTURE.md +++ b/CODING_STANDARDS_AND_STRUCTURE.md @@ -110,9 +110,13 @@ export DKU_API_KEY="your-api-key" ## Fixed Tool Surface - `runtime/run_mcp.py` is the single launcher. It requires `--transport stdio` for the local plugin or `--transport http` for the authenticated Streamable HTTP deployment. Keep the transports explicitly selected and preserve the same registered tool catalog. -- Register one directly visible tool catalog. Do not add an MCP search mode. -- Classify explicit tools with `PROJECT_READ_POLICY`, `PROJECT_EXECUTION_POLICY`, `ADMIN_POLICY` or `PROJECT_EDIT_POLICY`; use `INSTANCE_MANAGEMENT_POLICY` for connection controls. Execution includes status/polling, administration includes instance inspection and project folders, and project edits include project Cobuild. Only connection management is exempt from requiring a selected profile. -- `SUPPORTED_DSS_INSTANCE_TYPES` defines the currently supported backend types independently of all valid `InstanceType` values. Agent Management, Govern and Deployer remain configurable profiles with instance controls only until their product support lands. +- Register one stable, directly visible MCP tool catalog. Do not change registration when instances switch. The bounded operation-registry pilot exposes fixed search/describe/read/write tools; operation discovery does not dynamically register MCP tools. +- Classify v1 tools with `PROJECT_READ_POLICY`, `PROJECT_EXECUTION_POLICY`, `ADMIN_POLICY` or `PROJECT_EDIT_POLICY`; use `API_OPERATIONS_POLICY` for shared operation tools and `INSTANCE_MANAGEMENT_POLICY` for connection controls. The categories describe purpose: execution includes status/polling, administration includes instance inspection and project folders, and project edits include project Cobuild. +- Only the `INSTANCE_MANAGEMENT_POLICY` group is exempt from requiring a selected profile. All v1 and v2 Dataiku tools, including operation search/describe, require one. Read/write execution handlers validate their selected operation directly. +- Every tool declares allowed_instance_types metadata. Middleware enforces it against the pinned request instance before backend token exchange. Operation handlers additionally enforce the selected operation’s instance policy, read/write effect, arguments and transport before invoking its SDK adapter; missing declarations fail closed. +- `SUPPORTED_DSS_INSTANCE_TYPES` identifies types served by the DSS client independently of project-tool policies; Govern uses its own SDK client. +- Product operations use public Python SDK methods only; do not call private transport helpers or add direct endpoint fallbacks. Govern execution is stdio-only; Agent Management uses the SDK’s native HTTP bearer support. +- Operation arguments use one Pydantic model for discovery and execution. Never accept caller-supplied HTTP methods or paths. Keep backend requirements and response projections explicit; catalog presence does not verify live server support. Preserve Cobuild routing for in-project asset construction. - The registered set is the contract; `tests/test_tool_surface.py` pins the exact catalog. Any tool add/remove/rename updates that pinned set in the same change. ## Guardrails diff --git a/README.md b/README.md index f1c84c3d..19a3d9b6 100644 --- a/README.md +++ b/README.md @@ -122,6 +122,19 @@ Dataiku Headless is an MCP server and agent skill library for operating Dataiku Cobuild runs as a retained conversation through MCP tools. This repository intentionally keeps its own tool surface small: inspection tools, three deterministic executions of existing assets (`build_datasets`, `run_recipe`, and `run_scenario`), narrow configuration actions such as selecting which container an existing object runs in, and a few bootstrap actions that Cobuild cannot perform, such as creating a project or uploading a local file. +Project and API Deployer operations discover infrastructures, published versions and deployments, +create deployment configurations, change target versions, apply updates asynchronously, +and inspect health, governance codes and update history. They operate on already published +bundles and API services. See the [Deployer guide](skills/dataiku-headless/references/deployer.md) +for node selection, production effects and coverage limits. + +Direct project reads, execution and administration support Design, Automation, Agent Management. +Project edits and Cobuild require Design and Agent Management. The four shared operation tools expose +145 public SDK actions for Deployer, Govern, Agent Management, restricted to their matching profiles. +Search and describe load schemas on demand; separate read/write tools execute them. +See the [operation guide](skills/dataiku-headless/references/operations.md). + + ## Capability reference For a quick reference to what Headless can inspect, what Cobuild builds, and the @@ -133,7 +146,7 @@ To build or update a Dataiku plugin, follow the ## MCP Server -`dataiku_mcp` is a FastMCP server that exposes Dataiku operations as typed, async MCP tools. Tools are organized by domain: projects, project folders, flow, connections, datasets, data quality, managed folders, recipes, machine learning, insights, dashboards, scenarios, WebApps, wikis, agents, LLMs and knowledge banks, instance plugins, job management, administrative tasks, and Cobuild conversations. +`dataiku_mcp` is a FastMCP server that exposes Dataiku operations as typed, async MCP tools. Tools are organized by domain: projects, project folders, flow, connections, datasets, data quality, managed folders, recipes, machine learning, insights, dashboards, scenarios, WebApps, wikis, agents, LLMs and knowledge banks, instance plugins, Project and API Deployer, job management, administrative tasks, and Cobuild conversations. - Async execution for all Dataiku API calls - Progress notifications for long-running operations @@ -180,11 +193,8 @@ Each profile requires `instance_type`: `design`, `automation`, `deployer`, `gove `agent-management`. Existing stdio profiles missing this field are automatically assigned `design` and rewritten on disk before loading. This temporary migration is scheduled for deprecation by 0.9.0; existing values are preserved and validated. -The configured type determines which tools can run; see the -[instance-type policy](docs/capabilities.md#instance-types). Agent Management, Govern and Deployer -currently support instance controls only. Their status is `unsupported`, without -a DSS connection check. The setup page allows these profiles to be saved without testing. -Design and Automation require a successful connection test. +The declared type selects tool policy; backend features and permissions are validated by Dataiku. A local Deployer on a Design or Automation node requires a separate `deployer` profile for that URL. +Design, Automation and Deployer and Agent Management use the DSS client. Govern uses its own client in stdio and reports unsupported_authentication over HTTP. **Environment override.** To select an explicit target, including inside a Code Studio, set these three variables in your environment or copy @@ -258,6 +268,10 @@ The Dataiku Headless marketplace plugin uses stdio transport. For customer-manag │ │ ├── data_quality.py # Dataset Data Quality rule inspection tools │ │ ├── dashboards.py # Dashboard inspection tools │ │ ├── datasets.py # Dataset inspection tools + local-file upload writes +│ │ ├── deployer.py # Internal Deployer operation adapters +│ │ ├── agent_management.py # Fixed Agent Management operation adapters +│ │ ├── govern.py # Govern SDK operation adapters +│ │ ├── operations.py # Shared discovery/read/write MCP tools │ │ ├── evaluation_stores.py # Evaluation Store inspection tools │ │ ├── flow.py # Flow inspection tools │ │ ├── instances.py # Multi-instance switching tools diff --git a/dataiku_mcp/__init__.py b/dataiku_mcp/__init__.py index bd19b45b..9ecf0b30 100644 --- a/dataiku_mcp/__init__.py +++ b/dataiku_mcp/__init__.py @@ -42,6 +42,7 @@ licensing, llms_and_knowledge_banks, managed_folders, + operations, plugins, project_folders, project_libraries, diff --git a/dataiku_mcp/auth.py b/dataiku_mcp/auth.py index 58419cb8..e1739b1c 100644 --- a/dataiku_mcp/auth.py +++ b/dataiku_mcp/auth.py @@ -57,6 +57,31 @@ def get_dss_client() -> dataikuapi.DSSClient: return client +def get_govern_client() -> dataikuapi.GovernClient: + """Construct a Govern client with the pinned profile’s request credentials.""" + instance = request.get_pinned_instance() + if instance.instance_type != "govern": + raise ValueError( + "Select an instance of type govern before using Govern operations." + ) + if request.is_http_request(): + raise ValueError( + "Govern operations require stdio; HTTP authentication is unsupported." + ) + client = dataikuapi.GovernClient( + instance.url, + api_key=instance.api_key, + internal_ticket=instance.api_ticket, + extra_headers={"X-DKU-Client-Application": "dataiku-headless"}, + ) + client._session.verify = ( + False + if instance.no_check_certificate + else instance.encrypted_rpc_cert_path or True + ) + return client + + def get_dataiku_version(client: dataikuapi.DSSClient) -> str: """Return the instance's Dataiku version, or an empty string if unavailable.""" try: diff --git a/dataiku_mcp/instance_policy.py b/dataiku_mcp/instance_policy.py index 2e77f5e0..b969d743 100644 --- a/dataiku_mcp/instance_policy.py +++ b/dataiku_mcp/instance_policy.py @@ -18,17 +18,25 @@ from .config.models import InstanceType -# Product-node support is added separately from this Design/Automation foundation. -SUPPORTED_DSS_INSTANCE_TYPES = frozenset({"design", "automation"}) +# Types served by DSSClient; Govern uses its own SDK client. +SUPPORTED_DSS_INSTANCE_TYPES = frozenset( + {"design", "automation", "agent-management", "deployer"} +) # These are @mcp.tool metadata policies. V1 tools expose individual actions; -# project authoring still routes through Cobuild and its fixed direct exceptions. -PROJECT_READ_POLICY = {"allowed_instance_types": sorted(SUPPORTED_DSS_INSTANCE_TYPES)} +# v2 tools discover and execute a fixed SDK operation catalog. Project authoring +# still routes through Cobuild and its fixed direct exceptions. +PROJECT_READ_POLICY = { + "allowed_instance_types": ["design", "agent-management", "automation"] +} PROJECT_EXECUTION_POLICY = { - "allowed_instance_types": sorted(SUPPORTED_DSS_INSTANCE_TYPES) + "allowed_instance_types": ["design", "agent-management", "automation"] +} +ADMIN_POLICY = {"allowed_instance_types": ["design", "agent-management", "automation"]} +PROJECT_EDIT_POLICY = {"allowed_instance_types": ["design", "agent-management"]} +API_OPERATIONS_POLICY = { + "allowed_instance_types": ["agent-management", "deployer", "govern"] } -ADMIN_POLICY = {"allowed_instance_types": sorted(SUPPORTED_DSS_INSTANCE_TYPES)} -PROJECT_EDIT_POLICY = {"allowed_instance_types": ["design"]} INSTANCE_MANAGEMENT_POLICY = {"allowed_instance_types": list(get_args(InstanceType))} diff --git a/dataiku_mcp/operation_registry.py b/dataiku_mcp/operation_registry.py new file mode 100644 index 00000000..cd58c0ee --- /dev/null +++ b/dataiku_mcp/operation_registry.py @@ -0,0 +1,166 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""One schema and policy contract for discovery, middleware and dispatch.""" + +import inspect +from dataclasses import dataclass +from types import MappingProxyType +from typing import Awaitable, Callable, Literal, get_type_hints + +from pydantic import BaseModel, ConfigDict, ValidationError, create_model + +from .config.models import InstanceType +from .tools.utils.redaction import contains_redacted_values + +Effect = Literal["read", "write"] + + +class OperationArguments(BaseModel): + model_config = ConfigDict( + extra="forbid", strict=True, hide_input_in_errors=True, allow_inf_nan=False + ) + + +@dataclass(frozen=True) +class Operation: + operation_id: str + domain: str + description: str + effect: Effect + allowed_instance_types: tuple[InstanceType, ...] + arguments: type[BaseModel] + handler: Callable[..., Awaitable[str]] + requirements: str + response_contract: str + example: dict + destructive: bool = False + idempotent: bool = False + category: str = "" + backend: Literal["dss", "govern"] = "dss" + transports: tuple[str, ...] = ("stdio", "http") + local_file_fields: tuple[str, ...] = () + + def validate(self, arguments: dict) -> dict: + if self.effect == "write" and contains_redacted_values(arguments): + raise ValueError( + "Do not submit redacted result placeholders as write values. Use a scoped update that preserves hidden fields, or report the coverage gap." + ) + try: + value = self.arguments.model_validate(arguments) + except ValidationError as exc: + # Pydantic errors otherwise echo potentially sensitive input payloads. + problems = [ + ".".join(map(str, item["loc"])) + ": " + item["type"] + for item in exc.errors(include_input=False) + ] + raise ValueError( + "Invalid operation arguments: " + "; ".join(problems) + ) from None + # Keep nested models intact for typed handlers and retain default values. + return {name: getattr(value, name) for name in value.model_fields_set} + + def summary(self) -> dict: + return { + "operation_id": self.operation_id, + "domain": self.domain, + "category": self.category, + "description": self.description, + "effect": self.effect, + "allowed_instance_types": list(self.allowed_instance_types), + "destructive": self.destructive, + "idempotent": self.idempotent, + "transports": list(self.transports), + } + + def describe(self) -> dict: + return { + **self.summary(), + "input_schema": self.arguments.model_json_schema(), + "example_arguments": self.example, + "requirements": self.requirements, + "server_support": "unknown", + "response_contract": self.response_contract, + "transports": list(self.transports), + "backend": self.backend, + "local_file_fields": list(self.local_file_fields), + } + + +def arguments_for( + handler: Callable, *, exclude: tuple[str, ...] = () +) -> type[BaseModel]: + hints = get_type_hints(handler, include_extras=True) + return create_model( + handler.__name__ + "Arguments", + __base__=OperationArguments, + **{ + name: ( + hints[name], + parameter.default + if parameter.default is not inspect.Parameter.empty + else ..., + ) + for name, parameter in inspect.signature(handler).parameters.items() + if name not in exclude + }, + ) + + +def get_operations(): + # Lazy import keeps server construction independent of tool registration. + from .operations_catalog import OPERATIONS + + return OPERATIONS + + +def get_operation(operation_id: str) -> Operation: + try: + return get_operations()[operation_id] + except (KeyError, TypeError): + raise ValueError("Unknown operation_id. Use search_operations.") from None + + +def require_operation( + operation_id: str, effect: Effect, instance_type: InstanceType +) -> Operation: + operation = get_operation(operation_id) + if operation.effect != effect: + raise ValueError( + f"Operation '{operation_id}' requires the {operation.effect} dispatcher." + ) + if instance_type not in operation.allowed_instance_types: + raise ValueError( + f"Operation '{operation_id}' is unavailable for instance type '{instance_type}'. Use switch_instance." + ) + return operation + + +def make_catalog(operations: list[Operation]): + result = {operation.operation_id: operation for operation in operations} + if len(result) != len(operations): + raise ValueError("Duplicate operation_id") + return MappingProxyType(result) + + +def require_transport(operation: Operation, arguments: dict, is_http: bool) -> None: + transport = "http" if is_http else "stdio" + if transport not in operation.transports: + raise ValueError( + f"Operation '{operation.operation_id}' requires local stdio authentication; HTTP delegated tokens are not supported." + ) + if is_http and any(arguments.get(field) for field in operation.local_file_fields): + raise ValueError( + "Local-file upload/download is unavailable in HTTP mode because it accesses the MCP host filesystem. Use stdio." + ) diff --git a/dataiku_mcp/operation_schemas.py b/dataiku_mcp/operation_schemas.py new file mode 100644 index 00000000..e12f8ce6 --- /dev/null +++ b/dataiku_mcp/operation_schemas.py @@ -0,0 +1,98 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Structural schemas for Govern operation adapters.""" + +from typing import Annotated, Any, Literal + +from pydantic import Field, create_model, model_validator + +from .operation_registry import OperationArguments +from .tools.utils.validation import PathSegment + +NonEmpty = Annotated[str, Field(min_length=1)] + + +class UserContainer(OperationArguments): + type: Literal["user"] + login: PathSegment + + +class DeleteTimeSeriesArguments(OperationArguments): + time_series_id: PathSegment + min_timestamp: int | None = None + max_timestamp: int | None = None + delete_all: bool = False + + @model_validator(mode="after") + def explicit_range(self): + unbounded = self.min_timestamp is None and self.max_timestamp is None + if unbounded != self.delete_all: + raise ValueError("Use a timestamp range or delete_all=true, not both") + if ( + self.min_timestamp is not None + and self.max_timestamp is not None + and self.min_timestamp > self.max_timestamp + ): + raise ValueError("min_timestamp must not exceed max_timestamp") + return self + + +def govern_arguments(name, spec): + if name == "delete_time_series_values": + return DeleteTimeSeriesArguments + kinds = { + "string": str, + "boolean": bool, + "integer": int, + "object": dict[str, Any], + "list": list[Any], + "any": Any, + } + fields = {} + for param in spec.params: + kind = PathSegment if param.path else kinds[param.kind] + if param.kind == "string" and param.required and not param.path: + kind = NonEmpty + if param.name == "users_container": + kind = UserContainer + if param.name == "status": + if name == "update_signoff_status": + kind = Literal[ + "NOT_STARTED", + "WAITING_FOR_FEEDBACK", + "WAITING_FOR_APPROVAL", + "APPROVED", + "REJECTED", + "ABANDONED", + ] + elif name in {"add_signoff_feedback", "update_signoff_feedback"}: + kind = Literal["APPROVED", "MINOR_ISSUE", "MAJOR_ISSUE"] + elif name in {"add_signoff_approval", "update_signoff_approval"}: + kind = Literal["APPROVED", "REJECTED", "ABANDONED"] + if not param.required: + kind = kind | None + description = param.description + if param.kind in {"object", "list"}: + description += " Backend-defined contents; inspect current definitions before mutation." + constraints = {"ge": 1} if param.name in {"page_size", "max_results"} else {} + fields[param.name] = ( + kind, + Field( + default=... if param.required else None, + description=description, + **constraints, + ), + ) + return create_model(name + "Arguments", __base__=OperationArguments, **fields) diff --git a/dataiku_mcp/operations_catalog.py b/dataiku_mcp/operations_catalog.py new file mode 100644 index 00000000..e46ad6e1 --- /dev/null +++ b/dataiku_mcp/operations_catalog.py @@ -0,0 +1,183 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Audited operation declarations for Deployer, Govern and Agent Management.""" + +from .operation_registry import Operation, arguments_for, make_catalog +from .tools import deployer, govern, agent_management +from .operation_schemas import govern_arguments + + +def _deployer( + name: str, example: dict, *, effect="read", destructive=False, idempotent=False +): + handler = getattr(deployer, name) + return Operation( + operation_id="deployer." + name.replace("_deployer", ""), + domain="deployer", + description=handler.__doc__, + effect=effect, + allowed_instance_types=("deployer",), + arguments=arguments_for(handler), + handler=handler, + requirements="Selected node must expose Project/API Deployer. The configured type does not verify feature availability or permissions.", + response_contract="Selected operational fields; credentials, raw settings, free-text diagnostics and update logs are omitted. Configuration saves do not apply deployments; start_update returns a future_id for deployer.get_future_status on the same instance.", + example=example, + destructive=destructive, + idempotent=idempotent, + ) + + +_DEPLOYER_OPERATIONS = list( + [ + _deployer("list_deployer_stages", {"deployer_type": "project"}), + _deployer("list_deployer_infrastructures", {"deployer_type": "project"}), + _deployer( + "get_deployer_infrastructure", + {"deployer_type": "project", "infra_id": "discovered-infra"}, + ), + _deployer("list_deployer_published_items", {"deployer_type": "api"}), + _deployer( + "get_deployer_published_item", + {"deployer_type": "api", "source_id": "discovered-service"}, + ), + _deployer("list_deployer_deployments", {"deployer_type": "project"}), + _deployer( + "get_deployer_deployment", + {"deployer_type": "project", "deployment_id": "discovered-deployment"}, + ), + _deployer( + "list_deployer_updates", + {"deployer_type": "project", "deployment_id": "discovered-deployment"}, + ), + _deployer( + "get_deployer_update", + {"deployer_type": "project", "deployment_id": "discovered-deployment"}, + ), + _deployer( + "create_deployer_deployment", + { + "deployer_type": "project", + "deployment_id": "requested-deployment", + "infra_id": "discovered-infra", + "source_id": "discovered-project", + "version": "discovered-bundle", + }, + effect="write", + ), + _deployer( + "update_deployer_deployment", + { + "deployer_type": "api", + "deployment_id": "discovered-deployment", + "version": "discovered-version", + }, + effect="write", + destructive=True, + idempotent=True, + ), + _deployer( + "start_deployer_update", + {"deployer_type": "project", "deployment_id": "discovered-deployment"}, + effect="write", + destructive=True, + ), + _deployer("get_deployer_future_status", {"future_id": "returned-future"}), + ] +) + + +def _agent_management_operation(name, spec): + arguments = ( + agent_management.AlertEvaluationArguments + if name == "evaluate_alerts" + else arguments_for(spec.handler, exclude=("client",)) + ) + return Operation( + operation_id="agent-management." + name, + domain="agent-management", + category=spec.domain, + description=spec.handler.__doc__, + effect=spec.effect, + allowed_instance_types=("agent-management",), + arguments=arguments, + handler=agent_management.operation_handler(spec), + requirements="Requires dataiku-api-client 15.2.0 or newer and a compatible Agent Management release. Uses public SDK methods only. Nested settings are backend-defined. Catalog presence does not verify live permissions or server features.", + response_contract="Public SDK JSON with credentials recursively redacted. Save inspected fetchedRevision as expected_revision for writes. Async starts return platform_id, agent_id and operation_id for polling without waiting. No automatic retries. Alert evaluation returns queued_agent_count, not newly created alerts. Local-file transfers require stdio.", + example=spec.example, + destructive=spec.destructive, + transports=("stdio",) + if "output_path" in spec.local_file_fields + else ("stdio", "http"), + local_file_fields=spec.local_file_fields, + ) + + +def _govern_operation(name, spec): + arguments = govern_arguments(name, spec) + example = {} + for param in spec.params: + if param.required: + example[param.name] = { + "string": "discovered-" + param.name.replace("_", "-"), + "object": {}, + "list": [], + "boolean": False, + "integer": 1, + "any": "value", + }[param.kind] + if name == "delete_time_series_values": + example["delete_all"] = True + if "status" in example and name in { + "update_signoff_status", + "add_signoff_feedback", + "update_signoff_feedback", + "add_signoff_approval", + "update_signoff_approval", + }: + example["status"] = "APPROVED" + if "users_container" in example: + example["users_container"] = {"type": "user", "login": "discovered-user"} + return Operation( + operation_id="govern." + name, + domain="govern", + category=spec.domain, + description=spec.description, + effect="read" if spec.kind == "read" else "write", + allowed_instance_types=("govern",), + arguments=arguments, + handler=govern.operation_handler(spec), + backend="govern", + transports=("stdio",), + requirements="Requires a compatible Govern release and a configured Govern profile with stdio credentials. Govern HTTP authentication is unsupported. Blueprint-specific definitions remain backend-defined; inspect current definitions before replacing them.", + response_contract="Public Govern SDK JSON with credential fields recursively redacted. Write and delete actions operate on Govern records, not DSS projects. Definition replacements require preserving unrelated fields; inspect and re-read after mutations. No retry on uncertain outcome.", + example=example, + destructive=spec.kind != "read" and not name.startswith("create_"), + local_file_fields=("file_path", "output_path"), + ) + + +OPERATIONS = make_catalog( + [ + *_DEPLOYER_OPERATIONS, + *[ + _agent_management_operation(name, spec) + for name, spec in agent_management.AGENT_MANAGEMENT_OPERATIONS.items() + ], + *[ + _govern_operation(name, spec) + for name, spec in govern.GOVERN_OPERATIONS.items() + ], + ] +) diff --git a/dataiku_mcp/server.py b/dataiku_mcp/server.py index eb8500d2..503d2f70 100644 --- a/dataiku_mcp/server.py +++ b/dataiku_mcp/server.py @@ -128,6 +128,10 @@ async def on_call_tool( instance.instance_type in SUPPORTED_DSS_INSTANCE_TYPES ) if access_token is not None and requires_dss_token: + if instance.instance_type == "govern": + raise ValueError( + "Govern operations require stdio; HTTP authentication is unsupported." + ) dss_token = await exchange_http_token(access_token.token) dss_token_reset_token = request.bind_http_dss_token(dss_token) return await call_next(context) diff --git a/dataiku_mcp/setup_server.py b/dataiku_mcp/setup_server.py index eead4654..ce36a5f9 100644 --- a/dataiku_mcp/setup_server.py +++ b/dataiku_mcp/setup_server.py @@ -26,7 +26,6 @@ import dataikuapi from .config import request, stdio -from .instance_policy import SUPPORTED_DSS_INSTANCE_TYPES from .config.models import InstanceType SESSION_LIFETIME_SECONDS = 10 * 60 @@ -113,6 +112,7 @@ def _page( error: str = "", status: str = "", values: dict | None = None, + connection_validated: bool = False, ) -> str: error_markup = ( f'' if error else "" @@ -135,6 +135,7 @@ def _page( api_key = escape(values.get("api_key", "")) set_default = " checked" if is_initial_page or values.get("set_default") else "" no_check_certificate = " checked" if values.get("no_check_certificate") else "" + save_disabled = "" if connection_validated else " disabled" return f""" @@ -196,7 +197,7 @@ def _page(
A short name used when switching instances.
-
Agent Management, Govern and Deployer profiles can be saved without testing. Their API tools are not available yet.
+
Enter the URL of your Dataiku instance.
Create one in Dataiku under Profile & Settings → API keys.
@@ -205,7 +206,7 @@ def _page(
Advanced options
{status_markup} -
+
@@ -245,12 +246,13 @@ def _success_page(name: str) -> str: def _test_connection(values: dict) -> None: - if values["instance_type"] not in SUPPORTED_DSS_INSTANCE_TYPES: - raise ValueError( - f"Connection testing is unavailable for {values['instance_type']}. Save the profile without testing." - ) try: - client = dataikuapi.DSSClient(values["url"], values["api_key"]) + factory = ( + dataikuapi.GovernClient + if values["instance_type"] == "govern" + else dataikuapi.DSSClient + ) + client = factory(values["url"], values["api_key"]) client._session.verify = not values["no_check_certificate"] client.get_auth_info() except Exception as exc: @@ -328,12 +330,13 @@ def do_POST(self) -> None: _page( status="Connection successful. You can now save this instance.", values=values, + connection_validated=True, ), ) return if action != "save": raise ValueError("Unknown setup action.") - if values["instance_type"] in SUPPORTED_DSS_INSTANCE_TYPES and ( + if ( not session_state or session_state.validated_connection != _connection_settings(values) diff --git a/dataiku_mcp/tools/agent_management.py b/dataiku_mcp/tools/agent_management.py new file mode 100644 index 00000000..152e7e63 --- /dev/null +++ b/dataiku_mcp/tools/agent_management.py @@ -0,0 +1,955 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Agent Management adapters using only public dataiku-api-client methods.""" + +from __future__ import annotations + +import hashlib +import os +import tempfile +from contextlib import ExitStack +from dataclasses import dataclass +from datetime import datetime, timezone +from typing import Annotated, Any, Callable, Literal + +from dataikuapi.agentmanagement.agent import AgentFilter +from dataikuapi.utils import DataikuException +from pydantic import Field, StringConstraints, model_validator +from requests import RequestException + +from ..auth import get_dss_client +from ..executors import run_blocking +from ..operation_registry import OperationArguments +from .utils.serialization import compact_json +from .utils.redaction import redact_sensitive_data +from .utils.validation import PathSegment + +NonEmpty = Annotated[str, StringConstraints(strip_whitespace=True, min_length=1)] +PositiveLimit = Annotated[int, Field(ge=1, le=10000)] +Definition = Annotated[ + dict[str, Any], + Field( + description="Backend-defined settings. Inspect current definitions and preserve unrelated fields." + ), +] + + +class AgentSelection(OperationArguments): + text_filter: NonEmpty | None = None + agent_ids: list[PathSegment] | None = None + scope_ids: list[NonEmpty] | None = None + platform_types: list[NonEmpty] | None = None + platform_ids: list[PathSegment] | None = None + discovery_statuses: list[NonEmpty] | None = None + risk_sign_offs: list[NonEmpty] | None = None + tags: list[NonEmpty] | None = None + states: list[NonEmpty] | None = None + favorites: bool | None = None + + @model_validator(mode="after") + def nonempty_criteria(self): + for name in self.model_fields_set: + if getattr(self, name) == []: + raise ValueError("Filter lists must not be empty") + return self + + def to_sdk(self): + selection = AgentFilter() + for name, value in self.model_dump(exclude_none=True).items(): + # Each mapping is a declared public AgentFilter builder method. + method = getattr(selection, "with_" + name) + method(*value) if isinstance(value, list) else method(value) + return selection + + def is_filtered(self): + return bool(self.model_dump(exclude_none=True)) + + +class AlertEvaluationArguments(OperationArguments): + agent_filter: AgentSelection | None = None + scope: Literal["filtered", "all_writable_agents"] = "filtered" + mode: Literal["DUE", "FORCE_LATEST", "FORCE_CATCH_UP"] = "DUE" + + @model_validator(mode="after") + def explicit_scope(self): + if ( + not (self.agent_filter and self.agent_filter.is_filtered()) + and self.scope != "all_writable_agents" + ): + raise ValueError("An empty filter requires scope=all_writable_agents") + return self + + +@dataclass(frozen=True) +class AgentManagementOperation: + handler: Callable + domain: str + effect: str + example: dict + destructive: bool = False + local_file_fields: tuple[str, ...] = () + + +AGENT_MANAGEMENT_OPERATIONS: dict[str, AgentManagementOperation] = {} + + +def _register( + domain, effect="read", *, example=None, destructive=False, local_file_fields=() +): + def register(handler): + AGENT_MANAGEMENT_OPERATIONS[handler.__name__] = AgentManagementOperation( + handler, domain, effect, example or {}, destructive, local_file_fields + ) + return handler + + return register + + +def _selection(value): + return value.to_sdk() if value is not None else None + + +def _agent(client, platform_id, agent_id): + return client.get_agent(platform_id, agent_id) + + +def _started(operation): + return { + "platform_id": operation.platform_id, + "agent_id": operation.agent_id, + "operation_id": operation.id, + } + + +def _save_settings(settings, changes, expected_revision): + raw = settings.get_raw() if hasattr(settings, "get_raw") else settings.raw + if not changes or "fetchedRevision" in changes: + raise ValueError("Supply setting changes separately from expected_revision") + # Patch named top-level fields of the freshly fetched definition, retaining + # hidden credentials and unrelated configuration. Nested values are replacements. + raw.update(changes) + raw["fetchedRevision"] = expected_revision + return settings + + +@_register("platforms") +def list_platforms(client): + """List readable Agent Management platforms.""" + return [dict(item) for item in client.list_platforms()] + + +@_register("platforms", example={"platform_id": "discovered-platform"}) +def get_platform(client, platform_id: PathSegment): + """Read platform status.""" + return client.get_platform(platform_id).get_status().get_raw() + + +@_register("platforms", example={"platform_id": "discovered-platform"}) +def get_platform_settings(client, platform_id: PathSegment): + """Read platform settings and fetchedRevision.""" + return client.get_platform(platform_id).get_settings().raw + + +@_register( + "platforms", + "write", + example={ + "platform_id": "requested-platform", + "platform_type": "type-from-product", + "display_name": "Requested platform", + }, +) +def create_platform( + client, + platform_id: PathSegment, + platform_type: NonEmpty, + display_name: NonEmpty, + connection_name: NonEmpty | None = None, + default_project_key: NonEmpty | None = None, + permissions: list[Definition] | None = None, + default_agent_permissions: list[Definition] | None = None, +): + """Create an Agent Management platform with the public SDK.""" + platform = client.create_platform( + platform_id, + platform_type, + display_name, + connection_name=connection_name, + default_project_key=default_project_key, + permissions=permissions, + default_agent_permissions=default_agent_permissions, + ) + return {"platform_id": platform.id} + + +@_register( + "platforms", + "write", + destructive=True, + example={ + "platform_id": "discovered-platform", + "changes": {"displayName": "Requested name"}, + "expected_revision": "revision-from-read", + }, +) +def update_platform_settings( + client, platform_id: PathSegment, changes: Definition, expected_revision: NonEmpty +): + """Replace named top-level platform settings fields, checking the inspected revision.""" + settings = _save_settings( + client.get_platform(platform_id).get_settings(), changes, expected_revision + ) + return settings.save().raw + + +@_register( + "platforms", + "write", + destructive=True, + example={"platform_id": "discovered-platform"}, +) +def delete_platform( + client, + platform_id: PathSegment, + allow_deletion_if_platform_contains_agents: bool = False, +): + """Delete a platform, optionally including discovered or managed agents.""" + client.get_platform(platform_id).delete( + allow_deletion_if_platform_contains_agents=allow_deletion_if_platform_contains_agents + ) + return {"deleted": platform_id} + + +@_register("agents") +def search_agents(client, agent_filter: AgentSelection | None = None): + """Search readable agents and contextual facet counts with typed SDK filters.""" + return client.search_agents(agent_filter=_selection(agent_filter)) + + +@_register( + "agents", + example={"platform_id": "discovered-platform", "agent_id": "discovered-agent"}, +) +def get_agent(client, platform_id: PathSegment, agent_id: PathSegment): + """Read an agent summary using the public status accessor.""" + return _agent(client, platform_id, agent_id).get_status().get_light() + + +@_register( + "agents", + example={"platform_id": "discovered-platform", "agent_id": "discovered-agent"}, +) +def get_agent_status(client, platform_id: PathSegment, agent_id: PathSegment): + """Read full agent status using the public SDK.""" + return _agent(client, platform_id, agent_id).get_status().get_heavy() + + +@_register( + "agents", + example={"platform_id": "discovered-platform", "agent_id": "discovered-agent"}, +) +def get_agent_settings(client, platform_id: PathSegment, agent_id: PathSegment): + """Read agent settings and fetchedRevision.""" + return _agent(client, platform_id, agent_id).get_settings().get_raw() + + +@_register( + "agents", + "write", + example={"platform_id": "discovered-platform", "agent_id": "requested-agent"}, +) +def create_agent( + client, + platform_id: PathSegment, + agent_id: PathSegment, + display_name: str | None = None, + description: str | None = None, + tags: list[NonEmpty] | None = None, + configuration: Definition | None = None, + permissions: list[Definition] | None = None, + agent_owner: Definition | None = None, +): + """Create a monitored Agent Management record; remote discovery is usually preferred.""" + agent = client.get_platform(platform_id).create_agent( + agent_id, + display_name=display_name, + description=description, + tags=tags, + configuration=configuration, + permissions=permissions, + agent_owner=agent_owner, + ) + return {"platform_id": platform_id, "agent_id": agent.id} + + +@_register( + "agents", + "write", + destructive=True, + example={ + "platform_id": "discovered-platform", + "agent_id": "discovered-agent", + "changes": { + "configuration": { + "displayName": "Requested name", + "description": "Preserved description from the inspected configuration", + } + }, + "expected_revision": "revision-from-read", + }, +) +def update_agent_settings( + client, + platform_id: PathSegment, + agent_id: PathSegment, + changes: Definition, + expected_revision: NonEmpty, +): + """Replace named top-level agent settings fields using the inspected revision. + + To rename, copy the complete inspected configuration, change its displayName, + and supply it as changes.configuration, preserving every unrelated field. + """ + return ( + _save_settings( + _agent(client, platform_id, agent_id).get_settings(), + changes, + expected_revision, + ) + .save() + .get_raw() + ) + + +@_register( + "agents", + "write", + destructive=True, + example={"platform_id": "discovered-platform", "agent_id": "discovered-agent"}, +) +def delete_agent(client, platform_id: PathSegment, agent_id: PathSegment): + """Delete a monitored agent record.""" + _agent(client, platform_id, agent_id).delete() + return {"deleted": agent_id, "platform_id": platform_id} + + +@_register( + "metrics", + example={"platform_id": "discovered-platform", "agent_id": "discovered-agent"}, +) +def get_agent_uptime_results( + client, + platform_id: PathSegment, + agent_id: PathSegment, + from_time: int | None = None, + to_time: int | None = None, +): + """Read uptime results within optional epoch-millisecond bounds.""" + + def iso_utc(value): + return ( + datetime.fromtimestamp(value / 1000, tz=timezone.utc) + .isoformat() + .replace("+00:00", "Z") + if value is not None + else None + ) + + return _agent(client, platform_id, agent_id).get_uptime_results( + from_time=iso_utc(from_time), to_time=iso_utc(to_time) + ) + + +@_register("metrics") +def get_available_metric_ids(client, agent_filter: AgentSelection | None = None): + """Discover qualified metric IDs supported by readable agents.""" + return client.get_available_metric_ids(agent_filter=_selection(agent_filter)) + + +@_register( + "metrics", example={"metrics": ["qualified-metric-from-read"], "latest_only": True} +) +def get_metric_time_series( + client, + metrics: Annotated[list[NonEmpty], Field(min_length=1)], + agent_filter: AgentSelection | None = None, + from_time: int | None = None, + to_time: int | None = None, + latest_only: bool = False, +): + """Read stored metric time series; discover qualified metric IDs first.""" + return client.get_metric_time_series( + metrics, + agent_filter=_selection(agent_filter), + from_time=from_time, + to_time=to_time, + latest_only=latest_only, + ) + + +@_register( + "metrics", + example={ + "metrics": ["qualified-metric-from-read"], + "bucket": "DAY", + "from_time": 0, + "to_time": 86400000, + }, +) +def get_metric_buckets( + client, + metrics: Annotated[list[NonEmpty], Field(min_length=1)], + bucket: Literal["HOUR", "DAY", "WEEK", "MONTH"], + from_time: int, + agent_filter: AgentSelection | None = None, + to_time: int | None = None, +): + """Read bucketed metric values using qualified IDs and epoch-millisecond bounds.""" + result = client.get_metric_buckets( + metrics, + bucket, + agent_filter=_selection(agent_filter), + from_time=from_time, + to_time=to_time, + ) + return [ + series + for platform, agent in result.agents() + for series in result.raw_agent_data(platform, agent) + ] + + +@_register( + "operations", "write", destructive=True, example={"scope": "all_writable_agents"} +) +def evaluate_alerts( + client, + agent_filter: AgentSelection | None = None, + scope: Literal["filtered", "all_writable_agents"] = "filtered", + mode: Literal["DUE", "FORCE_LATEST", "FORCE_CATCH_UP"] = "DUE", +): + """Queue native alert evaluation; forcing correction can invalidate historical alerts.""" + if ( + not (agent_filter and agent_filter.is_filtered()) + and scope != "all_writable_agents" + ): + raise ValueError("An empty filter requires scope=all_writable_agents") + return { + "queued_agent_count": client.evaluate_alerts( + agent_filter=_selection(agent_filter), mode=mode + ) + } + + +@_register("operations", "write", example={"platform_id": "discovered-platform"}) +def scan_platform(client, platform_id: PathSegment): + """Start platform discovery and return the operation identity for polling.""" + return _started(client.get_platform(platform_id).start_scan()) + + +@_register("operations", "write", example={"platform_id": "discovered-platform"}) +def run_platform_uptime_tests( + client, + platform_id: PathSegment, + agent_ids: Annotated[list[PathSegment], Field(min_length=1)] | None = None, +): + """Start uptime tests for selected agents or all eligible agents on a platform.""" + return _started( + client.get_platform(platform_id).start_run_uptime_tests(agent_ids=agent_ids) + ) + + +@_register( + "operations", + "write", + example={"platform_id": "discovered-platform", "agent_id": "discovered-agent"}, +) +def fetch_agent_logs(client, platform_id: PathSegment, agent_id: PathSegment): + """Queue agent log ingestion and return the operation identity.""" + return _started(_agent(client, platform_id, agent_id).start_fetch_logs()) + + +@_register( + "operations", + "write", + example={"platform_id": "discovered-platform", "agent_id": "discovered-agent"}, +) +def fetch_agent_operational_metrics( + client, platform_id: PathSegment, agent_id: PathSegment +): + """Queue operational metric ingestion and return the operation identity.""" + return _started( + _agent(client, platform_id, agent_id).start_fetch_platform_operational_metrics() + ) + + +@_register( + "operations", + "write", + destructive=True, + example={"platform_id": "discovered-platform", "agent_id": "discovered-agent"}, +) +def setup_agent_monitoring( + client, + platform_id: PathSegment, + agent_id: PathSegment, + run_first_fetch: bool = True, + include_in_uptime_tests: bool = True, + project_key: NonEmpty | None = None, + connection_name: NonEmpty | None = None, + partitioning: Literal["NONE", "DAY"] | None = None, +): + """Provision monitoring resources and enable supported ingestion and uptime capabilities.""" + options = { + key: value + for key, value in dict( + project_key=project_key, + connection_name=connection_name, + partitioning=partitioning, + ).items() + if value is not None + } + return _agent(client, platform_id, agent_id).setup_monitoring( + run_first_fetch=run_first_fetch, + include_in_uptime_tests=include_in_uptime_tests, + **options, + ) + + +@_register("operations") +def search_operations( + client, + agent_filter: AgentSelection | None = None, + operation_types: list[NonEmpty] | None = None, + statuses: list[NonEmpty] | None = None, + triggers: list[NonEmpty] | None = None, + from_timestamp: int | None = None, + to_timestamp: int | None = None, + limit: PositiveLimit | None = None, + current_only: bool = False, +): + """Search bounded operation history and report whether narrower filters are needed.""" + result = client.list_operations( + agent_filter=_selection(agent_filter), + operation_types=operation_types, + statuses=statuses, + triggers=triggers, + from_timestamp=from_timestamp, + to_timestamp=to_timestamp, + limit=limit, + current_only=current_only, + ) + return { + "operations": [dict(item) for item in result.operations], + "has_more": result.has_more, + } + + +@_register( + "operations", + example={ + "platform_id": "discovered-platform", + "agent_id": "discovered-agent", + "operation_id": "returned-operation", + }, +) +def get_agent_operation( + client, platform_id: PathSegment, agent_id: PathSegment, operation_id: PathSegment +): + """Read one agent operation status without waiting.""" + return ( + _agent(client, platform_id, agent_id) + .get_operation(operation_id) + .get_status() + .get_raw() + ) + + +@_register( + "operations", + example={ + "platform_id": "discovered-platform", + "agent_id": "discovered-agent", + "operation_id": "returned-operation", + }, +) +def get_agent_operation_result( + client, platform_id: PathSegment, agent_id: PathSegment, operation_id: PathSegment +): + """Read a completed agent operation result.""" + return ( + _agent(client, platform_id, agent_id).get_operation(operation_id).get_result() + ) + + +@_register( + "operations", + "write", + destructive=True, + example={ + "platform_id": "discovered-platform", + "agent_id": "discovered-agent", + "operation_id": "returned-operation", + }, +) +def abort_agent_operation( + client, platform_id: PathSegment, agent_id: PathSegment, operation_id: PathSegment +): + """Request cancellation of an agent operation.""" + _agent(client, platform_id, agent_id).get_operation(operation_id).abort() + return {"abort_requested": operation_id} + + +@_register( + "operations", + example={ + "platform_id": "discovered-platform", + "operation_id": "returned-operation", + }, +) +def get_platform_operation(client, platform_id: PathSegment, operation_id: PathSegment): + """Read one platform operation status without waiting.""" + return ( + client.get_platform(platform_id) + .get_operation(operation_id) + .get_status() + .get_raw() + ) + + +@_register( + "operations", + example={ + "platform_id": "discovered-platform", + "operation_id": "returned-operation", + }, +) +def get_platform_operation_result( + client, platform_id: PathSegment, operation_id: PathSegment +): + """Read a completed platform operation result.""" + return client.get_platform(platform_id).get_operation(operation_id).get_result() + + +@_register( + "operations", + "write", + destructive=True, + example={ + "platform_id": "discovered-platform", + "operation_id": "returned-operation", + }, +) +def abort_platform_operation( + client, platform_id: PathSegment, operation_id: PathSegment +): + """Request cancellation of a platform operation.""" + client.get_platform(platform_id).get_operation(operation_id).abort() + return {"abort_requested": operation_id} + + +@_register("risk") +def get_risk_exposure(client): + """Read portfolio risk exposure, joining agents and riskExposures by agentKey.""" + exposure = client.get_risk_exposure().raw + return { + **exposure, + "agents": [ + { + ("agentKey" if key == "key" else key): value + for key, value in agent.items() + } + for agent in exposure["agents"] + ], + } + + +@_register("risk") +def get_risk_taxonomy(client): + """Read risk taxonomy and fetchedRevision.""" + return client.get_risk_taxonomy().get_raw() + + +@_register("risk") +def get_default_risk_taxonomy(client): + """Read the default risk taxonomy.""" + return client.get_default_risk_taxonomy().get_raw() + + +@_register("risk", example={"taxonomy": {}}) +def compute_risk_taxonomy_replacement_impact(client, taxonomy: Definition): + """Compute replacement impact for a proposed taxonomy without saving it.""" + settings = client.get_risk_taxonomy() + settings.raw["taxonomy"] = taxonomy + return settings.compute_replacement_impact() + + +@_register( + "risk", + "write", + destructive=True, + example={ + "taxonomy": {}, + "expected_revision": "revision-from-read", + "impact": {"fingerprint": "fingerprint-from-impact"}, + }, +) +def update_risk_taxonomy( + client, taxonomy: Definition, expected_revision: NonEmpty, impact: Definition +): + """Save a proposed taxonomy with the inspected revision and reviewed impact fingerprint.""" + settings = client.get_risk_taxonomy() + settings.raw.update(taxonomy=taxonomy, fetchedRevision=expected_revision) + return settings.save(impact).get_raw() + + +@_register( + "risk", + example={"platform_id": "discovered-platform", "agent_id": "discovered-agent"}, +) +def get_agent_risk_assessment(client, platform_id: PathSegment, agent_id: PathSegment): + """Read agent risk assessment and fetchedRevision.""" + return _agent(client, platform_id, agent_id).get_risk_assessment().get_raw() + + +@_register( + "risk", + "write", + destructive=True, + local_file_fields=("documents",), + example={ + "platform_id": "discovered-platform", + "agent_id": "discovered-agent", + "assessment": {}, + "expected_revision": "revision-from-read", + }, +) +def update_agent_risk_assessment( + client, + platform_id: PathSegment, + agent_id: PathSegment, + assessment: Definition, + expected_revision: NonEmpty, + documents: dict[PathSegment, NonEmpty] | None = None, +): + """Replace the complete assessment using its inspected revision and optional evidence uploads. + + Omitted evidence references cause permanent deletion of the corresponding + evidence after commit. Preserve all wanted assessment fields and evidence + references. The empty assessment example deliberately clears the assessment + and its evidence. documents maps evidence IDs to local paths. + """ + settings = _agent(client, platform_id, agent_id).get_risk_assessment() + settings.raw.update(assessment=assessment, fetchedRevision=expected_revision) + with ExitStack() as stack: + uploads = { + key: ( + os.path.basename(path), + stack.enter_context( + open(os.path.realpath(os.path.expanduser(path)), "rb") + ), + ) + for key, path in (documents or {}).items() + } + return settings.save(documents=uploads or None).get_raw() + + +@_register( + "risk", + "write", + destructive=True, + example={"platform_id": "discovered-platform", "agent_id": "discovered-agent"}, +) +def sign_off_agent_risk_assessment( + client, platform_id: PathSegment, agent_id: PathSegment +): + """Sign off the current agent risk assessment.""" + return ( + _agent(client, platform_id, agent_id) + .set_risk_assessment_signed_off(True) + .get_raw() + ) + + +@_register( + "risk", + "write", + destructive=True, + example={"platform_id": "discovered-platform", "agent_id": "discovered-agent"}, +) +def remove_agent_risk_sign_off(client, platform_id: PathSegment, agent_id: PathSegment): + """Remove current agent risk-assessment sign-off.""" + return ( + _agent(client, platform_id, agent_id) + .set_risk_assessment_signed_off(False) + .get_raw() + ) + + +def _download_binary(response, output_path, overwrite): + absolute_path = os.path.realpath(os.path.expanduser(output_path)) + parent = os.path.dirname(absolute_path) + temporary_path = None + digest = hashlib.sha256() + size = 0 + try: + if not os.path.isdir(parent): + raise FileNotFoundError("Destination directory does not exist") + if os.path.lexists(absolute_path) and not overwrite: + raise FileExistsError( + "Destination exists; use overwrite=true to replace it" + ) + with tempfile.NamedTemporaryFile( + prefix=".agent-management-evidence-", dir=parent, delete=False + ) as handle: + temporary_path = handle.name + for chunk in response.iter_content(chunk_size=1024 * 1024): + if chunk: + handle.write(chunk) + digest.update(chunk) + size += len(chunk) + if overwrite: + os.replace(temporary_path, absolute_path) + else: + os.link(temporary_path, absolute_path) + os.unlink(temporary_path) + temporary_path = None + finally: + response.close() + if temporary_path is not None: + os.unlink(temporary_path) + return { + "output_path": absolute_path, + "size_bytes": size, + "sha256": digest.hexdigest(), + "content_type": response.headers.get("Content-Type"), + } + + +@_register( + "risk", + "write", + destructive=True, + local_file_fields=("output_path",), + example={ + "platform_id": "discovered-platform", + "agent_id": "discovered-agent", + "evidence_id": "discovered-evidence", + "output_path": "/absolute/path/evidence.bin", + }, +) +def download_agent_risk_evidence( + client, + platform_id: PathSegment, + agent_id: PathSegment, + evidence_id: PathSegment, + output_path: NonEmpty, + overwrite: bool = False, +): + """Download risk evidence locally; overwrite requires explicit intent.""" + response = ( + _agent(client, platform_id, agent_id) + .get_risk_assessment() + .get_risk_evidence_content(evidence_id) + ) + return _download_binary(response, output_path, overwrite) + + +@_register("settings") +def get_settings(client): + """Read global settings, including business KPI definitions and fetchedRevision.""" + return client.get_settings().get_raw() + + +@_register( + "settings", + "write", + destructive=True, + example={ + "changes": {"businessKPIs": []}, + "expected_revision": "revision-from-read", + }, +) +def update_settings( + client, + changes: Definition, + expected_revision: NonEmpty, + reset_incompatible_metrics: bool = False, +): + """Replace named global settings fields; optionally allow deletion of incompatible metrics. + + The example sets businessKPIs=[] to remove all business KPI definitions. + Use it only when clearing all definitions is intended. + """ + return ( + _save_settings(client.get_settings(), changes, expected_revision) + .save(confirm_metric_reset=reset_incompatible_metrics) + .get_raw() + ) + + +@_register("tags") +def list_tags(client): + """List visible tag names and colors.""" + return [{"name": item.name, "color": item.color} for item in client.list_tags()] + + +@_register( + "tags", + "write", + destructive=True, + example={"tag_name": "discovered-tag", "color": "requested-color"}, +) +def update_tag(client, tag_name: PathSegment, color: NonEmpty): + """Update an existing tag color; the SDK cannot create a tag handle for an absent tag.""" + tag = client.list_tags().tag(tag_name) + tag.color = color + tag.save() + return {"name": tag.name, "color": tag.color} + + +@_register( + "tags", + "write", + destructive=True, + example={"tag_name": "discovered-tag", "new_name": "requested-tag"}, +) +def rename_tag(client, tag_name: PathSegment, new_name: PathSegment): + """Rename a tag everywhere it is used.""" + tag = client.list_tags().tag(tag_name).rename(new_name) + return {"name": tag.name, "color": tag.color} + + +@_register("tags", "write", destructive=True, example={"tag_name": "discovered-tag"}) +def delete_tag(client, tag_name: PathSegment): + """Delete a tag everywhere it is used.""" + client.list_tags().tag(tag_name).delete() + return {"deleted": tag_name} + + +def operation_handler(spec: AgentManagementOperation): + async def execute(**params): + def run(): + try: + client = get_dss_client().get_agent_management_client() + return redact_sensitive_data(spec.handler(client, **params)) + except (DataikuException, RequestException): + raise RuntimeError( + "Agent Management request failed. Check SDK/server compatibility, permissions and IDs. Re-read state before retrying a write; it may have reached Dataiku. Conflicting revisions must not be retried automatically." + ) from None + + return compact_json(await run_blocking(run)) + + return execute diff --git a/dataiku_mcp/tools/connections.py b/dataiku_mcp/tools/connections.py index 90912a3c..e68cee29 100644 --- a/dataiku_mcp/tools/connections.py +++ b/dataiku_mcp/tools/connections.py @@ -14,8 +14,6 @@ """Dataiku connection discovery and inspection tools.""" -from typing import Any - from typing import Annotated, Literal from fastmcp import Context @@ -30,51 +28,7 @@ require_non_empty_string as _require_non_empty_string, ) -_CONNECTION_SECRET_REDACTION = "__DATAIKU_REDACTED__" -_SENSITIVE_SUFFIXES = ( - "apikey", - "credential", - "credentials", - "password", - "secret", - "token", -) -_SENSITIVE_EXACT_KEYS = ("key",) -_SENSITIVE_KEY_FRAGMENTS = ( - "accesskey", - "appsecret", - "keybase64data", - "keyjsondata", - "passphrase", - "privatekey", - "secretkey", - "tokenkey", -) - - -def _is_sensitive_key(key: str) -> bool: - normalized = key.replace("_", "").replace("-", "").lower() - return ( - normalized in _SENSITIVE_EXACT_KEYS - or normalized.endswith(_SENSITIVE_SUFFIXES) - or any(fragment in normalized for fragment in _SENSITIVE_KEY_FRAGMENTS) - ) - - -def _redact_sensitive_data(value: Any) -> Any: - if isinstance(value, dict): - redacted = {} - for key, item in value.items(): - if _is_sensitive_key(str(key)): - redacted[key] = _CONNECTION_SECRET_REDACTION - else: - redacted[key] = _redact_sensitive_data(item) - return redacted - - if isinstance(value, list): - return [_redact_sensitive_data(item) for item in value] - - return value +from .utils.redaction import redact_sensitive_data as _redact_sensitive_data _CONNECTION_TYPES_BY_CATEGORY = { diff --git a/dataiku_mcp/tools/deployer.py b/dataiku_mcp/tools/deployer.py new file mode 100644 index 00000000..d3da5c8d --- /dev/null +++ b/dataiku_mcp/tools/deployer.py @@ -0,0 +1,379 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Project and API Deployer inspection and deployment operations.""" + +from typing import Annotated, Literal + +from dataikuapi.utils import DataikuException +from pydantic import Field +from requests import RequestException + +from ..auth import get_dss_client +from ..executors import run_blocking +from .utils.serialization import columnar, compact_json +from .utils.validation import require_non_empty_string +from .utils.validation import PathSegment as Identifier + + +DeployerType = Literal["project", "api"] +Version = Annotated[ + str, Field(description="Published bundle ID (project) or service version ID (api).") +] + +# Raw Deployer status includes API keys, OAuth settings, scripts and endpoint code. +# Keep these allowlists explicit, including for nested objects. +_INFRA_FIELDS = ("id", "stage", "infraType", "governCheckPolicy", "canListHeavyStatus") +_ITEM_FIELDS = ("id", "name") +_DEPLOYMENT_FIELDS = ( + "id", + "infraId", + "publishedProjectKey", + "deployedProjectKey", + "bundleId", + "publishedServiceId", + "deployedServiceId", + "enabled", + "endpointId", +) +_PERMISSION_FIELDS = ( + "canDeploy", + "canWrite", + "isAdmin", + "hasFullDeployPermissions", + "isInfraAdmin", + "neverEverDeployed", +) +_UPDATE_FIELDS = ("startTimestamp", "endTimestamp", "type", "status", "requester") + + +def _pick(raw: dict, fields: tuple) -> dict: + return {key: raw[key] for key in fields if key in raw} + + +def _deployment_info(raw: dict) -> dict: + result = _pick(raw, _DEPLOYMENT_FIELDS) + mapping = raw.get("generationsMapping") + if mapping is not None: + result["generationsMapping"] = _pick(mapping, ("mode", "generation")) + if "entries" in mapping: + result["generationsMapping"]["entries"] = [ + _pick(entry, ("generation", "proba")) for entry in mapping["entries"] + ] + return result + + +def _summary(raw: dict) -> dict: + result = _pick(raw, _PERMISSION_FIELDS) + for key, fields in ( + ("infraBasicInfo", _INFRA_FIELDS), + ("projectBasicInfo", _ITEM_FIELDS), + ("serviceBasicInfo", _ITEM_FIELDS), + ): + if key in raw: + result[key] = _pick(raw[key], fields) + if "deploymentBasicInfo" in raw: + result["deploymentBasicInfo"] = _deployment_info(raw["deploymentBasicInfo"]) + if "packages" in raw: + result["packages"] = columnar( + raw["packages"], ["id", "publishedOn", "publishedBy"] + ) + if "deployments" in raw: + result["deployments"] = [_deployment_info(item) for item in raw["deployments"]] + if "infras" in raw: + result["infras"] = [_pick(item, _INFRA_FIELDS) for item in raw["infras"]] + return result + + +async def _execute(deployer_type: DeployerType, operation) -> str: + def _run(): + client = get_dss_client() + deployer = ( + client.get_projectdeployer() + if deployer_type == "project" + else client.get_apideployer() + ) + try: + return operation(deployer) + except (DataikuException, RequestException): + # SDK exception bodies may include node credentials or raw settings. + raise RuntimeError( + "Deployer request failed. Check node capabilities, permissions and object IDs. " + "For a write, inspect current state and update history before retrying; " + "the request may have reached Dataiku." + ) from None + + return compact_json(await run_blocking(_run)) + + +async def list_deployer_stages(deployer_type: DeployerType) -> str: + """Discover configured infrastructure stages for a Project or API Deployer.""" + return await _execute( + deployer_type, + lambda deployer: {"stages": columnar(deployer.list_stages(), ["id", "desc"])}, + ) + + +async def list_deployer_infrastructures(deployer_type: DeployerType) -> str: + """Discover deployment targets and the current user’s deployment permissions.""" + return await _execute( + deployer_type, + lambda deployer: { + "infrastructures": [ + _summary(raw) for raw in deployer.list_infras(as_objects=False) + ] + }, + ) + + +async def get_deployer_infrastructure( + deployer_type: DeployerType, infra_id: Identifier +) -> str: + """Inspect an infrastructure’s summary and associated deployments without credentials.""" + infra_id = require_non_empty_string(infra_id, "infra_id") + return await _execute( + deployer_type, + lambda deployer: _summary(deployer.get_infra(infra_id).get_status().get_raw()), + ) + + +async def list_deployer_published_items(deployer_type: DeployerType) -> str: + """Discover published projects or API services available for deployment.""" + + def _run(deployer): + items = ( + deployer.list_projects(as_objects=False) + if deployer_type == "project" + else deployer.list_services(as_objects=False) + ) + return {"items": [_summary(raw) for raw in items]} + + return await _execute(deployer_type, _run) + + +async def get_deployer_published_item( + deployer_type: DeployerType, source_id: Identifier +) -> str: + """Inspect a published project or API service and its available bundles or versions.""" + source_id = require_non_empty_string(source_id, "source_id") + + def _run(deployer): + item = ( + deployer.get_project(source_id) + if deployer_type == "project" + else deployer.get_service(source_id) + ) + return _summary(item.get_status().get_raw()) + + return await _execute(deployer_type, _run) + + +async def list_deployer_deployments(deployer_type: DeployerType) -> str: + """Discover deployment IDs and configured target versions; use detail for live health.""" + return await _execute( + deployer_type, + lambda deployer: { + "deployments": [ + _summary(raw) for raw in deployer.list_deployments(as_objects=False) + ] + }, + ) + + +async def get_deployer_deployment( + deployer_type: DeployerType, deployment_id: Identifier +) -> str: + """Inspect configured deployment version, current health and governance status.""" + deployment_id = require_non_empty_string(deployment_id, "deployment_id") + + def _run(deployer): + deployment = deployer.get_deployment(deployment_id) + status = deployment.get_status() + governance = deployment.get_governance_status() + # Only structured codes and severity: diagnostic free text can contain secrets. + return { + **_summary(status.get_light()), + "settings": _deployment_info(deployment.get_settings().get_raw()), + "health": status.get_health(), + "health_messages": [ + _pick(message, ("severity", "isFatal", "code")) + for message in status.get_health_messages().get("messages", []) + ], + "governance_messages": [ + _pick(message, ("severity", "isFatal", "code")) + for message in governance.get("messages", []) + ], + } + + return await _execute(deployer_type, _run) + + +async def list_deployer_updates( + deployer_type: DeployerType, deployment_id: Identifier +) -> str: + """Find deployment update timestamps and outcomes for recovery and auditing.""" + deployment_id = require_non_empty_string(deployment_id, "deployment_id") + return await _execute( + deployer_type, + lambda deployer: { + "updates": [ + _pick(update.get_raw(), _UPDATE_FIELDS) + for update in deployer.get_deployment(deployment_id).list_updates() + ] + }, + ) + + +async def get_deployer_update( + deployer_type: DeployerType, + deployment_id: Identifier, + timestamp: Identifier | None = None, +) -> str: + """Inspect one deployment update outcome, using a timestamp or the latest update.""" + deployment_id = require_non_empty_string(deployment_id, "deployment_id") + if timestamp is not None: + timestamp = require_non_empty_string(timestamp, "timestamp") + return await _execute( + deployer_type, + lambda deployer: _pick( + deployer.get_deployment(deployment_id).get_update(timestamp).get_raw(), + _UPDATE_FIELDS, + ), + ) + + +async def create_deployer_deployment( + deployer_type: DeployerType, + deployment_id: Identifier, + source_id: Identifier, + infra_id: Identifier, + version: Version, + deployed_project_key: Identifier | None = None, + project_folder_id: Identifier | None = None, + endpoint_id: Identifier | None = None, +) -> str: + """Create deployment configuration from a published version; start an update to apply it.""" + deployment_id = require_non_empty_string(deployment_id, "deployment_id") + source_id = require_non_empty_string(source_id, "source_id") + infra_id = require_non_empty_string(infra_id, "infra_id") + version = require_non_empty_string(version, "version") + if deployed_project_key is not None: + deployed_project_key = require_non_empty_string( + deployed_project_key, "deployed_project_key" + ) + if project_folder_id is not None: + project_folder_id = require_non_empty_string( + project_folder_id, "project_folder_id" + ) + if endpoint_id is not None: + endpoint_id = require_non_empty_string(endpoint_id, "endpoint_id") + if deployer_type == "project" and endpoint_id is not None: + raise ValueError("endpoint_id is only supported for API deployments") + if deployer_type == "api" and ( + deployed_project_key is not None or project_folder_id is not None + ): + raise ValueError( + "deployed_project_key and project_folder_id are only supported for project deployments" + ) + + def _run(deployer): + if deployer_type == "project": + deployment = deployer.create_deployment( + deployment_id, + source_id, + infra_id, + version, + deployed_project_key=deployed_project_key, + project_folder_id=project_folder_id, + ) + else: + deployment = deployer.create_deployment( + deployment_id, source_id, infra_id, version, endpoint_id=endpoint_id + ) + return { + "deployment_id": deployment.id, + "status": "configuration_created", + "update_started": False, + } + + return await _execute(deployer_type, _run) + + +async def update_deployer_deployment( + deployer_type: DeployerType, + deployment_id: Identifier, + version: Version | None = None, + enabled: bool | None = None, +) -> str: + """Save a target version or API enablement; an update is still needed to apply it.""" + deployment_id = require_non_empty_string(deployment_id, "deployment_id") + if version is None and enabled is None: + raise ValueError("Provide version or enabled") + if version is not None: + version = require_non_empty_string(version, "version") + if enabled is not None and deployer_type != "api": + raise ValueError("enabled is only supported for API deployments") + + def _run(deployer): + settings = deployer.get_deployment(deployment_id).get_settings() + if version is not None: + if deployer_type == "project": + settings.bundle_id = version + else: + settings.set_single_version(version) + if enabled is not None: + settings.set_enabled(enabled) + settings.save() + return { + "deployment_id": deployment_id, + "status": "configuration_saved", + "update_started": False, + } + + return await _execute(deployer_type, _run) + + +async def start_deployer_update( + deployer_type: DeployerType, deployment_id: Identifier +) -> str: + """Apply saved deployment settings asynchronously, potentially replacing running production assets.""" + deployment_id = require_non_empty_string(deployment_id, "deployment_id") + + def _run(deployer): + future = deployer.get_deployment(deployment_id).start_update() + return { + "deployment_id": deployment_id, + "future_id": future.job_id, + "status": "update_submitted", + "hint": "Poll get_future_status on this same instance, then re-read deployment health and update history. If future_id is null, inspect update history. Do not resubmit an uncertain update.", + } + + return await _execute(deployer_type, _run) + + +async def get_deployer_future_status(future_id: Identifier) -> str: + """Poll a deployment future without fetching potentially sensitive result payloads.""" + from dataikuapi.dss.future import DSSFuture + + def _run(): + try: + state = DSSFuture(get_dss_client(), future_id).peek_state() + except (DataikuException, RequestException): + raise RuntimeError( + "Deployer future polling failed. Inspect deployment health and update history on this instance; do not resubmit the deployment." + ) from None + fields = ("jobId", "hasResult", "aborted", "alive", "startTime", "endTime") + return {"future_id": future_id, "state": _pick(state, fields)} + + return compact_json(await run_blocking(_run)) diff --git a/dataiku_mcp/tools/govern.py b/dataiku_mcp/tools/govern.py new file mode 100644 index 00000000..ff55a87d --- /dev/null +++ b/dataiku_mcp/tools/govern.py @@ -0,0 +1,1866 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Govern public SDK adapters, integrated with the shared operation registry. + +Adapted from PR #174; all actions are exposed through shared discovery and read/write dispatch. +""" + +from __future__ import annotations + +import hashlib +import os +import tempfile +from collections.abc import Callable +from dataclasses import dataclass +from typing import Any + +from dataikuapi.govern.artifact_search import ( + GovernArtifactFilterArchivedStatus, + GovernArtifactFilterArtifacts, + GovernArtifactFilterBlueprints, + GovernArtifactFilterBlueprintVersions, + GovernArtifactFilterFieldValue, + GovernArtifactSearchQuery, + GovernArtifactSearchSortField, + GovernArtifactSearchSortFieldDefinition, + GovernArtifactSearchSortName, + GovernArtifactSearchSortWorkflow, +) +from dataikuapi.utils import DataikuException +from requests import RequestException +from pydantic import BaseModel +from .utils.redaction import redact_sensitive_data + +from ..auth import get_govern_client +from ..executors import run_blocking +from .utils.serialization import compact_json +from .utils.validation import require_non_empty_string + +DOMAINS = ( + "artifacts", + "signoffs", + "blueprints", + "roles", + "custom_pages", + "time_series", + "files", + "users", + "instance", +) + +_CHUNK_SIZE = 1024 * 1024 + + +@dataclass(frozen=True) +class GovernParam: + name: str + kind: str # string, boolean, integer, object, list, any + description: str + required: bool = False + path: bool = False # the SDK puts the value into the URL path + + +@dataclass(frozen=True) +class GovernOperation: + domain: str + kind: str # read, write, delete + description: str + sdk: str + params: tuple[GovernParam, ...] + run: Callable[[Any, dict[str, Any]], Any] + + +GOVERN_OPERATIONS: dict[str, GovernOperation] = {} + + +def _register( + operation_id: str, + *, + domain: str, + kind: str, + description: str, + sdk: str, + params: tuple[GovernParam, ...] = (), +): + def decorator(func): + GOVERN_OPERATIONS[operation_id] = GovernOperation( + domain, kind, description, sdk, tuple(params), func + ) + return func + + return decorator + + +def _p( + name: str, kind: str, description: str, required: bool = False, path: bool = False +) -> GovernParam: + return GovernParam(name, kind, description, required, path) + + +ARTIFACT_ID = _p( + "artifact_id", "string", "Artifact ID, for example ar.42.", True, path=True +) +STEP_ID = _p( + "step_id", "string", "Workflow step ID that carries the signoff.", True, path=True +) +BLUEPRINT_ID = _p( + "blueprint_id", + "string", + "Blueprint ID, for example bp.system.govern_project.", + True, + path=True, +) +VERSION_ID = _p( + "version_id", + "string", + "Blueprint version ID, for example bv.system.default.", + True, + path=True, +) +ROLE_ID = _p("role_id", "string", "Role ID, for example ro.reviewer.", True, path=True) +CUSTOM_PAGE_ID = _p( + "custom_page_id", "string", "Custom page ID, for example cp.1.", True, path=True +) +TIME_SERIES_ID = _p( + "time_series_id", "string", "Time series ID, for example ts.7.", True, path=True +) +UPLOADED_FILE_ID = _p( + "uploaded_file_id", "string", "Uploaded file ID, for example uf.3.", True, path=True +) +LOGIN = _p("login", "string", "User login.", True, path=True) +GROUP_NAME = _p("name", "string", "Group name.", True, path=True) +NEW_IDENTIFIER = _p( + "new_identifier", + "string", + "Bare identifier for the new object; the server adds the type prefix.", + True, + path=True, +) +USERS_CONTAINER = _p( + "users_container", + "object", + "Delegate: {type: user, login}. Delegation accepts a single user only.", + True, +) +COMMENT = _p("comment", "string", "Optional comment recorded with the decision.") + + +def _users_container(value: Any) -> dict[str, Any]: + """Validate a raw users-container dict; the SDK sends it as the request body.""" + if ( + not isinstance(value, dict) + or value.get("type") != "user" + or not value.get("login") + ): + raise ValueError( + "users_container must be {type: 'user', login: }; delegation " + "accepts a single user only" + ) + return value + + +def _replace_raw(definition, new_raw: dict[str, Any]): + """Replace a mutable SDK definition in place so ``save()`` sends ``new_raw``.""" + raw = definition.get_raw() + raw.clear() + raw.update(new_raw) + return definition + + +def _with_id(body: dict[str, Any], object_id: str) -> dict[str, Any]: + """Add the ID the operation already names to a small metadata body. + + Govern refuses a blueprint, role or custom page body without its ID. Only + these metadata bodies get it: for larger definitions the missing ID makes + Govern refuse a wrong payload instead of saving it. + """ + if "id" in body and body["id"] != object_id: + raise ValueError("Definition ID must match the selected object") + return {**body, "id": object_id} + + +def _version_state(version) -> dict[str, Any]: + return { + "definition": version.get_definition().get_raw(), + "trace": version.get_trace().get_raw(), + } + + +def _signoff(client, params: dict[str, Any]): + return client.get_artifact(params["artifact_id"]).get_signoff(params["step_id"]) + + +def _admin_blueprint(client, params: dict[str, Any]): + return client.get_blueprint_designer().get_blueprint(params["blueprint_id"]) + + +def _admin_version(client, params: dict[str, Any]): + return _admin_blueprint(client, params).get_version(params["version_id"]) + + +def _roles(client): + return client.get_roles_permissions_handler() + + +def _pages(client): + return client.get_custom_pages_handler() + + +# --------------------------------------------------------------------------- +# artifacts +# --------------------------------------------------------------------------- + + +_SEARCH_ID_FILTERS = ("blueprint_ids", "blueprint_version_ids", "artifact_ids") +_FIELD_FILTER_KEYS = { + "condition_type", + "condition", + "field_id", + "negate", + "case_sensitive", +} +_SORT_KEYS = {"type", "direction", "fields"} +_SORT_FIELD_KEYS = {"blueprint_id", "field_id"} + + +def _require_known_keys(value: Any, allowed: set[str], label: str) -> None: + if not isinstance(value, dict): + raise ValueError(f"Each {label} must be an object") + unknown = sorted(set(value) - allowed) + if unknown: + raise ValueError( + f"Unknown keys in {label}: {unknown}. Allowed: {sorted(allowed)}" + ) + + +@_register( + "search_artifacts", + domain="artifacts", + kind="read", + description="Search artifacts with optional blueprint, version, artifact, field and archived filters. Each hit holds id, name, blueprintVersionId and status; get_artifact returns fields and workflow.", + sdk="GovernClient.new_artifact_search_request", + params=( + _p("blueprint_ids", "list", "Blueprint IDs to keep."), + _p( + "blueprint_version_ids", + "list", + "Blueprint version IDs to keep, each {blueprintId, versionId}.", + ), + _p("artifact_ids", "list", "Artifact IDs to keep."), + _p( + "field_filters", + "list", + "Field conditions, each {condition_type: EQUALS|CONTAINS|START_WITH|END_WITH, condition, field_id, negate, case_sensitive}; no field_id filters on the name.", + ), + _p("archived", "boolean", "true includes archived artifacts."), + _p( + "sort", + "object", + "Sort: {type: name|workflow|field, direction: ASC|DESC, fields: [{blueprint_id, field_id}]}.", + ), + _p("page_size", "integer", "Batch size per request (default 20)."), + _p("max_results", "integer", "Maximum hits to return (default 100)."), + ), +) +def _search_artifacts(client, p): + # An empty ID list matches nothing; dropping the filter would match everything. + if any(p.get(key) == [] for key in _SEARCH_ID_FILTERS): + return {"count": 0, "has_more": False, "hits": []} + filters = [] + if p.get("blueprint_ids"): + filters.append(GovernArtifactFilterBlueprints(list(p["blueprint_ids"]))) + if p.get("blueprint_version_ids"): + filters.append( + GovernArtifactFilterBlueprintVersions(list(p["blueprint_version_ids"])) + ) + if p.get("artifact_ids"): + filters.append(GovernArtifactFilterArtifacts(list(p["artifact_ids"]))) + for field_filter in p.get("field_filters") or []: + if not isinstance(field_filter, dict) or not field_filter.get("condition_type"): + raise ValueError("Each field_filters entry needs a condition_type") + _require_known_keys(field_filter, _FIELD_FILTER_KEYS, "field_filters entry") + filters.append( + GovernArtifactFilterFieldValue( + field_filter["condition_type"], + condition=field_filter.get("condition"), + field_id=field_filter.get("field_id"), + negate_condition=field_filter.get("negate"), + case_sensitive=field_filter.get("case_sensitive"), + ) + ) + if p.get("archived") is not None: + filters.append(GovernArtifactFilterArchivedStatus(bool(p["archived"]))) + + sort = None + sort_spec = p.get("sort") + if sort_spec: + _require_known_keys(sort_spec, _SORT_KEYS, "sort") + sort_type = sort_spec.get("type", "name") + direction = sort_spec.get("direction", "ASC") + if sort_type == "name": + sort = GovernArtifactSearchSortName(direction) + elif sort_type == "workflow": + sort = GovernArtifactSearchSortWorkflow(direction) + elif sort_type == "field": + for field in sort_spec.get("fields") or []: + _require_known_keys(field, _SORT_FIELD_KEYS, "sort.fields entry") + # The SDK puts these into the request body as given, so build them. + fields = [ + GovernArtifactSearchSortFieldDefinition( + field["blueprint_id"], field["field_id"] + ).build() + for field in sort_spec.get("fields") or [] + ] + if not fields: + raise ValueError("sort.fields is required when sort.type is field") + sort = GovernArtifactSearchSortField(fields=fields, direction=direction) + else: + raise ValueError("sort.type must be name, workflow or field") + + page_size = int(p["page_size"]) if p.get("page_size") is not None else 20 + max_results = int(p["max_results"]) if p.get("max_results") is not None else 100 + if page_size <= 0 or max_results <= 0: + raise ValueError("page_size and max_results must be positive") + + request = client.new_artifact_search_request( + GovernArtifactSearchQuery(artifact_filters=filters, artifact_search_sort=sort) + ) + hits: list[dict[str, Any]] = [] + has_more = False + while True: + batch = request.fetch_next_batch(page_size=page_size).get_response_hits() + if not batch: + break + for hit in batch: + if len(hits) >= max_results: + has_more = True + break + # A raw hit embeds the whole blueprint version (about 20 KB). + artifact = hit.get_raw().get("artifact", {}) + hits.append( + { + key: artifact[key] + for key in ("id", "name", "blueprintVersionId", "status") + if key in artifact + } + ) + if has_more or len(batch) < page_size: + break + return {"count": len(hits), "has_more": has_more, "hits": hits} + + +@_register( + "get_artifact", + domain="artifacts", + kind="read", + description="Get an artifact definition: name, fields, workflow state and blueprint version.", + sdk="GovernArtifact.get_definition", + params=(ARTIFACT_ID,), +) +def _get_artifact(client, p): + return client.get_artifact(p["artifact_id"]).get_definition().get_raw() + + +@_register( + "create_artifact", + domain="artifacts", + kind="write", + description="Create an artifact on an ACTIVE blueprint version.", + sdk="GovernClient.create_artifact", + params=( + _p( + "blueprint_id", + "string", + "Blueprint ID. Required unless definition is given.", + ), + _p( + "version_id", + "string", + "ACTIVE blueprint version ID. Required unless definition is given.", + ), + _p("name", "string", "Artifact name. Required unless definition is given."), + _p( + "fields", + "object", + "Field values keyed by field ID; list fields take arrays.", + ), + _p( + "definition", + "object", + "Complete artifact payload; overrides the other params.", + ), + ), +) +def _create_artifact(client, p): + definition = p.get("definition") + if definition is None: + for key in ("blueprint_id", "version_id", "name"): + if not p.get(key): + raise ValueError(f"'{key}' is required when 'definition' is not given") + definition = { + "blueprintVersionId": { + "blueprintId": p["blueprint_id"], + "versionId": p["version_id"], + }, + "name": p["name"], + "fields": p.get("fields") or {}, + } + return client.create_artifact(definition).get_definition().get_raw() + + +@_register( + "update_artifact", + domain="artifacts", + kind="write", + description="Replace the complete artifact definition; read it first and send everything back.", + sdk="GovernArtifactDefinition.save", + params=( + ARTIFACT_ID, + _p( + "definition", + "object", + "Complete artifact definition from get_artifact.", + True, + ), + ), +) +def _update_artifact(client, p): + artifact = client.get_artifact(p["artifact_id"]) + _replace_raw(artifact.get_definition(), p["definition"]).save() + return artifact.get_definition().get_raw() + + +@_register( + "update_artifact_fields", + domain="artifacts", + kind="write", + description="Merge field values (and optionally the name) into an artifact, preserving the rest.", + sdk="GovernArtifactDefinition.save", + params=( + ARTIFACT_ID, + _p("fields", "object", "Field values to set, keyed by field ID.", True), + _p("name", "string", "New artifact name."), + ), +) +def _update_artifact_fields(client, p): + artifact = client.get_artifact(p["artifact_id"]) + definition = artifact.get_definition() + raw = definition.get_raw() + raw.setdefault("fields", {}).update(p["fields"]) + if p.get("name"): + raw["name"] = p["name"] + definition.save() + return artifact.get_definition().get_raw() + + +@_register( + "delete_artifact", + domain="artifacts", + kind="delete", + description="Delete an artifact.", + sdk="GovernArtifact.delete", + params=(ARTIFACT_ID,), +) +def _delete_artifact(client, p): + client.get_artifact(p["artifact_id"]).delete() + return {"deleted": p["artifact_id"]} + + +# --------------------------------------------------------------------------- +# signoffs +# --------------------------------------------------------------------------- + + +@_register( + "list_signoffs", + domain="signoffs", + kind="read", + description="List the signoffs of an artifact with their step and status.", + sdk="GovernArtifact.list_signoffs", + params=(ARTIFACT_ID,), +) +def _list_signoffs(client, p): + return [ + item.get_raw() for item in client.get_artifact(p["artifact_id"]).list_signoffs() + ] + + +@_register( + "get_signoff", + domain="signoffs", + kind="read", + description="Get a signoff definition: status, feedback responses and approver response.", + sdk="GovernArtifactSignoff.get_definition", + params=(ARTIFACT_ID, STEP_ID), +) +def _get_signoff(client, p): + return _signoff(client, p).get_definition().get_raw() + + +@_register( + "get_signoff_details", + domain="signoffs", + kind="read", + description="Get a signoff with reviewer and approver membership resolved to users.", + sdk="GovernArtifactSignoff.get_details", + params=(ARTIFACT_ID, STEP_ID), +) +def _get_signoff_details(client, p): + return _signoff(client, p).get_details().get_raw() + + +@_register( + "create_signoff", + domain="signoffs", + kind="write", + description="Create the signoff of an ONGOING step whose blueprint version has a signoff configuration.", + sdk="GovernArtifact.create_signoff", + params=(ARTIFACT_ID, STEP_ID), +) +def _create_signoff(client, p): + artifact = client.get_artifact(p["artifact_id"]) + return artifact.create_signoff(p["step_id"]).get_definition().get_raw() + + +@_register( + "update_signoff_status", + domain="signoffs", + kind="write", + description="Move a signoff to NOT_STARTED, WAITING_FOR_FEEDBACK, WAITING_FOR_APPROVAL, APPROVED, REJECTED or ABANDONED.", + sdk="GovernArtifactSignoff.update_status", + params=( + ARTIFACT_ID, + STEP_ID, + _p("status", "string", "Target signoff status.", True), + _p( + "users_to_notify", + "list", + "Users to email, each {userLogin, groupId}; omitted or [] sends no notifications.", + ), + _p( + "reload_conf_for_reset", + "boolean", + "On NOT_STARTED, reload the configuration from the blueprint version and drop delegations.", + ), + ), +) +def _update_signoff_status(client, p): + signoff = _signoff(client, p) + signoff.update_status( + p["status"], + users_to_notify=p.get("users_to_notify") or [], + reload_conf_for_reset=bool(p.get("reload_conf_for_reset", False)), + ) + return signoff.get_definition().get_raw() + + +@_register( + "list_signoff_feedbacks", + domain="signoffs", + kind="read", + description="List the feedback responses recorded on a signoff.", + sdk="GovernArtifactSignoff.list_feedbacks", + params=(ARTIFACT_ID, STEP_ID), +) +def _list_signoff_feedbacks(client, p): + return [item.get_raw() for item in _signoff(client, p).list_feedbacks()] + + +FEEDBACK_ID = _p("feedback_id", "string", "Feedback response ID.", True, path=True) +FEEDBACK_STATUS = _p( + "status", "string", "Feedback status: APPROVED, MINOR_ISSUE or MAJOR_ISSUE.", True +) +APPROVAL_STATUS = _p( + "status", "string", "Approval status: APPROVED, REJECTED or ABANDONED.", True +) + + +@_register( + "get_signoff_feedback", + domain="signoffs", + kind="read", + description="Get one feedback response of a signoff.", + sdk="GovernArtifactSignoffFeedback.get_definition", + params=(ARTIFACT_ID, STEP_ID, FEEDBACK_ID), +) +def _get_signoff_feedback(client, p): + return _signoff(client, p).get_feedback(p["feedback_id"]).get_definition().get_raw() + + +@_register( + "add_signoff_feedback", + domain="signoffs", + kind="write", + description="Record a feedback response as the authenticated user, in a feedback group.", + sdk="GovernArtifactSignoff.add_feedback", + params=( + ARTIFACT_ID, + STEP_ID, + _p( + "group_id", + "string", + "Feedback group ID from the signoff configuration.", + True, + ), + FEEDBACK_STATUS, + COMMENT, + ), +) +def _add_signoff_feedback(client, p): + feedback = _signoff(client, p).add_feedback( + p["group_id"], p["status"], comment=p.get("comment") + ) + return feedback.get_definition().get_raw() + + +@_register( + "update_signoff_feedback", + domain="signoffs", + kind="write", + description="Change the status or comment of an existing feedback response.", + sdk="GovernArtifactSignoffFeedbackDefinition.save", + params=(ARTIFACT_ID, STEP_ID, FEEDBACK_ID, FEEDBACK_STATUS, COMMENT), +) +def _update_signoff_feedback(client, p): + definition = _signoff(client, p).get_feedback(p["feedback_id"]).get_definition() + raw = definition.get_raw() + raw["status"] = p["status"] + if p.get("comment") is not None: + raw["comment"] = p["comment"] + definition.save() + return definition.get_raw() + + +@_register( + "delete_signoff_feedback", + domain="signoffs", + kind="delete", + description="Delete a feedback response; the signoff must be WAITING_FOR_FEEDBACK.", + sdk="GovernArtifactSignoffFeedback.delete", + params=(ARTIFACT_ID, STEP_ID, FEEDBACK_ID), +) +def _delete_signoff_feedback(client, p): + _signoff(client, p).get_feedback(p["feedback_id"]).delete() + return {"deleted": p["feedback_id"]} + + +@_register( + "delegate_signoff_feedback", + domain="signoffs", + kind="write", + description="Add a delegated reviewer (one user) to a feedback group of a signoff.", + sdk="GovernArtifactSignoff.delegate_feedback", + params=( + ARTIFACT_ID, + STEP_ID, + _p("group_id", "string", "Feedback group ID.", True), + USERS_CONTAINER, + ), +) +def _delegate_signoff_feedback(client, p): + signoff = _signoff(client, p) + signoff.delegate_feedback(p["group_id"], _users_container(p["users_container"])) + return signoff.get_details().get_raw() + + +@_register( + "get_signoff_approval", + domain="signoffs", + kind="read", + description="Get the approver response of a signoff.", + sdk="GovernArtifactSignoffApproval.get_definition", + params=(ARTIFACT_ID, STEP_ID), +) +def _get_signoff_approval(client, p): + return _signoff(client, p).get_approval().get_definition().get_raw() + + +@_register( + "add_signoff_approval", + domain="signoffs", + kind="write", + description="Record the final approval decision as the authenticated user.", + sdk="GovernArtifactSignoff.add_approval", + params=(ARTIFACT_ID, STEP_ID, APPROVAL_STATUS, COMMENT), +) +def _add_signoff_approval(client, p): + signoff = _signoff(client, p) + signoff.add_approval(p["status"], comment=p.get("comment")) + return signoff.get_approval().get_definition().get_raw() + + +@_register( + "update_signoff_approval", + domain="signoffs", + kind="write", + description="Change the status or comment of the recorded approval.", + sdk="GovernArtifactSignoffApprovalDefinition.save", + params=(ARTIFACT_ID, STEP_ID, APPROVAL_STATUS, COMMENT), +) +def _update_signoff_approval(client, p): + definition = _signoff(client, p).get_approval().get_definition() + raw = definition.get_raw() + raw["status"] = p["status"] + if p.get("comment") is not None: + raw["comment"] = p["comment"] + definition.save() + return definition.get_raw() + + +@_register( + "delete_signoff_approval", + domain="signoffs", + kind="delete", + description="Delete the recorded approval; the signoff must be APPROVED, REJECTED or ABANDONED.", + sdk="GovernArtifactSignoffApproval.delete", + params=(ARTIFACT_ID, STEP_ID), +) +def _delete_signoff_approval(client, p): + _signoff(client, p).get_approval().delete() + return { + "deleted": "approval", + "artifact_id": p["artifact_id"], + "step_id": p["step_id"], + } + + +@_register( + "delegate_signoff_approval", + domain="signoffs", + kind="write", + description="Add a delegated approver (one user) to a signoff.", + sdk="GovernArtifactSignoff.delegate_approval", + params=(ARTIFACT_ID, STEP_ID, USERS_CONTAINER), +) +def _delegate_signoff_approval(client, p): + signoff = _signoff(client, p) + signoff.delegate_approval(_users_container(p["users_container"])) + return signoff.get_details().get_raw() + + +@_register( + "get_signoff_recurrence", + domain="signoffs", + kind="read", + description="Get the recurrence (scheduled reset) configuration of a signoff.", + sdk="GovernArtifactSignoff.get_recurrence_configuration", + params=(ARTIFACT_ID, STEP_ID), +) +def _get_signoff_recurrence(client, p): + return _signoff(client, p).get_recurrence_configuration().get_raw() + + +@_register( + "update_signoff_recurrence", + domain="signoffs", + kind="write", + description="Replace the recurrence configuration of a signoff: {activated, days, weeks, months, years, reloadConf}.", + sdk="GovernArtifactSignoffRecurrenceConfiguration.save", + params=( + ARTIFACT_ID, + STEP_ID, + _p("configuration", "object", "Complete recurrence configuration.", True), + ), +) +def _update_signoff_recurrence(client, p): + recurrence = _signoff(client, p).get_recurrence_configuration() + _replace_raw(recurrence, p["configuration"]).save() + return recurrence.get_raw() + + +# --------------------------------------------------------------------------- +# blueprints +# --------------------------------------------------------------------------- + + +@_register( + "list_blueprints", + domain="blueprints", + kind="read", + description="List readable blueprints; each item nests the blueprint under 'blueprint'.", + sdk="GovernClient.list_blueprints", +) +def _list_blueprints(client, p): + return [item.get_raw() for item in client.list_blueprints()] + + +@_register( + "get_blueprint", + domain="blueprints", + kind="read", + description="Get a blueprint: name, icon and colors.", + sdk="GovernBlueprint.get_definition", + params=(BLUEPRINT_ID,), +) +def _get_blueprint(client, p): + return client.get_blueprint(p["blueprint_id"]).get_definition().get_raw() + + +@_register( + "list_blueprint_versions", + domain="blueprints", + kind="read", + description="List the versions of a blueprint with their status.", + sdk="GovernBlueprint.list_versions", + params=(BLUEPRINT_ID,), +) +def _list_blueprint_versions(client, p): + return [ + item.get_raw() + for item in client.get_blueprint(p["blueprint_id"]).list_versions() + ] + + +@_register( + "get_blueprint_version", + domain="blueprints", + kind="read", + description="Get a blueprint version definition (fields, workflow, views, hooks) and its trace (status, origin).", + sdk="GovernBlueprintVersion.get_definition", + params=(BLUEPRINT_ID, VERSION_ID), +) +def _get_blueprint_version(client, p): + version = client.get_blueprint(p["blueprint_id"]).get_version(p["version_id"]) + return _version_state(version) + + +@_register( + "create_blueprint", + domain="blueprints", + kind="write", + description="Create a blueprint (designer). The body holds name, icon, color and backgroundColor, no id.", + sdk="GovernAdminBlueprintDesigner.create_blueprint", + params=(NEW_IDENTIFIER, _p("blueprint", "object", "Blueprint metadata.", True)), +) +def _create_blueprint(client, p): + designer = client.get_blueprint_designer() + blueprint = designer.create_blueprint(p["new_identifier"], p["blueprint"]) + return blueprint.get_definition().get_raw() + + +@_register( + "update_blueprint", + domain="blueprints", + kind="write", + description="Replace the blueprint metadata (name, icon, colors); versions are edited separately.", + sdk="GovernAdminBlueprintDefinition.save", + params=( + BLUEPRINT_ID, + _p("blueprint", "object", "Complete blueprint metadata.", True), + ), +) +def _update_blueprint(client, p): + blueprint = _admin_blueprint(client, p) + _replace_raw( + blueprint.get_definition(), _with_id(p["blueprint"], p["blueprint_id"]) + ).save() + return blueprint.get_definition().get_raw() + + +@_register( + "delete_blueprint", + domain="blueprints", + kind="delete", + description="Delete a blueprint; its versions must hold no artifacts.", + sdk="GovernAdminBlueprint.delete", + params=(BLUEPRINT_ID,), +) +def _delete_blueprint(client, p): + _admin_blueprint(client, p).delete() + return {"deleted": p["blueprint_id"]} + + +@_register( + "create_blueprint_version", + domain="blueprints", + kind="write", + description="Create a DRAFT blueprint version, optionally forked from an origin version.", + sdk="GovernAdminBlueprint.create_version", + params=( + BLUEPRINT_ID, + NEW_IDENTIFIER, + _p("name", "string", "Display name of the version."), + _p( + "origin_version_id", + "string", + "Version to copy fields, workflow and views from.", + ), + ), +) +def _create_blueprint_version(client, p): + version = _admin_blueprint(client, p).create_version( + p["new_identifier"], + name=p.get("name"), + origin_version_id=p.get("origin_version_id"), + ) + return _version_state(version) + + +@_register( + "update_blueprint_version", + domain="blueprints", + kind="write", + description="Replace a blueprint version definition; danger_zone_accepted lets a field removal or type change discard artifact data.", + sdk="GovernAdminBlueprintVersionDefinition.save", + params=( + BLUEPRINT_ID, + VERSION_ID, + _p( + "definition", + "object", + "The definition object of get_blueprint_version (not the whole result), edited.", + True, + ), + _p( + "danger_zone_accepted", + "boolean", + "Accept data loss on artifacts of this version.", + ), + ), +) +def _update_blueprint_version(client, p): + if "definition" in p["definition"] and "trace" in p["definition"]: + raise ValueError( + "Pass the 'definition' object of get_blueprint_version, not the whole " + "result: saving the result would erase the version's fields and workflow." + ) + version = _admin_version(client, p) + definition = _replace_raw(version.get_definition(), p["definition"]) + definition.save(danger_zone_accepted=p.get("danger_zone_accepted")) + return _version_state(version) + + +@_register( + "set_blueprint_version_status", + domain="blueprints", + kind="write", + description="Set a blueprint version status: DRAFT, ACTIVE or ARCHIVED.", + sdk="GovernAdminBlueprintVersionTrace.set_status", + params=(BLUEPRINT_ID, VERSION_ID, _p("status", "string", "Target status.", True)), +) +def _set_blueprint_version_status(client, p): + version = _admin_version(client, p) + version.get_trace().set_status(p["status"]) + return version.get_trace().get_raw() + + +@_register( + "delete_blueprint_version", + domain="blueprints", + kind="delete", + description="Delete a blueprint version; it must hold no artifacts.", + sdk="GovernAdminBlueprintVersion.delete", + params=(BLUEPRINT_ID, VERSION_ID), +) +def _delete_blueprint_version(client, p): + _admin_version(client, p).delete() + return {"deleted": p["version_id"], "blueprint_id": p["blueprint_id"]} + + +@_register( + "list_signoff_configurations", + domain="blueprints", + kind="read", + description="List the signoff configurations (review gates) of a blueprint version.", + sdk="GovernAdminBlueprintVersion.list_signoff_configurations", + params=(BLUEPRINT_ID, VERSION_ID), +) +def _list_signoff_configurations(client, p): + return [ + item.get_raw() + for item in _admin_version(client, p).list_signoff_configurations() + ] + + +@_register( + "get_signoff_configuration", + domain="blueprints", + kind="read", + description="Get the signoff configuration of one workflow step.", + sdk="GovernAdminSignoffConfiguration.get_definition", + params=(BLUEPRINT_ID, VERSION_ID, STEP_ID), +) +def _get_signoff_configuration(client, p): + configuration = _admin_version(client, p).get_signoff_configuration(p["step_id"]) + return configuration.get_definition().get_raw() + + +@_register( + "create_signoff_configuration", + domain="blueprints", + kind="write", + description="Create a review gate on a workflow step: {title, mandatory, feedbackUsersGroups, approvers, recurrenceConfiguration}, no id.", + sdk="GovernAdminBlueprintVersion.create_signoff_configuration", + params=( + BLUEPRINT_ID, + VERSION_ID, + STEP_ID, + _p("configuration", "object", "Signoff configuration without an id.", True), + ), +) +def _create_signoff_configuration(client, p): + configuration = _admin_version(client, p).create_signoff_configuration( + p["step_id"], p["configuration"] + ) + return configuration.get_definition().get_raw() + + +@_register( + "update_signoff_configuration", + domain="blueprints", + kind="write", + description="Replace the signoff configuration of a workflow step.", + sdk="GovernAdminSignoffConfigurationDefinition.save", + params=( + BLUEPRINT_ID, + VERSION_ID, + STEP_ID, + _p("configuration", "object", "Complete signoff configuration.", True), + ), +) +def _update_signoff_configuration(client, p): + configuration = _admin_version(client, p).get_signoff_configuration(p["step_id"]) + _replace_raw(configuration.get_definition(), p["configuration"]).save() + return configuration.get_definition().get_raw() + + +@_register( + "delete_signoff_configuration", + domain="blueprints", + kind="delete", + description="Delete the signoff configuration of a workflow step.", + sdk="GovernAdminSignoffConfiguration.delete", + params=(BLUEPRINT_ID, VERSION_ID, STEP_ID), +) +def _delete_signoff_configuration(client, p): + _admin_version(client, p).get_signoff_configuration(p["step_id"]).delete() + return { + "deleted": p["step_id"], + "blueprint_id": p["blueprint_id"], + "version_id": p["version_id"], + } + + +# --------------------------------------------------------------------------- +# roles +# --------------------------------------------------------------------------- + + +@_register( + "list_roles", + domain="roles", + kind="read", + description="List roles.", + sdk="GovernAdminRolesPermissionsHandler.list_roles", +) +def _list_roles(client, p): + return [item.get_raw() for item in _roles(client).list_roles()] + + +@_register( + "get_role", + domain="roles", + kind="read", + description="Get a role definition.", + sdk="GovernAdminRole.get_definition", + params=(ROLE_ID,), +) +def _get_role(client, p): + return _roles(client).get_role(p["role_id"]).get_definition().get_raw() + + +@_register( + "create_role", + domain="roles", + kind="write", + description="Create a role: {label, description}.", + sdk="GovernAdminRolesPermissionsHandler.create_role", + params=( + NEW_IDENTIFIER, + _p("role", "object", "Role definition without an id.", True), + ), +) +def _create_role(client, p): + return ( + _roles(client) + .create_role(p["new_identifier"], p["role"]) + .get_definition() + .get_raw() + ) + + +@_register( + "update_role", + domain="roles", + kind="write", + description="Replace a role definition.", + sdk="GovernAdminRoleDefinition.save", + params=(ROLE_ID, _p("role", "object", "Complete role definition.", True)), +) +def _update_role(client, p): + role = _roles(client).get_role(p["role_id"]) + _replace_raw(role.get_definition(), _with_id(p["role"], p["role_id"])).save() + return role.get_definition().get_raw() + + +@_register( + "delete_role", + domain="roles", + kind="delete", + description="Delete a role.", + sdk="GovernAdminRole.delete", + params=(ROLE_ID,), +) +def _delete_role(client, p): + _roles(client).get_role(p["role_id"]).delete() + return {"deleted": p["role_id"]} + + +@_register( + "list_role_assignments", + domain="roles", + kind="read", + description="List the role assignments of every blueprint.", + sdk="GovernAdminRolesPermissionsHandler.list_role_assignments", +) +def _list_role_assignments(client, p): + return [item.get_raw() for item in _roles(client).list_role_assignments()] + + +@_register( + "get_role_assignments", + domain="roles", + kind="read", + description="Get the role assignments of a blueprint: roleAssignmentsRules keyed by role ID.", + sdk="GovernAdminBlueprintRoleAssignments.get_definition", + params=(BLUEPRINT_ID,), +) +def _get_role_assignments(client, p): + return ( + _roles(client) + .get_role_assignments(p["blueprint_id"]) + .get_definition() + .get_raw() + ) + + +@_register( + "create_role_assignments", + domain="roles", + kind="write", + description="Create the role assignments of a blueprint: {blueprintId, roleAssignmentsRules}.", + sdk="GovernAdminRolesPermissionsHandler.create_role_assignments", + params=( + _p( + "role_assignments", + "object", + "Role assignments including blueprintId.", + True, + ), + ), +) +def _create_role_assignments(client, p): + assignments = _roles(client).create_role_assignments(p["role_assignments"]) + return assignments.get_definition().get_raw() + + +@_register( + "update_role_assignments", + domain="roles", + kind="write", + description="Replace the role assignments of a blueprint; preserve the other roles' rules.", + sdk="GovernAdminBlueprintRoleAssignmentsDefinition.save", + params=( + BLUEPRINT_ID, + _p("role_assignments", "object", "Complete role assignments.", True), + ), +) +def _update_role_assignments(client, p): + assignments = _roles(client).get_role_assignments(p["blueprint_id"]) + _replace_raw(assignments.get_definition(), p["role_assignments"]).save() + return assignments.get_definition().get_raw() + + +@_register( + "delete_role_assignments", + domain="roles", + kind="delete", + description="Delete every role assignment of a blueprint.", + sdk="GovernAdminBlueprintRoleAssignments.delete", + params=(BLUEPRINT_ID,), +) +def _delete_role_assignments(client, p): + _roles(client).get_role_assignments(p["blueprint_id"]).delete() + return {"deleted": "role_assignments", "blueprint_id": p["blueprint_id"]} + + +@_register( + "get_default_blueprint_permissions", + domain="roles", + kind="read", + description="Get the default permissions applied to blueprints without their own.", + sdk="GovernAdminRolesPermissionsHandler.get_default_permissions_definition", +) +def _get_default_blueprint_permissions(client, p): + return _roles(client).get_default_permissions_definition().get_raw() + + +@_register( + "update_default_blueprint_permissions", + domain="roles", + kind="write", + description="Replace the default blueprint permissions.", + sdk="GovernAdminDefaultPermissionsDefinition.save", + params=(_p("permissions", "object", "Complete default permissions.", True),), +) +def _update_default_blueprint_permissions(client, p): + definition = _roles(client).get_default_permissions_definition() + _replace_raw(definition, p["permissions"]).save() + return _roles(client).get_default_permissions_definition().get_raw() + + +@_register( + "list_blueprint_permissions", + domain="roles", + kind="read", + description="List the blueprints that carry their own permissions.", + sdk="GovernAdminRolesPermissionsHandler.list_blueprint_permissions", +) +def _list_blueprint_permissions(client, p): + return [item.get_raw() for item in _roles(client).list_blueprint_permissions()] + + +@_register( + "get_blueprint_permissions", + domain="roles", + kind="read", + description="Get the permissions of a blueprint.", + sdk="GovernAdminBlueprintPermissions.get_definition", + params=(BLUEPRINT_ID,), +) +def _get_blueprint_permissions(client, p): + return ( + _roles(client) + .get_blueprint_permissions(p["blueprint_id"]) + .get_definition() + .get_raw() + ) + + +@_register( + "create_blueprint_permissions", + domain="roles", + kind="write", + description="Create the permissions of a blueprint: {blueprintId, permissions}.", + sdk="GovernAdminRolesPermissionsHandler.create_blueprint_permissions", + params=( + _p( + "blueprint_permissions", + "object", + "Permissions including blueprintId.", + True, + ), + ), +) +def _create_blueprint_permissions(client, p): + permissions = _roles(client).create_blueprint_permissions( + p["blueprint_permissions"] + ) + return permissions.get_definition().get_raw() + + +@_register( + "update_blueprint_permissions", + domain="roles", + kind="write", + description="Replace the permissions of a blueprint.", + sdk="GovernAdminBlueprintPermissionsDefinition.save", + params=( + BLUEPRINT_ID, + _p("blueprint_permissions", "object", "Complete permissions.", True), + ), +) +def _update_blueprint_permissions(client, p): + permissions = _roles(client).get_blueprint_permissions(p["blueprint_id"]) + _replace_raw(permissions.get_definition(), p["blueprint_permissions"]).save() + return permissions.get_definition().get_raw() + + +@_register( + "delete_blueprint_permissions", + domain="roles", + kind="delete", + description="Delete the permissions of a blueprint so the defaults apply again.", + sdk="GovernAdminBlueprintPermissions.delete", + params=(BLUEPRINT_ID,), +) +def _delete_blueprint_permissions(client, p): + _roles(client).get_blueprint_permissions(p["blueprint_id"]).delete() + return {"deleted": "blueprint_permissions", "blueprint_id": p["blueprint_id"]} + + +# --------------------------------------------------------------------------- +# custom pages +# --------------------------------------------------------------------------- + + +@_register( + "list_custom_pages", + domain="custom_pages", + kind="read", + description="List the custom pages visible to the authenticated user.", + sdk="GovernClient.list_custom_pages", +) +def _list_custom_pages(client, p): + return [item.get_raw() for item in client.list_custom_pages()] + + +@_register( + "get_custom_page", + domain="custom_pages", + kind="read", + description="Get a custom page as the authenticated user sees it.", + sdk="GovernCustomPage.get_definition", + params=(CUSTOM_PAGE_ID,), +) +def _get_custom_page(client, p): + return client.get_custom_page(p["custom_page_id"]).get_definition().get_raw() + + +@_register( + "get_custom_page_definition", + domain="custom_pages", + kind="read", + description="Get the editable definition of a custom page (designer).", + sdk="GovernAdminCustomPage.get_definition", + params=(CUSTOM_PAGE_ID,), +) +def _get_custom_page_definition(client, p): + return ( + _pages(client).get_custom_page(p["custom_page_id"]).get_definition().get_raw() + ) + + +@_register( + "create_custom_page", + domain="custom_pages", + kind="write", + description="Create a custom page; copy the definition shape from an existing page of the same type.", + sdk="GovernAdminCustomPagesHandler.create_custom_page", + params=( + NEW_IDENTIFIER, + _p("custom_page", "object", "Custom page definition without an id.", True), + ), +) +def _create_custom_page(client, p): + page = _pages(client).create_custom_page(p["new_identifier"], p["custom_page"]) + return page.get_definition().get_raw() + + +@_register( + "update_custom_page", + domain="custom_pages", + kind="write", + description="Replace a custom page definition.", + sdk="GovernAdminCustomPageDefinition.save", + params=( + CUSTOM_PAGE_ID, + _p("custom_page", "object", "Complete custom page definition.", True), + ), +) +def _update_custom_page(client, p): + page = _pages(client).get_custom_page(p["custom_page_id"]) + _replace_raw( + page.get_definition(), _with_id(p["custom_page"], p["custom_page_id"]) + ).save() + return page.get_definition().get_raw() + + +@_register( + "delete_custom_page", + domain="custom_pages", + kind="delete", + description="Delete a custom page.", + sdk="GovernAdminCustomPage.delete", + params=(CUSTOM_PAGE_ID,), +) +def _delete_custom_page(client, p): + _pages(client).get_custom_page(p["custom_page_id"]).delete() + return {"deleted": p["custom_page_id"]} + + +@_register( + "get_custom_pages_order", + domain="custom_pages", + kind="read", + description="Get the display order of the custom pages as a list of IDs.", + sdk="GovernAdminCustomPagesHandler.get_custom_pages_order", +) +def _get_custom_pages_order(client, p): + return _pages(client).get_custom_pages_order() + + +@_register( + "update_custom_pages_order", + domain="custom_pages", + kind="write", + description="Set the display order of the custom pages; the list must contain every page ID.", + sdk="GovernAdminCustomPagesHandler.save_custom_pages_order", + params=(_p("order", "list", "Custom page IDs in display order.", True),), +) +def _update_custom_pages_order(client, p): + return _pages(client).save_custom_pages_order(list(p["order"])) + + +# --------------------------------------------------------------------------- +# time series +# --------------------------------------------------------------------------- + +DATAPOINTS = _p( + "datapoints", + "list", + "Datapoints, each {timestamp: epoch milliseconds, value: number}.", +) +MIN_TIMESTAMP = _p("min_timestamp", "integer", "Lower bound, epoch milliseconds.") +MAX_TIMESTAMP = _p("max_timestamp", "integer", "Upper bound, epoch milliseconds.") + + +@_register( + "create_time_series", + domain="time_series", + kind="write", + description="Create a time series, optionally with initial datapoints; returns its ID.", + sdk="GovernClient.create_time_series", + params=(DATAPOINTS,), +) +def _create_time_series(client, p): + datapoints = list(p.get("datapoints") or []) + time_series = client.create_time_series(datapoints=datapoints) + return {"time_series_id": time_series.time_series_id, "datapoints": len(datapoints)} + + +@_register( + "get_time_series_values", + domain="time_series", + kind="read", + description="Get the datapoints of a time series, optionally within a timestamp window.", + sdk="GovernTimeSeries.get_values", + params=(TIME_SERIES_ID, MIN_TIMESTAMP, MAX_TIMESTAMP), +) +def _get_time_series_values(client, p): + return client.get_time_series(p["time_series_id"]).get_values( + min_timestamp=p.get("min_timestamp"), max_timestamp=p.get("max_timestamp") + ) + + +@_register( + "push_time_series_values", + domain="time_series", + kind="write", + description="Push datapoints into a time series; upsert overwrites existing timestamps.", + sdk="GovernTimeSeries.push_values", + params=( + TIME_SERIES_ID, + _p("datapoints", "list", "Datapoints, each {timestamp, value}.", True), + _p("upsert", "boolean", "Overwrite existing timestamps (default false)."), + ), +) +def _push_time_series_values(client, p): + datapoints = list(p["datapoints"]) + client.get_time_series(p["time_series_id"]).push_values( + datapoints, upsert=p.get("upsert") is True + ) + return {"time_series_id": p["time_series_id"], "pushed": len(datapoints)} + + +@_register( + "delete_time_series_values", + domain="time_series", + kind="delete", + description="Delete datapoints of a time series within a window, or all of them without bounds.", + sdk="GovernTimeSeries.delete", + params=(TIME_SERIES_ID, MIN_TIMESTAMP, MAX_TIMESTAMP), +) +def _delete_time_series_values(client, p): + client.get_time_series(p["time_series_id"]).delete( + min_timestamp=p.get("min_timestamp"), max_timestamp=p.get("max_timestamp") + ) + return { + "time_series_id": p["time_series_id"], + "deleted": True, + "min_timestamp": p.get("min_timestamp"), + "max_timestamp": p.get("max_timestamp"), + } + + +# --------------------------------------------------------------------------- +# files +# --------------------------------------------------------------------------- + + +@_register( + "upload_file", + domain="files", + kind="write", + description="Upload a local file; attach the returned ID to an UPLOADED_FILE field afterwards.", + sdk="GovernClient.upload_file", + params=( + _p("file_path", "string", "Local path of the file to upload.", True), + _p( + "file_name", + "string", + "Name stored in Govern (default: the local file name).", + ), + ), +) +def _upload_file(client, p): + path = os.path.realpath( + os.path.expanduser(require_non_empty_string(p["file_path"], "file_path")) + ) + if not os.path.isfile(path): + raise FileNotFoundError(f"File not found: {path}") + file_name = p.get("file_name") or os.path.basename(path) + with open(path, "rb") as handle: + uploaded = client.upload_file(file_name, handle) + return uploaded.get_description() + + +@_register( + "get_uploaded_file", + domain="files", + kind="read", + description="Get the description (name, size, type) of an uploaded file.", + sdk="GovernUploadedFile.get_description", + params=(UPLOADED_FILE_ID,), +) +def _get_uploaded_file(client, p): + return client.get_uploaded_file(p["uploaded_file_id"]).get_description() + + +@_register( + "download_uploaded_file", + domain="files", + kind="write", + description="Download an uploaded file to a local path; never overwrites unless overwrite is true.", + sdk="GovernUploadedFile.download", + params=( + UPLOADED_FILE_ID, + _p("output_path", "string", "Local destination path.", True), + _p("overwrite", "boolean", "Replace an existing destination file."), + ), +) +def _download_uploaded_file(client, p): + output_path = require_non_empty_string(p["output_path"], "output_path") + overwrite = bool(p.get("overwrite", False)) + absolute_path = os.path.realpath(os.path.expanduser(output_path)) + parent = os.path.dirname(absolute_path) + if not os.path.isdir(parent): + raise FileNotFoundError(f"Destination directory does not exist: {parent}") + if os.path.lexists(absolute_path) and not overwrite: + raise FileExistsError( + f"Destination already exists: {absolute_path}. Set overwrite=true to replace it." + ) + uploaded = client.get_uploaded_file(p["uploaded_file_id"]) + description = uploaded.get_description() + stream = uploaded.download() + temporary_path = None + digest = hashlib.sha256() + size = 0 + try: + with tempfile.NamedTemporaryFile( + prefix=".govern-file-", suffix=".tmp", dir=parent, delete=False + ) as handle: + temporary_path = handle.name + while True: + # The raw stream keeps any gzip encoding unless asked to decode. + chunk = stream.read(_CHUNK_SIZE, decode_content=True) + if not chunk: + break + handle.write(chunk) + digest.update(chunk) + size += len(chunk) + if os.path.lexists(absolute_path) and not overwrite: + raise FileExistsError( + f"Destination already exists: {absolute_path}. Set overwrite=true to replace it." + ) + if overwrite: + os.replace(temporary_path, absolute_path) + else: + os.link(temporary_path, absolute_path) + os.unlink(temporary_path) + temporary_path = None + finally: + close = getattr(stream, "close", None) + if close is not None: + close() + if temporary_path is not None: + try: + os.unlink(temporary_path) + except FileNotFoundError: + pass + return { + "uploaded_file_id": p["uploaded_file_id"], + "output_path": absolute_path, + "size_bytes": size, + "sha256": digest.hexdigest(), + "description": description, + } + + +@_register( + "delete_uploaded_file", + domain="files", + kind="delete", + description="Delete an uploaded file.", + sdk="GovernUploadedFile.delete", + params=(UPLOADED_FILE_ID,), +) +def _delete_uploaded_file(client, p): + client.get_uploaded_file(p["uploaded_file_id"]).delete() + return {"deleted": p["uploaded_file_id"]} + + +# --------------------------------------------------------------------------- +# users and groups +# --------------------------------------------------------------------------- + + +@_register( + "list_users", + domain="users", + kind="read", + description="List Govern users; include_settings adds each user's full settings.", + sdk="GovernClient.list_users", + params=(_p("include_settings", "boolean", "Include full settings per user."),), +) +def _list_users(client, p): + return client.list_users( + as_objects=False, include_settings=bool(p.get("include_settings", False)) + ) + + +@_register( + "get_user", + domain="users", + kind="read", + description="Get a user's settings (admin).", + sdk="GovernUser.get_settings", + params=(LOGIN,), +) +def _get_user(client, p): + return client.get_user(p["login"]).get_settings().get_raw() + + +@_register( + "create_user", + domain="users", + kind="write", + description="Create a Govern user (admin).", + sdk="GovernClient.create_user", + params=( + LOGIN, + _p("password", "string", "Password; required for LOCAL users."), + _p("display_name", "string", "Display name."), + _p( + "source_type", + "string", + "LOCAL (default), LDAP, or another identity source.", + ), + _p("groups", "list", "Group names."), + _p( + "profile", + "string", + "User profile. When omitted, the SDK sends DATA_SCIENTIST; the tool does " + "not check it against the license.", + ), + _p("email", "string", "Email address."), + ), +) +def _create_user(client, p): + kwargs: dict[str, Any] = {} + for key in ("display_name", "source_type", "groups", "profile", "email"): + if p.get(key) is not None: + kwargs[key] = p[key] + user = client.create_user(p["login"], p.get("password"), **kwargs) + return user.get_settings().get_raw() + + +@_register( + "update_user", + domain="users", + kind="write", + description="Merge top-level keys (displayName, email, groups, enabled, userProfile, ...) into a user's settings.", + sdk="GovernUserSettings.save", + params=(LOGIN, _p("settings", "object", "Settings keys to set.", True)), +) +def _update_user(client, p): + user = client.get_user(p["login"]) + settings = user.get_settings() + settings.get_raw().update(p["settings"]) + settings.save() + return user.get_settings().get_raw() + + +@_register( + "delete_user", + domain="users", + kind="delete", + description="Delete a Govern user (admin).", + sdk="GovernUser.delete", + params=( + LOGIN, + _p("allow_self_deletion", "boolean", "Allow deleting the authenticated user."), + ), +) +def _delete_user(client, p): + client.get_user(p["login"]).delete( + allow_self_deletion=bool(p.get("allow_self_deletion", False)) + ) + return {"deleted": p["login"]} + + +@_register( + "get_own_user", + domain="users", + kind="read", + description="Get the settings of the authenticated user.", + sdk="GovernOwnUser.get_settings", +) +def _get_own_user(client, p): + return client.get_own_user().get_settings().get_raw() + + +@_register( + "list_groups", + domain="users", + kind="read", + description="List Govern groups (admin).", + sdk="GovernClient.list_groups", +) +def _list_groups(client, p): + return client.list_groups() + + +@_register( + "get_group", + domain="users", + kind="read", + description="Get a group definition (admin).", + sdk="GovernGroup.get_definition", + params=(GROUP_NAME,), +) +def _get_group(client, p): + return client.get_group(p["name"]).get_definition() + + +@_register( + "create_group", + domain="users", + kind="write", + description="Create a Govern group (admin).", + sdk="GovernClient.create_group", + params=( + GROUP_NAME, + _p("description", "string", "Group description."), + _p("source_type", "string", "LOCAL (default) or LDAP."), + ), +) +def _create_group(client, p): + kwargs: dict[str, Any] = {} + for key in ("description", "source_type"): + if p.get(key) is not None: + kwargs[key] = p[key] + return client.create_group(p["name"], **kwargs).get_definition() + + +@_register( + "update_group", + domain="users", + kind="write", + description="Merge top-level keys (description, isGovernArchitect, ...) into a group definition.", + sdk="GovernGroup.set_definition", + params=(GROUP_NAME, _p("definition", "object", "Definition keys to set.", True)), +) +def _update_group(client, p): + group = client.get_group(p["name"]) + definition = group.get_definition() + definition.update(p["definition"]) + group.set_definition(definition) + return group.get_definition() + + +@_register( + "delete_group", + domain="users", + kind="delete", + description="Delete a Govern group (admin).", + sdk="GovernGroup.delete", + params=(GROUP_NAME,), +) +def _delete_group(client, p): + client.get_group(p["name"]).delete() + return {"deleted": p["name"]} + + +# --------------------------------------------------------------------------- +# instance +# --------------------------------------------------------------------------- + + +@_register( + "get_instance_info", + domain="instance", + kind="read", + description="Get the Govern node identity and version; confirms the URL targets a GOVERN node.", + sdk="GovernClient.get_instance_info", +) +def _get_instance_info(client, p): + return client.get_instance_info().raw + + +@_register( + "get_auth_info", + domain="instance", + kind="read", + description="Get the identity behind the configured API key.", + sdk="GovernClient.get_auth_info", +) +def _get_auth_info(client, p): + return client.get_auth_info() + + +def operation_handler(spec: GovernOperation): + """Bind one SDK operation without accepting arbitrary methods or paths.""" + + async def execute(**params): + params = { + name: value.model_dump(exclude_unset=True) + if isinstance(value, BaseModel) + else value + for name, value in params.items() + } + + def _run(): + client = get_govern_client() + try: + return redact_sensitive_data(spec.run(client, params)) + except (DataikuException, RequestException): + raise RuntimeError( + "Govern request failed. Check permissions, backend version and IDs. For a write, re-read the affected object before retrying; it may have reached Govern." + ) from None + + return compact_json(await run_blocking(_run)) + + return execute diff --git a/dataiku_mcp/tools/instances.py b/dataiku_mcp/tools/instances.py index 13603254..8b826cae 100644 --- a/dataiku_mcp/tools/instances.py +++ b/dataiku_mcp/tools/instances.py @@ -22,10 +22,10 @@ from fastmcp import Context from pydantic import Field -from ..instance_policy import INSTANCE_MANAGEMENT_POLICY, SUPPORTED_DSS_INSTANCE_TYPES -from ..auth import get_dss_client +from ..auth import get_dss_client, get_govern_client from ..config import request, stdio from ..executors import run_blocking +from ..instance_policy import INSTANCE_MANAGEMENT_POLICY from ..server import DSS_INDEPENDENT_TOOL_TAG, mcp from ..setup_server import SESSION_LIFETIME_SECONDS, start_setup_server from .utils.serialization import columnar, compact_json, omit_empty @@ -120,20 +120,29 @@ async def delete_instance(name: InstanceName, ctx: Context) -> str: }, ) async def get_current_instance(ctx: Context) -> str: - """Inspect the active profile and its DSS connection, or report an unsupported node type.""" + """Confirm which instance is active, whether it can be reached, and its version.""" # Strip both credential types from the response, including on failure. current_instance = asdict(request.get_pinned_instance()) current_instance.pop("api_key", None) current_instance.pop("api_ticket", None) current_instance.pop("encrypted_rpc_cert_path", None) - if current_instance["instance_type"] not in SUPPORTED_DSS_INSTANCE_TYPES: - current_instance["connection_status"] = "unsupported" + if current_instance["instance_type"] == "govern" and request.is_http_request(): + current_instance["connection_status"] = "unsupported_authentication" return compact_json(omit_empty(current_instance)) current_instance["connection_status"] = "failed" try: version = await run_blocking( - lambda: get_dss_client().get_instance_info().raw.get("dssVersion") or "" + lambda: ( + ( + get_govern_client() + if current_instance["instance_type"] == "govern" + else get_dss_client() + ) + .get_instance_info() + .raw.get("dssVersion") + or "" + ) ) except Exception: pass diff --git a/dataiku_mcp/tools/operations.py b/dataiku_mcp/tools/operations.py new file mode 100644 index 00000000..e43583fb --- /dev/null +++ b/dataiku_mcp/tools/operations.py @@ -0,0 +1,153 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Stable discovery and effect-separated dispatch for audited operations.""" + +import json +from typing import Literal + +from ..config import request +from ..instance_policy import API_OPERATIONS_POLICY +from ..operation_registry import ( + get_operation, + get_operations, + require_operation, + require_transport, +) +from ..server import DSS_INDEPENDENT_TOOL_TAG, mcp +from .utils.serialization import compact_json + + +@mcp.tool( + meta=API_OPERATIONS_POLICY, + title="Search Operations", + annotations={ + "readOnlyHint": True, + "destructiveHint": False, + "openWorldHint": False, + }, + tags={DSS_INDEPENDENT_TOOL_TAG}, +) +async def search_operations( + query: str = "", + domain: Literal["deployer", "govern", "agent-management"] | None = None, + effect: Literal["read", "write"] | None = None, + limit: int = 10, + offset: int = 0, +) -> str: + """Find relevant operations for the selected instance before loading argument schemas.""" + if not 1 <= limit <= 50: + raise ValueError("limit must be between 1 and 50") + if offset < 0: + raise ValueError("offset must be non-negative") + instance = request.get_pinned_instance() + words = query.casefold().split() + matches = [ + op + for op in get_operations().values() + if instance.instance_type in op.allowed_instance_types + and (domain is None or domain == op.domain) + and (effect is None or effect == op.effect) + and all( + word in (op.operation_id + " " + op.description).casefold() + for word in words + ) + ] + return compact_json( + { + "instance_name": instance.name, + "server_support": "unknown", + "total": len(matches), + "operations": [op.summary() for op in matches[offset : offset + limit]], + "truncated": len(matches) > offset + limit, + "next_offset": offset + limit if len(matches) > offset + limit else None, + } + ) + + +@mcp.tool( + meta=API_OPERATIONS_POLICY, + title="Describe Operation", + annotations={ + "readOnlyHint": True, + "destructiveHint": False, + "openWorldHint": False, + }, + tags={DSS_INDEPENDENT_TOOL_TAG}, +) +async def describe_operation(operation_id: str) -> str: + """Load one operation's input schema, example, policy and backend requirements.""" + operation = get_operation(operation_id) + instance = request.get_pinned_instance() + return compact_json( + { + **operation.describe(), + "instance_name": instance.name, + "allowed_on_selected_instance": instance.instance_type + in operation.allowed_instance_types, + } + ) + + +async def _dispatch( + operation_id: str, + arguments: dict, + effect: Literal["read", "write"], + expected_instance: str | None, +) -> str: + instance = request.get_pinned_instance() + if expected_instance is not None and instance.name != expected_instance: + raise ValueError( + "Selected instance changed. Inspect get_current_instance before retrying." + ) + operation = require_operation(operation_id, effect, instance.instance_type) + values = operation.validate(arguments) + require_transport(operation, values, request.is_http_request()) + result = json.loads(await operation.handler(**values)) + return compact_json( + {"operation_id": operation_id, "instance_name": instance.name, "result": result} + ) + + +@mcp.tool( + meta=API_OPERATIONS_POLICY, + title="Run Read Operation", + annotations={ + "readOnlyHint": True, + "destructiveHint": False, + "openWorldHint": False, + }, +) +async def run_read_operation( + operation_id: str, arguments: dict, expected_instance: str | None = None +) -> str: + """Run a catalogued read; expected_instance rejects a stale instance selection.""" + return await _dispatch(operation_id, arguments, "read", expected_instance) + + +@mcp.tool( + meta=API_OPERATIONS_POLICY, + title="Run Write Operation", + annotations={ + "readOnlyHint": False, + "destructiveHint": True, + "idempotentHint": False, + "openWorldHint": False, + }, +) +async def run_write_operation( + operation_id: str, arguments: dict, expected_instance: str | None = None +) -> str: + """Run a catalogued mutation after reviewing its schema, scope and execution effects.""" + return await _dispatch(operation_id, arguments, "write", expected_instance) diff --git a/dataiku_mcp/tools/utils/redaction.py b/dataiku_mcp/tools/utils/redaction.py new file mode 100644 index 00000000..b63ba91b --- /dev/null +++ b/dataiku_mcp/tools/utils/redaction.py @@ -0,0 +1,111 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Recursive credential redaction for backend operation results.""" + +from typing import Any +from urllib.parse import urlsplit, urlunsplit, parse_qsl, urlencode + +_CONNECTION_SECRET_REDACTION = "__DATAIKU_REDACTED__" +_SENSITIVE_SUFFIXES = ( + "apikey", + "apikeys", + "credential", + "credentials", + "password", + "passwords", + "secret", + "secrets", + "token", + "tokens", +) +_SENSITIVE_EXACT_KEYS = ( + "key", + "authorization", + "cookie", + "setcookie", + "logs", + "stacktrace", + "traceback", + "errordetails", + "errormessage", + "error", + "errors", +) +_SENSITIVE_KEY_FRAGMENTS = ( + "accesskey", + "appsecret", + "keybase64data", + "keyjsondata", + "passphrase", + "privatekey", + "secretkey", + "tokenkey", +) + + +def _is_sensitive_key(key: str) -> bool: + normalized = key.replace("_", "").replace("-", "").lower() + return ( + normalized in _SENSITIVE_EXACT_KEYS + or normalized.endswith(_SENSITIVE_SUFFIXES) + or any(fragment in normalized for fragment in _SENSITIVE_KEY_FRAGMENTS) + ) + + +def redact_sensitive_data(value: Any) -> Any: + if isinstance(value, dict): + redacted = {} + for key, item in value.items(): + if _is_sensitive_key(str(key)): + redacted[key] = _CONNECTION_SECRET_REDACTION + else: + redacted[key] = redact_sensitive_data(item) + return redacted + + if isinstance(value, list): + return [redact_sensitive_data(item) for item in value] + + if isinstance(value, str) and value.startswith(("https://", "http://")): + try: + parts = urlsplit(value) + query = parse_qsl(parts.query, keep_blank_values=True) + if "@" in parts.netloc or any(_is_sensitive_key(key) for key, _ in query): + host = parts.netloc.rsplit("@", 1)[-1] + query = [ + ( + key, + _CONNECTION_SECRET_REDACTION + if _is_sensitive_key(key) + else item, + ) + for key, item in query + ] + return urlunsplit( + (parts.scheme, host, parts.path, urlencode(query), parts.fragment) + ) + except ValueError: + return _CONNECTION_SECRET_REDACTION + return value + + +def contains_redacted_values(value: Any) -> bool: + """Detect result placeholders that must never become replacement credentials.""" + if isinstance(value, str): + return _CONNECTION_SECRET_REDACTION in value + if isinstance(value, dict): + return any(contains_redacted_values(item) for item in value.values()) + if isinstance(value, list): + return any(contains_redacted_values(item) for item in value) + return False diff --git a/dataiku_mcp/tools/utils/validation.py b/dataiku_mcp/tools/utils/validation.py index 99a3cf62..21d2eaf7 100644 --- a/dataiku_mcp/tools/utils/validation.py +++ b/dataiku_mcp/tools/utils/validation.py @@ -15,6 +15,9 @@ """Shared validation helpers for tool modules.""" import re +from typing import Annotated + +from pydantic import AfterValidator, Field from pathlib import PurePosixPath from urllib.parse import urlsplit @@ -147,3 +150,29 @@ def require_relative_posix_path(value: str, field_name: str) -> str: if path.is_absolute() or ".." in path.parts or "\\" in value: raise ValueError(f"'{field_name}' must be a relative path without '..'") return value + + +def require_path_segment(value: str) -> str: + """Reject separators, double-decoding escapes and URL-normalized dot segments.""" + value = value.strip() + if ( + not value + or value in {".", ".."} + or any(char in value for char in "/\\%") + or any(ord(char) < 32 or ord(char) == 127 for char in value) + ): + raise ValueError( + "Expected a non-empty path segment without separators, escapes or dot segments" + ) + return value + + +PathSegment = Annotated[ + str, + Field( + min_length=1, + pattern=r"^[^/\\%\x00-\x1f\x7f]+$", + description="One identifier segment; dot segments, slashes, backslashes and percent escapes are rejected.", + ), + AfterValidator(require_path_segment), +] diff --git a/docs/capabilities.md b/docs/capabilities.md index 7461058c..54b877d0 100644 --- a/docs/capabilities.md +++ b/docs/capabilities.md @@ -22,44 +22,15 @@ Local profile actions are listed separately because they only change which Datai instance the local client targets. -## Instance types - -The configured instance type controls which tools can run in both stdio and HTTP. -The catalog stays the same; each tool advertises its allowed types in metadata, -and dispatch rejects unsupported calls before authentication delegation or execution. -Dataiku permissions and API availability still apply to permitted calls. - -| Instance type | Available tools | -|---|---| -| design | All existing explicit tools | -| automation | Reads, existing asset execution, instance administration, and project-folder operations; no Cobuild or project definition/content writes | -| agent-management, govern, deployer | Instance controls only; API tools for these node types are not implemented yet | - -Automation excludes all six Cobuild tools, even conversation listing and polling, -and `create_project`, `update_project_settings`, `set_project_variables`, -`set_container_exec_config`, `create_upload_dataset`, `create_managed_folder`, -`upload_file_to_managed_folder`, and `write_project_library_file`. -It permits `build_datasets`, `run_recipe`, `run_scenario`, `abort_job`, and every -project-folder operation, including creation, deletion, and moving projects. -Existing execution can modify outputs; this policy does not make Automation read-only. - -Instance controls are `list_instances`, `switch_instance`, `configure_instance`, -`delete_instance`, and `get_current_instance`. The first four also work without an -active selection, subject to the existing HTTP restrictions on local configuration. -For Agent Management, Govern and Deployer, `get_current_instance` returns sanitized profile metadata -with connection_status set to unsupported, without contacting DSS or reporting a version. -Agent Management, Govern and Deployer profiles can be saved without a connection -test. Design and Automation require a successful DSS connection test in local setup. - ## Surface -**131 tools** · 93 read · 24 direct Dataiku write · 6 Cobuild · 4 execute · 3 local +**135 tools** · 96 read · 25 direct Dataiku write · 6 Cobuild · 4 execute · 3 local profile · 1 connection test | Bucket | # | Scope | |---|---|---| -| Read / inspect | 93 | Never mutates | -| Direct Dataiku write | 24 | Bootstrap, project configuration, cross-project, admin | +| Read / inspect | 96 | Never mutates | +| Direct Dataiku write | 25 | Bootstrap, project configuration, cross-project, admin | | Cobuild conversation | 6 | All flow and analytic building | | Execute | 4 | `build_datasets`, `run_recipe`, `run_scenario`, `abort_job` | | Local profile action | 3 | `configure_instance`, `switch_instance`, `delete_instance` | @@ -116,6 +87,7 @@ No Cobuild involved. Scope says what kind of access, and where a write lands. | Managed folders | `list_managed_folders`, `get_managed_folder_info`, `get_managed_folder_contents` | `create_managed_folder`, `upload_file_to_managed_folder` | Bootstrap, **in-project** — local-file upload is stdio only | | Project libraries | `list_project_library`, `read_project_library_file`, `search_project_library`, `validate_project_library_file` | `write_project_library_file` | Bootstrap, **in-project** — local-file write is stdio only | | Project folders | `list_project_folders`, `get_project_folder` | `create_project_folder`, `move_project_to_folder`, `delete_project_folder` | Cross-project | +| Operation registry pilot | `search_operations`, `describe_operation`, `run_read_operation` | `run_write_operation` | Fixed MCP interface over 145 operations: 13 Deployer, 86 Govern, 46 Agent Management. Backend-defined definition payloads are labelled explicitly. Tool instance policy is enforced before token exchange; read/write handlers validate operations before SDK execution. | | Code environments | `list_code_envs` | `create_code_env`, `update_code_env`, `delete_code_env` | Instance-level | | Plugins | `list_plugins`, `list_plugin_usages` | `install_plugin`, `ensure_plugin_code_env`, `update_plugin`, `delete_plugin` | Instance-level — local-path install/update are stdio only | | Users | `list_users` | `create_user`, `update_user`, `delete_user` | Instance-level | @@ -144,3 +116,24 @@ Local client configuration, not Dataiku objects. | `switch_instance` | Change the active instance; in HTTP mode saves only the authenticated user’s selected catalog instance | | `list_instances`, `get_current_instance` | Show configured instances, their declared types, and the active one; `get_current_instance` includes the Dataiku version when available | | `delete_instance` | Removes a **saved connection profile from the local config file**. Does not touch the Dataiku instance. | + +## Instance policies and operation discovery + +| Policy group | Allowed configured instance types | +| --- | --- | +| V1 explicit project reads, execution and administration | Design, Automation, Agent Management | +| V1 explicit project edits and project Cobuild | Design and Agent Management | +| V2 general API operation discovery and execution | Deployer, Govern, Agent Management | +| Instance management | All five types | + +All four v2 tools require a selected profile. Each catalogued operation requires its +matching instance type. Design and Automation cannot use v2. A local Deployer on +a Design or Automation node requires a separate Deployer profile for that URL. +The catalog remains stable across instance switches; new product actions are +operation IDs rather than individually registered MCP tools. + +Design, Automation and Deployer and Agent Management use the DSS client. Govern uses its own client in stdio and reports unsupported_authentication over HTTP. + +Discovery exposes schemas, effects, backend requirements and transport restrictions. +All adapters use public SDK methods; catalog presence does not verify live server +support. See [the operation guide](../skills/dataiku-headless/references/operations.md). diff --git a/docs/http-deployment.md b/docs/http-deployment.md index 8a796916..58a0b54f 100644 --- a/docs/http-deployment.md +++ b/docs/http-deployment.md @@ -110,11 +110,7 @@ The settings file has four sections: - `user_selections` records each authenticated user's current instance. Start with an empty object; the server manages it. -Instance types are operator-supplied and enforce the same -[tool policy](capabilities.md#instance-types) as stdio. Unsupported tool calls are -rejected before DSS token exchange. Agent Management, Govern and Deployer support instance controls -only; `get_current_instance` reports `unsupported` without contacting DSS. The -configured type does not trigger API discovery or select a different SDK client. +Instance types are operator-supplied and enforce the same [tool policy](capabilities.md#instance-policies-and-operation-discovery) as stdio. Unsupported tool calls fail before backend token exchange. Design, Automation and Deployer and Agent Management use the DSS client. Govern uses its own client in stdio and reports unsupported_authentication over HTTP. Authentication, transport, and instance settings are loaded at startup, so restart the server after changing them. User selections are updated in memory and persisted @@ -385,3 +381,7 @@ string-or-array format found in the exchanged token. > delegated token's `sub` must equal the selected Dataiku login or email. Validate this > constraint before production rollout. See Microsoft's > [access-token claims reference](https://learn.microsoft.com/en-us/entra/identity-platform/access-token-claims-reference). + +## Product authentication boundaries + +The shared operation catalog includes Deployer, Govern, Agent Management. Deployer supports delegated HTTP authentication. Govern execution is stdio-only and rejected before backend token exchange. Its HTTP connection status is unsupported_authentication. Agent Management uses the SDK’s native delegated bearer support; local-file transfers require stdio. diff --git a/pyproject.toml b/pyproject.toml index 6022cd81..03bf0164 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -21,7 +21,7 @@ classifiers = [ ] dependencies = [ "fastmcp>=3.4.5,<4", - "dataiku-api-client>=14.7.2", + "dataiku-api-client>=15.2.0", "pydantic>=2.11,<3", "python-dotenv", "requests>=2.32", diff --git a/runtime/run_mcp.py b/runtime/run_mcp.py index d6901f8a..1babee3d 100755 --- a/runtime/run_mcp.py +++ b/runtime/run_mcp.py @@ -3,7 +3,7 @@ # requires-python = ">=3.10" # dependencies = [ # "fastmcp==3.4.5", -# "dataiku-api-client==14.7.2", +# "dataiku-api-client==15.2.0", # "pydantic==2.13.4", # "python-dotenv==1.2.2", # "requests==2.34.2", diff --git a/runtime/run_mcp.py.lock b/runtime/run_mcp.py.lock index f9ff7087..7208d7d0 100644 --- a/runtime/run_mcp.py.lock +++ b/runtime/run_mcp.py.lock @@ -1,5 +1,5 @@ version = 1 -revision = 5 +revision = 3 requires-python = ">=3.10" resolution-markers = [ "python_full_version >= '3.14' and sys_platform == 'win32'", @@ -12,7 +12,7 @@ resolution-markers = [ [manifest] requirements = [ - { name = "dataiku-api-client", specifier = "==14.7.2" }, + { name = "dataiku-api-client", specifier = "==15.2.0" }, { name = "fastmcp", specifier = "==3.4.5" }, { name = "pydantic", specifier = "==2.13.4" }, { name = "python-dotenv", specifier = "==1.2.2" }, @@ -620,13 +620,13 @@ wheels = [ [[package]] name = "dataiku-api-client" -version = "14.7.2" +version = "15.2.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "python-dateutil" }, { name = "requests" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/96/e1/336e135820f414301839e57b7d337925f955fd5bc02ba1eb39c4f46d21e0/dataiku_api_client-14.7.2.tar.gz", hash = "sha256:476efce84ce6fd69dd226c85690ea83fd7cbba77276c3d2c0afa00ce7dba5031", size = 416594, upload-time = "2026-07-22T07:54:50.998Z" } +sdist = { url = "https://files.pythonhosted.org/packages/fb/0f/43d2be13e28c2278e2bfd88e4d35f412fb5f99b0839f8b539daa67cdcca0/dataiku_api_client-15.2.0.tar.gz", hash = "sha256:44e7d68339e7d5fb04f28eac0fc7117c9c434f2f349a688f9bb0b7043ba78517", size = 489461, upload-time = "2026-10-16T12:08:01.425Z" } [[package]] name = "dnspython" diff --git a/skills/dataiku-headless-setup/SKILL.md b/skills/dataiku-headless-setup/SKILL.md index d774cc89..d2724d0e 100644 --- a/skills/dataiku-headless-setup/SKILL.md +++ b/skills/dataiku-headless-setup/SKILL.md @@ -22,7 +22,7 @@ Bring a new or broken plugin installation to a verified Dataiku connection. A re A startup line on stderr followed by exit status 0 is expected. Do not substitute `uv sync --locked --script`: it caches downloads but leaves environment creation for the first server launch. 7. Retry `list_instances`. If the tools are still unavailable, reload the plugin or restart the agent once, then retry before diagnosing a Dataiku connection problem. -8. When `list_instances` succeeds, if no instance is configured, run `configure_instance` and have the user complete the local setup page. Design and Automation require a successful connection test before saving. Agent Management, Govern and Deployer testing is unavailable; save their locally validated profiles directly. If multiple instances exist without an active one, ask which to use and run `switch_instance`. -9. Verify the active profile with `get_current_instance`. If its `connection_status` is `failed`, the discovered profile cannot connect to Dataiku; run `configure_instance` to replace or add a working profile instead of treating it as set up. If it is `unsupported`, the Agent Management, Govern or Deployer profile is configured but its API tools are not implemented; explain this and offer to switch to a supported instance instead of repeating setup or probing DSS. When it is `connected`, make a lightweight read-only Dataiku call such as `list_projects` to validate the available Dataiku access. Never request or repeat the API key in chat. +8. When `list_instances` succeeds, if no instance is configured, run `configure_instance` and have the user complete the local setup page. Supported types require a successful connection test before saving. Govern uses its own SDK client. If multiple instances exist without an active one, ask which to use and run `switch_instance`. +9. Verify the active profile with `get_current_instance`. If its `connection_status` is `failed`, the discovered profile cannot connect to Dataiku; run `configure_instance` to replace or add a working profile instead of treating it as set up. When it is `connected`, validate access with a read allowed on that type: use the shared operation interface with deployer.list_stages for Deployer and `list_projects` for Design or Automation or Agent Management. Govern uses its own connection check in stdio; HTTP reports unsupported_authentication. Never request or repeat the API key in chat. -Report completion with the active instance name, URL, and configured type, Dataiku version when available, and whether the Dataiku read succeeded or the selected node type has no API tools yet. Include the uv version only when runtime recovery was needed. If a restart is required, say that setup is incomplete and give the single next action. +Report completion with the active instance name, URL, and configured type, Dataiku version when available, and whether the Dataiku read succeeded. Include the uv version only when runtime recovery was needed. If a restart is required, say that setup is incomplete and give the single next action. diff --git a/skills/dataiku-headless/SKILL.md b/skills/dataiku-headless/SKILL.md index 1353336c..39612953 100644 --- a/skills/dataiku-headless/SKILL.md +++ b/skills/dataiku-headless/SKILL.md @@ -1,16 +1,30 @@ --- name: dataiku-headless -description: Use for Dataiku tasks including projects, flows, datasets, recipes, jobs, machine learning, agents, and Cobuild conversations. Choose the right reference guide, inspect state first, route project changes through Cobuild by default, and validate results after execution. +description: Use for Dataiku tasks including projects, flows, datasets, recipes, jobs, machine learning, agents, deployments, Govern, Agent Management, and Cobuild conversations. Choose the right reference guide, inspect state first, route project changes through Cobuild by default, and validate results after execution. --- # Dataiku Headless -Use this for any Dataiku task. Choose the right reference guide first, inspect the current state before acting, route project changes through Cobuild by default, and validate by re-reading the resulting state. +Start by identifying the configured instance type. The default project workflow is for Design and supporting projects on Agent Management. Inspect before acting, use Cobuild for project construction, and verify results. + +## Instance Types + +| Configured type | Purpose | Headless workflow | +| --- | --- | --- | +| `design` | Development environment for building projects, pipelines, models, applications and agents. | Use the Design workflow below; project construction goes through Cobuild. | +| `automation` | Production environment for running deployed project bundles, schedules and monitoring. | Inspect and execute existing assets; return to Design for project edits. | +| `deployer` | Control plane for deploying published project bundles and API service versions. | Use [Deployer operations](./references/deployer.md), including for a local Deployer selected through its own profile. | +| `govern` | Central governance of analytics, models and agents through governed items and sign-off workflows. | Use [Govern operations](./references/govern.md); stdio only. | +| `agent-management` | Agent inventory, performance/value monitoring and risk management, with Design projects for supporting pipelines. | Use [Agent Management operations](./references/agent-management.md) for the estate and the Design workflow for supporting projects. | + +Descriptions follow Dataiku’s [deployment documentation](https://doc.dataiku.com/dss/latest/deployment/index.html), [deployment knowledge base](https://knowledge.dataiku.com/latest/deploy/project-deployment/concept-project-deployment.html), [Govern architecture knowledge base](https://knowledge.dataiku.com/latest/govern/overview/concept-govern-in-dataiku-architecture.html), and [Agent Management product reference](https://www.dataiku.com/product/agent-management). + +Configured type selects Headless policy; it does not establish live features or permissions. Use `list_instances` and `get_current_instance` first. Design, Automation and Deployer and Agent Management use the DSS client. Govern uses its own client in stdio and reports unsupported_authentication over HTTP. ## Shared Operating Rules 1. If the user asks to install, set up, connect, or repair Dataiku Headless, or its MCP tools are unavailable just after installation, read `../dataiku-headless-setup/SKILL.md`. Choose either local stdio or customer-managed HTTP setup and never enable both. -2. Ensure an instance is configured before instance operations. In local stdio mode, if `get_current_instance` errors or `list_instances` is empty, run `configure_instance` first. In HTTP mode, use `list_instances` then `switch_instance`; the instance catalog is platform-managed. Keep the reported `dataiku_version` in context for version-sensitive requests. Check the configured `instance_type` against the policy below before choosing a tool. It controls dispatch; Dataiku permissions and API availability still apply. +2. Ensure an instance is configured before instance operations. In local stdio mode, if `get_current_instance` errors or `list_instances` is empty, run `configure_instance` first. In HTTP mode, use `list_instances` then `switch_instance`; the instance catalog is platform-managed. Keep the reported `dataiku_version` in context for version-sensitive requests. The required `instance_type` selects dispatch policy; follow the instance-type table rather than applying the Design workflow to every node. 3. Discover project keys and object identifiers through tools; do not invent them. 4. Read before write. Inspect the current object, flow context, jobs, or run history before changing anything. 5. Treat the matching reference guide as the source of truth for object-specific concepts, inspection steps, and required references. @@ -20,28 +34,10 @@ Use this for any Dataiku task. Choose the right reference guide first, inspect t 9. Preserve surrounding flow, storage, and operational context unless the user requests a change. 10. If MCP or Cobuild coverage is insufficient, stop and report the gap rather than falling back to raw Python, `dataikuapi`, or ad hoc REST calls. Attribute the gap to the Dataiku version only when that requirement is known; otherwise, do not guess. -## Instance Types - -| Configured type | Available work | -| --- | --- | -| `design` | All tools | -| `automation` | Reads, execution of existing assets, instance administration, and all project-folder operations | -| `agent-management`, `govern`, `deployer` | Instance controls only: list, switch, configure, delete, and current profile | - -Automation excludes all Cobuild tools, including conversation listing and polling, -and project creation, project settings/variables updates, container placement, -dataset/folder creation, file upload, and library writes. Builds, recipe/scenario -runs, and job aborts remain available and can change outputs. For project -construction or edits, ask which Design instance to use and -switch to it; do not bypass the policy through raw APIs. - -Agent Management, Govern and Deployer report `connection_status=unsupported` without a DSS probe. -Explain that their API tools are not implemented, then switch to a supported -configured instance for Dataiku work. Local configuration actions remain subject -to the existing HTTP restrictions. The catalog stays visible across all types; -unsupported calls fail before execution. +## Default Workflow — Design Node -## Default Workflow +Use this workflow on a Design node or for supporting projects on an Agent Management node. For other types, follow the instance-type table. +On Automation, use the relevant inspection/execution guide without creating or editing project definitions. 1. Identify the target project and the user's main object of work. 2. Read the matching reference guide from the routing table below. @@ -53,7 +49,7 @@ unsupported calls fail before execution. 8. For tasks that span multiple object types, keep one primary guide and read supporting guides for dependent objects or job/run follow-up. 9. Completion status is not enough. Validate by re-reading the changed object and comparing it with what the user asked for. Where the request is specific about names, titles, or values, check them in the saved settings or a data sample, and mention any requirement you could not confirm. Use `./references/jobs.md` when execution may still be in progress. -## Guide Routing +## Design Guide Routing | User intent | Guide to read next | | --- | --- | @@ -89,4 +85,4 @@ unsupported calls fail before execution. - `./references/cobuild.md` is the default write path for in-project assets, but it should be grounded by the relevant guide first. - Direct writes may grow over time for documented non-project, cross-project, instance-level, or administrative operations that Cobuild does not handle. -- `./references/jobs.md` is the follow-up guide after builds, runs, training, deployment, or other uncertain execution. +- `./references/jobs.md` is the follow-up guide after builds, runs, training, or other uncertain execution. For Deployer updates, follow `./references/deployer.md` for future polling and deployment history. diff --git a/skills/dataiku-headless/references/agent-management.md b/skills/dataiku-headless/references/agent-management.md new file mode 100644 index 00000000..29a7535e --- /dev/null +++ b/skills/dataiku-headless/references/agent-management.md @@ -0,0 +1,100 @@ +--- +name: agent-management +description: Inspect and manage the monitored agent estate through public Agent Management Python SDK methods: platforms, agents, metrics, ingestion operations, risk, settings and tags. +--- + +# Dataiku Agent Management + +Use this guide for the Agent Management product and its monitored agent estate. +Project-level Dataiku agents use [the agents guide](./agents.md) and Cobuild. + +## Operation interface + +Read [the shared operation guide](./operations.md). Search with +`domain="agent-management"`, describe the selected operation, then use the matching +read/write dispatcher with an `agent-management.` operation ID. The catalog uses +public `dataiku-api-client` methods only and requires version 15.2.0 or newer. +There is no direct endpoint fallback. Catalog presence does not verify a server's +version, permissions or enabled features. + +The SDK calls infrastructure records **platforms**. Discover `platform_id` with +`list_platforms`, then `search_agents`; never invent identifiers. Typed +`agent_filter` fields map to public `AgentFilter` builders. Values within one +criterion combine with OR, while different criteria combine with AND. Empty +criterion lists are rejected. Omit the filter for an unfiltered read. + +The 46 operations cover platform and agent records/settings, native metric +retrieval, alert evaluation, discovery, monitoring setup, ingestion, operation +status/results/abort, risk assessments and evidence, taxonomy, global settings +and existing tags. Search/describe supplies exact schemas and examples. + +## Inspect, change, verify + +1. Inspect the selected instance and discover platform and agent IDs. +2. Read the relevant settings, assessment or taxonomy; retain `fetchedRevision`. +3. Describe the write operation and pass that revision as `expected_revision`. +4. Re-read after the write. On conflict, inspect changed state; never retry stale + intent automatically. + +Settings updates use `changes` to replace named **top-level fields** of a freshly +fetched definition. Unnamed fields, including hidden credentials, are retained. +Nested objects and arrays supplied in `changes` are complete replacements: inspect +and preserve their unrelated fields. To rename an agent, copy the complete inspected +`configuration`, change its `displayName`, and pass the copy in +`changes.configuration`; `displayName` is not a top-level agent setting. +Do not submit redaction placeholders or put +`fetchedRevision` inside `changes`. Global business KPI definitions are available +through `get_settings` and `update_settings` as `businessKPIs`. Changing the timezone +deletes incompatible platform metrics; changing the week-start day deletes weekly +platform metrics. `reset_incompatible_metrics=true` explicitly permits that deletion. +The default rejects it. + +The `update_settings` example sets `businessKPIs` to an empty list, removing all +business KPI definitions. Use that example only when the user intends to remove +all definitions; otherwise preserve the definitions they want to keep. + +Risk-taxonomy replacement requires reviewing +`compute_risk_taxonomy_replacement_impact` for the exact proposed taxonomy, then +passing the returned impact and inspected revision to `update_risk_taxonomy`. +Risk-assessment updates replace the **complete** `assessment` and accept an optional +`documents` mapping from evidence IDs to local paths. Preserve all wanted assessment +fields and evidence references: omitted references can permanently delete evidence +after commit. The `assessment: {}` example deliberately clears the assessment and +its evidence. Risk exposure returns the public agent identifier as `agents[].agentKey` +so it can be joined to `riskExposures[].agentKey` without credential redaction. +Evidence download needs `output_path` and +explicit `overwrite=true` to replace an existing destination. + +## Monitoring and asynchronous work + +Discover qualified metric identifiers with `get_available_metric_ids`; use +`get_metric_time_series` or `get_metric_buckets` with those IDs. Time-series reads +require `from_time` unless `latest_only=true`; latest-only reads forbid time bounds. +Bucketed reads require `from_time`. Time bounds are epoch milliseconds. SDK metric +responses represent timestamps as UTC date/time strings after JSON serialization. + +`setup_agent_monitoring` provisions monitoring resources and enables supported +capabilities. Inspect warnings in the returned result; setup does not guarantee +telemetry exists. Discovery, uptime tests and log/operational-metric ingestion +return `platform_id`, optional `agent_id`, and `operation_id`. Keep those identities +on the same instance and poll the corresponding agent/platform status and result +operations. A queued operation is not complete. Aborts are mutations. + +`evaluate_alerts` queues evaluation and returns `queued_agent_count`, which counts +agents included in the request, not newly created alerts or completed evaluations. +Review the agent filter first. An absent or empty filter requires +`scope="all_writable_agents"`. The default mode is `DUE`; `FORCE_LATEST` and +`FORCE_CATCH_UP` can invalidate historical alerts and create replacements that do +not inherit acknowledgements. Verify outcomes separately. + +## Transport and credentials + +Stdio uses configured credentials. HTTP uses the SDK's native delegated bearer +support through `DSSClient.get_agent_management_client()`. Host-file transfers +require stdio; requests with `documents` or `output_path` are rejected after HTTP +token exchange but before SDK execution. Risk-assessment updates without files can +run over HTTP. + +Responses recursively redact credentials and sensitive diagnostics. Never share +more business data than needed. Downloaded files use an explicit destination and +atomic no-clobber behavior unless overwrite is requested. diff --git a/skills/dataiku-headless/references/api-deployer.md b/skills/dataiku-headless/references/api-deployer.md new file mode 100644 index 00000000..daf2135d --- /dev/null +++ b/skills/dataiku-headless/references/api-deployer.md @@ -0,0 +1,42 @@ +--- +name: api-deployer +description: Deploy already published API service versions, select a single target version and verify asynchronous API deployment updates. +--- + +# API Deployer + +Read [the shared Deployer guide](./deployer.md) first for instance selection, +coverage and recovery. Pass arguments through the shared read/write dispatcher. All operations below use `deployer_type="api"`. + +API services are authored in a Design/Automation project's API Designer and +published as versioned packages. API Deployer manages those packages and sends them +to an infrastructure. An API node serves requests; it is not the Deployer profile. +The tool does not configure or invoke prediction endpoints. + +1. Discover infrastructures and stages, then inspect the chosen target and its + reported deployment permissions and governance policy. Infrastructure type + determines available deployment behavior. +2. Discover published services and inspect the chosen source_id to identify its + package versions. Do not confuse a saved-model version with an API package version. +3. Inspect existing deployment configuration, traffic mapping, health and history. +4. Create configuration with deployment_id, source_id (published service ID), + infra_id and version. endpoint_id is optional for Deploy Anywhere infrastructures + that deploy a particular endpoint; use a known endpoint from the package/UI. + deployed_project_key and project_folder_id are project-only and rejected here. +5. Use `deployer.update_deployment(version=...)` to select one service version. + This replaces any existing multi-version traffic mapping with SINGLE_GENERATION. + Check the existing mapping and ensure a single-version transition matches intent. + An older published version can be selected for rollback. +6. Use `deployer.update_deployment(enabled=false)` to save a disabled target, or + enabled=true to save re-enablement. A save alone does not change serving state. + Authentication and other settings are preserved. +7. Apply with `deployer.start_update`, track the future and exact update record, and + re-read deployment health. A successful update does not prove prediction quality; + endpoint invocation and test queries are outside this surface. + +API service authoring/publication, advanced traffic splits, endpoint authentication, +API key handling, infrastructure provisioning and deployment deletion are not +exposed. Inspect the Dataiku UI for these operations and detailed error logs. +Never return raw light status: it can contain service and inherited API keys. + +[Official Python API reference](https://developer.dataiku.com/latest/api-reference/python/api-deployer.html) diff --git a/skills/dataiku-headless/references/cobuild.md b/skills/dataiku-headless/references/cobuild.md index 31e8dcda..ee44c26f 100644 --- a/skills/dataiku-headless/references/cobuild.md +++ b/skills/dataiku-headless/references/cobuild.md @@ -5,7 +5,7 @@ description: Use Dataiku Cobuild for project-level asset creation, modification, # Cobuild -All Cobuild tools require `design`, including read-only conversations, listing, and polling. Automation cannot use Cobuild; switch to a supported instance before this workflow. +All Cobuild tools require `design` or `agent-management`, including read-only conversations, listing, and polling. Automation cannot use Cobuild; switch to a supported instance before this workflow. Cobuild is Dataiku's AI building agent for co-developing Dataiku projects. It can build and modify flows, recipes, models, dashboards, webapps, notebooks, wiki content, agents, scenarios, and other project assets through a retained conversation. diff --git a/skills/dataiku-headless/references/container-execution.md b/skills/dataiku-headless/references/container-execution.md index a9dc36b0..c76a30b8 100644 --- a/skills/dataiku-headless/references/container-execution.md +++ b/skills/dataiku-headless/references/container-execution.md @@ -5,7 +5,7 @@ description: Select where an existing recipe, ML task, WebApp backend, Knowledge # Container Execution Placement -`set_container_exec_config` requires `design`; placement changes are unavailable on Automation. +`set_container_exec_config` requires `design` or `agent-management`; placement changes are unavailable on Automation. `set_container_exec_config` changes only the container execution selection of one existing object. It preserves the object's logic, code, inputs, outputs, and every other diff --git a/skills/dataiku-headless/references/datasets.md b/skills/dataiku-headless/references/datasets.md index c13a0516..5e4dd482 100644 --- a/skills/dataiku-headless/references/datasets.md +++ b/skills/dataiku-headless/references/datasets.md @@ -5,7 +5,7 @@ description: Understand and inspect Dataiku datasets, including their storage, s # Datasets -Dataset inspection, export, and builds are available on `design` and `automation`. New dataset creation, including direct uploads and Cobuild, requires `design`. +Dataset inspection, export, and builds are available on `design`, `automation`, and `agent-management`. New dataset creation, including direct uploads and Cobuild, requires `design` or `agent-management`. Use this guide to understand, inspect, and locally export Dataiku datasets, gather context for Cobuild, and handle the direct Uploaded Files creation exception. diff --git a/skills/dataiku-headless/references/datasets/uploaded-files-datasets.md b/skills/dataiku-headless/references/datasets/uploaded-files-datasets.md index 53df6f5f..9ba7722d 100644 --- a/skills/dataiku-headless/references/datasets/uploaded-files-datasets.md +++ b/skills/dataiku-headless/references/datasets/uploaded-files-datasets.md @@ -1,6 +1,6 @@ # Uploaded Files Datasets -`create_upload_dataset` requires `design`; Automation cannot create uploaded datasets. +`create_upload_dataset` requires `design` or `agent-management`; Automation cannot create uploaded datasets. Read this reference only when the user wants to bring supplied data into a project as a new Uploaded Files dataset. diff --git a/skills/dataiku-headless/references/deployer.md b/skills/dataiku-headless/references/deployer.md new file mode 100644 index 00000000..e3cbcef7 --- /dev/null +++ b/skills/dataiku-headless/references/deployer.md @@ -0,0 +1,82 @@ +--- +name: deployer +description: Inspect and operate Project and API Deployer deployments, selecting the correct node and tracking asynchronous updates. +--- + +# Deployer + +Deployer is an instance-level control plane with two independent components: +Project Deployer sends published project bundles to Automation nodes; API Deployer +sends published API service versions to API nodes or supported managed infrastructures. +Deployer may be standalone or enabled locally on a Design or Automation node. +All 13 operations, including future polling, use the schema-backed registry described in [Operations](./operations.md). Deployer operations require a profile declared as type `deployer`. +These deployment operations are direct instance-level exceptions to Cobuild routing. +Building the underlying Flow, models or API endpoint logic remains project work. + +## Select the target first + +1. Inspect `list_instances` and `get_current_instance`; select the intended Deployer + using `switch_instance`. Keep its name and reported Dataiku version in context. +2. The configured `instance_type` is metadata. A profile marked deployer does not + verify that Deployer is enabled or the credential has deployment privileges. + For a local Deployer on a Design or Automation node, configure a separate + `deployer` profile pointing at that URL. Design and Automation profiles cannot + call the v2 operation tools. Tools address the selected node; they do not follow its remote Deployer configuration automatically. +3. Read [Project Deployer](./project-deployer.md) for bundles or + [API Deployer](./api-deployer.md) for real-time services. +4. Supply `deployer_type="project"` or `deployer_type="api"` on every call. IDs + belong to that component on that instance, even when names happen to match. + +## Shared inspection and execution + +- `deployer.list_stages` discovers configured stage IDs and descriptions. +- `deployer.list_infrastructures` and `deployer.get_infrastructure` show safe target + summaries, governance policy and associated deployments. Permissions are reported + when the SDK response includes them; absence is not permission to deploy. +- `deployer.list_published_items` discovers projects/services. + `deployer.get_published_item(source_id=...)` returns available packages as a + columnar table with id, publishedOn and publishedBy. Use those IDs as versions. +- `deployer.list_deployments` reports configuration, not live health. + `deployer.get_deployment` adds health, saved version settings, and structured + health/governance codes. Review these before mutation. +- `deployer.create_deployment` creates configuration only. Existing IDs fail through + Dataiku; this tool has no overwrite mode. +- `deployer.update_deployment` saves only a target version and/or API enablement. + Saving is not execution. Other settings are preserved; API version selection + replaces the existing traffic mapping with a single-version mapping. +- `deployer.start_update` applies the saved settings asynchronously. It can replace + production assets, alter API traffic, or disable an API deployment. Inspect the + target and saved version first, and execute only within the user's request. + +## Track the same update + +Keep the selected instance, deployer_type, deployment_id and returned future_id. +Poll `deployer.get_future_status(future_id=...)` on that same instance. Do not use Flow job +IDs or assume a configuration save means an update completed. Then inspect +`deployer.list_updates`, pin the matching startTimestamp with +`deployer.get_update(timestamp=...)`, and re-read `deployer.get_deployment`. +Omitting timestamp reads the latest update, which may belong to another caller. +Completion of a future alone does not establish a healthy deployed service/project. + +If a request times out, is cancelled, returns no future ID, or fails after submission, +inspect deployment state and update history before retrying. Do not resubmit an +uncertain update. Reconnect to the same configured instance when resuming. + +## Coverage limits + +Tools intentionally expose selected operational fields. Raw infrastructure/settings, +API keys, OAuth settings, endpoint code, free-text diagnostics and update logs are +omitted because SDK payloads may contain credentials. Use the Dataiku UI for detailed +diagnostics. Unknown response fields are omitted rather than passed through. + +There are no tools here for infrastructure provisioning, remapping, authentication, +permissions, advanced traffic mappings, deletion, bundle export/upload, API package +publication, or deployment test execution. Work with already published packages. +Report missing coverage; do not bypass it with raw SDK or REST calls. Governance +warnings are not bypassed by create/save operations. Use Cobuild for supported +in-project asset construction; if it cannot publish an artifact, report that gap. + +## Official documentation + +- [Production deployments and bundles](https://doc.dataiku.com/dss/latest/deployment/index.html) +- [API Node and API Deployer](https://doc.dataiku.com/dss/latest/apinode/introduction.html) diff --git a/skills/dataiku-headless/references/govern.md b/skills/dataiku-headless/references/govern.md new file mode 100644 index 00000000..983ae4b0 --- /dev/null +++ b/skills/dataiku-headless/references/govern.md @@ -0,0 +1,153 @@ +--- +name: govern +description: Operate Dataiku Govern through the shared operation registry, with task guides for artifacts, blueprints, signoffs, and supporting objects. +--- + +# Govern + +Govern runs on a separate node. Its records are artifacts described by blueprint +versions, with workflow steps and signoffs. A governed project is an artifact; +it is not a DSS project containing datasets and recipes. + +Use the shared operation registry for all Govern reads and writes. Its 86 Govern +operations wrap the public SDK through the same interface as Deployer and Agent Management. No standalone govern tool or credential-bearing +shell scripts are needed. Missing operations remain coverage gaps, not permission +to call raw SDK/REST interfaces. DSS project construction still uses Cobuild on Design. + +## Operation interface + +Read [the shared operation guide](./operations.md). Search with domain="govern", +describe a returned operation_id, then call run_read_operation or +run_write_operation with operation IDs prefixed by govern. The arguments object +is flat: identifiers, definitions and options use the names in the input schema. +Category distinguishes artifacts, signoffs, blueprints, roles, custom_pages, +time_series, files, users and instance administration. Search supports pagination. + +Results use public SDK structures with credential fields recursively redacted. +Blueprint-specific definitions remain backend-defined; the structural schema does +not replace inspecting the full target before a whole-definition replacement. + +## Connect + +The Govern node is a configured instance of type `govern`, with its own URL and +local API key or ticket. The registry uses the active instance and refuses an instance of another +type. `list_instances` shows the type of each instance; when none has type +`govern`, use `configure_instance` in stdio and choose the Govern type. + +Govern operations require stdio with the configured API key or ticket. HTTP +execution is unsupported and rejected before token exchange. Static discovery +remains available; `get_current_instance` reports `unsupported_authentication` for +a Govern HTTP profile. Do not supply credentials in operation arguments or try +another authentication path. + +Start a Govern task with `get_instance_info`: it confirms the node type is +`GOVERN` and reports the server version. Then `get_auth_info` tells which identity +signs the work. A global API key has no user behind it: `get_own_user` fails and +decisions are recorded under the key identity; a personal API key records them +under that user. If the tool reports a missing Govern configuration or a +permission error, report that specific blocker; do not repurpose DSS credentials. + +Blueprint design and custom page administration need the Govern Blueprint Designer +license option. Without it the server answers "Your license does not allow you to +use the Govern Blueprint Designer" for `create_blueprint`, version and signoff +configuration writes, and custom page writes and ordering. Report that as a +licensing blocker, not as a tool gap. + +## Workflow + +1. Search domain="govern" and describe the selected operation when its arguments + are not in context. Never guess an operation ID. +2. Discover identifiers through `list_blueprints`, `list_blueprint_versions`, + `search_artifacts`, `list_roles`, `list_custom_pages`, `list_users`, and + `list_groups`. Do not invent IDs. +3. Read the full target before changing it: `get_artifact`, + `get_blueprint_version`, `get_signoff`, `get_role`, `get_custom_page_definition`. +4. Run the write with the exact operation_id and arguments. Update operations + replace the complete definition unless the guide says merge. +5. Re-read after every write. The tool returns the persisted state, but compare it + with the intended change; a successful save does not prove the values held. +6. For multi-step work, verify each unit and retain the created IDs. After an + uncertain write, inspect before retrying a create. + +## Parameter rules + +- `update_artifact`, `update_blueprint`, `update_blueprint_version`, + `update_signoff_configuration`, `update_role`, `update_role_assignments`, + `update_blueprint_permissions`, `update_default_blueprint_permissions`, + `update_custom_page`, and `update_signoff_recurrence` replace the whole + definition. Send the full object read from the matching get operation with only + the requested keys changed. +- `update_artifact_fields`, `update_user`, and `update_group` merge the given + keys into the current definition and preserve the rest. +- `create_*` payloads never contain an `id`; the server derives it from + `new_identifier` or the parent. +- Users containers in signoff configurations and role rules are + `{"type": "user", "login": ...}`, `{"type": "group", "groupName": ...}`, + `{"type": "role", "roleId": ...}`, or `{"type": "global-api-key", "globalAPIKeyId": ...}`. + The type is lowercase. The `users_container` param of the two delegation + operations accepts the `user` form only. +- `update_signoff_status` notifies only an explicit users_to_notify list. Omitted or empty lists send no + notifications; specify recipients only when the task authorizes notifying them. +- `update_blueprint_version` refuses a save that removes a field or changes its + type on a version with artifacts. `danger_zone_accepted: true` discards that + data everywhere; use it only after an explicit user decision. +- `search_artifacts` returns at most `max_results` hits (default 100) and + `has_more`. Raise `max_results` or narrow the filters for a complete inventory, + and say so when a result is a sample. + +## Operations + +| Domain | Operations | +| --- | --- | +| artifacts | `search_artifacts`, `get_artifact`, `create_artifact`, `update_artifact`, `update_artifact_fields`, `delete_artifact` | +| signoffs | `list_signoffs`, `get_signoff`, `get_signoff_details`, `create_signoff`, `update_signoff_status`, `list_signoff_feedbacks`, `get_signoff_feedback`, `add_signoff_feedback`, `update_signoff_feedback`, `delete_signoff_feedback`, `delegate_signoff_feedback`, `get_signoff_approval`, `add_signoff_approval`, `update_signoff_approval`, `delete_signoff_approval`, `delegate_signoff_approval`, `get_signoff_recurrence`, `update_signoff_recurrence` | +| blueprints | `list_blueprints`, `get_blueprint`, `list_blueprint_versions`, `get_blueprint_version`, `create_blueprint`, `update_blueprint`, `delete_blueprint`, `create_blueprint_version`, `update_blueprint_version`, `set_blueprint_version_status`, `delete_blueprint_version`, `list_signoff_configurations`, `get_signoff_configuration`, `create_signoff_configuration`, `update_signoff_configuration`, `delete_signoff_configuration` | +| roles | `list_roles`, `get_role`, `create_role`, `update_role`, `delete_role`, `list_role_assignments`, `get_role_assignments`, `create_role_assignments`, `update_role_assignments`, `delete_role_assignments`, `get_default_blueprint_permissions`, `update_default_blueprint_permissions`, `list_blueprint_permissions`, `get_blueprint_permissions`, `create_blueprint_permissions`, `update_blueprint_permissions`, `delete_blueprint_permissions` | +| custom_pages | `list_custom_pages`, `get_custom_page`, `get_custom_page_definition`, `create_custom_page`, `update_custom_page`, `delete_custom_page`, `get_custom_pages_order`, `update_custom_pages_order` | +| time_series | `create_time_series`, `get_time_series_values`, `push_time_series_values`, `delete_time_series_values` | +| files | `upload_file`, `get_uploaded_file`, `download_uploaded_file`, `delete_uploaded_file` | +| users | `list_users`, `get_user`, `create_user`, `update_user`, `delete_user`, `get_own_user`, `list_groups`, `get_group`, `create_group`, `update_group`, `delete_group` | +| instance | `get_instance_info`, `get_auth_info` | + +Blueprint version import, bulk user provisioning, API keys, licensing, logs, and +identity-provider settings are not in the catalog. Report them as gaps and hand off +to the Govern UI. + +## Choose the task guide + +| Task | Read next | +| --- | --- | +| Discover, create, or update governed records; inspect field schemas | [Artifacts](./govern/artifacts.md) | +| Design blueprint versions, fields, workflow, views, or hooks | [Blueprints](./govern/blueprints.md) | +| Configure review gates or work with feedback, approvals, and workflow state | [Signoffs](./govern/signoffs.md) | +| Manage roles, permissions, custom pages, users, groups, files, or time series | [Supporting objects](./govern/supporting-objects.md) | + +## Design the governance process + +For process design, identify the item being governed, its parent, required evidence, +responsible roles, and the decision that each review gate records. Reuse a suitable +standard blueprint before designing a custom item type. Derive risk criteria and +approval rules from the user's policy; a plausible template is not an established policy. + +Synced Dataiku assets and their governance layer are distinct. Discover an asset's +existing Govern record before creating another; a Govern project may also exist +before it is linked to a Dataiku project. A parent must be governed before its +children can be governed. Hiding a parent hides its children in Govern; it does +not delete them. See [Govern items](https://doc.dataiku.com/dss/latest/governance/types-govern-items.html) +and [governance actions](https://doc.dataiku.com/dss/latest/governance/governance-features.html). + +Workflow progress, signoff approval, and deployment authorization are separate. +Use the configured signoff to record a decision. For deployment eligibility, inspect +the [Deployer infrastructure policy](https://doc.dataiku.com/dss/latest/governance/deployment-policies.html); +an `approved` field or a finished workflow step is not a deployment gate. + +## Safety + +- Every `write` and `delete` operation is a mutation. Execute only within the user's authorized scope; the host applies its approval policy. +- Deleting a blueprint version requires deleting its artifacts first. Do not + cascade-delete artifacts to unblock a schema change. +- Decisions (`add_signoff_feedback`, `add_signoff_approval`) are recorded under the + authenticated identity. Do not alter reviewer membership to bypass a permission + error; delegation changes who reviews and is not impersonation. +- Keep local paths explicit for `upload_file` and `download_uploaded_file`. Never + expose API keys in payloads or responses. diff --git a/skills/dataiku-headless/references/govern/artifacts.md b/skills/dataiku-headless/references/govern/artifacts.md new file mode 100644 index 00000000..c3100362 --- /dev/null +++ b/skills/dataiku-headless/references/govern/artifacts.md @@ -0,0 +1,75 @@ +# Govern artifacts + +Use the shared Govern operation registry described in [Govern](../govern.md). Identifiers such as +`blueprint_id`, `version_id`, and `artifact_id` come from discovery or the user. + +## Discover and inspect + +Blueprints and versions: + +- `list_blueprints` returns envelopes; the blueprint is nested under `blueprint`. +- `list_blueprint_versions` with `blueprint_id` lists versions and their status. +- `get_blueprint_version` with `blueprint_id` and `version_id` returns + `definition` (the schema, with `fieldDefinitions`) and `trace` (status, origin). + +Artifacts: + +- `search_artifacts` with `blueprint_ids`, `blueprint_version_ids`, `artifact_ids`, + `field_filters`, `archived`, and `sort`. Each hit holds the artifact `id`, + `name`, `blueprintVersionId`, and `status`; call `get_artifact` for its fields + and workflow. The result carries `count` and `has_more`; when `has_more` is true, + raise `max_results` or narrow the filters before calling the inventory complete. + A field filter is `{"condition_type": "CONTAINS", "condition": "churn"}` for the + name, or adds `field_id` for one field. +- `get_artifact` with `artifact_id` returns the full definition: `name`, `fields`, + `workflow`, and `blueprintVersionId`. + +For an existing artifact, inspect its own blueprint version; another version of +the same blueprint can have different fields. + +Govern 15 also exposes creation and last-modification metadata, including the actor +and timestamp. Inspect `creationRevision` and `lastModificationRevision` when the +task needs provenance; preserve server-managed metadata when editing other fields. +These values describe creation and the latest change, not a complete audit history. + +## Create or update + +Create on an ACTIVE version after reading its `fieldDefinitions`: + +```json +{"operation": "create_artifact", + "params": {"blueprint_id": "bp.system.govern_project", "version_id": "bv.system.default", + "name": "Churn model", "fields": {"description": "Quarterly churn scoring"}}} +``` + +Retain the returned `id` and re-read with `get_artifact`. + +For a partial edit, use `update_artifact_fields` with `artifact_id` and `fields` +(and `name` when it changes). It merges the given fields and preserves the rest. +Compare the returned `fields` with the request; a field that did not persist means +the value was rejected silently, for example a scalar sent to a list field. + +For a structural edit (workflow state, archived flag, blueprint reference), use +`update_artifact` with the complete `definition` read from `get_artifact`, changed +only where requested. A partial definition loses unrelated state. + +`delete_artifact` removes the record; confirm the ID first. + +## Field rules + +- Fields are keyed by ID in `fieldDefinitions`; the type key is `fieldType`. + Use the actual field ID, not its display label. User-entered values belong in + `sourceType: STORE` fields; preserve computed fields. +- `listConfig` makes a field a list, even if the configuration is empty. Send an + array for one value as well. CATEGORY values must match the defined categories, + case-sensitively. +- REFERENCE values are artifact IDs, including references to user and group + artifacts; they are not logins or group names. Inspect `allowedBlueprints` + before selecting them. +- DATE values use an ISO-8601 datetime with timezone, for example + `2026-06-01T00:00:00.000Z`. Uploaded files and time series use IDs from + `upload_file` and `create_time_series`; see [Supporting objects](./supporting-objects.md). +- `required` is enforced on every artifact whatever the workflow step or the view. + +For workflow transitions or approvals, follow [Signoffs](./signoffs.md). An artifact +field edit is not a substitute for submitting feedback or approval. diff --git a/skills/dataiku-headless/references/govern/blueprints.md b/skills/dataiku-headless/references/govern/blueprints.md new file mode 100644 index 00000000..7a557b6a --- /dev/null +++ b/skills/dataiku-headless/references/govern/blueprints.md @@ -0,0 +1,99 @@ +# Govern blueprint authoring + +Use the shared Govern operation registry described in [Govern](../govern.md). A blueprint stores identity +and presentation metadata. Its versions hold fields, workflow, views, hooks, and +actions. Blueprint design operations need blueprint designer rights on the API key. + +## Inspect, fork, edit, publish + +Inspect with `get_blueprint`, `list_blueprint_versions`, `get_blueprint_version` +(definition plus trace), and `list_signoff_configurations`. + +Prefer forking a suitable existing version when changing the structure, so the +required system fields and behavior survive: + +1. `create_blueprint_version` with `blueprint_id`, `new_identifier`, `name`, and + `origin_version_id`. The new version is DRAFT and invisible to users. +2. `get_blueprint_version` and edit the returned `definition` locally, preserving + everything not requested. +3. `update_blueprint_version` with the complete `definition`. Leave + `danger_zone_accepted` unset. +4. `create_signoff_configuration` per gated step; see [Signoffs](./signoffs.md). +5. `set_blueprint_version_status` to `ACTIVE` when activation is part of the task. + Check `trace.status` in the result. + +For a new blueprint, `create_blueprint` with `new_identifier` and a `blueprint` +body holding `name`, `icon`, `color`, and `backgroundColor`, without an `id`. Do +not assume a system version exists on a new blueprint or invent an origin ID. +`update_blueprint` replaces that metadata only. + +On a version used by artifacts, removing fields or changing types blocks the save. +A blocked save calls for a new version or an explicit decision covering that data +loss; do not retry with `danger_zone_accepted: true` on your own. Deleting a +version (`delete_blueprint_version`) requires removing its artifacts first; do not +cascade-delete them just to unblock a schema operation. + +## Definition structure + +| Key | Purpose and checks | +| --- | --- | +| `id` | Preserve the `{blueprintId, versionId}` of the target version. | +| `fieldDefinitions` | Dictionary keyed by field ID; inspect `fieldType`, `sourceType`, and list or required constraints. See [Artifacts](./artifacts.md). | +| `workflowDefinition` | Contains ordered `stepDefinitions`; preserve stable step IDs referenced by signoffs and UI definitions. | +| `uiDefinition` | Layout structure varies by release: inspect `views`, `uiStepDefinitions`, and the artifact page or tab configuration. Workflow and UI step IDs must match. | +| `logicalHookList` | Pre-phase hooks for validation and computed fields before commit. | +| `postLogicalHookList` | Govern 15 post-phase hooks for work after commit. Preserve both hook lists when editing unrelated settings. | +| `actions` | Action definitions keyed by action ID; a UI action component must reference the action for it to be visible. | + +An accepted definition can still render an empty artifact page. Check that the +configured artifact tabs or page and workflow steps resolve to nonempty views, and +that editable fields appear in the intended views. Older definitions may use +`artifactPageViewId`; Govern 15 definitions can use `tabs` and +`artifactStructureTabIds` instead. The optional `customRightPanel` and +`rightPanelTabIds` configure the selection panel independently. A custom tab can +be shared by the artifact page and right panel; its ID and view remain shared. +System tabs such as Workflow, Timeline, and Role assignments belong only on the +artifact page. Preserve stable tab IDs used in navigation, and check both orders +and separators. See [page structure](https://doc.dataiku.com/dss/latest/governance/blueprint-designer/blueprint-version-design.html#design-the-item-page-structure). + +Reuse components from an inspected definition rather than inventing UI keys. +Govern 15 text components have separate display settings (Markdown or plain text) +and editor settings (rich text, single-line, or multiline); configure these on the +view component without changing the field's `TEXT` type. State separately whether +layout was inspected in the UI or only checked structurally. + +Required field constraints apply globally; hiding a required field does not make +it optional. A hidden workflow step can bypass its mandatory signoff. Review view +and step visibility conditions when they affect the requested approval behavior. + +Field and workflow saves alone do not create review gates; use the signoff +configuration operations in [Signoffs](./signoffs.md). + +## Choose the hook phase + +Hooks select lifecycle events (`CREATE`, `UPDATE`, `DELETE`) and when to run: + +- Pre-phase hooks run before commit. Use them to validate changes or populate + computed fields. The operation can still fail afterward; keep external side + effects and API writes to other items out of these hooks. To schedule related + items' UPDATE hooks after commit, use `handler.artifactIdsToUpdate`. +- Govern 15 post-phase hooks run after commit. Use them for work that requires + persisted state, such as an authorized notification or external synchronization. + They cannot validate or veto an action that has already committed. Check the + saved item and any downstream effect separately before retrying. + +Use the target release's [hook documentation and editor samples](https://doc.dataiku.com/dss/latest/governance/blueprint-designer/blueprint-version-design.html#set-rules-with-hooks) +for the handler context. Older examples may emulate post-create work with threads +and polling; use native post-phase hooks on Govern 15 when that is the intent. +Treat scripts as executable changes, and verify execution as well as saved configuration. + +## Export and import boundary + +A local JSON snapshot can hold the full version definition, trace, and signoff +configurations read above. Keep it outside the plugin and protect sensitive content. +A definition alone is not a complete transferable blueprint package. + +The catalog has no blueprint version import operation, because the public SDK has +none. For a cross-instance import or artifact-version migration, report the missing +operation and use the Govern UI as the handoff. Do not claim that a JSON snapshot +supplies import parity. diff --git a/skills/dataiku-headless/references/govern/signoffs.md b/skills/dataiku-headless/references/govern/signoffs.md new file mode 100644 index 00000000..3caf264f --- /dev/null +++ b/skills/dataiku-headless/references/govern/signoffs.md @@ -0,0 +1,94 @@ +# Govern workflow and signoffs + +Use the shared Govern operation registry described in [Govern](../govern.md). A blueprint version's +signoff configuration defines a review gate. An artifact's signoff is the runtime +review, including feedback and approval. Inspect both before changing either. + +## Workflow state + +Read `get_artifact` and its blueprint version. On Govern 13.5+, use +`workflow.steps[step_id]` for state and visibility; order comes from the version's +`workflowDefinition.stepDefinitions`. `status.stepId` is deprecated and may name a +step that is not actually ongoing. See the +[official workflow reference](https://developer.dataiku.com/latest/concepts-and-examples/govern/govern-artifacts/govern-artifact-workflow.html). + +When asked to advance the workflow, read the current steps and required signoffs, +change only the intended step states in the full definition, save with +`update_artifact`, and re-read. Preserve visibility and unrelated steps. Do not skip +a required review or rewrite the legacy status field to make signoff creation succeed. + +## Configure a review gate + +- `list_signoff_configurations` with `blueprint_id` and `version_id` lists gates. +- `get_signoff_configuration` with `step_id` returns one. +- `create_signoff_configuration` with `step_id` and a `configuration` without a + top-level `id`; the server derives it from the step and version. +- `update_signoff_configuration` replaces the complete `configuration`. +- `delete_signoff_configuration` removes the gate. + +Configuration keys: + +- `title`, `mandatory`, `feedbackUsersGroups`, `approvers`, and + `recurrenceConfiguration`. Preserve unrelated settings when updating. +- Each feedback group has an `id`, a non-empty `title`, and a `users` list. The + group's ID is used by `add_signoff_feedback` and `delegate_signoff_feedback`; + it is not necessarily a Govern role ID. +- Reviewers and approvers are `{"usersContainer": {...}}` entries. The container + type is lowercase: `user` (`login`), `group` (`groupName`), `role` (`roleId`), + or `global-api-key` (`globalAPIKeyId`). Discover the user, group, or role + first. Approvers form a flat list. +- `recurrenceConfiguration` is `{activated, days, weeks, months, years, reloadConf}`; + activation needs a positive interval. + +Example: + +```json +{"title": "Review gate", "mandatory": true, + "feedbackUsersGroups": [{"id": "reviewers", "title": "Reviewers", + "users": [{"usersContainer": {"type": "user", "login": "alice"}}]}], + "approvers": [{"usersContainer": {"type": "user", "login": "bob"}}], + "recurrenceConfiguration": {"activated": false, "days": 0, "weeks": 0, "months": 0, "years": 0, "reloadConf": false}} +``` + +Re-read the configuration and check reviewers, approvers, `mandatory`, and the step. +Existing runtime signoffs keep their previous configuration until reset; a +configuration edit alone does not update every review. + +## Run a signoff + +- `list_signoffs` with `artifact_id` lists the runtime signoffs. +- `get_signoff` returns status, feedback responses, and the approver response. +- `get_signoff_details` resolves reviewer and approver membership to users. +- A signoff exists only for an ONGOING step. Setting `workflow.steps..status` + to `ONGOING` through `update_artifact` creates the signoff when the step has a + configuration; `create_signoff` covers the case where it is missing. Both fail + on a step that is not ONGOING. + +| Intent | Operation | +| --- | --- | +| Start feedback or approval collection | `update_signoff_status` with `status: WAITING_FOR_FEEDBACK` or `WAITING_FOR_APPROVAL` and `users_to_notify: []` | +| Read feedback | `list_signoff_feedbacks`, `get_signoff_feedback` | +| Record requested feedback | `add_signoff_feedback` with `group_id`, `status`, `comment` | +| Change or remove a feedback | `update_signoff_feedback`, `delete_signoff_feedback` | +| Read approval | `get_signoff_approval` | +| Record requested approval decision | `add_signoff_approval` with `status`, `comment` | +| Change or remove the approval | `update_signoff_approval`, `delete_signoff_approval` | +| Delegate reviewers | `delegate_signoff_feedback` with `group_id` and `users_container`, or `delegate_signoff_approval` with `users_container`; both take one `{"type": "user", "login": ...}` | +| Scheduled reset | `get_signoff_recurrence`, `update_signoff_recurrence` | + +Feedback statuses are `APPROVED`, `MINOR_ISSUE`, and `MAJOR_ISSUE`; approval statuses +are `APPROVED`, `REJECTED`, and `ABANDONED`. Signoff statuses move +`NOT_STARTED → WAITING_FOR_FEEDBACK → WAITING_FOR_APPROVAL → APPROVED | REJECTED | ABANDONED`. +Submit decisions only when requested, under the authenticated identity. +Administrative API access does not make that identity a reviewer or approver; check +the configured membership, and do not alter it to bypass a permission error. +Delegation changes who reviews and is not impersonation. + +Omitting `users_to_notify` or passing `[]` sends no notifications. Supply an +explicit recipient list only when the task authorizes notifications; when notifying, list `{userLogin, groupId}` entries for the feedback +stage and `{userLogin}` entries for the approval stage. Resetting an in-progress +review requires `ABANDONED` before `NOT_STARTED`; setting `APPROVED` through +`update_signoff_status` is not a substitute for recording an approval. +`reload_conf_for_reset: true` also clears delegations; do not set it incidentally. +A feedback can be deleted only while the signoff is `WAITING_FOR_FEEDBACK`, and +the approval only once the signoff is `APPROVED`, `REJECTED`, or `ABANDONED`. diff --git a/skills/dataiku-headless/references/govern/supporting-objects.md b/skills/dataiku-headless/references/govern/supporting-objects.md new file mode 100644 index 00000000..0cbf3c54 --- /dev/null +++ b/skills/dataiku-headless/references/govern/supporting-objects.md @@ -0,0 +1,93 @@ +# Govern supporting objects + +Use the shared Govern operation registry described in [Govern](../govern.md). These operations use the +Govern credentials and permissions. The identically named DSS user and group MCP +tools target DSS and must not be used here. + +## Roles, permissions, and identities + +| Task | Operations | +| --- | --- | +| Roles | `list_roles`, `get_role`, `create_role`, `update_role`, `delete_role` | +| Role assignments per blueprint | `list_role_assignments`, `get_role_assignments`, `create_role_assignments`, `update_role_assignments`, `delete_role_assignments` | +| Blueprint permissions | `get_default_blueprint_permissions`, `update_default_blueprint_permissions`, `list_blueprint_permissions`, `get_blueprint_permissions`, `create_blueprint_permissions`, `update_blueprint_permissions`, `delete_blueprint_permissions` | +| Users | `list_users`, `get_user`, `create_user`, `update_user`, `delete_user`, `get_own_user` | +| Groups | `list_groups`, `get_group`, `create_group`, `update_group`, `delete_group` | + +Read the existing definition before saving and re-read afterward. Role definitions +and blueprint role assignments are separate resources. Updating one role's binding +must preserve the other roles; `delete_role_assignments` removes every binding of +the blueprint. For global permissions on Govern 15, use `isGovernArchitect`; the +deprecated `mayManageGovern` key was removed. Discover existing permissions before +changing them. + +Role bindings live in `roleAssignmentsRules[role_id]`, a list of rules with +`criteria`, `userContainers`, and `fieldIds`. An unconditional binding has empty +`criteria` and `fieldIds` lists and the selected user or group containers in +`userContainers`. Inspect a matching rule and verify the saved membership; change +one role's rules inside the full assignment definition, never replace the whole +definition to change one role. + +`update_user` and `update_group` merge the given keys into the current settings or +definition. `create_user` needs a `password` for LOCAL users; other keys are optional. + +## Custom pages + +- `list_custom_pages` and `get_custom_page` show pages as the authenticated user + sees them. +- `get_custom_page_definition`, `create_custom_page`, `update_custom_page`, + `delete_custom_page`, `get_custom_pages_order`, and `update_custom_pages_order` + are designer operations. + +Custom pages can display artifact views or custom HTML. Discover the target +release's page types from an existing page; built-in `standard-page` entries are not +creatable custom pages. Preserve the page type and definition, and treat HTML and +scripts as executable content. An external embed needs a browser-accessible URL and +compatible authentication; a saved definition does not prove that the embedded page +renders. + +### Govern 15 grid pages and charts + +Prefer native grid pages for dashboards over Govern items. A `grid` page can contain +multiple tabs with table, chart, HTML, and nested subgrid tiles. Copy the payload +structure from a matching page read with `get_custom_page_definition` instead of +guessing nested keys. + +Build the chart's data settings before its appearance: + +1. Filter the intended items and add projections for the needed fields. By default, + only item IDs are projected. Table tiles fetch their own columns, so a working + table does not prove that a chart has the data it needs. +2. Define breakdown dimensions and measures, then select that breakdown and measure + in the chart. A subgrid inherits its parent's data settings unless overridden. +3. Use standard charts when they fit. Chart selections filter tables and charts + sharing the same data settings. Converting to a custom JavaScript or ECharts + chart cannot be reverted to the standard chart configuration. + +Re-read the saved definition, then check the page with real data: displayed counts +and measures, drill-down targets, and selection-panel behavior. A preview using +sample data does not validate the real query. Create pages hidden while validating +them, and make them visible when publication is part of the task. See +[custom page design](https://doc.dataiku.com/dss/latest/governance/custom-pages.html) +for the current tile, projection, aggregation, and chart options. + +## Files and time series + +- `upload_file` with a local `file_path` returns the file description; retain its + `id`. `get_uploaded_file` reads the description, `download_uploaded_file` writes + the content to an explicit `output_path` and never overwrites unless `overwrite` + is true, `delete_uploaded_file` removes it. +- Attach the returned file ID to the appropriate artifact field through + `update_artifact_fields`, then re-read the field. Uploading alone does not attach it. +- `create_time_series` returns `time_series_id`; retain it. Use + `get_time_series_values`, `push_time_series_values`, and + `delete_time_series_values` afterward. +- Datapoints contain `timestamp` in epoch **milliseconds** and `value`. + `push_time_series_values` with `upsert: true` (explicit overwrite intent; the default is false) overwrites existing + timestamps; choose the mode deliberately. Read the affected interval after a + write. `delete_time_series_values` removes values in its timestamp window, or all + values when neither bound is supplied; it does not delete the time-series object. + Verify boundary behavior before a range deletion: Govern 15.0.1 excludes both + endpoint timestamps from reads and deletes. + +Deleting all time-series values requires delete_all=true; otherwise provide a timestamp range. diff --git a/skills/dataiku-headless/references/managed_folders.md b/skills/dataiku-headless/references/managed_folders.md index 8dbe3710..1bde39c4 100644 --- a/skills/dataiku-headless/references/managed_folders.md +++ b/skills/dataiku-headless/references/managed_folders.md @@ -5,7 +5,7 @@ description: Understand and inspect Dataiku managed folders and, when explicitly # Managed Folders -Folder reads are available on `design` and `automation`. Creation, file upload, and Cobuild changes require `design`. +Folder reads are available on `design`, `automation`, and `agent-management`. Creation, file upload, and Cobuild changes require `design` or `agent-management`. Use this guide to understand and inspect managed folders, and to handle the narrow direct exceptions for folder creation and local-file upload. diff --git a/skills/dataiku-headless/references/migrations.md b/skills/dataiku-headless/references/migrations.md index e6ba43f1..85895417 100644 --- a/skills/dataiku-headless/references/migrations.md +++ b/skills/dataiku-headless/references/migrations.md @@ -6,7 +6,7 @@ description: Translate business logic from third-party tools (e.g. Alteryx, Tabl # Migrations -Migration construction requires a `design` instance. Automation can inspect and execute existing assets but cannot build the migrated project or use Cobuild. +Migration construction requires a `design` or `agent-management` instance. Automation can inspect and execute existing assets but cannot build the migrated project or use Cobuild. Translate business logic from third-party tools (e.g. Alteryx, Tableau Prep, SAS, Excel) into runnable Dataiku flows. diff --git a/skills/dataiku-headless/references/operations.md b/skills/dataiku-headless/references/operations.md new file mode 100644 index 00000000..fb265858 --- /dev/null +++ b/skills/dataiku-headless/references/operations.md @@ -0,0 +1,54 @@ +--- +name: operations +description: Discover and execute Deployer, Govern and Agent Management operations with typed arguments, node policy and transport checks. +--- + +# Shared operation interface + +Deployer, Govern, Agent Management profiles use four fixed MCP tools: search_operations, +describe_operation, run_read_operation and run_write_operation. Product actions +are operation IDs, not individually registered MCP tools. The catalog stays stable +across instance switches. Other profile types cannot use these tools. + +## Workflow + +1. Select the intended profile using list_instances, get_current_instance and + switch_instance. A configured node type is policy, not live feature detection. +2. Search by query, domain (deployer, govern, agent-management), and effect. + Search filters by selected type. Results default to 10 (maximum 50); use the + returned next_offset to page with offset, or narrow the query. All four v2 + tools require a selected profile; search and describe inspect the local catalog + without contacting Dataiku. +3. Describe a returned operation_id. Inspect input_schema, category, effects, + allowed types, requirements, transports, local_file_fields and response_contract. + Definitions and settings vary by backend release, infrastructure or blueprint; + nested objects explicitly described as backend-defined are not fully specified + by the structural schema. Read their current definitions before replacing them. +Never submit redacted placeholders from a read as replacement values; the write +dispatcher rejects them. Prefer a scoped update that preserves hidden fields, or +report the coverage gap. + +4. Discover IDs through reads, then execute with the matching read/write tool and + an arguments object. Example IDs are structural placeholders, not live defaults. + Supply expected_instance with the selected instance name to reject stale context. +5. Inspect the returned result and re-read after mutation. Retain operation/future + IDs and revisions for recovery. Never resubmit an uncertain write automatically. + +Middleware checks the tool’s allowed instance types before exchanging credentials. +Execution handlers check operation IDs, node type, effects, argument shape, selected +instance and transport before calling the SDK. A read tool cannot +execute a mutation. Backend permissions remain authoritative. server_support is +unknown until actual read-only evidence establishes availability; do not infer a +missing feature from a single 404 or probe with writes. + +## Coverage and transport + +- [Deployer](./deployer.md): 13 public SDK operations for Project/API Deployer, + including deployment future polling. Select a Deployer profile even when it + shares a URL with a Design or Automation node. Supports stdio and HTTP. +- [Govern](./govern.md): 86 public SDK operations for artifacts, blueprints, signoffs and supporting administration. Govern requires stdio; HTTP execution is rejected before backend token exchange. +- [Agent Management](./agent-management.md): 46 public SDK operations for platforms, agents, metrics, monitoring, risk and settings. Supports stdio and HTTP; local-file transfers require stdio. + +Adapters use public Python SDK methods only. Do not fall back to direct endpoint +calls or another identity when coverage is missing. Project construction remains +with Cobuild on supported Design profiles and supporting Agent Management projects. No dynamic tool registration or code sandbox is used. diff --git a/skills/dataiku-headless/references/project-deployer.md b/skills/dataiku-headless/references/project-deployer.md new file mode 100644 index 00000000..18586fe8 --- /dev/null +++ b/skills/dataiku-headless/references/project-deployer.md @@ -0,0 +1,43 @@ +--- +name: project-deployer +description: Deploy already published project bundles to Automation infrastructures and verify the resulting project deployment. +--- + +# Project Deployer + +Read [the shared Deployer guide](./deployer.md) first for instance selection, +coverage and recovery. Pass arguments through the shared read/write dispatcher. All operations below use `deployer_type="project"`. + +A published project on Deployer is a collection of immutable bundle versions, not +a Design-node project that Cobuild can edit. A deployment associates a published +project and bundle with an Automation infrastructure and target project key. +Stages are infrastructure metadata, not configured instance types. + +1. Discover infrastructures and stages. Inspect the intended infrastructure, + associated deployments, governance policy and reported permissions. +2. Discover published projects and inspect the selected source_id to identify its + package IDs. Do not infer a bundle ID from a Design project name or version. +3. For an existing deployment, inspect its saved settings, health and update history. +4. Create configuration with deployment_id, source_id (published project key), + infra_id and version (bundle ID). Optional deployed_project_key and + project_folder_id select the Automation target key and folder. Without these, + Dataiku uses the published project key and root folder. Discover any folder ID + on the target Automation instance; switch back to Deployer before creating. + Never choose an existing target project key without understanding the overwrite + effects of applying the bundle. +5. To promote or roll back an existing deployment, save a discovered bundle ID using + `deployer.update_deployment(version=...)`. This preserves other deployment settings. + The enabled argument and endpoint_id are API-only and rejected here. +6. Run `deployer.start_update` to apply the saved bundle. This may replace the + Automation project's assets. Follow the future and exact update record, then + re-read deployment health as described in the shared guide. +7. When project/output validation is needed, switch to the configured Automation + instance, discover the actual deployed project and inspect it with the project, + dataset and scenario guides. Deployer success does not establish output parity. + +Bundle creation/publication, deployment remapping, code-environment behavior, +scenario activation and deployment tests are outside this tool surface. They +must already be configured in Dataiku. Do not use an arbitrary settings patch +or Cobuild on the Deployer to emulate those operations. + +[Official Python API reference](https://developer.dataiku.com/latest/api-reference/python/project-deployer.html) diff --git a/skills/dataiku-headless/references/project-libraries.md b/skills/dataiku-headless/references/project-libraries.md index c6db4aef..a2ef9293 100644 --- a/skills/dataiku-headless/references/project-libraries.md +++ b/skills/dataiku-headless/references/project-libraries.md @@ -5,7 +5,7 @@ description: Understand and inspect a Dataiku project's library tree and, when e # Project Libraries -Library reads, search, and validation are available on `design` and `automation`. Direct file writes and Cobuild changes require `design`. +Library reads, search, and validation are available on `design`, `automation`, and `agent-management`. Direct file writes and Cobuild changes require `design` or `agent-management`. Use this guide to understand and inspect the project library, and to handle the direct local-source-file write exception. diff --git a/skills/dataiku-headless/references/projects.md b/skills/dataiku-headless/references/projects.md index a1c59ef2..2759cea7 100644 --- a/skills/dataiku-headless/references/projects.md +++ b/skills/dataiku-headless/references/projects.md @@ -5,7 +5,7 @@ description: Understand and inspect Dataiku projects, their metadata, variables, # Projects -Project creation and variable writes require `design`. Project and Flow reads are also available on `automation`; Cobuild is not. +Project creation and variable writes require `design` or `agent-management`. Project and Flow reads are also available on `automation`; Cobuild is not. Use this guide to understand and inspect Dataiku projects, update project variables, and gather grounded context for Cobuild. diff --git a/skills/dataiku-headless/references/projects/settings.md b/skills/dataiku-headless/references/projects/settings.md index eb2555e7..80ca775f 100644 --- a/skills/dataiku-headless/references/projects/settings.md +++ b/skills/dataiku-headless/references/projects/settings.md @@ -5,7 +5,7 @@ description: Inspect and update Dataiku project settings. Use for Flow display a # Project Settings -`update_project_settings` requires `design`. Automation can inspect settings but cannot update them. +`update_project_settings` requires `design` or `agent-management`. Automation can inspect settings but cannot update them. Project settings are a narrow direct-write exception for configuration that Cobuild does not manage. diff --git a/tests/test_agent_management_operations.py b/tests/test_agent_management_operations.py new file mode 100644 index 00000000..adfecbce --- /dev/null +++ b/tests/test_agent_management_operations.py @@ -0,0 +1,930 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Expanded product coverage, SDK routing, authentication and local-file boundaries.""" + +import asyncio +import copy +import json +import threading +from dataclasses import replace +from types import SimpleNamespace + +import pytest +import requests +from dataikuapi.utils import DataikuException +from fastmcp import Client + +from dataiku_mcp import mcp, auth, server +from dataiku_mcp.config import http, request, stdio +from dataiku_mcp.operations_catalog import OPERATIONS +from dataiku_mcp.operation_registry import get_operation +from dataiku_mcp.tools import agent_management, deployer +from tests.test_instance_policy import _instance +from tests.test_operations import _call, _http_setup + +SECRET = "do-not-return-this-secret" + +EXPECTED_AGENT_MANAGEMENT_IDS = { + "delete_agent", + "abort_platform_operation", + "get_agent_operation", + "scan_platform", + "rename_tag", + "update_platform_settings", + "create_agent", + "setup_agent_monitoring", + "get_agent_risk_assessment", + "list_tags", + "create_platform", + "get_agent_operation_result", + "delete_tag", + "get_platform", + "list_platforms", + "delete_platform", + "get_platform_operation_result", + "update_settings", + "update_risk_taxonomy", + "run_platform_uptime_tests", + "abort_agent_operation", + "search_operations", + "evaluate_alerts", + "get_agent_uptime_results", + "get_metric_buckets", + "search_agents", + "update_agent_settings", + "get_agent", + "fetch_agent_operational_metrics", + "fetch_agent_logs", + "sign_off_agent_risk_assessment", + "update_agent_risk_assessment", + "get_settings", + "update_tag", + "remove_agent_risk_sign_off", + "get_default_risk_taxonomy", + "get_platform_settings", + "compute_risk_taxonomy_replacement_impact", + "get_risk_taxonomy", + "get_agent_settings", + "get_available_metric_ids", + "download_agent_risk_evidence", + "get_platform_operation", + "get_metric_time_series", + "get_risk_exposure", + "get_agent_status", +} + + +class EvidenceResponse: + headers = {"Content-Type": "application/octet-stream"} + + def __init__(self): + self.closed = False + + def iter_content(self, chunk_size): + yield b"evidence" + + def close(self): + self.closed = True + + +class AgentManagementBackend: + """Fake HTTP boundary beneath the real public SDK objects.""" + + def __init__(self): + self.calls = [] + self.main_thread = threading.get_ident() + self.settings = { + "fetchedRevision": "revision-from-read", + "displayName": "Original", + "connection": {"password": SECRET}, + "untouched": 7, + "businessKPIs": [], + } + self.agent_settings = { + "id": "discovered-agent", + "infraId": "discovered-platform", + "fetchedRevision": "revision-from-read", + "configuration": { + "displayName": "Original", + "description": "Preserved description from the inspected configuration", + }, + "permissions": [], + } + + def send(self, session, prepared, **kwargs): + from urllib.parse import urlsplit + + assert threading.get_ident() != self.main_thread + path = urlsplit(prepared.url).path + assert path.startswith("/dip/publicapi/agent-management/") + self.calls.append(prepared) + payload = {"ok": True, "password": SECRET} + if path.endswith("/platforms"): + payload = ( + [{"id": "discovered-platform"}] + if prepared.method == "GET" + else {"id": "requested-platform"} + ) + elif path.endswith("/agents/search"): + payload = { + "agents": [{"id": "discovered-agent", "password": SECRET}], + "facets": {}, + } + elif path.endswith("/operations/search"): + payload = { + "operations": [ + { + "id": "operation", + "infraId": "discovered-platform", + "password": SECRET, + } + ], + "hasMore": True, + } + elif "/agents/metrics/" in path: + payload = [] + elif path.endswith("/tags"): + payload = [{"name": "discovered-tag", "color": "red"}] + elif "/tags/" in path: + payload = {"name": "requested-tag", "color": "blue"} + elif path.endswith("/settings"): + if "/agents/" in path: + if prepared.method == "PUT": + body = json.loads(prepared.body) + # The public update DTO ignores unknown root fields. + for field in ("configuration", "permissions", "fetchedRevision"): + if field in body: + self.agent_settings[field] = body[field] + payload = self.agent_settings + else: + if prepared.method == "PUT": + self.settings = json.loads(prepared.body) + payload = self.settings + elif path.endswith("/risk-exposure"): + payload = { + "exposure": { + "agents": [ + { + "key": "discovered-platform:discovered-agent", + "id": "discovered-agent", + "infrastructureName": "discovered-platform", + "password": SECRET, + } + ], + "riskExposures": [ + {"agentKey": "discovered-platform:discovered-agent", "score": 2} + ], + "password": SECRET, + } + } + elif path.endswith("/uptime-results"): + payload = [] + elif "/risk-taxonomy" in path: + payload = { + "taxonomy": {}, + "fetchedRevision": "revision-from-read", + "fingerprint": "fingerprint-from-impact", + } + elif "/risk-assessment/evidences/" in path: + response = requests.Response() + response.status_code = 200 + response._content = b"evidence" + response._content_consumed = True + return response + elif "/risk-assessment" in path: + payload = {"assessment": {}, "fetchedRevision": "revision-from-read"} + elif path.endswith("/agents") and prepared.method == "POST": + payload = {"id": "requested-agent"} + elif path.endswith("/evaluate-alerts"): + payload = {"queuedAgentCount": 2} + elif "/actions/" in path: + payload = { + "operation": {"id": "returned-operation", "agentId": "discovered-agent"} + } + elif path.endswith("/result"): + payload = {"result": {"value": 1, "password": SECRET}} + response = requests.Response() + response.status_code = 200 + response._content = json.dumps(payload).encode() + return response + + +@pytest.mark.parametrize("name", sorted(EXPECTED_AGENT_MANAGEMENT_IDS)) +def test_every_agent_management_action_uses_real_public_sdk( + monkeypatch, tmp_path, name +): + operation = OPERATIONS["agent-management." + name] + monkeypatch.setattr(stdio, "_current_instance", _instance("agent-management")) + backend = AgentManagementBackend() + monkeypatch.setattr( + requests.Session, + "send", + lambda session, prepared, **kwargs: backend.send(session, prepared, **kwargs), + ) + arguments = copy.deepcopy(operation.example) + if "output_path" in operation.local_file_fields: + arguments["output_path"] = str(tmp_path / "evidence.bin") + result = asyncio.run( + _call( + "run_read_operation" + if operation.effect == "read" + else "run_write_operation", + operation_id=operation.operation_id, + arguments=arguments, + ) + ) + assert backend.calls + assert SECRET not in json.dumps(result) + if "output_path" in operation.local_file_fields: + assert (tmp_path / "evidence.bin").read_bytes() == b"evidence" + if name in { + "scan_platform", + "fetch_agent_logs", + "fetch_agent_operational_metrics", + "run_platform_uptime_tests", + }: + assert result["result"]["operation_id"] == "returned-operation" + if name == "evaluate_alerts": + assert result["result"] == {"queued_agent_count": 2} + if name == "update_agent_settings": + assert result["result"]["configuration"] == { + "displayName": "Requested name", + "description": "Preserved description from the inspected configuration", + } + assert "displayName" not in json.loads(backend.calls[-1].body) + + +def test_agent_management_post_searches_are_reads_and_alerts_require_scope(): + assert OPERATIONS["agent-management.search_agents"].effect == "read" + assert OPERATIONS["agent-management.search_operations"].effect == "read" + operation = OPERATIONS["agent-management.evaluate_alerts"] + assert operation.effect == "write" + with pytest.raises(ValueError): + operation.validate({}) + operation.validate({"scope": "all_writable_agents"}) + operation.validate({"agent_filter": {"favorites": False}}) + with pytest.raises(ValueError): + operation.validate({"agent_filter": {"platform_ids": []}}) + + +@pytest.mark.parametrize( + "arguments, expected_query", + [ + ({}, {}), + ({"from_time": 0}, {"from": ["1970-01-01T00:00:00Z"]}), + ({"to_time": 1767225600123}, {"to": ["2026-01-01T00:00:00.123000Z"]}), + ( + {"from_time": 1767225600000, "to_time": 1767312000000}, + {"from": ["2026-01-01T00:00:00Z"], "to": ["2026-01-02T00:00:00Z"]}, + ), + ], +) +def test_uptime_bounds_use_iso_utc_on_the_wire(monkeypatch, arguments, expected_query): + from urllib.parse import parse_qs, urlsplit + + monkeypatch.setattr(stdio, "_current_instance", _instance("agent-management")) + backend = AgentManagementBackend() + monkeypatch.setattr( + requests.Session, + "send", + lambda session, prepared, **kwargs: backend.send(session, prepared, **kwargs), + ) + result = asyncio.run( + _call( + "run_read_operation", + operation_id="agent-management.get_agent_uptime_results", + arguments={ + "platform_id": "discovered-platform", + "agent_id": "discovered-agent", + **arguments, + }, + ) + ) + assert result["result"] == [] + assert parse_qs(urlsplit(backend.calls[-1].url).query) == expected_query + + +def test_risk_exposure_preserves_agent_join_and_redacts_credentials(monkeypatch): + monkeypatch.setattr(stdio, "_current_instance", _instance("agent-management")) + backend = AgentManagementBackend() + monkeypatch.setattr( + requests.Session, + "send", + lambda session, prepared, **kwargs: backend.send(session, prepared, **kwargs), + ) + exposure = asyncio.run( + _call( + "run_read_operation", + operation_id="agent-management.get_risk_exposure", + arguments={}, + ) + )["result"] + joined = [ + agent + for agent in exposure["agents"] + if agent["agentKey"] == exposure["riskExposures"][0]["agentKey"] + ] + assert len(joined) == 1 + assert joined[0]["id"] == "discovered-agent" + assert "key" not in joined[0] + assert joined[0]["password"] == "__DATAIKU_REDACTED__" + assert exposure["password"] == "__DATAIKU_REDACTED__" + + +def test_agent_rename_preserves_inspected_configuration_and_permissions(monkeypatch): + monkeypatch.setattr(stdio, "_current_instance", _instance("agent-management")) + backend = AgentManagementBackend() + backend.agent_settings["configuration"]["tags"] = ["preserve-tag"] + backend.agent_settings["permissions"] = [{"group": "operators", "read": True}] + monkeypatch.setattr( + requests.Session, + "send", + lambda session, prepared, **kwargs: backend.send(session, prepared, **kwargs), + ) + current = asyncio.run( + _call( + "run_read_operation", + operation_id="agent-management.get_agent_settings", + arguments={ + "platform_id": "discovered-platform", + "agent_id": "discovered-agent", + }, + ) + )["result"] + configuration = copy.deepcopy(current["configuration"]) + configuration["displayName"] = "Requested name" + result = asyncio.run( + _call( + "run_write_operation", + operation_id="agent-management.update_agent_settings", + arguments={ + "platform_id": "discovered-platform", + "agent_id": "discovered-agent", + "changes": {"configuration": configuration}, + "expected_revision": current["fetchedRevision"], + }, + ) + )["result"] + assert result["configuration"] == { + **current["configuration"], + "displayName": "Requested name", + } + assert result["permissions"] == current["permissions"] + assert ( + json.loads(backend.calls[-1].body)["fetchedRevision"] + == current["fetchedRevision"] + ) + + +def test_backend_defined_settings_require_objects_not_scalars(): + for name, field in [ + ("update_platform_settings", "changes"), + ("update_agent_settings", "changes"), + ("update_risk_taxonomy", "taxonomy"), + ]: + operation = OPERATIONS["agent-management." + name] + with pytest.raises(ValueError): + operation.validate({**operation.example, field: 42}) + assert "Backend-defined" in json.dumps(operation.describe()["input_schema"]) + + +def test_agent_management_local_files_rejected_before_sdk_execution(monkeypatch): + events = _http_setup(monkeypatch, "agent-management") + args = { + "platform_id": "platform", + "agent_id": "agent", + "assessment": {}, + "expected_revision": "revision-from-read", + "documents": {"evidence": "/private/local-file"}, + } + + async def run(): + async with Client(mcp) as client: + result = await client.call_tool( + "run_write_operation", + { + "operation_id": "agent-management.update_agent_risk_assessment", + "arguments": args, + }, + raise_on_error=False, + ) + assert result.is_error + assert "filesystem" in result.content[0].text + + asyncio.run(run()) + assert events == ["exchange"] + + +def test_agent_management_multipart_without_files_uses_native_http_auth(monkeypatch): + events = _http_setup(monkeypatch, "agent-management") + backend = AgentManagementBackend() + + def send(session, prepared, **kwargs): + assert prepared.headers["Authorization"] == "Bearer backend-token" + assert "X-DKU-APITicket" not in prepared.headers + return backend.send(session, prepared, **kwargs) + + monkeypatch.setattr(requests.Session, "send", send) + asyncio.run( + _call( + "run_write_operation", + operation_id="agent-management.update_agent_risk_assessment", + arguments={ + "platform_id": "platform", + "agent_id": "agent", + "assessment": {}, + "expected_revision": "revision-from-read", + }, + ) + ) + assert events == ["exchange"] + assert len(backend.calls) == 2 + assert b"revision-from-read" in backend.calls[-1].body + assert b"filename=" not in backend.calls[-1].body + + +def test_binary_download_does_not_clobber_destination_created_during_transfer( + monkeypatch, tmp_path +): + destination = tmp_path / "evidence.bin" + response = EvidenceResponse() + + def chunks(chunk_size): + destination.write_bytes(b"created-concurrently") + yield b"replacement" + + response.iter_content = chunks + with pytest.raises(FileExistsError): + agent_management._download_binary( + response, + str(destination), + False, + ) + assert destination.read_bytes() == b"created-concurrently" + assert response.closed + assert list(tmp_path.iterdir()) == [destination] + + +def test_redacted_credentials_cannot_be_written_back(monkeypatch): + events = _http_setup(monkeypatch, "agent-management") + + async def run(): + async with Client(mcp) as client: + result = await client.call_tool( + "run_write_operation", + { + "operation_id": "agent-management.update_settings", + "arguments": { + "body": {"connection": {"password": "__DATAIKU_REDACTED__"}} + }, + }, + raise_on_error=False, + ) + assert result.is_error + assert "redacted result placeholders" in result.content[0].text + + asyncio.run(run()) + assert events == ["exchange"] + + +@pytest.mark.parametrize("kind", ["key", "ticket", "http"]) +def test_native_agent_management_sdk_preserves_credentials_and_certificates( + monkeypatch, kind +): + instance = replace( + _instance("agent-management"), + api_key=SECRET if kind == "key" else None, + api_ticket=SECRET if kind == "ticket" else None, + encrypted_rpc_cert_path="/private/certificate.pem", + ) + monkeypatch.setattr(request, "get_pinned_instance", lambda: instance) + monkeypatch.setattr(request, "is_http_request", lambda: kind == "http") + monkeypatch.setattr(request, "get_http_dss_token", lambda: SECRET) + client = auth.get_dss_client().get_agent_management_client() + prepared = client._session.prepare_request(requests.Request("GET", instance.url)) + assert client._session.verify == "/private/certificate.pem" + assert prepared.headers["X-DKU-Client-Application"] == "dataiku-headless" + if kind == "http": + assert prepared.headers["Authorization"] == "Bearer " + SECRET + assert "X-DKU-APITicket" not in prepared.headers + elif kind == "ticket": + assert prepared.headers["X-DKU-APITicket"] == SECRET + assert "Authorization" not in prepared.headers + else: + assert prepared.headers["Authorization"].startswith("Basic ") + + +@pytest.mark.parametrize( + "operation_name", ["get_metric_time_series", "get_metric_buckets"] +) +def test_native_metric_results_and_sdk_filter_mapping(monkeypatch, operation_name): + monkeypatch.setattr(stdio, "_current_instance", _instance("agent-management")) + calls = [] + bucketed = operation_name == "get_metric_buckets" + + def send(session, prepared, **kwargs): + calls.append(json.loads(prepared.body)) + response = requests.Response() + response.status_code = 200 + response._content = json.dumps( + [ + { + "infrastructureName": "platform", + "agentId": "agent", + "columns": ["start", "end", "activity.interactions"] + if bucketed + else ["timestamp", "activity.interactions"], + "rows": [[0, 86400000, 0]] if bucketed else [[0, 0]], + } + ] + ).encode() + return response + + monkeypatch.setattr(requests.Session, "send", send) + args = { + "metrics": ["activity.interactions"], + "from_time": 0, + "to_time": 86400000, + "agent_filter": { + "platform_ids": ["platform"], + "agent_ids": ["agent"], + "favorites": False, + }, + } + if bucketed: + args["bucket"] = "DAY" + result = asyncio.run( + _call( + "run_read_operation", + operation_id="agent-management." + operation_name, + arguments=args, + ) + ) + assert calls[0]["agentFilter"] == { + "platformName": ["platform"], + "agentId": ["agent"], + "favorites": False, + } + assert result["result"][0]["rows"][0][-1] == 0 + assert result["result"][0]["rows"][0][0] == "1970-01-01 00:00:00+00:00" + + +def test_risk_evidence_upload_uses_sdk_revision_and_document_mapping( + monkeypatch, tmp_path +): + monkeypatch.setattr(stdio, "_current_instance", _instance("agent-management")) + backend = AgentManagementBackend() + monkeypatch.setattr( + requests.Session, + "send", + lambda session, prepared, **kwargs: backend.send(session, prepared, **kwargs), + ) + document = tmp_path / "evidence.txt" + document.write_text("evidence contents") + asyncio.run( + _call( + "run_write_operation", + operation_id="agent-management.update_agent_risk_assessment", + arguments={ + "platform_id": "platform", + "agent_id": "agent", + "assessment": {"answer": 0}, + "expected_revision": "inspected-revision", + "documents": {"evidence-id": str(document)}, + }, + ) + ) + sent = backend.calls[-1] + assert sent.method == "PUT" + assert b"inspected-revision" in sent.body + assert b'name="fetchedRevision"' in sent.body + assert b"evidence-id" in sent.body + assert b"evidence.txt" in sent.body + assert b"evidence contents" in sent.body + + +@pytest.fixture +def selected(monkeypatch): + instance = _instance("agent-management") + monkeypatch.setattr(stdio, "_current_instance", instance) + return instance + + +@pytest.mark.parametrize( + "kind, total", + [ + ("deployer", 13), + ("agent-management", 46), + ("govern", 86), + ], +) +def test_search_filters_without_mutating_catalog(monkeypatch, kind, total): + monkeypatch.setattr(stdio, "_current_instance", _instance(kind)) + + async def run(): + before = {tool.name for tool in await mcp.list_tools()} + result = await _call("search_operations", limit=50) + assert result["total"] == total + assert {tool.name for tool in await mcp.list_tools()} == before + + asyncio.run(run()) + + +def test_search_query_effect_and_describe_disallowed(selected): + result = asyncio.run( + _call("search_operations", query="agent settings", effect="write") + ) + assert "agent-management.update_agent_settings" in [ + op["operation_id"] for op in result["operations"] + ] + detail = asyncio.run( + _call("describe_operation", operation_id="deployer.start_update") + ) + assert detail["allowed_on_selected_instance"] is False + + +@pytest.mark.parametrize( + "tool, operation_id, arguments, kind", + [ + ( + "run_read_operation", + "agent-management.update_agent_settings", + {}, + "agent-management", + ), + ( + "run_write_operation", + "agent-management.get_platform", + {"platform_id": "platform"}, + "agent-management", + ), + ( + "run_read_operation", + "deployer.list_stages", + {"deployer_type": "project"}, + "agent-management", + ), + ("run_read_operation", "agent-management.list_platforms", {}, "deployer"), + ("run_read_operation", "not-in-catalog", {}, "design"), + ( + "run_read_operation", + "agent-management.get_platform", + {"platform_id": "platform", "path": "/admin"}, + "agent-management", + ), + ( + "run_read_operation", + "agent-management.get_platform", + 42, + "agent-management", + ), + ], +) +def test_invalid_operation_rejected_before_sdk_execution( + monkeypatch, tool, operation_id, arguments, kind +): + events = _http_setup(monkeypatch, kind) + sdk_calls = [] + + def unexpected_client(): + sdk_calls.append(True) + raise AssertionError("Invalid operations must not reach the SDK") + + monkeypatch.setattr(agent_management, "get_dss_client", unexpected_client) + monkeypatch.setattr(deployer, "get_dss_client", unexpected_client) + + async def run(): + async with Client(mcp) as client: + result = await client.call_tool( + tool, + {"operation_id": operation_id, "arguments": arguments}, + raise_on_error=False, + ) + assert result.is_error + assert not request.is_http_request() + assert request._pinned_instance.get() is None + + asyncio.run(run()) + assert events == ([] if kind in {"design", "automation"} else ["exchange"]) + assert not sdk_calls + + +@pytest.mark.parametrize( + "bad_id", [".", "..", " .. ", "x/y", "x\\y", "%2e%2e", "%252f", "a\n", " "] +) +@pytest.mark.parametrize( + "operation_id, arguments", + [ + ("agent-management.get_platform", {}), + ("deployer.get_infrastructure", {"deployer_type": "project"}), + ], +) +def test_path_escape_rejected_before_sdk_execution( + monkeypatch, bad_id, operation_id, arguments +): + kind = ( + "agent-management" + if operation_id.startswith("agent-management.") + else "deployer" + ) + events = _http_setup(monkeypatch, kind) + + async def run(): + async with Client(mcp) as client: + result = await client.call_tool( + "run_read_operation", + { + "operation_id": operation_id, + "arguments": { + **arguments, + ( + "platform_id" + if operation_id.startswith("agent-management.") + else "infra_id" + ): bad_id, + }, + }, + raise_on_error=False, + ) + assert result.is_error + + asyncio.run(run()) + assert events == ["exchange"] + + +def test_agent_management_settings_preserve_hidden_fields_and_revision( + selected, monkeypatch +): + owner_thread = threading.get_ident() + raw = { + "fetchedRevision": "revision-from-read", + "apiKey": SECRET, + "untouched": 7, + "value": 9, + } + saved = [] + + def save(**kwargs): + assert threading.get_ident() != owner_thread + saved.append(raw.copy()) + return settings + + settings = SimpleNamespace(get_raw=lambda: raw, save=save) + sdk = SimpleNamespace(get_settings=lambda: settings) + monkeypatch.setattr( + agent_management, + "get_dss_client", + lambda: SimpleNamespace(get_agent_management_client=lambda: sdk), + ) + result = asyncio.run( + _call( + "run_write_operation", + operation_id="agent-management.update_settings", + arguments={ + "changes": {"value": 0, "nullable": None}, + "expected_revision": "revision-from-read", + }, + expected_instance=selected.name, + ) + ) + assert saved == [ + { + "fetchedRevision": "revision-from-read", + "apiKey": SECRET, + "untouched": 7, + "value": 0, + "nullable": None, + } + ] + assert SECRET not in json.dumps(result) + + +def test_sdk_conflict_is_sanitized_and_not_retried(selected, monkeypatch): + calls = [] + + def conflict(*args, **kwargs): + calls.append(args) + raise DataikuException(SECRET) + + monkeypatch.setattr( + agent_management, + "get_dss_client", + lambda: SimpleNamespace( + get_agent_management_client=lambda: SimpleNamespace(get_settings=conflict) + ), + ) + + async def run(): + async with Client(mcp) as client: + result = await client.call_tool( + "run_write_operation", + { + "operation_id": "agent-management.update_settings", + "arguments": { + "changes": {"value": 0}, + "expected_revision": "stale", + }, + }, + raise_on_error=False, + ) + assert result.is_error + assert SECRET not in result.content[0].text + assert "Re-read" in result.content[0].text + + asyncio.run(run()) + assert len(calls) == 1 + + +@pytest.mark.parametrize("changes", [42, None, SECRET]) +def test_invalid_settings_schema_does_not_echo_payload(changes): + operation = get_operation("agent-management.update_settings") + with pytest.raises(ValueError) as error: + operation.validate({"changes": changes, "expected_revision": SECRET}) + assert SECRET not in str(error.value) + + +def test_http_pinning_survives_switch_during_exchange(monkeypatch): + profiles = {kind: _instance(kind) for kind in ("agent-management", "govern")} + selection = {"issuer": {"alice": "agent-management"}} + monkeypatch.setattr( + http, "get_instances_and_selections", lambda: (profiles, selection) + ) + monkeypatch.setattr( + server, + "get_access_token", + lambda: SimpleNamespace( + token="mcp-token", claims={"iss": "issuer", "sub": "alice"} + ), + ) + + async def exchange(_): + selection["issuer"]["alice"] = "govern" + await asyncio.sleep(0) + return "backend-token" + + monkeypatch.setattr(server, "exchange_http_token", exchange) + + def get_client(): + assert request.get_pinned_instance().name == "agent-management" + assert request.get_http_dss_token() == "backend-token" + return SimpleNamespace( + get_agent_management_client=lambda: SimpleNamespace( + get_platform=lambda _: SimpleNamespace( + get_status=lambda: SimpleNamespace( + get_raw=lambda: {"id": "platform"} + ) + ) + ) + ) + + monkeypatch.setattr(agent_management, "get_dss_client", get_client) + result = asyncio.run( + _call( + "run_read_operation", + operation_id="agent-management.get_platform", + arguments={"platform_id": "platform"}, + ) + ) + assert result["instance_name"] == "agent-management" + assert not request.is_http_request() + + +def test_agent_management_catalog_covers_expected_actions_without_new_mcp_tools(): + ids = { + op.operation_id.split(".", 1)[1] + for op in OPERATIONS.values() + if op.domain == "agent-management" + } + assert ids == EXPECTED_AGENT_MANAGEMENT_IDS + tools = asyncio.run(mcp.list_tools()) + assert len(tools) == 135 + assert not any( + tool.name == "govern" or "mira" in tool.name or "deployer" in tool.name + for tool in tools + ) + assert not any( + "mira" in op.operation_id or "MIRA" in op.description + for op in OPERATIONS.values() + ) + + +def test_agent_management_adapter_uses_only_public_sdk_methods(): + import ast + import inspect + + for node in ast.walk(ast.parse(inspect.getsource(agent_management))): + if isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute): + assert not node.func.attr.startswith("_perform") + assert node.func.attr not in {"request", "urlopen"} diff --git a/tests/test_deployer.py b/tests/test_deployer.py new file mode 100644 index 00000000..035820b6 --- /dev/null +++ b/tests/test_deployer.py @@ -0,0 +1,447 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Deployer SDK contract, async boundary and secret-exposure regressions.""" + +import asyncio +import copy +import json +import threading +from types import SimpleNamespace + +import pytest +from dataikuapi.dss.apideployer import DSSAPIDeployer +from dataikuapi.dss.projectdeployer import DSSProjectDeployer +from dataikuapi.utils import DataikuException +from fastmcp import Client +from requests import Timeout + +from dataiku_mcp import mcp +from dataiku_mcp.tools import deployer + +SECRET = "DO_NOT_RETURN_THIS_SECRET" + + +class Backend: + """Use real SDK handles against an in-memory HTTP boundary.""" + + def __init__(self, kind): + self.kind = kind + self.calls = [] + self.owner_thread = threading.get_ident() + self.settings = { + "id": "deployment", + "infraId": "target", + "bundleId": "old", + "publishedProjectKey": "source", + "publishedServiceId": "source", + "enabled": True, + "generationsMapping": { + "mode": "MULTI_GENERATION", + "entries": [ + {"generation": "old", "proba": 0.5}, + {"generation": "other", "proba": 0.5}, + ], + }, + "auth": {"apiKeys": [{"key": SECRET}]}, + "localVariables": {"credential": SECRET}, + "untouched": {"nested": "preserved"}, + } + self.info = { + "infraBasicInfo": { + "id": "target", + "stage": "stage", + "infraType": "STATIC", + "apiKey": SECRET, + "prePushScript": SECRET, + }, + "projectBasicInfo": {"id": "source", "name": "Source"}, + "serviceBasicInfo": {"id": "source", "name": "Source"}, + "deploymentBasicInfo": {**self.settings, "futureSecret": SECRET}, + "packages": [ + { + "id": "version", + "publishedOn": 42, + "publishedBy": "user", + "endpoints": [{"code": SECRET}], + } + ], + "deployments": [self.settings], + "infras": [{"id": "target", "password": SECRET}], + "canDeploy": True, + "canWrite": False, + "apiKeys": [{"key": SECRET}], + "authorizationInherited": {"apiKeys": [{"key": SECRET}]}, + "oauth2Config": {"clientSecret": SECRET}, + } + self.update = { + "startTimestamp": 123, + "endTimestamp": 456, + "status": "DONE", + "type": "UPDATE", + "logs": {"lines": [SECRET]}, + "error": SECRET, + } + + def _request(self, method, path, kwargs): + assert threading.get_ident() != self.owner_thread, "SDK called on event loop" + self.calls.append((method, path, copy.deepcopy(kwargs))) + if method == "PUT": + self.settings = copy.deepcopy(kwargs["body"]) + return None + if method == "POST" and path.endswith("/actions/update"): + return {"jobId": "future", "hasResult": False} + if method == "POST": + return {} + if path.endswith("/settings"): + return copy.deepcopy(self.settings) + if path.endswith("/governance-status"): + return { + "messages": [ + {"code": "GOVERN", "severity": "WARNING", "details": SECRET} + ] + } + if path.endswith("/status"): + return { + "health": "OUT_OF_SYNC", + "healthMessages": { + "messages": [ + {"code": "SYNC", "severity": "WARNING", "message": SECRET} + ] + }, + "apiNodes": [{"apiKey": SECRET}], + } + if path.endswith("/stages"): + return [{"id": "stage", "desc": "Stage"}] + if path.endswith("/update"): + return [self.update] + if path.endswith("/last-update") or "/update/" in path: + return self.update + if path.rsplit("/", 1)[-1] in {"deployments", "infras", "projects", "services"}: + return [copy.deepcopy(self.info)] + return copy.deepcopy(self.info) + + def _perform_json(self, method, path, **kwargs): + return self._request(method, path, kwargs) + + def _perform_empty(self, method, path, **kwargs): + return self._request(method, path, kwargs) + + +@pytest.fixture(params=["project", "api"]) +def backend(request, monkeypatch): + fake = Backend(request.param) + client = SimpleNamespace( + get_projectdeployer=lambda: DSSProjectDeployer(fake), + get_apideployer=lambda: DSSAPIDeployer(fake), + ) + + def get_client(): + assert threading.get_ident() != fake.owner_thread + return client + + monkeypatch.setattr(deployer, "get_dss_client", get_client) + return fake + + +def call(function, backend, **kwargs): + result = asyncio.run(function(deployer_type=backend.kind, **kwargs)) + assert SECRET not in result + assert all( + path.startswith(f"/{backend.kind}-deployer/") for _, path, _ in backend.calls + ) + return json.loads(result) + + +@pytest.mark.parametrize( + "function, kwargs", + [ + (deployer.list_deployer_stages, {}), + (deployer.list_deployer_infrastructures, {}), + (deployer.get_deployer_infrastructure, {"infra_id": "target"}), + (deployer.list_deployer_published_items, {}), + (deployer.get_deployer_published_item, {"source_id": "source"}), + (deployer.list_deployer_deployments, {}), + (deployer.get_deployer_deployment, {"deployment_id": "deployment"}), + (deployer.list_deployer_updates, {"deployment_id": "deployment"}), + (deployer.get_deployer_update, {"deployment_id": "deployment"}), + ( + deployer.get_deployer_update, + {"deployment_id": "deployment", "timestamp": "123"}, + ), + ], +) +def test_read_tools_use_sdk_off_loop_and_omit_secrets(backend, function, kwargs): + result = call(function, backend, **kwargs) + assert result + assert all(method == "GET" for method, _, _ in backend.calls) + + +def test_deployment_detail_reports_health_mapping_and_codes(backend): + result = call(deployer.get_deployer_deployment, backend, deployment_id="deployment") + assert result["health"] == "OUT_OF_SYNC" + assert result["settings"]["enabled"] is True + assert result["settings"]["generationsMapping"]["entries"][1] == { + "generation": "other", + "proba": 0.5, + } + assert result["health_messages"] == [{"severity": "WARNING", "code": "SYNC"}] + assert result["governance_messages"] == [{"severity": "WARNING", "code": "GOVERN"}] + + +def test_published_versions_are_discoverable(backend): + result = call(deployer.get_deployer_published_item, backend, source_id="source") + assert result["packages"] == { + "columns": ["id", "publishedOn", "publishedBy"], + "rows": [["version", 42, "user"]], + } + assert result["canDeploy"] is True + assert result["canWrite"] is False + + +def test_create_only_saves_configuration(backend): + options = ( + {"deployed_project_key": "destination", "project_folder_id": "folder"} + if backend.kind == "project" + else {"endpoint_id": "endpoint"} + ) + result = call( + deployer.create_deployer_deployment, + backend, + deployment_id="new", + source_id="source", + infra_id="target", + version="version", + **options, + ) + assert result == { + "deployment_id": "new", + "status": "configuration_created", + "update_started": False, + } + [(method, path, params)] = backend.calls + assert method == "POST" + assert path.endswith("/deployments") + assert params["params"]["ignoreWarnings"] is False + assert ( + params["body"]["bundleId" if backend.kind == "project" else "version"] + == "version" + ) + assert ( + params["body"][ + "publishedProjectKey" if backend.kind == "project" else "publishedServiceId" + ] + == "source" + ) + if backend.kind == "project": + assert params["body"]["deployedProjectKey"] == "destination" + assert params["body"]["projectFolderId"] == "folder" + else: + assert params["body"]["endpointId"] == "endpoint" + + +def test_version_change_preserves_other_settings_and_governance(backend): + before = copy.deepcopy(backend.settings) + result = call( + deployer.update_deployer_deployment, + backend, + deployment_id="deployment", + version="version", + ) + assert result["update_started"] is False + after = backend.settings + assert after["untouched"] == before["untouched"] + assert after["auth"] == before["auth"] + assert after["localVariables"] == before["localVariables"] + if backend.kind == "project": + assert after["bundleId"] == "version" + assert after["generationsMapping"] == before["generationsMapping"] + else: + assert after["generationsMapping"] == { + "mode": "SINGLE_GENERATION", + "generation": "version", + } + assert [method for method, _, _ in backend.calls] == ["GET", "PUT"] + assert backend.calls[-1][2]["params"]["ignoreWarnings"] is False + + +def test_api_disable_preserves_version_mapping(backend): + if backend.kind == "project": + with pytest.raises(ValueError, match="API"): + call( + deployer.update_deployer_deployment, + backend, + deployment_id="deployment", + enabled=False, + ) + assert not backend.calls + else: + mapping = copy.deepcopy(backend.settings["generationsMapping"]) + call( + deployer.update_deployer_deployment, + backend, + deployment_id="deployment", + enabled=False, + ) + assert backend.settings["enabled"] is False + assert backend.settings["generationsMapping"] == mapping + + +def test_start_returns_durable_future_without_polling(backend): + result = call(deployer.start_deployer_update, backend, deployment_id="deployment") + assert result["future_id"] == "future" + assert result["status"] == "update_submitted" + assert [(method, path) for method, path, _ in backend.calls] == [ + ("POST", f"/{backend.kind}-deployer/deployments/deployment/actions/update") + ] + + +@pytest.mark.parametrize("options", [{}, {"version": ""}, {"version": " "}]) +def test_invalid_changes_do_not_call_sdk(backend, options): + with pytest.raises(ValueError): + call( + deployer.update_deployer_deployment, + backend, + deployment_id="deployment", + **options, + ) + assert not backend.calls + + +def test_rejects_component_specific_create_arguments(backend): + options = ( + {"endpoint_id": "endpoint"} + if backend.kind == "project" + else {"project_folder_id": "folder"} + ) + with pytest.raises(ValueError): + call( + deployer.create_deployer_deployment, + backend, + deployment_id="new", + source_id="source", + infra_id="target", + version="version", + **options, + ) + assert not backend.calls + + +@pytest.mark.parametrize("exception", [DataikuException, Timeout]) +def test_sdk_error_has_no_sensitive_text_or_cause(backend, monkeypatch, exception): + def fail(*args, **kwargs): + raise exception(SECRET) + + monkeypatch.setattr(backend, "_perform_json", fail) + with pytest.raises(RuntimeError) as error: + call(deployer.start_deployer_update, backend, deployment_id="deployment") + assert SECRET not in str(error.value) + assert "before retrying" in str(error.value) + assert error.value.__suppress_context__ + + +def test_mcp_schema_rejects_unknown_deployer_type(monkeypatch): + from dataiku_mcp.config import stdio + + monkeypatch.setattr( + stdio, + "_current_instance", + SimpleNamespace(name="test", instance_type="deployer"), + ) + + async def run(): + async with Client(mcp) as client: + with pytest.raises(Exception, match="Invalid operation arguments"): + await client.call_tool( + "run_read_operation", + { + "operation_id": "deployer.list_stages", + "arguments": {"deployer_type": "automation"}, + }, + ) + + asyncio.run(run()) + + +def test_mcp_keeps_deployer_target_when_active_instance_changes(backend, monkeypatch): + from dataiku_mcp.config import request, stdio + from dataiku_mcp.config.models import DSSInstance + + original = DSSInstance( + name="local-deployer", + url="https://local.example", + instance_type="deployer", + api_key="placeholder", + source="config", + no_check_certificate=False, + ) + other = DSSInstance( + name="other", + url="https://other.example", + instance_type="design", + api_key="placeholder", + source="config", + no_check_certificate=False, + ) + monkeypatch.setattr(stdio, "_current_instance", original) + observed = [] + + def get_client(): + stdio._current_instance = other + observed.append(request.get_pinned_instance().name) + assert request.get_pinned_instance().instance_type == "deployer" + return SimpleNamespace( + get_projectdeployer=lambda: DSSProjectDeployer(backend), + get_apideployer=lambda: DSSAPIDeployer(backend), + ) + + monkeypatch.setattr(deployer, "get_dss_client", get_client) + + async def run(): + async with Client(mcp) as client: + result = await client.call_tool( + "run_read_operation", + { + "operation_id": "deployer.list_stages", + "arguments": {"deployer_type": backend.kind}, + }, + ) + assert not result.is_error + + asyncio.run(run()) + assert observed == ["local-deployer"] + assert stdio.get_current_instance() == other + + +def test_deployer_is_exposed_only_as_operations(): + from dataiku_mcp.operations_catalog import OPERATIONS + + tools = asyncio.run(mcp.list_tools()) + assert not any("deployer" in tool.name for tool in tools) + operations = [op for op in OPERATIONS.values() if op.domain == "deployer"] + assert len(operations) == 13 + assert all(op.allowed_instance_types == ("deployer",) for op in operations) + assert OPERATIONS["deployer.start_update"].destructive + assert not OPERATIONS["deployer.list_stages"].destructive + + +def test_start_without_future_id_does_not_fetch_or_leak_result(backend, monkeypatch): + monkeypatch.setattr( + backend, + "_perform_json", + lambda *args, **kwargs: {"hasResult": True, "result": SECRET}, + ) + result = call(deployer.start_deployer_update, backend, deployment_id="deployment") + assert result["future_id"] is None + assert "inspect update history" in result["hint"] diff --git a/tests/test_govern_connection.py b/tests/test_govern_connection.py new file mode 100644 index 00000000..9cc1a630 --- /dev/null +++ b/tests/test_govern_connection.py @@ -0,0 +1,94 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +from dataclasses import replace +from types import SimpleNamespace + +import pytest + +from dataiku_mcp import auth, setup_server +from dataiku_mcp.config import request +from tests.test_instance_policy import _instance + +SECRET = "do-not-return-this-secret" + + +@pytest.mark.parametrize("field", ["api_key", "api_ticket"]) +@pytest.mark.parametrize("verification", [True, False, "/private/certificate.pem"]) +def test_govern_client_preserves_pinned_auth_and_certificate( + monkeypatch, field, verification +): + instance = replace( + _instance("govern"), + api_key=SECRET if field == "api_key" else None, + api_ticket=SECRET if field == "api_ticket" else None, + no_check_certificate=verification is False, + encrypted_rpc_cert_path=verification if isinstance(verification, str) else None, + ) + monkeypatch.setattr(request, "get_pinned_instance", lambda: instance) + monkeypatch.setattr(request, "is_http_request", lambda: False) + captured = {} + fake = SimpleNamespace(_session=SimpleNamespace(verify=None)) + + def create(url, **kwargs): + captured.update(url=url, **kwargs) + return fake + + monkeypatch.setattr(auth.dataikuapi, "GovernClient", create) + assert auth.get_govern_client() is fake + assert fake._session.verify == verification + assert captured["api_key" if field == "api_key" else "internal_ticket"] == SECRET + assert captured["extra_headers"]["X-DKU-Client-Application"] == "dataiku-headless" + + +def test_govern_client_rejects_http(monkeypatch): + monkeypatch.setattr(request, "get_pinned_instance", lambda: _instance("govern")) + monkeypatch.setattr(request, "is_http_request", lambda: True) + with pytest.raises(ValueError, match="stdio"): + auth.get_govern_client() + + +def test_govern_client_rejects_wrong_profile(monkeypatch): + monkeypatch.setattr(request, "get_pinned_instance", lambda: _instance("design")) + with pytest.raises(ValueError, match="govern"): + auth.get_govern_client() + + +def test_govern_setup_tests_the_correct_client_and_invalidates_type_change(monkeypatch): + captured = [] + + def create(url, key): + captured.append("govern") + return SimpleNamespace( + _session=SimpleNamespace(verify=None), get_auth_info=lambda: {} + ) + + monkeypatch.setattr(setup_server.dataikuapi, "GovernClient", create) + monkeypatch.setattr( + setup_server.dataikuapi, + "DSSClient", + lambda *a, **k: pytest.fail("Wrong client"), + ) + values = { + "url": "https://govern.example", + "api_key": "placeholder", + "no_check_certificate": False, + "instance_type": "govern", + } + setup_server._test_connection(values) + assert captured == ["govern"] + assert setup_server._connection_settings( + values + ) != setup_server._connection_settings({**values, "instance_type": "design"}) diff --git a/tests/test_govern_operations.py b/tests/test_govern_operations.py new file mode 100644 index 00000000..f6c435fb --- /dev/null +++ b/tests/test_govern_operations.py @@ -0,0 +1,353 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Expanded product coverage, SDK routing, authentication and local-file boundaries.""" + +import asyncio +import copy +import json +import threading +from types import SimpleNamespace + +import dataikuapi +import pytest +from dataikuapi.utils import DataikuException +from fastmcp import Client + +from dataiku_mcp import mcp, auth +from dataiku_mcp.config import request, stdio +from dataiku_mcp.operations_catalog import OPERATIONS +from dataiku_mcp.tools import govern +from tests.test_instance_policy import _instance +from tests.test_operations import _call, _http_setup + +SECRET = "do-not-return-this-secret" + +EXPECTED_GOVERN_IDS = { + "list_signoff_configurations", + "get_signoff_recurrence", + "list_users", + "update_custom_pages_order", + "get_time_series_values", + "list_role_assignments", + "search_artifacts", + "get_blueprint", + "delete_blueprint", + "update_custom_page", + "list_custom_pages", + "get_signoff_details", + "delete_signoff_configuration", + "list_signoffs", + "create_signoff_configuration", + "create_signoff", + "delete_artifact", + "get_group", + "delegate_signoff_approval", + "delete_role_assignments", + "create_blueprint", + "get_auth_info", + "create_time_series", + "delete_signoff_approval", + "update_user", + "list_blueprint_versions", + "list_signoff_feedbacks", + "delete_user", + "create_custom_page", + "get_role_assignments", + "list_blueprint_permissions", + "create_role", + "push_time_series_values", + "update_group", + "create_blueprint_version", + "add_signoff_approval", + "update_signoff_feedback", + "get_signoff_configuration", + "update_artifact", + "get_custom_page_definition", + "get_blueprint_version", + "delegate_signoff_feedback", + "download_uploaded_file", + "update_role_assignments", + "add_signoff_feedback", + "delete_custom_page", + "get_artifact", + "upload_file", + "get_user", + "update_role", + "get_uploaded_file", + "update_signoff_recurrence", + "list_roles", + "get_signoff_feedback", + "delete_signoff_feedback", + "list_groups", + "update_default_blueprint_permissions", + "create_role_assignments", + "delete_role", + "get_own_user", + "update_signoff_configuration", + "delete_uploaded_file", + "update_signoff_status", + "get_signoff", + "update_artifact_fields", + "list_blueprints", + "update_blueprint_permissions", + "get_blueprint_permissions", + "create_artifact", + "get_role", + "create_blueprint_permissions", + "get_signoff_approval", + "create_user", + "delete_group", + "get_instance_info", + "delete_time_series_values", + "update_blueprint", + "get_default_blueprint_permissions", + "delete_blueprint_version", + "set_blueprint_version_status", + "get_custom_pages_order", + "get_custom_page", + "create_group", + "update_blueprint_version", + "update_signoff_approval", + "delete_blueprint_permissions", +} + + +@pytest.mark.parametrize( + "id", + ["govern.get_instance_info", "govern.search_artifacts", "govern.delete_artifact"], +) +def test_govern_http_denies_before_exchange(monkeypatch, id): + events = _http_setup(monkeypatch, "govern") + op = OPERATIONS[id] + + async def run(): + async with Client(mcp) as client: + result = await client.call_tool( + "run_read_operation" if op.effect == "read" else "run_write_operation", + {"operation_id": id, "arguments": op.example}, + raise_on_error=False, + ) + assert result.is_error + assert "stdio" in result.content[0].text + assert not request.is_http_request() + + asyncio.run(run()) + assert not events + + +def test_govern_http_current_instance_reports_unsupported_without_exchange(monkeypatch): + events = _http_setup(monkeypatch, "govern") + monkeypatch.setattr( + auth.dataikuapi, + "GovernClient", + lambda *a, **k: pytest.fail("No Govern HTTP client"), + ) + result = asyncio.run(_call("get_current_instance")) + assert result["connection_status"] == "unsupported_authentication" + assert not events + + +def test_all_govern_operations_are_stdio_only(): + assert all( + op.transports == ("stdio",) + for op in OPERATIONS.values() + if op.domain == "govern" + ) + + +def test_govern_artifact_field_merge_uses_real_sdk_and_preserves_definition( + monkeypatch, +): + monkeypatch.setattr(stdio, "_current_instance", _instance("govern")) + client = dataikuapi.GovernClient("https://govern.example", api_key="placeholder") + state = { + "id": "ar.example", + "name": "Artifact", + "fields": {"old": {"value": 1}}, + "workflow": {"state": "preserved"}, + "password": SECRET, + } + calls = [] + owner_thread = threading.get_ident() + + def perform(method, path, **kwargs): + assert threading.get_ident() != owner_thread + calls.append((method, path, copy.deepcopy(kwargs))) + if method == "PUT": + state.clear() + state.update(copy.deepcopy(kwargs["body"])) + return {"artifact": copy.deepcopy(state)} + + monkeypatch.setattr(client, "_perform_json", perform) + monkeypatch.setattr(govern, "get_govern_client", lambda: client) + result = asyncio.run( + _call( + "run_write_operation", + operation_id="govern.update_artifact_fields", + arguments={"artifact_id": "ar.example", "fields": {"new": {"value": 0}}}, + ) + ) + assert state["fields"] == {"old": {"value": 1}, "new": {"value": 0}} + assert state["workflow"] == {"state": "preserved"} + assert state["password"] == SECRET + assert SECRET not in json.dumps(result) + assert [(method, path) for method, path, _ in calls] == [ + ("GET", "/artifact/ar.example"), + ("PUT", "/artifact/ar.example"), + ("GET", "/artifact/ar.example"), + ] + + +def test_govern_decision_defaults_do_not_send_notifications(monkeypatch): + monkeypatch.setattr(stdio, "_current_instance", _instance("govern")) + captured = {} + signoff = SimpleNamespace( + update_status=lambda *a, **k: captured.update(k), + get_definition=lambda: SimpleNamespace( + get_raw=lambda: {"status": "WAITING_FOR_APPROVAL"} + ), + ) + monkeypatch.setattr( + govern, + "get_govern_client", + lambda: SimpleNamespace( + get_artifact=lambda _: SimpleNamespace(get_signoff=lambda _: signoff) + ), + ) + asyncio.run( + _call( + "run_write_operation", + operation_id="govern.update_signoff_status", + arguments={ + "artifact_id": "ar.example", + "step_id": "review", + "status": "WAITING_FOR_APPROVAL", + }, + ) + ) + assert captured["users_to_notify"] == [] + assert captured["reload_conf_for_reset"] is False + + +def test_govern_time_series_destructive_intent_and_upsert_default(monkeypatch): + operation = OPERATIONS["govern.delete_time_series_values"] + with pytest.raises(ValueError): + operation.validate({"time_series_id": "ts.example"}) + operation.validate({"time_series_id": "ts.example", "delete_all": True}) + operation.validate({"time_series_id": "ts.example", "min_timestamp": 10}) + with pytest.raises(ValueError): + operation.validate( + {"time_series_id": "ts.example", "min_timestamp": 10, "max_timestamp": 5} + ) + monkeypatch.setattr(stdio, "_current_instance", _instance("govern")) + captured = {} + series = SimpleNamespace( + push_values=lambda values, **kwargs: captured.update(kwargs) + ) + monkeypatch.setattr( + govern, + "get_govern_client", + lambda: SimpleNamespace(get_time_series=lambda _: series), + ) + asyncio.run( + _call( + "run_write_operation", + operation_id="govern.push_time_series_values", + arguments={ + "time_series_id": "ts.example", + "datapoints": [{"timestamp": 1, "value": 2}], + }, + ) + ) + assert captured["upsert"] is False + + +def test_search_pages_all_operations_without_increasing_tool_catalog(monkeypatch): + monkeypatch.setattr(stdio, "_current_instance", _instance("govern")) + first = asyncio.run(_call("search_operations", domain="govern", limit=50)) + second = asyncio.run( + _call( + "search_operations", + domain="govern", + limit=50, + offset=first["next_offset"], + ) + ) + assert first["total"] == 86 + assert second["next_offset"] is None + assert ( + len( + { + item["operation_id"] + for item in first["operations"] + second["operations"] + } + ) + == 86 + ) + + +def test_govern_sdk_failure_is_sanitized_without_retry(monkeypatch): + monkeypatch.setattr(stdio, "_current_instance", _instance("govern")) + calls = [] + + def fail(): + calls.append(1) + raise DataikuException(SECRET) + + monkeypatch.setattr( + govern, "get_govern_client", lambda: SimpleNamespace(get_instance_info=fail) + ) + + async def run(): + async with Client(mcp) as client: + result = await client.call_tool( + "run_read_operation", + {"operation_id": "govern.get_instance_info", "arguments": {}}, + raise_on_error=False, + ) + assert result.is_error + assert SECRET not in result.content[0].text + + asyncio.run(run()) + assert len(calls) == 1 + + +def test_govern_catalog_covers_expected_actions_without_new_mcp_tools(): + ids = { + op.operation_id.split(".", 1)[1] + for op in OPERATIONS.values() + if op.domain == "govern" + } + assert ids == EXPECTED_GOVERN_IDS + tools = asyncio.run(mcp.list_tools()) + assert len(tools) == 135 + assert not any( + tool.name == "govern" or "mira" in tool.name or "deployer" in tool.name + for tool in tools + ) + assert not any( + "mira" in op.operation_id or "MIRA" in op.description + for op in OPERATIONS.values() + ) + + +def test_govern_adapter_uses_only_public_sdk_methods(): + import ast + import inspect + + for node in ast.walk(ast.parse(inspect.getsource(govern))): + if isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute): + assert not node.func.attr.startswith("_perform") + assert node.func.attr not in {"request", "urlopen"} diff --git a/tests/test_http_middleware.py b/tests/test_http_middleware.py index 35efafcb..beb2f7ac 100644 --- a/tests/test_http_middleware.py +++ b/tests/test_http_middleware.py @@ -180,6 +180,8 @@ async def get_tagged_tool_names(): "switch_instance", "delete_instance", "configure_instance", + "search_operations", + "describe_operation", "get_cobuild_turn_status", "list_cobuild_conversations", } @@ -238,7 +240,7 @@ def test_http_unknown_tool_fails_without_token_exchange( ] -@pytest.mark.parametrize("instance_type", ["govern", "deployer", "agent-management"]) +@pytest.mark.parametrize("instance_type", ["govern"]) def test_http_unsupported_current_instance_skips_exchange(monkeypatch, instance_type): events = _call_middleware( monkeypatch, "get_current_instance", instance_type=instance_type @@ -306,3 +308,17 @@ async def call_next(_context): asyncio.run(middleware.on_call_tool(context, call_next)) assert events == [("reset_identity", "identity")] + + +def test_http_deployer_status_exchanges_token(monkeypatch): + events = _call_middleware( + monkeypatch, "get_current_instance", instance_type="deployer" + ) + assert ("exchange", "mcp-token") in events + + +def test_http_agent_management_status_exchanges_token(monkeypatch): + events = _call_middleware( + monkeypatch, "get_current_instance", instance_type="agent-management" + ) + assert ("exchange", "mcp-token") in events diff --git a/tests/test_instance_policy.py b/tests/test_instance_policy.py index 0e549048..fd065ac5 100644 --- a/tests/test_instance_policy.py +++ b/tests/test_instance_policy.py @@ -59,9 +59,11 @@ def _instance(instance_type): def _allowed(tool_name): if tool_name in INSTANCE_TOOLS: return set(get_args(InstanceType)) + if tool_name in EXPECTED_TOOLS_BY_MODULE["operations"]: + return {"deployer", "govern", "agent-management"} if tool_name in PROJECT_EDIT_TOOLS: - return {"design"} - return {"design", "automation"} + return {"design", "agent-management"} + return {"design", "automation", "agent-management"} def test_every_registered_tool_declares_its_exact_instance_policy(): @@ -99,7 +101,7 @@ def test_invalid_policy_fails_closed_even_for_instance_controls(metadata): @pytest.mark.parametrize( - "instance_type", ["automation", "govern", "deployer", "agent-management"] + "instance_type", ["design", "automation", "govern", "deployer", "agent-management"] ) def test_mcp_dispatch_denies_every_disallowed_tool_before_validation_or_sdk( monkeypatch, instance_type @@ -124,7 +126,7 @@ async def dispatch(): asyncio.run(dispatch()) -@pytest.mark.parametrize("instance_type", ["design", "automation"]) +@pytest.mark.parametrize("instance_type", ["design", "automation", "agent-management"]) def test_mcp_allowed_read_and_recovery_after_denial(monkeypatch, instance_type): monkeypatch.setattr(stdio, "_current_instance", _instance(instance_type)) calls = [] @@ -152,26 +154,6 @@ async def dispatch(): assert calls == [instance_type] -@pytest.mark.parametrize("instance_type", ["govern", "deployer", "agent-management"]) -def test_mcp_unsupported_status_needs_no_client(monkeypatch, instance_type): - from dataiku_mcp.tools import instances - - monkeypatch.setattr(stdio, "_current_instance", _instance(instance_type)) - monkeypatch.setattr( - instances, "get_dss_client", lambda: pytest.fail("No SDK call is allowed") - ) - - async def dispatch(): - async with Client(mcp) as client: - result = await client.call_tool("get_current_instance", {}) - data = json.loads(result.content[0].text) - assert data["connection_status"] == "unsupported" - assert "dataiku_version" not in data - assert "private-key" not in result.content[0].text - - asyncio.run(dispatch()) - - @pytest.mark.parametrize( "tool_name", ["list_instances", "switch_instance", "configure_instance", "delete_instance"], diff --git a/tests/test_instances.py b/tests/test_instances.py index cc7e20e0..a473813f 100644 --- a/tests/test_instances.py +++ b/tests/test_instances.py @@ -93,15 +93,10 @@ def unexpected_client(): # API information is observational; it must not replace the configured type. client = _FakeClient({"dssVersion": "14.7.2", "nodeType": "AUTOMATION"}) monkeypatch.setattr(instances, "get_dss_client", lambda: client) + monkeypatch.setattr(instances, "get_govern_client", lambda: client) current = asyncio.run(instances.get_current_instance(FakeContext())) assert json.loads(current)["instance_type"] == instance_type - unsupported = instance_type in {"govern", "deployer", "agent-management"} - assert client.info_calls == (0 if unsupported else 1) - assert json.loads(current)["connection_status"] == ( - "unsupported" if unsupported else "connected" - ) - if unsupported: - assert "dataiku_version" not in json.loads(current) + assert client.info_calls == 1 for response in (listing, switching, current): assert API_KEY not in response assert "api_key" not in response @@ -152,10 +147,13 @@ def unavailable_instance_info(): assert client.info_calls == 0 +@pytest.mark.parametrize( + "instance_type", ["design", "automation", "deployer", "govern", "agent-management"] +) @pytest.mark.parametrize("credential", ["api_key", "api_ticket"]) @pytest.mark.parametrize("connected", [False, True]) def test_get_current_instance_never_returns_credentials( - monkeypatch, credential, connected + monkeypatch, credential, connected, instance_type ): secret = "do-not-return-this-credential" instance = DSSInstance( @@ -164,7 +162,7 @@ def test_get_current_instance_never_returns_credentials( no_check_certificate=False, source="code-studio-environment" if credential == "api_ticket" else "config", encrypted_rpc_cert_path="/private/test-certificate.pem", - instance_type="design", + instance_type=instance_type, **{credential: secret}, ) monkeypatch.setattr(request, "get_pinned_instance", lambda: instance) @@ -175,6 +173,7 @@ def get_client(): return _FakeClient({"dssVersion": "14.7.2"}) monkeypatch.setattr(instances, "get_dss_client", get_client) + monkeypatch.setattr(instances, "get_govern_client", get_client) response = asyncio.run(instances.get_current_instance(FakeContext())) result = json.loads(response) @@ -186,3 +185,20 @@ def get_client(): assert result["name"] == instance.name assert result["source"] == instance.source assert result["connection_status"] == ("connected" if connected else "failed") + + +def test_govern_http_status_reports_unsupported_authentication(monkeypatch): + instance = replace(_instance("primary"), instance_type="govern") + monkeypatch.setattr(request, "get_pinned_instance", lambda: instance) + monkeypatch.setattr(request, "is_http_request", lambda: True) + + def unexpected_client(): + pytest.fail("Govern HTTP status must not construct an SDK client") + + monkeypatch.setattr(instances, "get_govern_client", unexpected_client) + monkeypatch.setattr(instances, "get_dss_client", unexpected_client) + response = asyncio.run(instances.get_current_instance(FakeContext())) + result = json.loads(response) + assert result["connection_status"] == "unsupported_authentication" + assert "dataiku_version" not in result + assert API_KEY not in response diff --git a/tests/test_onboarding_ui.py b/tests/test_onboarding_ui.py index 2541615f..8707ffc5 100644 --- a/tests/test_onboarding_ui.py +++ b/tests/test_onboarding_ui.py @@ -28,21 +28,18 @@ def test_setup_page_uses_dataiku_branding_and_accessible_states(): assert "gradient" not in page -def test_setup_page_allows_govern_save_without_a_connection_test(): +def test_setup_page_requires_a_successful_connection_test(): page = _page() assert "Advanced options" in page assert 'name="no_check_certificate"' in page - assert 'name="action" value="save" disabled' not in page - assert ( - "Agent Management, Govern and Deployer profiles can be saved without testing." - in page - ) + assert 'name="action" value="save" disabled' in page + assert "Govern profiles can be saved without testing." not in page assert "button:disabled, button:disabled:hover" in page def test_setup_page_enables_save_after_a_successful_connection_test(): - page = _page(status="Connection successful.") + page = _page(status="Connection successful.", connection_validated=True) assert 'name="action" value="save" disabled' not in page diff --git a/tests/test_operation_redaction.py b/tests/test_operation_redaction.py new file mode 100644 index 00000000..4917bd6a --- /dev/null +++ b/tests/test_operation_redaction.py @@ -0,0 +1,63 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +import asyncio +import json +from types import SimpleNamespace + +import pytest + +from dataiku_mcp.config import stdio +from dataiku_mcp.tools import deployer +from tests.test_instance_policy import _instance +from tests.test_operations import _call + +SECRET = "do-not-return-this-secret" + + +def test_deployer_future_status_does_not_fetch_results(monkeypatch): + monkeypatch.setattr(stdio, "_current_instance", _instance("deployer")) + captured = [] + + def perform(method, path, **kwargs): + captured.append((method, path, kwargs)) + return {"hasResult": True, "result": {"password": SECRET}} + + monkeypatch.setattr( + deployer, "get_dss_client", lambda: SimpleNamespace(_perform_json=perform) + ) + result = asyncio.run( + _call( + "run_read_operation", + operation_id="deployer.get_future_status", + arguments={"future_id": "future"}, + ) + ) + assert result["result"]["state"] == {"hasResult": True} + assert captured == [("GET", "/futures/future", {"params": {"peek": True}})] + + +@pytest.mark.parametrize( + "value", + [ + {"apiKeys": [SECRET]}, + {"passwords": [SECRET]}, + {"url": "https://user:" + SECRET + "@example.com/path?api_key=" + SECRET}, + ], +) +def test_operation_redaction_covers_secret_containers_and_url_credentials(value): + from dataiku_mcp.tools.utils.redaction import redact_sensitive_data + + assert SECRET not in json.dumps(redact_sensitive_data(value)) diff --git a/tests/test_operations.py b/tests/test_operations.py new file mode 100644 index 00000000..d7759519 --- /dev/null +++ b/tests/test_operations.py @@ -0,0 +1,351 @@ +# Copyright 2026 Dataiku SAS +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +"""Operation discovery, schemas, execution policy and request pinning.""" + +import asyncio +import json +from types import SimpleNamespace + +import pytest +from fastmcp import Client +from dataikuapi.dss.apideployer import DSSAPIDeployer +from dataikuapi.dss.projectdeployer import DSSProjectDeployer + +from dataiku_mcp import mcp, server +from dataiku_mcp.config import http, request, stdio +from dataiku_mcp.operations_catalog import OPERATIONS +from dataiku_mcp.operation_registry import get_operation +from dataiku_mcp.tools import deployer +from tests.test_instance_policy import _instance +from tests.test_deployer import Backend + + +async def _call(name, **arguments): + async with Client(mcp) as client: + result = await client.call_tool(name, arguments) + return json.loads(result.content[0].text) + + +@pytest.mark.parametrize( + "tool, arguments", + [ + ("search_operations", {}), + ("describe_operation", {"operation_id": "deployer.list_stages"}), + ( + "run_read_operation", + {"operation_id": "deployer.list_stages", "arguments": {}}, + ), + ( + "run_write_operation", + { + "operation_id": "deployer.start_update", + "arguments": { + "deployer_type": "project", + "deployment_id": "deployment", + }, + }, + ), + ], +) +def test_all_v2_tools_require_selected_instance(monkeypatch, tool, arguments): + monkeypatch.setattr(stdio, "_current_instance", None) + monkeypatch.setattr(stdio, "get_instances", lambda: {}) + monkeypatch.setattr( + server, + "exchange_http_token", + lambda *_: pytest.fail("No exchange without a selected profile"), + ) + + async def run(): + async with Client(mcp) as client: + result = await client.call_tool(tool, arguments, raise_on_error=False) + assert result.is_error + assert "No Dataiku instances are configured" in result.content[0].text + assert request._pinned_instance.get() is None + + asyncio.run(run()) + + +def _http_setup(monkeypatch, kind="deployer"): + profiles = {kind: _instance(kind)} + monkeypatch.setattr( + http, + "get_instances_and_selections", + lambda: (profiles, {"issuer": {"alice": kind}}), + ) + monkeypatch.setattr( + server, + "get_access_token", + lambda: SimpleNamespace( + token="mcp-token", claims={"iss": "issuer", "sub": "alice"} + ), + ) + events = [] + + async def exchange(_): + events.append("exchange") + return "backend-token" + + monkeypatch.setattr(server, "exchange_http_token", exchange) + return events + + +def test_deployer_registry_uses_existing_sdk_handler(monkeypatch): + backend = Backend("project") + monkeypatch.setattr(stdio, "_current_instance", _instance("deployer")) + monkeypatch.setattr( + deployer, + "get_dss_client", + lambda: SimpleNamespace( + get_projectdeployer=lambda: DSSProjectDeployer(backend), + get_apideployer=lambda: DSSAPIDeployer(backend), + ), + ) + result = asyncio.run( + _call( + "run_write_operation", + operation_id="deployer.start_update", + arguments={"deployer_type": "project", "deployment_id": "deployment"}, + ) + ) + assert result["result"]["future_id"] == "future" + assert len(backend.calls) == 1 + + +@pytest.mark.parametrize("kind", ["design", "automation"]) +@pytest.mark.parametrize( + "tool, arguments", + [ + ("search_operations", {}), + ("describe_operation", {"operation_id": "deployer.list_stages"}), + ( + "run_read_operation", + { + "operation_id": "deployer.list_stages", + "arguments": {"deployer_type": "project"}, + }, + ), + ( + "run_write_operation", + { + "operation_id": "deployer.start_update", + "arguments": { + "deployer_type": "project", + "deployment_id": "deployment", + }, + }, + ), + ], +) +def test_v2_tools_reject_design_and_automation_before_exchange( + monkeypatch, kind, tool, arguments +): + events = _http_setup(monkeypatch, kind) + + async def run(): + async with Client(mcp) as client: + result = await client.call_tool(tool, arguments, raise_on_error=False) + assert result.is_error + assert f"instance type '{kind}'" in result.content[0].text + assert not request.is_http_request() + assert request._pinned_instance.get() is None + + asyncio.run(run()) + assert not events + + +def test_each_v2_operation_requires_its_matching_instance_type(): + from dataiku_mcp.operation_registry import require_operation + + for operation in OPERATIONS.values(): + assert operation.allowed_instance_types == (operation.domain,) + for kind in ("design", "automation", "agent-management", "deployer", "govern"): + if kind == operation.domain: + assert ( + require_operation(operation.operation_id, operation.effect, kind) + is operation + ) + else: + with pytest.raises(ValueError, match="unavailable for instance type"): + require_operation(operation.operation_id, operation.effect, kind) + + +def test_all_schemas_accept_examples_and_are_closed(): + assert sum(op.domain == "deployer" for op in OPERATIONS.values()) == 13 + for operation in OPERATIONS.values(): + operation.validate(operation.example) + description = operation.describe() + assert description["input_schema"]["additionalProperties"] is False + assert description["server_support"] == "unknown" + assert description["requirements"] + assert description["response_contract"] + assert description["effect"] in {"read", "write"} + + +def test_search_filters_paginates_and_preserves_tool_catalog(monkeypatch): + monkeypatch.setattr(stdio, "_current_instance", _instance("deployer")) + + async def run(): + before = {tool.name for tool in await mcp.list_tools()} + results = [] + offset = 0 + while offset is not None: + page = await _call( + "search_operations", domain="deployer", limit=5, offset=offset + ) + assert page["total"] == 13 + results.extend(op["operation_id"] for op in page["operations"]) + offset = page["next_offset"] + assert len(set(results)) == 13 + writes = await _call("search_operations", query="deployment", effect="write") + assert "deployer.update_deployment" in { + op["operation_id"] for op in writes["operations"] + } + detail = await _call("describe_operation", operation_id="deployer.start_update") + assert detail["allowed_on_selected_instance"] is True + assert {tool.name for tool in await mcp.list_tools()} == before + + asyncio.run(run()) + + +@pytest.mark.parametrize( + "tool,operation_id,arguments", + [ + ("run_read_operation", "deployer.start_update", {}), + ("run_write_operation", "deployer.list_stages", {"deployer_type": "project"}), + ("run_read_operation", "not-in-catalog", {}), + ( + "run_read_operation", + "deployer.list_stages", + {"deployer_type": "project", "path": "/admin"}, + ), + ("run_read_operation", "deployer.list_stages", 42), + ], +) +def test_invalid_operation_rejected_before_sdk_execution( + monkeypatch, tool, operation_id, arguments +): + events = _http_setup(monkeypatch) + monkeypatch.setattr( + deployer, + "get_dss_client", + lambda: pytest.fail("Invalid operations must not reach SDK"), + ) + + async def run(): + async with Client(mcp) as client: + result = await client.call_tool( + tool, + {"operation_id": operation_id, "arguments": arguments}, + raise_on_error=False, + ) + assert result.is_error + assert not request.is_http_request() + assert request._pinned_instance.get() is None + + asyncio.run(run()) + assert events == ["exchange"] + + +@pytest.mark.parametrize( + "bad_id", [".", "..", " .. ", "x/y", "x\\y", "%2e%2e", "%252f", "a\n", " "] +) +def test_path_escape_rejected_before_sdk_execution(monkeypatch, bad_id): + _http_setup(monkeypatch) + monkeypatch.setattr( + deployer, + "get_dss_client", + lambda: pytest.fail("Invalid identifier reached SDK"), + ) + + async def run(): + async with Client(mcp) as client: + result = await client.call_tool( + "run_read_operation", + { + "operation_id": "deployer.get_infrastructure", + "arguments": {"deployer_type": "project", "infra_id": bad_id}, + }, + raise_on_error=False, + ) + assert result.is_error + + asyncio.run(run()) + + +def test_expected_instance_rejects_stale_selection_before_sdk_execution(monkeypatch): + _http_setup(monkeypatch) + monkeypatch.setattr( + deployer, "get_dss_client", lambda: pytest.fail("Stale selection reached SDK") + ) + + async def run(): + async with Client(mcp) as client: + result = await client.call_tool( + "run_read_operation", + { + "operation_id": "deployer.list_stages", + "arguments": {"deployer_type": "project"}, + "expected_instance": "other", + }, + raise_on_error=False, + ) + assert result.is_error + assert "Selected instance changed" in result.content[0].text + + asyncio.run(run()) + + +def test_http_pinning_survives_switch_during_exchange(monkeypatch): + profiles = {kind: _instance(kind) for kind in ("deployer", "design")} + selection = {"issuer": {"alice": "deployer"}} + _http_setup(monkeypatch) + monkeypatch.setattr( + http, "get_instances_and_selections", lambda: (profiles, selection) + ) + + async def exchange(_): + selection["issuer"]["alice"] = "design" + await asyncio.sleep(0) + return "backend-token" + + monkeypatch.setattr(server, "exchange_http_token", exchange) + + def get_client(): + assert request.get_pinned_instance().name == "deployer" + assert request.get_http_dss_token() == "backend-token" + return SimpleNamespace( + get_projectdeployer=lambda: SimpleNamespace(list_stages=lambda: []) + ) + + monkeypatch.setattr(deployer, "get_dss_client", get_client) + result = asyncio.run( + _call( + "run_read_operation", + operation_id="deployer.list_stages", + arguments={"deployer_type": "project"}, + ) + ) + assert result["instance_name"] == "deployer" + assert not request.is_http_request() + + +def test_redacted_credentials_cannot_be_written_back(): + operation = get_operation("deployer.update_deployment") + with pytest.raises(ValueError, match="redacted result placeholders"): + operation.validate( + {"deployer_type": "project", "deployment_id": "__DATAIKU_REDACTED__"} + ) diff --git a/tests/test_setup_server.py b/tests/test_setup_server.py index e30114d2..83d219dd 100644 --- a/tests/test_setup_server.py +++ b/tests/test_setup_server.py @@ -169,7 +169,9 @@ def _post_setup_form(url: str, form: dict[str, str]) -> tuple[int, str]: return error.code, error.read().decode() -@pytest.mark.parametrize("instance_type", ["design", "automation"]) +@pytest.mark.parametrize( + "instance_type", ["design", "automation", "deployer", "govern", "agent-management"] +) def test_setup_server_requires_a_successful_test_before_saving( monkeypatch, instance_type ): @@ -235,48 +237,8 @@ def test_setup_server_invalidates_a_test_when_connection_settings_change( ) -@pytest.mark.parametrize("instance_type", ["govern", "deployer", "agent-management"]) -def test_unsupported_profile_can_be_saved_after_unavailable_test( - monkeypatch, instance_type -): - saved = [] - monkeypatch.setattr( - setup_server.dataikuapi, - "DSSClient", - lambda *args: pytest.fail( - "Unsupported profiles must not construct a DSS client" - ), - ) - monkeypatch.setattr( - setup_server.stdio, - "add_instance_to_config", - lambda **values: ( - saved.append(values) - or {"name": values["name"], "instance_type": values["instance_type"]} - ), - ) - monkeypatch.setattr(setup_server.request, "set_current_instance", lambda name: None) - session = setup_server.start_setup_server(open_browser=False) - form = { - "name": instance_type, - "url": "https://example.com", - "api_key": "secret", - "instance_type": instance_type, - } - try: - status, page = _post_setup_form(session.url, {**form, "action": "test"}) - assert status == 400 - assert f"Connection testing is unavailable for {instance_type}" in page - assert session.validated_connection is None - status, page = _post_setup_form(session.url, {**form, "action": "save"}) - assert status == 200 - assert "Instance saved" in page - assert saved[0]["instance_type"] == instance_type - finally: - session.close() - - -def test_design_connection_test_uses_dss_client(monkeypatch): +@pytest.mark.parametrize("instance_type", ["design", "deployer", "agent-management"]) +def test_supported_connection_test_uses_dss_client(monkeypatch, instance_type): calls = [] class Client: @@ -292,6 +254,8 @@ def get_auth_info(self): monkeypatch.setattr(setup_server.dataikuapi, "DSSClient", Client) setup_server._test_connection( - _validate_form({**_form("https://example.com"), "instance_type": ["design"]}) + _validate_form( + {**_form("https://example.com"), "instance_type": [instance_type]} + ) ) assert calls == ["client", "auth"] diff --git a/tests/test_tool_surface.py b/tests/test_tool_surface.py index 378dd222..2fb78f31 100644 --- a/tests/test_tool_surface.py +++ b/tests/test_tool_surface.py @@ -164,6 +164,14 @@ "upload_file_to_managed_folder", } ), + "operations": frozenset( + { + "search_operations", + "describe_operation", + "run_read_operation", + "run_write_operation", + } + ), "plugins": frozenset( { "install_plugin", diff --git a/uv.lock b/uv.lock index 20f32c58..deb871b7 100644 --- a/uv.lock +++ b/uv.lock @@ -1,5 +1,5 @@ version = 1 -revision = 5 +revision = 3 requires-python = ">=3.10" resolution-markers = [ "python_full_version >= '3.14' and sys_platform == 'emscripten'", @@ -481,13 +481,13 @@ wheels = [ [[package]] name = "dataiku-api-client" -version = "14.7.2" +version = "15.2.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "python-dateutil" }, { name = "requests" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/96/e1/336e135820f414301839e57b7d337925f955fd5bc02ba1eb39c4f46d21e0/dataiku_api_client-14.7.2.tar.gz", hash = "sha256:476efce84ce6fd69dd226c85690ea83fd7cbba77276c3d2c0afa00ce7dba5031", size = 416594, upload-time = "2026-07-22T07:54:50.998Z" } +sdist = { url = "https://files.pythonhosted.org/packages/fb/0f/43d2be13e28c2278e2bfd88e4d35f412fb5f99b0839f8b539daa67cdcca0/dataiku_api_client-15.2.0.tar.gz", hash = "sha256:44e7d68339e7d5fb04f28eac0fc7117c9c434f2f349a688f9bb0b7043ba78517", size = 489461, upload-time = "2026-10-16T12:08:01.425Z" } [[package]] name = "dataiku-headless" @@ -512,7 +512,7 @@ dev = [ [package.metadata] requires-dist = [ - { name = "dataiku-api-client", specifier = ">=14.7.2" }, + { name = "dataiku-api-client", specifier = ">=15.2.0" }, { name = "fastmcp", specifier = ">=3.4.5,<4" }, { name = "pydantic", specifier = ">=2.11,<3" }, { name = "python-dotenv" },