Skip to content

Databricks Toolkit for Coding Agents provided by Field Engineering

License

Notifications You must be signed in to change notification settings

databricks-solutions/ai-dev-kit

Repository files navigation

Databricks AI Dev Kit

Databricks Certified Gold Project


Overview

Vibecoding on Databricks just got a whole lot better. The AI Dev Kit gives your AI coding assistant (Claude Code, Cursor, Windsurf, etc.) the trusted sources it needs to build faster and smarter on Databricks.

Architecture


What Can I Build?

  • Spark Declarative Pipelines (streaming tables, CDC, SCD Type 2, Auto Loader)
  • Databricks Jobs (scheduled workflows, multi-task DAGs)
  • AI/BI Dashboards (visualizations, KPIs, analytics)
  • Unity Catalog (tables, volumes, governance)
  • Genie Spaces (natural language data exploration)
  • Knowledge Assistants (RAG-based document Q&A)
  • MLflow Experiments (evaluation, scoring, traces)
  • Databricks Apps (full-stack web applications)
  • ...and more

Choose Your Own Adventure

Adventure Best For Start Here
Starter Kit Start here! Claude Code + full Databricks integration ai-dev-project/
Visual Builder Web-based UI for Databricks development databricks-builder-app/
Skills Only Teaching your AI Databricks patterns (no actions) Install skills
MCP Tools Only Just executable actions (no guidance) Register MCP server
Core Library Building custom integrations (LangChain, OpenAI, etc.) pip install

Quick Start

Prerequisites

# Set your Databricks credentials
export DATABRICKS_HOST="https://your-workspace.cloud.databricks.com"
export DATABRICKS_TOKEN="your-token"

# Or use a profile from ~/.databrickscfg
export DATABRICKS_CONFIG_PROFILE="your-profile"

Starter Kit

The fastest way to get started with Claude Code + Databricks:

git clone https://github.com/databricks-solutions/ai-dev-kit.git
cd ai-dev-kit/ai-dev-project
./setup.sh
claude

This installs skills + MCP tools and opens Claude Code ready to build on Databricks.

Visual Builder App

Full-stack web application with chat UI for Databricks development:

cd ai-dev-kit/databricks-builder-app
./setup.sh
# Follow instructions to start the app

Skills Only

Just want to teach your AI assistant Databricks patterns?

cd ai-dev-kit
./databricks-skills/install_skills.sh

Skills include: Spark Declarative Pipelines, Asset Bundles, MLflow Evaluation, Synthetic Data Generation, and more.

MCP Tools Only

Just want executable Databricks actions?

cd ai-dev-kit
./databricks-mcp-server/setup.sh

# Register with Claude Code
claude mcp add-json databricks "{
  \"command\": \"$(pwd)/.venv/bin/python\",
  \"args\": [\"$(pwd)/run_server.py\"]
}"

Core Library

Use databricks-tools-core directly in your Python projects:

from databricks_tools_core.sql import execute_sql

results = execute_sql("SELECT * FROM my_catalog.schema.table LIMIT 10")

Works with LangChain, OpenAI Agents SDK, or any Python framework. See databricks-tools-core/ for details.


What's Included

Component Description
databricks-tools-core/ Python library with high-level Databricks functions
databricks-mcp-server/ MCP server exposing 45+ tools for AI assistants
databricks-skills/ 14 markdown skills teaching Databricks patterns
databricks-builder-app/ Full-stack web app with Claude Code integration
ai-dev-project/ Starter template for new projects

Star History

Star History Chart

License

(c) 2026 Databricks, Inc. All rights reserved.

The source in this project is provided subject to the Databricks License. See LICENSE.md for details.

Third-Party Licenses
Package Version License Project URL
fastmcp ≥0.1.0 MIT https://github.com/jlowin/fastmcp
mcp ≥1.0.0 MIT https://github.com/modelcontextprotocol/python-sdk
sqlglot ≥20.0.0 MIT https://github.com/tobymao/sqlglot
sqlfluff ≥3.0.0 MIT https://github.com/sqlfluff/sqlfluff
litellm ≥1.0.0 MIT https://github.com/BerriAI/litellm
pymupdf ≥1.24.0 AGPL-3.0 https://github.com/pymupdf/PyMuPDF
claude-agent-sdk ≥0.1.19 MIT https://github.com/anthropics/claude-code
fastapi ≥0.115.8 MIT https://github.com/fastapi/fastapi
uvicorn ≥0.34.0 BSD-3-Clause https://github.com/encode/uvicorn
httpx ≥0.28.0 BSD-3-Clause https://github.com/encode/httpx
sqlalchemy ≥2.0.41 MIT https://github.com/sqlalchemy/sqlalchemy
alembic ≥1.16.1 MIT https://github.com/sqlalchemy/alembic
asyncpg ≥0.30.0 Apache-2.0 https://github.com/MagicStack/asyncpg
greenlet ≥3.0.0 MIT https://github.com/python-greenlet/greenlet
psycopg2-binary ≥2.9.11 LGPL-3.0 https://github.com/psycopg/psycopg2

Acknowledgments

MCP Databricks Command Execution API from databricks-exec-code by Natyra Bajraktari and Henryk Borzymowski.

About

Databricks Toolkit for Coding Agents provided by Field Engineering

Topics

Resources

License

Contributing

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 11