ai-dev-kit

ai-dev-kit

Databricks Toolkit for Coding Agents provided by Field Engineering

Stars: 211

Visit
 screenshot

The AI Dev Kit is a comprehensive toolkit designed to enhance AI-driven development on Databricks. It provides trusted sources for AI coding assistants like Claude Code and Cursor to build faster and smarter on Databricks. The kit includes features such as Spark Declarative Pipelines, Databricks Jobs, AI/BI Dashboards, Unity Catalog, Genie Spaces, Knowledge Assistants, MLflow Experiments, Model Serving, Databricks Apps, and more. Users can choose from different adventures like installing the kit, using the visual builder app, teaching AI assistants Databricks patterns, executing Databricks actions, or building custom integrations with the core library. The kit also includes components like databricks-tools-core, databricks-mcp-server, databricks-skills, databricks-builder-app, and ai-dev-project.

README:

Databricks AI Dev Kit

Databricks Certified Gold Project


Overview

AI-Driven Development (vibe coding) on Databricks just got a whole lot better. The AI Dev Kit gives your AI coding assistant (Claude Code, Cursor, Windsurf, etc.) the trusted sources it needs to build faster and smarter on Databricks.

Architecture


What Can I Build?

  • Spark Declarative Pipelines (streaming tables, CDC, SCD Type 2, Auto Loader)
  • Databricks Jobs (scheduled workflows, multi-task DAGs)
  • AI/BI Dashboards (visualizations, KPIs, analytics)
  • Unity Catalog (tables, volumes, governance)
  • Genie Spaces (natural language data exploration)
  • Knowledge Assistants (RAG-based document Q&A)
  • MLflow Experiments (evaluation, scoring, traces)
  • Model Serving (deploy ML models and AI agents to endpoints)
  • Databricks Apps (full-stack web applications)
  • ...and more

Choose Your Own Adventure

Adventure Best For Start Here
Install ai-dev-kit Start here! Follow quick install instructions to add to your existing project folder Quick Start (install)
Starter Project Starter project for Claude Code + full Databricks integration ai-dev-project/
Visual Builder Web-based UI for Databricks development databricks-builder-app/
Skills Only Teaching your AI Databricks patterns (no actions) Install skills
MCP Tools Only Just executable actions (no guidance) Register MCP server
Core Library Building custom integrations (LangChain, OpenAI, etc.) pip install

Quick Start

Prerequisites

Install in existing project

By default this will install at a project level rather than a user level. This is often a good fit, but requires you to run your client from the exact directory that was used for the install.
Note: Project configuration files can be re-used in other projects. You find these configs under .claude or .cursor

Mac / Linux

  1. Open terminal
  2. Navigate to a local project directory (the root folder you will open with Cursor or Claude Code)
  3. Run curl -sL https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.sh | bash
  4. Respond to interactive prompts and follow next steps.
    • Note: Cursor and Co-pilot require updating settings manually after install.

Windows (Powershell)

  1. Open powershell terminal (requires git installed)
  2. Navigate to a local project directory (the root folder you will open with Cursor or Claude Code)
  3. Run irm https://raw.githubusercontent.com/databricks-solutions/ai-dev-kit/main/install.ps1 | iex

Starter Kit

If you prefer to clone the repo and start building, use the starter project.:

git clone https://github.com/databricks-solutions/ai-dev-kit.git
cd ai-dev-kit/ai-dev-project
./setup.sh
claude

This installs skills + MCP tools and opens Claude Code ready to build on Databricks.

Visual Builder App

Full-stack web application with chat UI for Databricks development:

cd ai-dev-kit/databricks-builder-app
./setup.sh
# Follow instructions to start the app

Skills Only

Just want to teach your local AI assistant Databricks patterns?

cd ai-dev-kit
./databricks-skills/install_skills.sh

Just want to teach your workspace AI assistant Databricks patterns?

cd ai-dev-kit
./databricks-skills/install_to_dbx_assistant.sh --profile <databricks cli profile>

Skills include: Spark Declarative Pipelines, Asset Bundles, MLflow Evaluation, Model Serving, Synthetic Data Generation, and more.

MCP Tools Only

Just want executable Databricks actions?

cd ai-dev-kit
./databricks-mcp-server/setup.sh

# Register with Claude Code
claude mcp add-json databricks "{
  \"command\": \"$(pwd)/.venv/bin/python\",
  \"args\": [\"$(pwd)/run_server.py\"]
}"

Core Library

Use databricks-tools-core directly in your Python projects:

from databricks_tools_core.sql import execute_sql

results = execute_sql("SELECT * FROM my_catalog.schema.table LIMIT 10")

Works with LangChain, OpenAI Agents SDK, or any Python framework. See databricks-tools-core/ for details.


What's Included

Component Description
databricks-tools-core/ Python library with high-level Databricks functions
databricks-mcp-server/ MCP server exposing 50+ tools for AI assistants
databricks-skills/ 15 markdown skills teaching Databricks patterns
databricks-builder-app/ Full-stack web app with Claude Code integration
ai-dev-project/ Starter template for new projects

Star History

Star History Chart

License

(c) 2026 Databricks, Inc. All rights reserved.

The source in this project is provided subject to the Databricks License. See LICENSE.md for details.

Third-Party Licenses
Package Version License Project URL
fastmcp ≥0.1.0 MIT https://github.com/jlowin/fastmcp
mcp ≥1.0.0 MIT https://github.com/modelcontextprotocol/python-sdk
sqlglot ≥20.0.0 MIT https://github.com/tobymao/sqlglot
sqlfluff ≥3.0.0 MIT https://github.com/sqlfluff/sqlfluff
litellm ≥1.0.0 MIT https://github.com/BerriAI/litellm
pymupdf ≥1.24.0 AGPL-3.0 https://github.com/pymupdf/PyMuPDF
claude-agent-sdk ≥0.1.19 MIT https://github.com/anthropics/claude-code
fastapi ≥0.115.8 MIT https://github.com/fastapi/fastapi
uvicorn ≥0.34.0 BSD-3-Clause https://github.com/encode/uvicorn
httpx ≥0.28.0 BSD-3-Clause https://github.com/encode/httpx
sqlalchemy ≥2.0.41 MIT https://github.com/sqlalchemy/sqlalchemy
alembic ≥1.16.1 MIT https://github.com/sqlalchemy/alembic
asyncpg ≥0.30.0 Apache-2.0 https://github.com/MagicStack/asyncpg
greenlet ≥3.0.0 MIT https://github.com/python-greenlet/greenlet
psycopg2-binary ≥2.9.11 LGPL-3.0 https://github.com/psycopg/psycopg2

Acknowledgments

MCP Databricks Command Execution API from databricks-exec-code by Natyra Bajraktari and Henryk Borzymowski.

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for ai-dev-kit

Similar Open Source Tools

For similar tasks

For similar jobs