databricks-workflows

Production-ready PySpark ETL template for Databricks — medallion architecture, DABs, tests, DQX, CI/CD, and agentic development with Claude Code.

Lightweight Python library and CLI that generates Databricks Workflows definition from your dbt project — each dbt model, test, seed, and snapshot runs as a separate task.

Developed a metadata-driven ETL framework that automates configurable data ingestion, SCD processing, and data quality using generic pipelines.

Built a real-time banking analytics platform for streaming transaction processing, fraud detection, and KPI monitoring.

Built an end-to-end retail lakehouse using the Medallion architecture with batch and streaming pipelines for scalable business analytics.

This azure databricks project implements a modern data engineering pipeline on Azure using the Medallion Architecture (Bronze → Silver → Gold) to process NYC taxi trip data.

AI-agent skill for migrating Matillion ETL pipelines to Databricks

Commercial sales analytics platform built on Databricks and dbt. Medallion architecture, Delta Lake, automated Workflows and a live AI/BI dashboard, with governed gold data marts covering the full deal lifecycle from pipeline stage through activity and rep performance.

Medallion pipeline over the public NYC TLC taxi dataset on Databricks Free Edition.