Build a data warehouse in SQL Server from two source systems, a CRM and an ERP, through bronze, silver and gold. The files and scripts of the free full project.
-
Updated
Nov 6, 2026 - TSQL
Build a data warehouse in SQL Server from two source systems, a CRM and an ERP, through bronze, silver and gold. The files and scripts of the free full project.
A data lakehouse on Databricks with the medallion architecture: PySpark, Spark SQL, Delta Lake and Unity Catalog. The project of the free Databricks bootcamp.
End-to-end Banking Analytics with Dataiku | Data Preparation, Machine Learning, Customer Scoring & Automated Pipelines
FinOps repository: An open-source framework for multi-cloud cost visibility. Extendable with dlt.
Building a next-generation hybrid data pipeline architecture that combines the power of Microsoft Fabric, Azure Cloud, and Power BI. This pipeline is engineered to tackle the challenges of real-time data ingestion, multi-layered processing, and analytics, delivering business-critical insights.
Production-grade data scraper for League of Legends ranked matches — Solo/Duo & Flex 5v5 — built with Python, async fetching, SQLite storage, and enterprise logging.
Build a data pipeline on Google Cloud using an event-driven architecture, leveraging GCS, Cloud Run functions, and BigQuery. Explore both VM and Composer options for Airflow management, and utilize Logging & Monitoring for pipeline health. Discover how SQL-based BigQuery ML can be used for initial ML implementation in specific scenarios.
Realtime Data Streaming | End-to-End Data Engineering Project
End-to-end Azure Data Engineering pipeline using ADF, Databricks (PySpark), ADLS Gen2, Azure SQL, and Power BI for COVID-19 analytics
Real-time flight data pipeline using Kafka, Spark Structured Streaming and Medallion Architecture on Databricks
End-to-end data platform for Emilia-Romagna air quality (ARPAE open data). Rust ingestion to partitioned Parquet, dbt transformations with data quality tests, Dagster orchestration. Runs locally on DuckDB and on BigQuery on GCP via Terraform, with identical marts. Tracks legal limit exceedances, pollutant trends and weather correlation.
MCP agent for semantic metrics governance with built-in trust scoring, lineage visualization, and conversational metric definition. Designed for data teams working with dbt, LookML, and modern semantic layers.
Фабрика DAG
ELT pipeline for Vietnam stock market (HOSE & HNX) — collecting OHLC data and news sentiment using Python, PostgreSQL, dbt, and Metabase.
Enterprise-grade ingestion blueprint for Postgres to Databricks powered by dlt. Features dual-mode operation (Full Load + CDC Load) and robust CI/CD via Databricks Asset Bundles.
End-to-end data engineering pipeline using Medallion Architecture (Bronze→Silver→Validation→Gold) on Databricks with PySpark, Apache Airflow orchestration and Power BI dashboard
Collection of projects inspired by Data Engineering Zoomcamp 2026, where each module is rebuilt, extended, and improved beyond the original course material.
Personal project - Build my own data lake to further deepen understanding and knowledge about data lakes, data lifecycle and data engineering.
End-to-end Azure Data Engineering project for Healthcare Revenue Cycle Management
To associate your repository with the data-engineering-project topic, visit your repo's landing page and select "manage topics."