DataForge helps data teams write functional transformation pipelines by leveraging software engineering principles

An end-to-end data pipeline which extracts divvy bikeshare data from web loads it into data lake and datawarehouse transforms it using dbt and finally , a dashboard to visualize the data using looker studio, the pipeline is orchestrated using prefect

Modeling tool like DBT to use SQL Alchemy core with a DataFrame interface like

A modern data platform implemented on Azure Synapse Analytics using ELT Framework - https://github.com/bennyaustin/elt-framework. Data platform infrastructure provisioned using https://github.com/bennyaustin/iac-synapse-dataplatform

💻💛Fundamental Data Engineering Course 2024 Week4 Learn DBT Transform Data with Models, Macro, ELT-Pipeline with Dagster 🌎

🛸 This project showcases an Extract, Load, Transform (ELT) pipeline built with Python, Apache Spark, Delta Lake, and Docker. The objective of the project is to scrape UFO sighting data from NUFORC and process it through the Medallion architecture to create a star schema in the Gold layer that is ready for analysis.

Public DBT instance to aid in data transformation for analytics purposes

🌄📈📉 A Data Engineering Project 🌈 that implements an ELT data pipeline using Dagster, Docker, Dbt, Polars, Snowflake, PostgreSQL. Data from kaggle website 🔥

This Repo contains all study, lab and supportive materials for Udemy course on "Google Cloud Professional Data Engineer - A Complete Guide".

🍺 A data engineering project showcasing an ELT pipeline using modern technologies such as Delta-rs, and Apache Airflow.

A deep dive into North American grocery e-commerce behaviour based on Instacart's open dataset. [ELT, EDA, ML clustering]

Data engineering projects

This is an ELT data pipeline setup to track the activities of an e-commerce website based on orders, reviews, deliveries and shipment date. This project utilized technologies like Airflow, AWS RDS-Postgres, Python etc.

SQL, Databases, warehouses, Data lake, cloud storage, MYSQL, Data Pipeline

An end to end ELT project that uses data from the Zomato Restaurant, an Indian multinational restaurant aggregator and food delivery company. The project extracts data from Kaggle dataset, loads it into Snowflake tables, then is transformed and modelled in dbt Labs.

Data Engineering project which involves ETL using PostgreSQL and Python

Irish Property Price Register transformed into a data warehouse via an EtLT pipeline.

Enterprise ELT Framework using Airbyte, dbt, Prefect, and Power BI for seamless data extraction, transformation, and visualization. This project showcases a scalable pipeline integrating SQL Server, GCP, and tabular models in Power BI for real-time analytics and business intelligence. Ideal for data engineers and analysts seeking efficient ETL/ELT.

Builded an ETL pipeline using Python, Pandas, Python dictionary methods and regular expressions to ETL data. It involves extracting data from multiple sources, cleaning and transforming the data using Jupyter Notebook with pandas, numpy, and datetime packages, and loading the cleaned data into a relational database using pgAdmin

"AutoImageCaption-CNNvsResNet" leverages the Flickr 8k Dataset to automate image captioning, comparing CNN+LSTM and ResNet+GRU models using BLEU scores for performance evaluation.