《Kafka技术内幕》代码

One click deploy docker-compose with Kafka, Spark Streaming, Zeppelin UI and Monitoring (Grafana + Kafka Manager)

Spark with Scala example projects

Samples for using Kafka within Spark Streaming and Akka Actors, Akka Streams

These are a select few projects related to Big Data Analytics and Management. The projects listed are a combination of both small and big projects but interesting ones.

End-to-end Kafka Streaming Examples on Databricks with Evolving Avro Schemas.

An ETL application which is written in Quarkus, Spark SQL Streaming, Neo4j and various types of Databases and stores. It also covers the devops frameworks like Jenkins CI/CD, docker and Kubernetes.

Design data streaming architecture and API for a real-life application called the Step Trending Electronic Data Interface (STEDI). It is a working application used to assess fall risk for seniors. When a senior takes a test, they are scored using an index which reflects the likelihood of falling, and potentially sustaining an injury in the cours…

Spark Examples

Structured Streaming Log Analysis

Research on legacy and structured streaming with Spark

In-Stream final project

Intro to streaming data with Kafka, Spark and AWS Glue

Playstore apps rating analysis - Machine Learning on Bigdata- Loading streaming Data using Kafka and Flume

Near real-time streaming using Apache Spark and Apache Kafka

spark-kafka-integration

The core objective of this project is to build an end-to-end data streaming pipeline that processes this dataset in real-time. By leveraging modern data engineering tools and techniques, we aim to connect, buffer, process, store, and visualize streaming data. This allows for better understanding of data flows, handling of large-scale real-time data

Repository for Spark structured streaming use case implementations.