Formatting Worker to Format Data from different Data Portals into POJOs. It drops all DataPoints older than 8 hours as they are not representative anymore. Furthermore it deduplicates Datapoints by dropping Points that were already ingested during the last 48 hours.
Architecture Overview:
- Rectangle: Kafka Topic
- Rhombus+Annotation: Operation
The Output Queues have the following Schema:
- HygonData
- Key: String (Stationname)
- Value: HygonDataPoint (from dataFormats)
- PegelOnlineData
- Key: String (Stationname)
- Value: PegelOnlineDataPoint (from dataFormats)
- Apache Maven for packaging.
mvn packageA jar File will be created in the /target directory.
For simplicity and portability the jar File has all dependencies included and can simply be run as a jar executable.
java -jar target/dataFormatter-1.1.0-jar-with-dependencies.jar We use Semamtic Versioning for versioning.
- Jan Speckamp - Initial work - speckij
This project is licensed under the GNU General Public License v3.0 - see the LICENSE.md file for details