How to monitor your kafka connectors

This a quick set up of kafka, kafka ui and kafka connect with a monitoring through Prometheus and Grafana using docker compose.

Architecture

Required software:

Here we are using a mongo kafka sink connector. Jar is downloaded from here and put in the kafka-jars.

We are using jmx exporter to collect kafka connect metrics.

Metrics config and grafana dashboard are download from here.

See this article for more explanation.