Skip to content

Latest commit

 

History

History
82 lines (56 loc) · 2.42 KB

README.md

File metadata and controls

82 lines (56 loc) · 2.42 KB
Raw

⚠️ THIS IS A PUBLIC REPO ⚠️

💀 Don't push private/confidantial code! 💀

Hadoop cluster - lite

Build a lightweight hadoop cluster with below mentioned hadoop services. These versions match what Hortonworks HDP 2.3.4 distribution has.

forked from: https://github.com/anair-it/hadoop-docker-lite

Dockerhub: https://hub.docker.com/r/anoopnair/

Versions

  • Alpine Linux: latest
  • Debian: 8.3
  • Java: Open JDK 7
  • hadoop : 2.7.1
  • Zookeeper: 3.4.6
  • Kafka: 0.9.0.1
  • Kafka manager: latest

Note: Hadoop is built on Debian Linux and all others on Alpine Linux

Prerequsities

The Docker project is tested on Ubuntu host. There is no guarantee that this will work on Windows.

Building all images

  • Pull from docker hub OR
  • Build each image individually

This will individually build the above mentioned images. This will take sometime and is required to be done only once.

  • Get zookeeper IP address using

docker inspect --format='{{.NetworkSettings.IPAddress}}' zookeeper

  • Get hbase IP address using

docker inspect --format='{{.NetworkSettings.IPAddress}}' hbase

  • Add zookeeper and hbase entries in the host /etc/hosts file

Verification

  • docker images to see all images created with proper tag version
  • docker logs -f _CONTAINER_NAME_ to follow logs for a specific container

If there is inconsistency in the image creation process, run the affected build script only. The build script is in each component directory.

Starting

./start.sh (background process) OR

docker-compose up (foreground process)

Stopping

docker-compose stop

Web UI

READMEs

Please go through below READMEs to get detailed information about each image

Reference