Name		Name	Last commit message	Last commit date
Latest commit History 34 Commits
.github		.github
crawler-common		crawler-common
crawler-core		crawler-core
crawler-web		crawler-web
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
arch.png		arch.png
renovate.json		renovate.json

Repository files navigation

Web Crawler

Web Crawler

Modules

Crawler common
Crawler core
Crawler web

Dependencies

Couchbase database: https://www.couchbase.com/downloads MinIO: https://min.io/download Kafka and Zookeeper: https://kafka.apache.org/

How to run

VM Argument: -Dspring.config.location=//config/

About

An Efficient and scalable Web Crawler

Apache-2.0 license

Report repository

Releases

No releases published

Packages

No packages published

Contributors 4

Languages