Latent-Dirichlet-Allocation

Implementation of Latent Dirichlet Allocation from scratch.

File description:

1. webCrawl.py has the python code to collect top 10k most recent Abstracts from arXiv.org under cs.LG category.
2. LDA.py has the implementation of Latent Dirichlet Allocation using colapsed Gibbs Sampling.
3. evaluate.py has code for various visualisations and topic distributions.
4. DataBase.csv has the web crawled data in csv format from arXiv.org cs.LG. (as of May 26,2021).
5. Plots- Contains plots of top 10 documents(among 10k) with their topic distributions and the plot of distibution of topics over the corpus.