Automatic Text Generation using Tensorflow, Keras and LSTM / LSTM+Attention
This project has been selected to provide me with valuable hands-on experience and allow me to apply the NLP knowledge I have recently acquired. It served as a practical way to put my skills to the test and further solidify my understanding of NLP concepts. The main objective of this project is to generate coherent and contextually relevant text using deep learning techniques. By leveraging the capabilities of LSTM, we can train a model to learn patterns and structures in a given text dataset, and then generate new text based on that learned knowledge. It presents two distinct approaches to text generation, each with its own unique features and capabilities.
In the first approach, we focus on the application of LSTM as a standalone model for text generation. This involves training the LSTM model on a dataset of text samples and then using it to generate new text based on the learned patterns. This approach is a great starting point to understand the fundamentals of text generation using LSTM.
The second approach takes text generation to the next level by integrating LSTM with a Multi-Head Attention mechanism. This attention mechanism allows the model to focus on different parts of the input text during the generation process, resulting in more contextually relevant and coherent text output. This approach is particularly useful for generating longer and more complex text sequences.