WhisperMesh is your cutting-edge chatbot that seamlessly blends voice and text interactions, creating a rich, intuitive conversational experience. With the power of LLM models and a sophisticated vector database, WhisperMesh understands your needs like never before, providing tailored responses that resonate with your queries.
Harnessing the RAG framework from Haystack, our app excels in extracting relevant information, ensuring that every interaction is not only engaging but also data-driven. Whether you prefer speaking or typing, WhisperMesh adapts to your style, transforming your input into insightful answers with a personal touch.
Join the conversation with WhisperMesh, where your voice matters, and let us guide you through a world of knowledge and discovery! 🌟💬✨
The architecture outlines a Streamlit application integrated with the HayStack framework to facilitate document uploading, querying, and response generation. Users upload PDF documents through the app, which are then processed and indexed by HayStack components, including document cleaners, splitters, and embedders that convert documents into vector representations using the Cohere model, stored in the Qdrant vector database. When a user submits a voice query, it is transcribed to text, embedded into a vector, and matched against the stored document vectors to retrieve relevant chunks. These chunks are fed into the gemma 7B generative model to generate a coherent response, which is then converted back to audio for user output. This system leverages advanced NLP techniques to ensure efficient and accurate document retrieval and response generation.
WhisperMesh offers a cutting-edge conversational experience by blending voice and text inputs, allowing you to communicate in the way that suits you best.
Powered by state-of-the-art large language models (LLMs), WhisperMesh understands your needs deeply and provides tailored responses that resonate with your queries.
Utilizing Qdrant as our vector database, WhisperMesh efficiently manages and retrieves relevant information, ensuring data-driven interactions that enhance user engagement.
Our application harnesses the Cohere embedding model for effective semantic understanding, ensuring that every interaction is insightful and contextually aware.
With the Whisper-large-v3 model from Groq Cloud, WhisperMesh excels at converting speech to text, making voice interactions smooth and accurate.
Employing the gemma-7b-it model from Groq Cloud, our app generates personalized responses, adding a unique touch to every conversation.
Harnessing the RAG (Retrieval-Augmented Generation) framework from Haystack, WhisperMesh excels in extracting relevant information, ensuring that your interactions are engaging and informative.
- Upload Options: Drag and drop your files or click on 'Browse Files' to select them. (Maximum file size: 200MB).
- Start Indexing: Click the '📄 Index Document' button to initiate the indexing process.
- Decide whether you prefer to chat via text or voice.
- Text Input: Use the text box below to type your question. 📝
- Voice Input: Click the 🎙️ microphone icon to ask questions using your voice.
- Python 3.11 or above 🐍
- Groq API for inference, which is currently available for free in its beta version with rate limits. You can obtain your API key here after creating an account: Groq API.
- Additionally, you have the option to use Qdrant either locally or via Qdrant Cloud. Its API is also free to use. Access it here: Qdrant Cloud.
- For embeddings, you can use the Cohere API. It offers a free tier and you can sign up and get your API key here: Cohere API.
git clone https://github.com/Mouez-Yazidi/WhisperMesh.git
cd WhisperMesh-
Create a
.envfile and add the following variables according to the credentials you obtained from the required platforms:COHERE_API_KEY= GROQ_API= GROQ_KEY= QDRANT_API= QDRANT_KEY=
Navigate to the local directory and install the necessary dependencies:
cd local
pip install -r requirements.txtTo run the app locally, execute the following command:
streamlit run ../app/main.py --environment localYou should now be able to access the app at http://localhost:8501 🌐.
If you prefer running the app in a Docker container, follow these steps:
- Make sure you have Docker installed 🐋.
- Build the Docker image:
docker build -t whispermesh -f Dockerfile ..- Run the container:
docker run -p 8501:8501 whispermesh streamlit run main.py --environment localEnsure that your code is pushed to a GitHub repository 📂.
- Visit Streamlit Cloud and sign in.
- Connect your GitHub repository 🔗.
- Choose your repository and branch.
- Go to the "Advanced settings" section of your app.
- In the "Secrets" section, input any sensitive information, such as API keys or other credentials.
- Make sure to add this variables according to the credentials you obtained from the required platforms.
COHERE_API_KEY=""
GROQ_API=""
GROQ_KEY=""
QDRANT_API=""
QDRANT_KEY=""Streamlit Cloud will:
- Install dependencies from cloud/requirements.txt 📦
🎉 You’re all set! Your app will now be live on Streamlit Cloud!
- Configurable: Separate configuration files for local and cloud deployment.
- Docker Support: Deploy the app using Docker for local containerization.
If you appreciate this project, I would be grateful if you could give it a star on GitHub. Your support motivates us to enhance and expand our work!
This project is licensed under the MIT License. See the LICENSE file for more details.
If you have any questions or suggestions, feel free to open an issue or contact us at mouez.yazidi2016@gmail.com.