Hi there! 👋 Welcome to flowise-private-doc-chat-rag-blog, where we explore how to build a fully private, conversational interface for your documents. Using Flowise, open-source LLMs, and retrieval-augmented generation (RAG), you can chat with your documents locally—no internet connection required.
This setup is perfect for those who value data privacy but still want the benefits of interacting with their documents using AI. This guide is part of a blog series, read it here
- Privacy First: All processing happens locally—your documents and queries never leave your machine.
- PDF Compatibility: Upload and process your PDFs effortlessly.
- Intelligent Splitting: Dynamically splits your documents into chunks for better embeddings and retrieval.
- Powerful Embeddings: Generate embeddings using the Ollama
nomic-embed-textmodel. - Conversational Memory: Maintain chat history for contextually aware responses.
- Locally Hosted LLMs: Use open-source models like Llama 2 via the Ollama server.
Here's a quick breakdown of the workflow:
- 📄 PDF Loader: The system uploads and parses your PDF into manageable text chunks.
- ✂️ Text Splitter: Chunks are split into smaller, overlapping sections to preserve context.
- 📊 Embedding Creation: Text is transformed into vector embeddings using the Ollama model.
- 💾 Vector Store: Embeddings are stored in memory for quick retrieval.
- 💬 Conversational Chain: A retrieval-based Q&A system connects with a local LLM for seamless, context-aware conversations.
Here's what the Flowise workflow looks like:
- Docker: For running the Ollama server.
- Node.js: To install and run Flowise.
- A PDF Document: Your chat-ready file.
git clone https://github.com/dwain-barnes/flowise-private-doc-chat-rag-blog.git
cd flowise-private-doc-chat-rag-blog### 3. Install and Start Flowise
Set up Flowise to manage your chatflow:
```bash
npx flowise
Visit http://localhost:3000 to open the Flowise dashboard.
- Navigate to Chatflows in Flowise.
- Import the file
rag.jsonfrom this repository. - Connect your nodes and adjust parameters (like chunk size) as needed.
- Upload your PDF file.
- Start chatting with your document!
Feel free to tweak these settings:
- Embedding Model: Update the embedding model in the Ollama Embeddings node to try different configurations.
- Chunking: Adjust
Chunk SizeandChunk Overlapin the Recursive Character Text Splitter node for better control over how your document is processed. - LLM Model: Switch the LLM in ChatOllama to explore different local models.
I’ve always wanted a way to interact with sensitive documents securely, without relying on third-party services. With Flowise and Ollama, I can process and chat with documents in a completely private environment. Whether you’re analyzing research papers, contracts, or notes, this setup is easy to use and incredibly powerful. Plus, it’s fun!
Ollama lets you host open-source LLMs locally, and Flowise makes it super easy to create workflows for document processing. Together, they’re a perfect match for private RAG systems.
This project uses Llama 3.2 3B via Ollama, but you can switch to other models.
Yes! As long as you have Docker and Node.js installed, this works on any major operating system.
This is a personal project, but I’d love for others to contribute! If you’ve got ideas for improving the workflow or want to add features, feel free to submit a pull request.
- Flowise for their amazing open-source tool.
- Ollama for enabling local LLM hosting.
- My fellow open-source enthusiasts for inspiring this project!
Let’s build something awesome! 🚀