Private Document QA System with Llama3, Postgres, Qdrant, and Google Drive

This n8n workflow automates the process of creating a private document Q&A system. It listens for new or updated files in a specified Google Drive folder, extracts their content, processes them using a Llama3 language model for embeddings and text splitting, stores them in a Qdrant vector store, and enables a conversational AI agent with Postgres chat memory to answer questions based on these documents.

What it does

This workflow simplifies the creation of a powerful, private Q&A system by:

1. Monitoring Google Drive: It triggers whenever new files are added or existing files are updated in a designated Google Drive folder.
2. Extracting File Content: It automatically extracts the text content from the uploaded or updated files.
3. Loading Documents: The extracted content is then loaded as documents for further processing.
4. Splitting Text: It uses a Recursive Character Text Splitter to break down large documents into smaller, manageable chunks, optimizing them for embedding and retrieval.
5. Generating Embeddings: It leverages an Ollama Embeddings model (likely running Llama3 or a similar model) to create vector embeddings for each text chunk.
6. Storing in Vector Database: These embeddings are then stored in a Qdrant Vector Store, making them searchable for semantic similarity.
7. Enabling Conversational AI: It sets up an AI Agent (powered by Ollama Chat Model, likely Llama3) that can answer questions.
8. Managing Chat History: It integrates Postgres Chat Memory to maintain conversation history, allowing the AI agent to understand context across multiple interactions.
9. Providing Q&A Tool: The AI agent is equipped with a Vector Store Question Answer Tool, enabling it to retrieve relevant information from the Qdrant store to answer user queries.

Prerequisites/Requirements

Before running this workflow, you will need:

• n8n Instance: A running n8n instance.
• Google Account: A Google account with access to Google Drive for the trigger and file operations. You'll need to set up Google Drive credentials in n8n.
• Ollama Installation: An Ollama server running locally or accessible via network, with the desired language model (e.g., Llama3) and embedding model downloaded.
• Qdrant Instance: A running Qdrant vector database instance.
• PostgreSQL Database: A PostgreSQL database for storing chat memory.
• Langchain Credentials: Configure Langchain credentials in n8n for Ollama, Qdrant, and Postgres.

Setup/Usage

1. Import the Workflow: Download the provided JSON and import it into your n8n instance.
2. Configure Google Drive Trigger:
   • Set up your Google Drive credential.
   • Specify the Google Drive folder ID you want to monitor for new or updated documents.
3. Configure Ollama Models:
   • In the "Embeddings Ollama" node, ensure the correct Ollama credential is set and the model name matches your downloaded embedding model (e.g., nomic-embed-text).
   • In the "Ollama Chat Model" node, ensure the correct Ollama credential is set and the model name matches your downloaded chat model (e.g., llama3).
   • In the "Ollama Model" node, ensure the correct Ollama credential is set and the model name matches your downloaded text completion model (e.g., llama3).
4. Configure Qdrant Vector Store:
   • Set up your Qdrant credential.
   • Specify the collection name where embeddings will be stored.
5. Configure Postgres Chat Memory:
   • Set up your PostgreSQL credential.
   • Ensure the database and table are correctly configured for chat history.
6. Activate the Workflow: Once all credentials and configurations are set, activate the workflow.

Now, any new or updated documents in your specified Google Drive folder will be automatically processed and added to your Q&A system. You can then interact with the AI Agent via the "When chat message received" trigger to ask questions about your private documents.