AI-Powered RAG Workflow for Document Analysis

This n8n workflow demonstrates a powerful Retrieval Augmented Generation (RAG) system designed to process documents, create embeddings, store them in a vector database, and then use an AI agent to answer questions based on the stored information. This can be particularly useful for analyzing large sets of documents like stock earnings reports, research papers, or legal documents.

What it does

This workflow automates the following steps:

1. Manual Trigger: Initiates the workflow execution manually.
2. Google Drive Integration: Retrieves a list of files (e.g., Google Docs, Google Sheets) from a specified Google Drive folder.
3. Loop Over Items: Processes each retrieved file individually or in batches.
4. Google Docs / Google Sheets Integration: Depending on the file type, it reads the content from Google Docs or Google Sheets.
5. Default Data Loader: Loads the content of the document into a format suitable for text processing.
6. Recursive Character Text Splitter: Breaks down the document content into smaller, manageable chunks (e.g., paragraphs or sentences) to optimize for embedding and retrieval.
7. Embeddings Google Gemini: Generates vector embeddings for each text chunk using the Google Gemini embedding model.
8. Pinecone Vector Store: Stores these text chunks and their corresponding embeddings in a Pinecone vector database, making them searchable for semantic similarity.
9. AI Agent: Utilizes an AI agent (configured with either an OpenAI Chat Model or Google Gemini Chat Model) to answer questions.
10. Vector Store Question Answer Tool: Equips the AI agent with a tool to query the Pinecone vector store, retrieve relevant document chunks, and use them as context to generate accurate and informed answers.

Prerequisites/Requirements

To use this workflow, you will need:

• n8n Instance: A running n8n instance.
• Google Account: For Google Drive, Google Docs, and Google Sheets integrations. Ensure your Google account has access to the relevant files and folders.
• Google Cloud Project (for Google Gemini): If using Google Gemini for embeddings and chat, you'll need a Google Cloud Project with the Gemini API enabled and appropriate credentials.
• OpenAI API Key: If using the OpenAI Chat Model for the AI Agent.
• Pinecone Account: An active Pinecone account with an API key and environment configured to store vector embeddings.

Setup/Usage

1. Import the Workflow: Download the provided JSON and import it into your n8n instance.
2. Configure Credentials:
   • Set up Google OAuth2 credentials for the Google Drive, Google Docs, and Google Sheets nodes.
   • Configure your Pinecone credentials (API Key and Environment).
   • Set up OpenAI credentials (API Key) if you plan to use the OpenAI Chat Model.
   • Set up Google Gemini credentials (API Key) if you plan to use the Google Gemini Chat Model and Embeddings.
3. Customize Nodes:
   • Google Drive: Specify the folder ID where your target documents are located.
   • Google Docs / Google Sheets: Ensure these nodes are correctly configured to read the content of your documents. You might need to adjust settings based on your document structure.
   • Recursive Character Text Splitter: Adjust chunkSize and chunkOverlap parameters based on the nature of your documents and desired retrieval granularity.
   • Pinecone Vector Store: Configure the indexName and other settings to match your Pinecone index.
   • AI Agent: Select your preferred Language Model (OpenAI Chat Model or Google Gemini Chat Model) and configure its settings (e.g., model name, temperature).
   • Vector Store Question Answer Tool: Ensure the tool is correctly linked to your Pinecone Vector Store.
4. Execute the Workflow: Click the "Execute workflow" button in the Manual Trigger node to run the workflow. The workflow will process your documents, populate the vector store, and then the AI agent will be ready to answer questions based on this knowledge base.