Upsert Huge Documents in a Vector Store with Supabase and Notion

This n8n workflow demonstrates how to efficiently process and store large documents from Notion into a Supabase vector store, enabling advanced AI capabilities like question-answering. It leverages Langchain nodes to handle document loading, splitting, embedding, and vector store management.

What it does

This workflow automates the following steps:

1. Triggers on a Schedule: Periodically checks for updates or new documents.
2. Fetches Notion Pages: Retrieves content from a specified Notion database.
3. Loads Document Data: Prepares the Notion page content as a document for processing.
4. Splits Documents into Chunks: Breaks down large documents into smaller, manageable chunks (tokens) suitable for embedding.
5. Generates Embeddings: Uses OpenAI to create vector embeddings for each document chunk.
6. Stores Embeddings in Supabase: Upserts the generated embeddings and associated metadata into a Supabase vector store.
7. (Optional) Handles Chat Messages: Includes a Chat Trigger and Question and Answer Chain for potential future integration with a chatbot to query the vector store. This part of the workflow is currently disconnected but shows potential for a conversational AI application.

Prerequisites/Requirements

To use this workflow, you will need:

• n8n Instance: A running n8n instance.
• Notion Account: With access to the database containing the documents you wish to process.
• Supabase Account: Configured with a vector store (e.g., pgvector extension enabled) to store the document embeddings.
• OpenAI API Key: For generating document embeddings.
• Notion Credentials: Configured in n8n.
• Supabase Credentials: Configured in n8n.
• OpenAI Credentials: Configured in n8n.

Setup/Usage

1. Import the Workflow: Download the JSON provided and import it into your n8n instance.
2. Configure Credentials:
   • Set up your Notion credentials.
   • Set up your Supabase credentials.
   • Set up your OpenAI credentials.
3. Configure Notion Trigger:
   • Specify the Notion database ID you want to monitor.
   • Adjust the polling interval as needed.
4. Configure Supabase Vector Store:
   • Specify your Supabase project URL and API key.
   • Define the table name and column names for your vector store.
5. Configure Embeddings OpenAI:
   • Ensure your OpenAI credentials are correctly linked.
6. Activate the Workflow: Once configured, activate the workflow to start processing your Notion documents.

Note on Chat Functionality: The Chat Trigger and Question and Answer Chain nodes are present but disconnected in the provided JSON. To enable a chat-based Q&A system, you would need to:

1. Connect the Chat Trigger to the Question and Answer Chain.
2. Configure the Question and Answer Chain with the OpenAI Chat Model and Vector Store Retriever (which would point to your Supabase vector store).
3. Integrate the Chat Trigger with your desired chat platform (e.g., Slack, Telegram) to receive messages.