Two-Stage Document Retrieval Chatbot with OpenAI and Supabase Vector Search

This n8n workflow creates a sophisticated chatbot that leverages OpenAI for conversational AI and Supabase for efficient vector-based document retrieval. It implements a two-stage retrieval process to provide more accurate and contextually relevant answers to user queries.

What it does

This workflow automates the following steps:

1. Listens for User Input: It starts by receiving a chat message from a user, which serves as the initial query.
2. Generates Search Query (Stage 1 Retrieval): Using OpenAI, it takes the user's initial chat message and generates an optimized search query specifically designed for document retrieval.
3. Performs Vector Search: It then uses the generated search query to perform a vector-based search in a Supabase database, retrieving relevant document chunks.
4. Refines Search Query (Stage 2 Retrieval): The retrieved document chunks, along with the original user query, are fed back into OpenAI to generate a more refined and focused search query. This second query aims to improve the relevance of the final retrieval.
5. Performs Second Vector Search: Another vector search is executed against the Supabase database using the refined query, aiming to fetch even more precise document chunks.
6. Aggregates Retrieved Content: All relevant document chunks from both retrieval stages are combined.
7. Generates Chatbot Response: Finally, the aggregated document content, along with the original user query, is sent to OpenAI to generate a comprehensive and contextually informed chatbot response.

Prerequisites/Requirements

To use this workflow, you will need:

• n8n Instance: A running n8n instance to import and execute the workflow.
• OpenAI API Key: An API key for OpenAI to access its language models for query generation and response generation.
• Supabase Project: A Supabase project configured with a vector database (e.g., using pg_vector) containing your document embeddings.
• Supabase API Key and URL: The API key and URL for your Supabase project to allow n8n to connect and perform vector searches.

Setup/Usage

1. Import the Workflow:
   • Download the provided JSON file for this workflow.
   • In your n8n instance, click "New" in the workflows section, then "Import from JSON" and upload the file.
2. Configure Credentials:
   • OpenAI: Set up an OpenAI credential in n8n with your API key.
   • Supabase: Set up a Supabase credential in n8n with your Supabase project URL and API key.
3. Configure Nodes:
   • Chat Trigger: This node is pre-configured to listen for chat messages. You might need to adjust its settings depending on how you integrate it (e.g., with a specific chat platform).
   • HTTP Request (Supabase calls):
     • Ensure the HTTP Request nodes (named "Supabase" in the workflow) are correctly configured to point to your Supabase vector search endpoint.
     • Verify the headers for apikey and Authorization use your Supabase API key.
     • Adjust the match_count and query_embedding parameters as needed for your specific Supabase setup and desired retrieval behavior.
   • OpenAI:
     • Ensure the OpenAI nodes are using your configured OpenAI credential.
     • Review the prompts in the OpenAI nodes (e.g., for generating search queries and the final response) and adjust them to best suit your use case and desired chatbot personality/behavior.
   • Code Tool: This node is likely used for custom logic related to embedding generation or data formatting. Review its JavaScript code and modify it if necessary to match your Supabase schema or embedding model requirements.
4. Activate the Workflow: Once all credentials and node configurations are complete, activate the workflow to start processing chat messages.

This workflow provides a robust foundation for building intelligent chatbots capable of answering complex questions by dynamically retrieving information from a vector database.