n8n Workflow: Evaluate Hybrid Search for Legal Question Answering

This n8n workflow is designed to facilitate the evaluation of a hybrid search system, likely for legal question-answering applications. It allows for the iterative testing and aggregation of results from an external API, processing data in batches, and filtering based on specific criteria.

What it does

This workflow automates the following steps:

1. Manual Trigger: Initiates the workflow upon manual execution.
2. Edit Fields (Set): Prepares the input data by setting or modifying fields before further processing.
3. Loop Over Items (Split in Batches): Divides the input data into manageable batches, allowing for iterative processing of items. This is crucial for handling large datasets or rate-limited APIs.
4. HTTP Request: For each item in a batch, it sends an HTTP request to an external API (likely a search or evaluation endpoint).
5. Filter: Filters the results received from the HTTP request based on a defined condition. This allows for focusing on relevant or successful responses.
6. Aggregate: Combines the filtered results from all batches back into a single dataset.
7. Split Out: Further processes the aggregated data, potentially splitting out nested arrays or objects into individual items for easier consumption or subsequent analysis.
8. Merge: Merges data from different branches of the workflow, likely combining the initial input with the processed results.

Prerequisites/Requirements

• n8n Instance: A running instance of n8n to import and execute the workflow.
• External API Endpoint: Access to an API that performs the hybrid search or evaluation, which the "HTTP Request" node will call.
• API Key/Authentication: Any necessary API keys or authentication details for the external API. These should be configured as n8n credentials.

Setup/Usage

1. Import the Workflow:
   • Download the provided JSON file for this workflow.
   • In your n8n instance, click on "Workflows" in the left sidebar.
   • Click "New" and then "Import from JSON".
   • Paste the JSON content or upload the file.
2. Configure Credentials:
   • Locate the "HTTP Request" node.
   • If the external API requires authentication (e.g., API Key, Bearer Token), configure the appropriate credentials within the "HTTP Request" node.
3. Adjust Node Settings:
   • Edit Fields (Set): Modify the fields being set or transformed to match your input data structure.
   • Loop Over Items (Split in Batches): Adjust the batch size as needed based on your data volume and API rate limits.
   • HTTP Request:
     • Set the correct URL for your hybrid search/evaluation API.
     • Configure the HTTP Method (e.g., POST, GET).
     • Define the Body of the request, mapping data from previous nodes (e.g., {{ $json.query }}).
   • Filter: Define the Conditions for filtering the API responses based on your evaluation criteria.
   • Aggregate: Ensure the aggregation method is suitable for combining your results.
   • Split Out: Configure how you want to split out nested data from the aggregated results.
4. Test the Workflow:
   • Click "Execute Workflow" to run a test.
   • Review the output of each node to ensure data is flowing and being transformed as expected.
5. Activate the Workflow:
   • Once configured and tested, activate the workflow to make it ready for use. Since this workflow uses a Manual Trigger, it will only run when explicitly executed.