n8n Recipe Recommendation Engine with Bright Data MCP & OpenAI 4o Mini

This n8n workflow acts as a sophisticated recipe recommendation engine. It leverages web scraping capabilities (via Bright Data's Web Scraper IDE) to gather recipe data, processes this data, and then uses OpenAI's Chat Model to generate personalized recipe recommendations. The workflow is designed to handle multiple recipe requests efficiently by batching and introducing delays to manage API rate limits.

What it does

1. Triggers Manually: The workflow is initiated manually, allowing for on-demand recipe recommendations.
2. Loads Bright Data Scraper Configuration: It reads a JSON configuration file from disk, which is expected to contain the setup for a Bright Data Web Scraper IDE project.
3. Sets Initial Scraper Parameters: A Function node dynamically sets the target URL for the Bright Data scraper based on the loaded configuration.
4. Executes Bright Data Scraper: It makes an HTTP request to the Bright Data Web Scraper IDE API to initiate a web scraping job, retrieving recipe data from the specified website.
5. Processes Scraped Data in Batches: The scraped recipe data is then processed in batches, with a Wait node introduced between batches to prevent overwhelming downstream APIs or to manage rate limits.
6. Prepares Data for OpenAI: For each batch of scraped data, a Function node transforms and formats the raw data into a structured prompt suitable for the OpenAI Chat Model.
7. Generates Recipe Recommendations with OpenAI: It utilizes the OpenAI Chat Model (specifically gpt-4o-mini) through a Basic LLM Chain to generate recipe recommendations based on the processed scraped data.
8. Parses Structured Output: A Structured Output Parser node extracts and formats the recommendations from OpenAI's response into a usable structured JSON format.
9. Final Data Transformation: A Code node performs any final data transformations or cleanup on the generated recommendations before the workflow concludes.

Prerequisites/Requirements

• n8n Instance: A running n8n instance.
• Bright Data Account: An account with Bright Data and an active Web Scraper IDE project configured to scrape recipe data.
  • Bright Data API Key: For authenticating HTTP requests to the Bright Data API.
  • Bright Data Scraper ID: The ID of your specific Web Scraper IDE project.
• OpenAI API Key: An API key for OpenAI, with access to the gpt-4o-mini model.
• Local File System Access: The n8n instance needs access to its local file system to read the Bright Data scraper configuration JSON file.
• Scraper Configuration File: A JSON file named bright_data_scraper_config.json located in the n8n execution environment, containing the Bright Data scraper's target URL and other necessary parameters.

Setup/Usage

1. Import the Workflow: Import the provided JSON into your n8n instance.
2. Configure Credentials:
   • Bright Data: Set up an HTTP Request credential for Bright Data, using your API key. The HTTP Request node (HTTP Request) will need to be configured with this credential.
   • OpenAI: Set up an OpenAI Chat Model credential using your OpenAI API key. The OpenAI Chat Model node will use this credential.
3. Create bright_data_scraper_config.json:
   • Create a JSON file named bright_data_scraper_config.json in a location accessible by your n8n instance (e.g., in the n8n data directory or a specified path).
   • This file should contain the initial configuration for your Bright Data scraper, including the targetUrl for the recipes.
   • Example content:

     {
       "targetUrl": "https://example.com/recipes",
       "scraperId": "YOUR_BRIGHT_DATA_SCRAPER_ID"
     }
     

     Replace https://example.com/recipes with your actual target recipe URL and YOUR_BRIGHT_DATA_SCRAPER_ID with your Bright Data Scraper ID.
4. Adjust Node Parameters:
   • Read/Write Files from Disk: Ensure the "File Path" in this node points to the correct location of your bright_data_scraper_config.json file.
   • HTTP Request (Bright Data): Verify the "URL" and "Authentication" settings are correct for your Bright Data Web Scraper IDE API endpoint.
   • Loop Over Items (Split in Batches): Adjust the "Batch Size" and "Delay Between Batches" as needed to optimize performance and respect API rate limits.
   • OpenAI Chat Model: Confirm that the "Model" is set to gpt-4o-mini or your preferred OpenAI chat model.
5. Execute the Workflow: Click "Execute Workflow" in the n8n editor to run the workflow manually and generate recipe recommendations.