AI-Powered Web Scraping with Jina, Google Sheets, and OpenAI

This n8n workflow provides an easy way to perform AI-powered web scraping. It allows you to extract structured information from web pages using Jina AI's reader service, summarize the content with OpenAI, and then store the extracted data in a Google Sheet.

What it does

1. Triggers Manually: The workflow starts when manually executed.
2. Reads URLs from Google Sheets: It fetches a list of URLs from a specified Google Sheet.
3. Scrapes Web Content with Jina AI: For each URL, it uses Jina AI's reader service to fetch the clean, readable content of the web page.
4. Extracts Structured Information with OpenAI: It then uses an OpenAI Chat Model (via the Langchain Information Extractor node) to extract specific, structured information (e.g., title, description, author, publication date) from the scraped content based on a defined schema.
5. Summarizes Content with OpenAI: It also uses another OpenAI Chat Model to generate a concise summary of the web page content.
6. Writes Data to Google Sheets: Finally, it appends the extracted structured data and the generated summary for each URL back into a Google Sheet.

Prerequisites/Requirements

• n8n Account: A running n8n instance.
• Google Sheets Account: To store and retrieve URLs and extracted data.
• Jina AI API Key: For the Jina AI Reader service (used implicitly by the HTTP Request node for web scraping).
• OpenAI API Key: For the OpenAI Chat Model nodes.

Setup/Usage

1. Import the workflow: Download the JSON provided and import it into your n8n instance.
2. Configure Credentials:
   • Google Sheets: Set up your Google Sheets OAuth2 credentials. Ensure the service account or user has read/write access to the target spreadsheet.
   • OpenAI: Set up your OpenAI API Key credentials.
3. Configure Google Sheets (Read):
   • In the first "Google Sheets" node, specify the "Spreadsheet ID" and "Sheet Name" where your URLs are listed.
   • Ensure your URLs are in a column that can be easily referenced (e.g., URL).
4. Configure HTTP Request (Jina AI):
   • The "HTTP Request" node is configured to use Jina AI's reader service. You might need to add your Jina AI API key as a header or modify the URL if Jina's API requires it differently.
   • The URL for Jina AI's reader typically looks like https://r.jina.ai/{your_target_url}.
5. Configure OpenAI Chat Model (Information Extractor):
   • In the "Information Extractor" node, define the JSON schema for the structured data you want to extract (e.g., {"title": "string", "description": "string", "author": "string", "published_date": "string"}).
   • Ensure the "Input Data" points to the content scraped by Jina AI.
6. Configure OpenAI Chat Model (Summary):
   • In the "OpenAI Chat Model" node for summarization, define a clear prompt for generating a summary.
   • Ensure the "Input Data" points to the content scraped by Jina AI.
7. Configure Google Sheets (Write):
   • In the second "Google Sheets" node, specify the "Spreadsheet ID" and "Sheet Name" where you want to write the extracted data.
   • Map the fields from the "Information Extractor" and "OpenAI Chat Model" (summary) nodes to the corresponding columns in your Google Sheet.
8. Execute the workflow: Click "Execute Workflow" to run it manually and process the URLs from your Google Sheet.