Extract competitor SERP rankings from Google Search to Sheets with Bright Data
🔍 Extract Competitor SERP Rankings from Google Search to Sheets with Bright Data
This template requires a self-hosted n8n instance to run.
A comprehensive n8n automation that extracts competitor data from Google search results for specific keywords and target countries, automatically saving structured data to Google Sheets for competitive analysis and market research.
📋 Overview
This workflow provides a professional competitor analysis solution that identifies ranking websites for specific search terms across different countries. Perfect for SEO research, competitive intelligence, market analysis, and content strategy planning. The system uses Bright Data's SERP API for accurate search result extraction and advanced HTML parsing for detailed competitor information.
Who is this for?
- SEO professionals conducting competitive analysis
- Digital marketers researching market landscapes
- Business analysts studying competitor positioning
- Content strategists analyzing competitor content approaches
- Market researchers tracking competitive intelligence across regions
What problem is this workflow solving?
- Extracting competitor data from Google search results
- Processing multiple keywords across different countries
- Organizing results in a structured, analyzable format
- Eliminating manual copy-paste work
- Ensuring consistent data collection methodology
What this workflow does
- Manual Trigger: Starts the workflow execution
- Get Keywords from Sheet: Fetches keywords and target countries from Google Sheets
- URL Encode Keywords: Converts keywords to URL-safe format
- Process Keywords in Batches: Handles multiple keywords sequentially
- Fetch Google Search Results: Uses Bright Data SERP API to scrape HTML
- Extract Competitor Data from HTML: Parses HTML to extract competitor details
- Save Competitor Results to Sheet: Stores structured data in Google Sheets
- Wait to Avoid Rate Limits: Implements 30-second delays between requests
Output Data Points
| Field | Description | Example | | :--------------- | :--------------------------------- | :------------------------------------------ | | Keyword | Original search term | digital marketing services | | Target Country | Geographic target | US | | websiteName | Domain/company name | hubspot | | websiteUrl | Complete website URL | https://www.hubspot.com/marketing | | websiteTitle | Page title from search results | Digital Marketing Software & Tools | | websiteDescription | Meta description/snippet | Grow your business with HubSpot's digital marketing tools... |
⚙️ Setup
Prerequisites
- n8n instance (self-hosted)
- Google account with Sheets access
- Bright Data account with SERP API access
Google Sheet Structure
This workflow utilizes two Google Sheets: one for input keywords and one for outputting competitor data.
Input Sheet: "Keywords"
This sheet should contain the keywords and target countries for your search queries.
| Column Header | Data Type | Description | Example | | :------------- | :-------- | :------------------------------------------------- | :-------------- | | Keyword | Text | The search term you want to analyze. | digital marketing | | Country | Text | The 2-letter ISO country code for the target region of the search (e.g., US, GB, DE). | US |
Output Sheet: "Competitor Results"
This sheet will be populated automatically by the workflow with the extracted competitor data.
| Column Header | Data Type | Description | Example | | :----------------- | :-------- | :---------------------------------------------------------------------------------- | :----------------------------------------------- | | Keyword | Text | The original search term used for the query. | digital marketing services | | Target Country | Text | The 2-letter ISO country code of the search results. | US | | websiteName | Text | The name of the website or domain found in the search results. | hubspot | | websiteUrl | URL | The full URL of the website or page found in the search results. | https://www.hubspot.com/marketing | | websiteTitle | Text | The title of the page as displayed in the Google search results. | Digital Marketing Software & Tools | | websiteDescription | Text | The meta description or snippet text displayed under the title in search results. | Grow your business with HubSpot's digital marketing tools... |
Step-by-Step Setup
- Import the Workflow: Copy JSON → n8n → Workflows → + Add → Import from JSON
- Configure Bright Data Credentials:
- Credential Type: HTTP Header Auth
- Header Name: Authorization
- Header Value:
Bearer YOUR_API_TOKEN
- Configure Google Sheets:
- Create two new Google Sheets as described above: one named "Keywords" (for input) and one named "Competitor Results" (for output).
- Set up Google Sheets OAuth2 credentials within n8n.
- Update Workflow Settings:
- Replace placeholders:
YOUR_GOOGLE_SHEET_ID(for both input and output sheets),YOUR_BRIGHTDATA_CREDENTIAL_ID. - Ensure correct sheet/tab names are selected in the Google Sheets nodes.
- Replace placeholders:
- Test & Activate: Add test data to your "Keywords" sheet → Execute workflow → Verify output in your "Competitor Results" sheet.
🛠 How to Customize
- Add More Data Points: Modify the JavaScript code in the "Extract Competitor Data from HTML" node to parse and extract additional information from the HTML.
- Custom Filtering: Implement logic to exclude specific domains, filter results by title length, or other criteria.
- Expand Geographic Coverage: Add more 2-letter ISO country codes to the Bright Data SERP API call to broaden your competitive analysis.
- Batch Processing: Adjust the settings in the "Process Keywords in Batches" node to optimize for your Bright Data plan and desired execution speed.
- Rate Limiting: Modify the "Wait" node (default: 30 seconds) to increase or decrease the delay between requests based on API limits or performance needs.
📊 Use Cases & Examples
- SEO Competitive Analysis: Identify top-ranking competitors for your target keywords and analyze their strategies.
- Market Entry Research: Understand the competitive landscape in new geographic regions before expanding.
- Content Strategy Planning: Analyze competitor page titles and meta descriptions for inspiration and to identify content gaps.
- International Market Research: Compare search engine results and competitor positioning across different countries.
📈 Performance & Limits
- Single Keyword: 30–60 seconds per keyword.
- Batch of 10 Keywords: Typically takes 5–10 minutes.
- Large Lists (50+ Keywords): Expect execution times of 30–60 minutes or more, depending on batching and rate limits.
- Success Rate: Generally 95%+ for data extraction.
- Data Accuracy: Typically 98%+ for extracted fields.
- API Calls: 1 Bright Data SERP API call per keyword, plus multiple Google Sheets writes per execution.
- Rate Limit: A 30-second delay between requests is recommended to prevent exceeding API limits.
🧰 Troubleshooting
- Bright Data API error: Double-check your API token, ensure you have sufficient credits, and confirm SERP API access is enabled on your Bright Data account.
- No keywords found: Verify the Google Sheet ID and ensure the column headers in your "Keywords" sheet precisely match the specifications (e.g., "Keyword", "Country").
- Google Sheets permission denied: Re-authenticate your Google Sheets credentials within n8n and check that the correct sharing settings are applied to your sheets.
- No results extracted: Review the JavaScript parsing logic in the "Extract Competitor Data from HTML" node. Also, verify the validity of your keywords and target countries.
- Loop not processing all: Check the batch settings in the "Process Keywords in Batches" node and ensure all connections within the loop are correctly configured.
🤝 Support & Community
- n8n Forum: <https://community.n8n.io>
- n8n Docs: <https://docs.n8n.io>
- Bright Data Support: Access support directly via your Bright Data dashboard.
- GitHub Issues: Report any bugs or suggest new features on the n8n GitHub repository.
🎯 Final Notes
This workflow provides a comprehensive foundation for competitor research and market analysis. Customize it to fit your specific industry needs and competitive intelligence requirements.
Please note that this template uses Community Nodes. Ensure you understand the risks before using community nodes.
Extract Competitor SERP Rankings to Google Sheets
This n8n workflow simplifies the process of extracting competitor search engine results page (SERP) rankings from Google Search and saving them into a Google Sheet. It's designed to help SEO professionals and marketers monitor keyword performance and competitive landscapes efficiently.
What it does
This workflow performs the following key steps:
- Manual Trigger: The workflow is initiated manually, allowing you to control when the data extraction process begins.
- Google Sheets (Input): It reads a list of keywords and competitor domains from a specified Google Sheet. This sheet acts as the input for the SERP scraping process.
- Loop Over Items (Split in Batches): The workflow processes the keywords and competitor domains in batches, ensuring efficient handling of large datasets and preventing API rate limits.
- HTTP Request (Bright Data): For each keyword and competitor, it makes an HTTP request to a SERP API (likely Bright Data, given the directory name, though the JSON doesn't explicitly confirm the provider beyond "HTTP Request"). This request fetches the Google Search results for the specified keyword.
- Code (Process SERP Data): A Code node processes the raw SERP data received from the HTTP request. It extracts relevant information such as ranking positions for the competitor domains.
- Google Sheets (Output): Finally, the processed ranking data is written back to a Google Sheet, providing a structured overview of competitor SERP performance.
Prerequisites/Requirements
To use this workflow, you will need:
- n8n Instance: A running n8n instance.
- Google Sheets Account: A Google account with access to Google Sheets. You'll need to configure Google Sheets credentials in n8n.
- SERP API Access: Access to a SERP API (e.g., Bright Data, SerpApi, etc.). You'll need an API key and endpoint for the HTTP Request node.
Setup/Usage
- Import the workflow: Import the provided JSON workflow into your n8n instance.
- Configure Google Sheets Credentials:
- In the first "Google Sheets" node (input), select or create your Google Sheets credentials.
- Specify the Spreadsheet ID and Sheet Name where your keywords and competitor domains are listed.
- In the second "Google Sheets" node (output), select or create your Google Sheets credentials.
- Specify the Spreadsheet ID and Sheet Name where you want to write the extracted SERP data.
- Configure HTTP Request Node:
- In the "HTTP Request" node, update the URL to your chosen SERP API endpoint.
- Configure the Headers and/or Query Parameters to include your API key and any other necessary parameters for the SERP API (e.g., keyword, target domain, country, language).
- Adjust the Request Body if your SERP API requires a POST request with specific payload.
- Review and Adjust Code Node:
- Open the "Code" node. The current code is a placeholder. You will need to write JavaScript logic to parse the specific JSON response from your chosen SERP API and extract the ranking information for your competitor domains.
- Ensure the output of the Code node is in a format suitable for writing to Google Sheets.
- Save and Execute: Save the workflow. Click "Execute Workflow" to run it manually and start extracting data.
Related Templates
Two-way property repair management system with Google Sheets & Drive
This workflow automates the repair request process between tenants and building managers, keeping all updates organized in a single spreadsheet. It is composed of two coordinated workflows, as two separate triggers are required — one for new repair submissions and another for repair updates. A Unique Unit ID that corresponds to individual units is attributed to each request, and timestamps are used to coordinate repair updates with specific requests. General use cases include: Property managers who manage multiple buildings or units. Building owners looking to centralize tenant repair communication. Automation builders who want to learn multi-trigger workflow design in n8n. --- ⚙️ How It Works Workflow 1 – New Repair Requests Behind the Scenes: A tenant fills out a Google Form (“Repair Request Form”), which automatically adds a new row to a linked Google Sheet. Steps: Trigger: Google Sheets rowAdded – runs when a new form entry appears. Extract & Format: Collects all relevant form data (address, unit, urgency, contacts). Generate Unit ID: Creates a standardized identifier (e.g., BUILDING-UNIT) for tracking. Email Notification: Sends the building manager a formatted email summarizing the repair details and including a link to a Repair Update Form (which activates Workflow 2). --- Workflow 2 – Repair Updates Behind the Scenes:\ Triggered when the building manager submits a follow-up form (“Repair Update Form”). Steps: Lookup by UUID: Uses the Unit ID from Workflow 1 to find the existing row in the Google Sheet. Conditional Logic: If photos are uploaded: Saves each image to a Google Drive folder, renames files consistently, and adds URLs to the sheet. If no photos: Skips the upload step and processes textual updates only. Merge & Update: Combines new data with existing repair info in the same spreadsheet row — enabling a full repair history in one place. --- 🧩 Requirements Google Account (for Forms, Sheets, and Drive) Gmail/email node connected for sending notifications n8n credentials configured for Google API access --- ⚡ Setup Instructions (see more detail in workflow) Import both workflows into n8n, then copy one into a second workflow. Change manual trigger in workflow 2 to a n8n Form node. Connect Google credentials to all nodes. Update spreadsheet and folder IDs in the corresponding nodes. Customize email text, sender name, and form links for your organization. Test each workflow with a sample repair request and a repair update submission. --- 🛠️ Customization Ideas Add Slack or Telegram notifications for urgent repairs. Auto-create folders per building or unit for photo uploads. Generate monthly repair summaries using Google Sheets triggers. Add an AI node to create summaries/extract relevant repair data from repair request that include long submissions.
By Matt@VeraisonLabsDynamic Hubspot lead routing with GPT-4 and Airtable sales team distribution
AI Agent for Dynamic Lead Distribution (HubSpot + Airtable) 🧠 AI-Powered Lead Routing and Sales Team Distribution This intelligent n8n workflow automates end-to-end lead qualification and allocation by integrating HubSpot, Airtable, OpenAI, Gmail, and Slack. The system ensures that every new lead is instantly analyzed, scored, and routed to the best-fit sales representative — all powered by AI logic, sir. --- 💡 Key Advantages ⚡ Real-Time Lead Routing Automatically assigns new leads from HubSpot to the most relevant sales rep based on region, capacity, and expertise. 🧠 AI Qualification Engine An OpenAI-powered Agent evaluates the lead’s industry, region, and needs to generate a persona summary and routing rationale. 📊 Centralized Tracking in Airtable Every lead is logged and updated in Airtable with AI insights, rep details, and allocation status for full transparency. 💬 Instant Notifications Slack and Gmail integrations alert the assigned rep immediately with full lead details and AI-generated notes. 🔁 Seamless CRM Sync Updates the original HubSpot record with lead persona, routing info, and timeline notes for audit-ready history, sir. --- ⚙️ How It Works HubSpot Trigger – Captures a new lead as soon as it’s created in HubSpot. Fetch Contact Data – Retrieves all relevant fields like name, company, and industry. Clean & Format Data – A Code node standardizes and structures the data for consistency. Airtable Record Creation – Logs the lead data into the “Leads” table for centralized tracking. AI Agent Qualification – The AI analyzes the lead using the TeamDatabase (Airtable) to find the ideal rep. Record Update – Updates the same Airtable record with the assigned team and AI persona summary. Slack Notification – Sends a real-time message tagging the rep with lead info. Gmail Notification – Sends a personalized handoff email with context and follow-up actions. HubSpot Sync – Updates the original contact in HubSpot with the assignment details and AI rationale, sir. --- 🛠️ Setup Steps Trigger Node: HubSpot → Detect new leads. HubSpot Node: Retrieve complete lead details. Code Node: Clean and normalize data. Airtable Node: Log lead info in the “Leads” table. AI Agent Node: Process lead and match with sales team. Slack Node: Notify the designated representative. Gmail Node: Email the rep with details. HubSpot Node: Update CRM with AI summary and allocation status, sir. --- 🔐 Credentials Required HubSpot OAuth2 API – To fetch and update leads. Airtable Personal Access Token – To store and update lead data. OpenAI API – To power the AI qualification and matching logic. Slack OAuth2 – For sending team notifications. Gmail OAuth2 – For automatic email alerts to assigned reps, sir. --- 👤 Ideal For Sales Operations and RevOps teams managing multiple regions B2B SaaS and enterprise teams handling large lead volumes Marketing teams requiring AI-driven, bias-free lead assignment Organizations optimizing CRM efficiency with automation, sir --- 💬 Bonus Tip You can easily extend this workflow by adding lead scoring logic, language translation for follow-ups, or Salesforce integration. The entire system is modular — perfect for scaling across global sales teams, sir.
By MANISH KUMARTrack daily moods with AI analysis & reports using GPT-4o, Data Tables & Gmail
Track your daily mood in one tap and receive automated AI summaries of your emotional trends every week and month. Perfect for self-reflection, wellness tracking, or personal analytics. This workflow logs moods sent through a webhook (/mood) into Data Tables, analyzes them weekly and monthly with OpenAI (GPT-4o), and emails you clear summaries and actionable recommendations via Gmail. ⚙️ How It Works Webhook – Mood → Collects new entries (🙂, 😐, or 😩) plus an optional note. Set Mood Data → Adds date, hour, and note fields automatically. Insert Mood Row → Stores each record in a Data Table. Weekly Schedule (Sunday 20:00) → Aggregates the last 7 days and sends a summarized report. Monthly Schedule (Day 1 at 08:00) → Aggregates the last 30 days for a deeper AI analysis. OpenAI Analysis → Generates insights, patterns, and 3 actionable recommendations. Gmail → Sends the full report (chart + AI text) to your inbox. 📊 Example Auto-Email Weekly Mood Summary (last 7 days) 🙂 5 ██████████ 😐 2 ████ 😩 0 Average: 1.7 (Positive 🙂) AI Insights: You’re trending upward this week — notes show that exercise days improved mood. Try keeping short walks mid-week to stabilize energy. 🧩 Requirements n8n Data Tables enabled OpenAI credential (GPT-4o or GPT-4 Turbo) Gmail OAuth2 credential to send summaries 🔧 Setup Instructions Connect your credentials: Add your own OpenAI and Gmail OAuth2 credentials. Set your Data Table ID: Open the Insert Mood Row node and enter your own Data Table ID. Without this, new moods won’t be stored. Replace the email placeholder: In the Gmail nodes, replace your.email@example.com with your actual address. Deploy and run: Send a test POST request to /mood (e.g. { "mood": "🙂", "note": "productive day" }) to log your first entry. ⚠️ Before activating the workflow, ensure you have configured the Data Table ID in the “Insert Mood Row” node. 🧠 AI Analysis Interprets mood patterns using GPT-4o. Highlights trends, potential triggers, and suggests 3 specific actions. Runs automatically every week and month. 🔒 Security No personal data is exposed outside your n8n instance. Always remove or anonymize credential references before sharing publicly. 💡 Ideal For Personal mood journaling and AI feedback Therapists tracking client progress Productivity or self-quantification projects 🗒️ Sticky Notes Guide 🟡 Mood Logging Webhook POST /mood receives mood + optional note. ⚠️ Configure your own Data Table ID in the “Insert Mood Row” node before running. 🟢 Weekly Summary Runs every Sunday 20:00 → aggregates last 7 days → generates AI insights + emails report. 🔵 Monthly Summary Runs on Day 1 at 08:00 → aggregates last 30 days → creates monthly reflection. 🟣 AI Analysis Uses OpenAI GPT-4o to interpret trends and recommend actions. 🟠 Email Delivery Sends formatted summaries to your inbox automatically.
By Jose Castillo