AI-powered web research in Google Sheets with GPT and Bright Data
π AI-Powered Web Research in Google Sheets with Bright Data
π Overview
Transform any Google Sheets cell into an intelligent web scraper! Type =BRIGHTDATA("cell", "search prompt") and get AI-filtered result from every website in ~20 seconds.
What happens automatically:
- AI optimizes your search query
- Bright Data scrapes the web (bypasses bot detection)
- AI analyzes and filters result
- Returns clean data directly to your cell
- Completes in <25 seconds
Cost: ~$0.02-0.05 per search | Time saved: 3-5 minutes per search
π₯ Who's it for
- Market researchers needing competitive intelligence
- E-commerce teams tracking prices
- Sales teams doing lead prospecting
- SEO specialists gathering content research
- Real estate agents monitoring listings
- Anyone tired of manual copy-paste
βοΈ How it works
- Webhook Call - Google Sheets function sends POST request
- Data Preparation - Organizes input structure
- AI Query Optimization - GPT-4.1 Mini refines search query
- Web Scraping - Bright Data fetches data while bypassing blocks
- AI Analysis - GPT-4o Mini filters and summarizes result
- Response - Returns plain text to your cell
- Logging - Updates logs for monitoring
π οΈ Setup Instructions
Time to deploy: 20 minutes
Requirements
- n8n instance with public URL
- Bright Data account + API key
- OpenAI API key
- Google account for Apps Script
Part 1: n8n Workflow Setup
- Import this template into your n8n instance
- Configure Webhook node:
- Copy your webhook URL:
https://n8n.yourdomain.com/webhook/brightdata-search - Set authentication: Header Auth
- Set API key:
12312346(or create your own)
- Copy your webhook URL:
- Add OpenAI credentials to AI nodes.
- Configure Bright Data: Add API credentials
- Configure Output Language: Manually edit the "Set Variables" Node.
- Test workflow with manual execution
- Activate the workflow
Part 2: Google Sheets Function
- Open Google Sheet β Extensions β Apps Script
- Paste this code:
function BRIGHTDATA(prompt, source) {
if (!prompt || prompt === "") {
return "β Must enter prompt";
}
source = source || "google";
// Update with YOUR webhook URL
const N8N_WEBHOOK_URL = "https://your-n8n-domain.com/webhook/brightdata-search";
// Update with YOUR password
const API_KEY = "12312346";
let spreadsheetId, sheetName, cellAddress;
try {
const sheet = SpreadsheetApp.getActiveSheet();
const activeCell = sheet.getActiveCell();
spreadsheetId = SpreadsheetApp.getActiveSpreadsheet().getId();
sheetName = sheet.getName();
cellAddress = activeCell.getA1Notation();
} catch (e) {
return "β Cannot identify cell";
}
const payload = {
prompt: prompt,
source: source.toLowerCase(),
context: {
spreadsheetId: spreadsheetId,
sheetName: sheetName,
cellAddress: cellAddress,
timestamp: new Date().toISOString()
}
};
const options = {
method: "post",
contentType: "application/json",
payload: JSON.stringify(payload),
muteHttpExceptions: true,
headers: {
"Accept": "text/plain",
"key": API_KEY
}
};
try {
const response = UrlFetchApp.fetch(N8N_WEBHOOK_URL, options);
const responseCode = response.getResponseCode();
if (responseCode !== 200) {
Logger.log("Error response: " + response.getContentText());
return "β Error " + responseCode;
}
return response.getContentText();
} catch (error) {
Logger.log("Exception: " + error.toString());
return "β Connection error: " + error.toString();
}
}
function doGet(e) {
return ContentService.createTextOutput(JSON.stringify({
status: "alive",
message: "Apps Script is running",
timestamp: new Date().toISOString()
})).setMimeType(ContentService.MimeType.JSON);
}
- Update
N8N_WEBHOOK_URLwith your webhook - Update
API_KEYwith your password - Save (Ctrl+S / Cmd+S) - Important!
- Close Apps Script editor
π‘ Usage Examples
=BRIGHTDATA("C3", "What is the current price of the product?")
=BRIGHTDATA("D30", "What is the size of this company?")
=BRIGHTDATA("A4", "Do this comapny is hiring Developers?")
π¨ Customization
Easy Tweaks
- AI Models - Switch to GPT-4o for better optimization
- Response Format - Modify prompt for specific outputs
- Speed - Optimize AI prompts to reduce time
- Language - Change prompts for any language
Advanced Options
- Implement rate limiting
- Add data validation
- Create async mode for long queries
- Add Slack notifications
π Pro Tips
- Be Specific - "What is iPhone 15 Pro 256GB US price?" beats "What is iPhone price?"
- Speed Matters - Keep prompts concise (30s timeout limit)
- Monitor Costs - Track Bright Data usage
- Debug - Check workflow logs for errors
β οΈ Important Notes
- Timeout: 30-second Google Sheets limit (aim for <20s)
- Plain Text Only: No JSON responses
- Costs: Monitor Bright Data at console.brightdata.com
- Security: Keep API keys secret
- No Browser Storage: Don't use localStorage/sessionStorage
π§ Troubleshooting
| Error | Solution | |-------|----------| | "Exceeded maximum execution time" | Optimize AI prompts or use async mode | | "Could not fetch data" | Verify Bright Data credentials | | Empty cell | Check n8n logs for AI parsing issues | | Broken characters | Verify UTF-8 encoding in webhook node |
π Resources
Built with β€οΈ by Elay Guez
AI-Powered Web Research in Google Sheets with GPT and Bright Data
This n8n workflow automates web research by leveraging AI agents (GPT) and web scraping tools (Bright Data) to extract information and organize it within Google Sheets. It's designed to streamline the process of gathering specific data from the web based on prompts.
What it does
This workflow acts as an AI-powered research assistant, performing the following steps:
- Triggers on demand: The workflow is initiated via a webhook, allowing it to be triggered by external systems or manually.
- Prepares Input: It takes the incoming data (likely a research query or topic) and structures it for the AI agent.
- AI Agent Orchestration: An AI Agent (powered by LangChain) takes the prepared input and orchestrates a research task.
- Web Scraping with Bright Data (MCP Client Tool): The AI Agent utilizes a "MCP Client Tool" (likely integrated with Bright Data or a similar web scraping service) to perform web searches and data extraction based on its research plan.
- Processes AI Output: The raw output from the AI Agent is then parsed and structured using a "Structured Output Parser" to ensure consistency and usability.
- Generates LLM Chain Response: A "Basic LLM Chain" (powered by an OpenAI Chat Model) further processes the scraped and parsed data, potentially summarizing, analyzing, or reformatting it according to the research objective.
- Responds to Webhook: The final, processed research results are sent back as a response to the initial webhook trigger.
- Data Table (Optional/Informational): A "Data table" node is present, likely for displaying intermediate or final research data within the n8n interface for debugging or review purposes.
- HTTP Request (Optional/Informational): An "HTTP Request" node is included, which could be used for advanced scenarios like calling another API or integrating with a custom service, though its specific configuration is not detailed in the provided JSON.
Prerequisites/Requirements
To use this workflow, you will need:
- n8n Instance: A running n8n instance.
- OpenAI API Key: For the "OpenAI Chat Model" to function, you'll need an API key from OpenAI.
- Bright Data (or similar web scraping service): The "MCP Client Tool" suggests integration with a web scraping service like Bright Data. You'll need an account and credentials for such a service.
- Google Sheets Account: While not explicitly shown in the data flow, the directory name "ai-powered-web-research-in-google-sheets-with-gpt-and-bright-data" implies a Google Sheets integration for storing the research results. You would typically add a Google Sheets node after the "Respond to Webhook" node to write data.
Setup/Usage
- Import the Workflow: Download the provided JSON and import it into your n8n instance.
- Configure Credentials:
- Set up your OpenAI API Key credential within n8n.
- Configure credentials for your Bright Data (or equivalent) service to be used by the "MCP Client Tool".
- Configure Webhook: The "Webhook" node will provide a unique URL. This is the endpoint you'll use to trigger the workflow.
- Customize AI Agent and LLM Chain:
- Review the configuration of the "AI Agent" and "Basic LLM Chain" nodes. You might need to adjust prompts, models, or other parameters to suit your specific research needs.
- The "Edit Fields (Set)" node allows you to define the input structure for your research queries.
- Add Google Sheets Integration (if desired): To save the research results to Google Sheets, add a "Google Sheets" node after the "Respond to Webhook" node. Configure it to append the processed data to your desired spreadsheet.
- Activate the Workflow: Once configured, activate the workflow in n8n.
- Trigger the Workflow: Send an HTTP POST request to the Webhook URL with your research query in the request body. The workflow will then execute, perform the research, and respond with the results.
Related Templates
Track competitor SEO keywords with Decodo + GPT-4.1-mini + Google Sheets
This workflow automates competitor keyword research using OpenAI LLM and Decodo for intelligent web scraping. Who this is for SEO specialists, content strategists, and growth marketers who want to automate keyword research and competitive intelligence. Marketing analysts managing multiple clients or websites who need consistent SEO tracking without manual data pulls. Agencies or automation engineers using Google Sheets as an SEO data dashboard for keyword monitoring and reporting. What problem this workflow solves Tracking competitor keywords manually is slow and inconsistent. Most SEO tools provide limited API access or lack contextual keyword analysis. This workflow solves that by: Automatically scraping any competitorβs webpage with Decodo. Using OpenAI GPT-4.1-mini to interpret keyword intent, density, and semantic focus. Storing structured keyword insights directly in Google Sheets for ongoing tracking and trend analysis. What this workflow does Trigger β Manually start the workflow or schedule it to run periodically. Input Setup β Define the website URL and target country (e.g., https://dev.to, france). Data Scraping (Decodo) β Fetch competitor web content and metadata. Keyword Analysis (OpenAI GPT-4.1-mini) Extract primary and secondary keywords. Identify focus topics and semantic entities. Generate a keyword density summary and SEO strength score. Recommend optimization and internal linking opportunities. Data Structuring β Clean and convert GPT output into JSON format. Data Storage (Google Sheets) β Append structured keyword data to a Google Sheet for long-term tracking. Setup Prerequisites If you are new to Decode, please signup on this link visit.decodo.com n8n account with workflow editor access Decodo API credentials OpenAI API key Google Sheets account connected via OAuth2 Make sure to install the Decodo Community node. Create a Google Sheet Add columns for: primarykeywords, seostrengthscore, keyworddensity_summary, etc. Share with your n8n Google account. Connect Credentials Add credentials for: Decodo API credentials - You need to register, login and obtain the Basic Authentication Token via Decodo Dashboard OpenAI API (for GPT-4o-mini) Google Sheets OAuth2 Configure Input Fields Edit the βSet Input Fieldsβ node to set your target site and region. Run the Workflow Click Execute Workflow in n8n. View structured results in your connected Google Sheet. How to customize this workflow Track Multiple Competitors β Use a Google Sheet or CSV list of URLs; loop through them using the Split In Batches node. Add Language Detection β Add a Gemini or GPT node before keyword analysis to detect content language and adjust prompts. Enhance the SEO Report β Expand the GPT prompt to include backlink insights, metadata optimization, or readability checks. Integrate Visualization β Connect your Google Sheet to Looker Studio for SEO performance dashboards. Schedule Auto-Runs β Use the Cron Node to run weekly or monthly for competitor keyword refreshes. Summary This workflow automates competitor keyword research using: Decodo for intelligent web scraping OpenAI GPT-4.1-mini for keyword and SEO analysis Google Sheets for live tracking and reporting Itβs a complete AI-powered SEO intelligence pipeline ideal for teams that want actionable insights on keyword gaps, optimization opportunities, and content focus trends, without relying on expensive SEO SaaS tools.
Generate song lyrics and music from text prompts using OpenAI and Fal.ai Minimax
Spark your creativity instantly in any chatβturn a simple prompt like "heartbreak ballad" into original, full-length lyrics and a professional AI-generated music track, all without leaving your conversation. π What This Template Does This chat-triggered workflow harnesses AI to generate detailed, genre-matched song lyrics (at least 600 characters) from user messages, then queues them for music synthesis via Fal.ai's minimax-music model. It polls asynchronously until the track is ready, delivering lyrics and audio URL back in chat. Crafts original, structured lyrics with verses, choruses, and bridges using OpenAI Submits to Fal.ai for melody, instrumentation, and vocals aligned to the style Handles long-running generations with smart looping and status checks Returns complete song package (lyrics + audio link) for seamless sharing π§ Prerequisites n8n account (self-hosted or cloud with chat integration enabled) OpenAI account with API access for GPT models Fal.ai account for AI music generation π Required Credentials OpenAI API Setup Go to platform.openai.com β API keys (sidebar) Click "Create new secret key" β Name it (e.g., "n8n Songwriter") Copy the key and add to n8n as "OpenAI API" credential type Test by sending a simple chat completion request Fal.ai HTTP Header Auth Setup Sign up at fal.ai β Dashboard β API Keys Generate a new API key β Copy it In n8n, create "HTTP Header Auth" credential: Name="Fal.ai", Header Name="Authorization", Header Value="Key [Your API Key]" Test with a simple GET to their queue endpoint (e.g., /status) βοΈ Configuration Steps Import the workflow JSON into your n8n instance Assign OpenAI API credentials to the "OpenAI Chat Model" node Assign Fal.ai HTTP Header Auth to the "Generate Music Track", "Check Generation Status", and "Fetch Final Result" nodes Activate the workflowβchat trigger will appear in your n8n chat interface Test by messaging: "Create an upbeat pop song about road trips" π― Use Cases Content Creators: YouTubers generating custom jingles for videos on the fly, streamlining production from idea to audio export Educators: Music teachers using chat prompts to create era-specific folk tunes for classroom discussions, fostering interactive learning Gift Personalization: Friends crafting anniversary R&B tracks from shared memories via quick chats, delivering emotional audio surprises Artist Brainstorming: Songwriters prototyping hip-hop beats in real-time during sessions, accelerating collaboration and iteration β οΈ Troubleshooting Invalid JSON from AI Agent: Ensure the system prompt stresses valid JSON; test the agent standalone with a sample query Music Generation Fails (401/403): Verify Fal.ai API key has minimax-music access; check usage quotas in dashboard Status Polling Loops Indefinitely: Bump wait time to 45-60s for complex tracks; inspect fal.ai queue logs for bottlenecks Lyrics Under 600 Characters: Tweak agent prompt to enforce fuller structures like [V1][C][V2][B][C]; verify output length in executions
Automate invoice processing with OCR, GPT-4 & Salesforce opportunity creation
PDF Invoice Extractor (AI) End-to-end pipeline: Watch Drive β Download PDF β OCR text β AI normalize to JSON β Upsert Buyer (Account) β Create Opportunity β Map Products β Create OLI via Composite API β Archive to OneDrive. --- Node by node (what it does & key setup) 1) Google Drive Trigger Purpose: Fire when a new file appears in a specific Google Drive folder. Key settings: Event: fileCreated Folder ID: google drive folder id Polling: everyMinute Creds: googleDriveOAuth2Api Output: Metadata { id, name, ... } for the new file. --- 2) Download File From Google Purpose: Get the file binary for processing and archiving. Key settings: Operation: download File ID: ={{ $json.id }} Creds: googleDriveOAuth2Api Output: Binary (default key: data) and original metadata. --- 3) Extract from File Purpose: Extract text from PDF (OCR as needed) for AI parsing. Key settings: Operation: pdf OCR: enable for scanned PDFs (in options) Output: JSON with OCR text at {{ $json.text }}. --- 4) Message a model (AI JSON Extractor) Purpose: Convert OCR text into strict normalized JSON array (invoice schema). Key settings: Node: @n8n/n8n-nodes-langchain.openAi Model: gpt-4.1 (or gpt-4.1-mini) Message role: system (the strict prompt; references {{ $json.text }}) jsonOutput: true Creds: openAiApi Output (per item): $.message.content β the parsed JSON (ensure itβs an array). --- 5) Create or update an account (Salesforce) Purpose: Upsert Buyer as Account using an external ID. Key settings: Resource: account Operation: upsert External Id Field: taxid_c External Id Value: ={{ $json.message.content.buyer.tax_id }} Name: ={{ $json.message.content.buyer.name }} Creds: salesforceOAuth2Api Output: Account record (captures Id) for downstream Opportunity. --- 6) Create an opportunity (Salesforce) Purpose: Create Opportunity linked to the Buyer (Account). Key settings: Resource: opportunity Name: ={{ $('Message a model').item.json.message.content.invoice.code }} Close Date: ={{ $('Message a model').item.json.message.content.invoice.issue_date }} Stage: Closed Won Amount: ={{ $('Message a model').item.json.message.content.summary.grand_total }} AccountId: ={{ $json.id }} (from Upsert Account output) Creds: salesforceOAuth2Api Output: Opportunity Id for OLI creation. --- 7) Build SOQL (Code / JS) Purpose: Collect unique product codes from AI JSON and build a SOQL query for PricebookEntry by Pricebook2Id. Key settings: pricebook2Id (hardcoded in script): e.g., 01sxxxxxxxxxxxxxxx Source lines: $('Message a model').first().json.message.content.products Output: { soql, codes } --- 8) Query PricebookEntries (Salesforce) Purpose: Fetch PricebookEntry.Id for each Product2.ProductCode. Key settings: Resource: search Query: ={{ $json.soql }} Creds: salesforceOAuth2Api Output: Items with Id, Product2.ProductCode (used for mapping). --- 9) Code in JavaScript (Build OLI payloads) Purpose: Join lines with PBE results and Opportunity Id β build OpportunityLineItem payloads. Inputs: OpportunityId: ={{ $('Create an opportunity').first().json.id }} Lines: ={{ $('Message a model').first().json.message.content.products }} PBE rows: from previous node items Output: { body: { allOrNone:false, records:[{ OpportunityLineItem... }] } } Notes: Converts discount_total β per-unit if needed (currently commented for standard pricing). Throws on missing PBE mapping or empty lines. --- 10) Create Opportunity Line Items (HTTP Request) Purpose: Bulk create OLIs via Salesforce Composite API. Key settings: Method: POST URL: https://<your-instance>.my.salesforce.com/services/data/v65.0/composite/sobjects Auth: salesforceOAuth2Api (predefined credential) Body (JSON): ={{ $json.body }} Output: Composite API results (per-record statuses). --- 11) Update File to One Drive Purpose: Archive the original PDF in OneDrive. Key settings: Operation: upload File Name: ={{ $json.name }} Parent Folder ID: onedrive folder id Binary Data: true (from the Download node) Creds: microsoftOneDriveOAuth2Api Output: Uploaded file metadata. --- Data flow (wiring) Google Drive Trigger β Download File From Google Download File From Google β Extract from File β Update File to One Drive Extract from File β Message a model Message a model β Create or update an account Create or update an account β Create an opportunity Create an opportunity β Build SOQL Build SOQL β Query PricebookEntries Query PricebookEntries β Code in JavaScript Code in JavaScript β Create Opportunity Line Items --- Quick setup checklist π Credentials: Connect Google Drive, OneDrive, Salesforce, OpenAI. π IDs: Drive Folder ID (watch) OneDrive Parent Folder ID (archive) Salesforce Pricebook2Id (in the JS SOQL builder) π§ AI Prompt: Use the strict system prompt; jsonOutput = true. π§Ύ Field mappings: Buyer tax id/name β Account upsert fields Invoice code/date/amount β Opportunity fields Product name must equal your Product2.ProductCode in SF. β Test: Drop a sample PDF β verify: AI returns array JSON only Account/Opportunity created OLI records created PDF archived to OneDrive --- Notes & best practices If PDFs are scans, enable OCR in Extract from File. If AI returns non-JSON, keep βReturn only a JSON arrayβ as the last line of the prompt and keep jsonOutput enabled. Consider adding validation on parsing.warnings to gate Salesforce writes. For discounts/taxes in OLI: Standard OLI fields donβt support per-line discount amounts directly; model them in UnitPrice or custom fields. Replace the Composite API URL with your orgβs domain or use the Salesforce nodeβs Bulk Upsert for simplicity.