n8n ScrapingBee and Google Sheets Integration Template

This n8n workflow provides a template for integrating web scraping results from ScrapingBee with Google Sheets. It demonstrates how to fetch data, process it, and conditionally store it based on specific criteria.

What it does

This workflow is designed to:

1. Receive a webhook trigger: It starts by listening for incoming data via a webhook. This webhook is expected to contain a binary file, likely a CSV or similar data export from a scraping process.
2. Decompress the received file: If the incoming binary data is compressed (e.g., a ZIP file), it decompresses it.
3. Extract data from the file: It then extracts structured data (e.g., CSV content) from the decompressed file.
4. Process extracted data with custom code: A Code node is used to manipulate the extracted data. This step is where you would implement specific logic to transform or filter the scraped data.
5. Conditionally filter data: An If node evaluates the processed data based on a defined condition.
6. Write data to Google Sheets (if condition is met): If the condition in the If node evaluates to true, the data is then appended to a specified Google Sheet.

Prerequisites/Requirements

To use this workflow, you will need:

• n8n instance: A running n8n instance.
• Webhook: An external service or application configured to send data to the n8n webhook URL. This data should ideally be a binary file (e.g., a compressed CSV).
• Google Account: A Google account with access to Google Sheets.
• Google Sheets Credential: An n8n credential configured for Google Sheets to allow the workflow to write data.
• ScrapingBee (Optional but implied by directory name): While not directly present in the JSON, the directory name suggests this workflow is intended to work with data from ScrapingBee. You would typically use ScrapingBee to generate the data that your webhook sends.

Setup/Usage

1. Import the workflow:
   • Download the provided JSON workflow definition.
   • In your n8n instance, go to "Workflows" and click "New".
   • Click the three dots next to "New Workflow" and select "Import from JSON".
   • Paste the JSON content or upload the file.
2. Configure the Webhook:
   • Open the "Webhook" node.
   • Copy the "Webhook URL". This is the URL you will use to send data to this workflow from your external application (e.g., your ScrapingBee post-processing script).
3. Configure Google Sheets Credential:
   • Open the "Google Sheets" node.
   • Click "Create New Credential" or select an existing Google Sheets OAuth2 credential.
   • Follow the n8n instructions to authenticate with your Google account and grant necessary permissions.
   • Specify the "Spreadsheet ID" and "Sheet Name" where you want the data to be written.
4. Customize the Compression Node:
   • If your incoming data is compressed, ensure the "Compression" node is configured for the correct "Operation" (e.g., "Decompress") and "File Format" (e.g., "ZIP").
5. Customize the Extract from File Node:
   • Configure this node to correctly parse the format of your extracted data (e.g., "CSV", "JSON").
6. Customize the Code Node:
   • Open the "Code" node and modify the JavaScript code to implement your specific data processing logic. This is where you would transform, clean, or enrich the scraped data before it's stored.
7. Customize the If Node:
   • Open the "If" node and define the condition(s) for filtering your data. For example, you might want to only store items where a certain field meets a specific value.
8. Activate the workflow: Once configured, activate the workflow by toggling the "Active" switch in the top right corner of the n8n editor.

Now, whenever data is sent to the webhook URL, the workflow will execute, process the data, and conditionally add it to your Google Sheet.