Prevent Duplicate Processing with Redis Item State Tracking

This n8n workflow demonstrates a robust method to prevent duplicate processing of items, particularly useful for scenarios where data sources might send the same item multiple times or when retries could lead to redundant operations. It leverages Redis to track the state of each item, ensuring that an item is processed only once.

What it does

This workflow is designed to:

1. Receive a Trigger: It starts by being triggered, likely from another workflow or an external system that initiates processing for an item.
2. Prepare Item ID: It extracts or generates a unique identifier for the item to be processed.
3. Check Redis for Existing Item: It queries Redis to see if this item ID has been processed before.
4. Route Based on State:
   • If the item ID is found in Redis, indicating it has already been processed, the workflow stops with an error to prevent re-processing.
   • If the item ID is not found in Redis, it proceeds to the processing logic.
5. Process the Item: In the example, it simulates processing by interacting with Google Sheets (e.g., adding a new row). This is where your actual business logic would reside.
6. Mark Item as Processed in Redis: After successful processing, it adds the item ID to Redis, marking it as processed to prevent future duplicates.

Prerequisites/Requirements

• n8n Instance: A running n8n instance.
• Redis Instance: Access to a Redis server. You'll need to configure a Redis credential in n8n.
• Google Sheets Account: Access to a Google Sheets spreadsheet. You'll need to configure a Google Sheets credential in n8n.
• Input Data: This workflow expects to be triggered by another workflow, which should provide the necessary item data to be processed.

Setup/Usage

1. Import the Workflow:
   • Copy the provided JSON code.
   • In your n8n instance, go to "Workflows" and click "New".
   • Click the three-dot menu in the top right and select "Import from JSON".
   • Paste the JSON code and click "Import".
2. Configure Credentials:
   • Redis: Locate the "Redis Trigger" and "Redis" nodes. Configure your Redis credential.
   • Google Sheets: Locate the "Google Sheets" node. Configure your Google Sheets credential.
3. Customize Item ID Generation:
   • The "Edit Fields" node (ID: 38) is currently a placeholder. You will need to modify this node to extract or generate the unique item_id from the incoming data that triggers this workflow. For example, if your trigger provides an id field, you might set a new field item_id to {{ $json.id }}.
4. Customize Processing Logic:
   • The "Google Sheets" node (ID: 18) is an example of an action to perform. Replace or modify this node with your actual processing logic (e.g., calling an API, sending an email, updating a database).
5. Activate the Workflow:
   • Once configured, activate the workflow by toggling the "Active" switch in the top right corner of the workflow editor.
6. Trigger the Workflow:
   • This workflow is designed to be executed by another workflow. You would typically use an "Execute Workflow" node in a parent workflow to call this workflow, passing the item data as input.

This setup ensures that even if the parent workflow or an external system attempts to process the same item multiple times, this workflow will only execute the core processing logic once, thanks to the Redis state tracking.