AI Agent That Can Scrape Webpages

This n8n workflow demonstrates how to set up an AI agent capable of interacting with a user, making decisions, and potentially calling other n8n workflows as tools. While the workflow's current JSON definition doesn't include a direct web scraping action, it lays the groundwork for an AI agent that can be extended to perform such tasks by integrating a web scraping tool.

What it does

This workflow sets up a basic AI agent interaction loop:

1. Listens for Chat Messages: It starts by waiting for a chat message from a user.
2. Initializes AI Agent: It then passes the received message to an AI Agent node.
3. Configures Language Model: The AI Agent uses an OpenAI Chat Model to process the input and determine the next steps.
4. Defines Tools: It provides the AI Agent with the ability to "Call n8n Workflow Tool". This means the AI can decide to execute another n8n workflow based on its understanding of the user's request.
5. Processes Agent Output: The output from the AI Agent is then processed.
6. Conditional Logic: An "If" node checks a condition (though the specific condition isn't defined in the provided JSON, it would typically check for a specific response or action from the AI).
7. Prepares Output (True Branch): If the condition is true, an "Edit Fields (Set)" node prepares the data.
8. Formats Output (Markdown): The prepared data is then formatted as Markdown.
9. Prepares Output (False Branch): If the condition is false, a different "Edit Fields (Set)" node prepares the data.
10. Formats Output (Markdown): This data is also formatted as Markdown.
11. Placeholder for External Execution: An "Execute Workflow Trigger" node suggests that this workflow might be called by another workflow or could trigger another workflow upon completion.
12. Documentation Note: A "Sticky Note" is included, likely for internal documentation or reminders within the workflow.

Prerequisites/Requirements

• n8n Instance: A running n8n instance.
• OpenAI API Key: Required for the "OpenAI Chat Model" node to function. This credential needs to be configured within n8n.
• Langchain Nodes: Ensure the @n8n/n8n-nodes-langchain package is installed in your n8n instance, as it provides the "AI Agent", "OpenAI Chat Model", and "Call n8n Workflow Tool" nodes.

Setup/Usage

1. Import the Workflow: Import the provided JSON into your n8n instance.
2. Configure Credentials:
   • Locate the "OpenAI Chat Model" node and configure your OpenAI API Key credential.
3. Configure AI Agent:
   • Review the "AI Agent" node settings. You may want to adjust the system message, model, temperature, and other parameters to fine-tune its behavior.
4. Define "Call n8n Workflow Tool":
   • The "Call n8n Workflow Tool" node is currently generic. To enable the AI to perform specific actions like web scraping, you would need to:
     • Create a separate n8n workflow dedicated to web scraping.
     • Configure the "Call n8n Workflow Tool" node to point to this web scraping workflow, providing a clear description of what the tool does so the AI can understand when to use it.
5. Configure Conditional Logic:
   • Adjust the "If" node's condition to match your desired branching logic based on the AI agent's output.
6. Activate the Workflow: Once configured, activate the workflow.
7. Test: Send a chat message to the "Manual Chat Trigger" to test the AI agent's responses and tool usage.