n8n AI Agent Tool Call Evaluation Workflow

This n8n workflow demonstrates a basic setup for evaluating whether an AI agent successfully calls a specific tool within its execution. It provides a foundational structure for testing and measuring the performance of AI agents, particularly in scenarios where tool usage is a key metric.

What it does

This workflow is designed to facilitate the evaluation of AI agents by providing a trigger and an evaluation step. While the core logic for the AI agent and its tools is present, the specific evaluation criteria would need to be defined within the "Evaluation" node.

1. Triggers an Evaluation: The workflow starts with an "Evaluation Trigger" node, indicating that it's designed to be used within an n8n evaluation dataset context.
2. Receives Chat Input: A "Chat Trigger" node is present, suggesting the AI agent is intended to respond to chat messages.
3. Initializes AI Agent: An "AI Agent" node is configured to use an OpenAI Chat Model and includes a "Calculator" tool. This agent is the core of the workflow, responsible for processing input and deciding whether to use the available tool.
4. Performs No Operation: A "No Operation, do nothing" node is included, which typically acts as a placeholder or a point where no action is taken, potentially for debugging or future expansion.
5. Edits Fields: An "Edit Fields (Set)" node allows for modification or setting of data, which could be used to prepare data for the AI agent or to process its output.
6. Evaluates Agent Performance: An "Evaluation" node is included to capture and assess the outcome of the AI agent's execution. This is where you would configure specific metrics, such as checking if the "Calculator" tool was invoked.
7. Provides a Sticky Note: A "Sticky Note" is present, likely for documentation or comments within the workflow itself.

Prerequisites/Requirements

• n8n Instance: An active n8n instance (cloud or self-hosted).
• OpenAI API Key: Required for the "OpenAI Chat Model" node to function. This should be configured as an n8n credential.
• LangChain Nodes: Ensure the @n8n/n8n-nodes-langchain package is installed on your n8n instance, as it provides the AI Agent, OpenAI Chat Model, Calculator, and Chat Trigger nodes.

Setup/Usage

1. Import the Workflow: Import the provided JSON into your n8n instance.
2. Configure OpenAI Credentials:
   • Navigate to "Credentials" in n8n.
   • Add a new credential of type "OpenAI API".
   • Enter your OpenAI API Key.
3. Activate the Workflow: Enable the workflow in n8n to start listening for triggers.
4. Configure Evaluation Node:
   • Open the "Evaluation" node.
   • Define the specific metrics and conditions you want to evaluate. For instance, to check if the "Calculator" tool was called, you would typically inspect the output of the "AI Agent" node for evidence of tool invocation (e.g., specific log messages, output format indicating tool use).
5. Run Evaluations: Use the n8n evaluation feature to run datasets through this workflow and collect metrics on the AI agent's tool-calling behavior.