Simulation: Test Your Agent

Simulation lets you run a batch of real customer questions through your agent to see how it performs — before those questions ever reach a live user. Upload a CSV of questions, watch the agent process them, and review detailed results including answer quality, confidence scores, and outcome breakdowns.

Who can use Simulation

Simulation is available to workspace admins on the agent configuration page. Navigate to Agents (via Distribution), select an agent, and switch to the Simulation tab.

Step-by-step: Create and run a simulation

Step 1 — Open the Simulation tab

1. From the left sidebar, click Agents.
2. Click on the agent you want to test. This opens the agent configuration page.
3. At the top of the page, you'll see tabs like Preview and Simulation. Click Simulation.

You'll now see the simulation panel with a list of past simulation runs (if any) and a + Simulation button.

Step 2 — Prepare your CSV file

Create a CSV file with one question per row in a single column. A header row (e.g. question) is optional — it will be auto-detected and skipped.

Example CSV:

question

How do I reset my password?
What are your pricing plans?
Can I cancel my subscription anytime?
How do I integrate with Slack?
Where can I find my API key?

You can include up to 100 questions per simulation. If your file contains more, only the first 100 will be used.For larger datasets, split your question list into batches of 100 and run multiple simulations.

Not sure about the format? Click Download sample CSV inside the upload dialog to get a ready-to-use template.

Step 3 — Upload and configure

1. Click the + Simulation button. A dialog will appear.
2. In the Simulation Name field, enter a descriptive name (e.g. "Q1 support ticket backtest").
3. Upload your CSV by either:
   • Dragging and dropping the file onto the upload area, or
   • Clicking the upload area to browse and select the file.
4. Once uploaded, you'll see a numbered preview of the parsed questions. Review them to make sure they look correct.
5. If you uploaded the wrong file, click the X next to the file name to remove it and upload a different one.

Step 4 — Start the simulation

1. Click Run Simulation (N) where N is the number of questions. The dialog will close and the simulation will appear in the run list.
2. The status will show as Pending briefly, then switch to Running with a progress bar.

Step 5 — Monitor progress

• While running, the progress bar updates in real-time showing how many questions have been processed (e.g. "44/100").
• You can cancel a running simulation at any time by clicking the cancel button (X icon) on the row.
• You can navigate away and come back — the simulation continues in the background.

Step 6 — View results

1. Once the status changes to Completed, click the row to open the results modal.
2. Review the summary metrics, charts, and individual question results (see "Understanding results" below for details).
3. Close the modal by clicking the X or pressing Escape.

Understanding results

Click any completed simulation to open the results modal. The results view is organized into three sections.

Summary metrics

Five KPI cards are shown at the top:

Metric	Description
Total	Total number of questions submitted in this simulation run.
Answered	Percentage of questions the agent directly answered using your knowledge base.
Clarification	Questions where the agent asked a follow-up to clarify intent rather than answering directly. These are not penalized in confidence scoring.
Handoff	Questions where the agent could not answer and would have required a human handoff.
Avg. Time	Average time per question to generate an answer.

Charts

Two radial charts provide a visual summary:

• Confidence Distribution — Breaks down results by confidence level: High, Medium, and Low.
• Outcome Breakdown — Shows how questions were categorized: Answered, Clarification, Unanswered, and Handoff.

Question-level results

A paginated table lists every question with its outcome status and confidence score. Click any row to expand it and see:

• The agent's full answer (rendered with formatting and markdown)
• Confidence justification — why the agent rated its own confidence at that level
• Context sufficiency — whether the knowledge base contained enough information to answer
• Citations — links to the knowledge base articles the agent used
• Processing time — how long the agent took to answer that question

Exporting and sharing results

From the results modal toolbar you can:

• Export CSV — Export the simulation output (including all answers) as a CSV file for offline analysis or record-keeping.
• Share via email — Enter an email address to send the results directly to a colleague.

Tips for getting the most out of Simulation

• Use real customer questions. Pull questions from your support ticket history or chat logs for the most realistic test.
• Run simulations after knowledge base changes. Whenever you update articles or add new sources, run a simulation to verify the agent still handles common questions correctly.
• Compare runs over time. Give simulations descriptive names (e.g. "After FAQ rewrite - Feb 2026") so you can track improvements.
• Review clarification results. If the agent frequently asks for clarification on a topic, that's a signal your knowledge base may need a clearer, more direct article on that subject.
• Check low-confidence answers carefully. Low confidence doesn't always mean a wrong answer, but it's worth reviewing the justification to see if the knowledge base needs improvement.