Your AI agent gave the wrong answer. How to find and fix the cause
Follow a wrong answer through source checks, document updates, syncing and repeat tests. Learn when to use the knowledge base, product catalog or a custom action.
A customer asks: “If I spend €40, do I get free delivery?” Your AI agent says yes. Your current threshold is €50. What should you change so that the same mistake does not return when someone phrases the question differently?
We will follow one fictional case from the conversation to the final checks. The prices and shipping rules are examples; replace them with your own. The aim is to identify the cause and check that customers also receive accurate answers to related questions.
1. Keep the question and the preceding conversation
Open the wrong answer in your conversation history and read the messages before it. The customer may have already mentioned another country, currency or an oversized item. The final sentence alone may not include everything the agent needed.
Record the exact question, actual answer, expected result and the document that confirms it. If the problem occurred on a specific product page, note its address too. For the first repeat test, keep the same agent, language and model. Changing sources, instructions and the model together makes it difficult to tell which change helped.
In our example, a correct answer needs to do three things: reject free delivery for a €40 basket, state the €50 threshold and apply it to the correct country and delivery method.
2. Check the material available to the agent
Select Show knowledge sources beneath the wrong answer in the conversation history. On a computer, the button appears when you hover over the message. If source excerpts were saved with that answer, the Resources panel shows their text and available source details. Compare the amount and its conditions with the agent's reply. An outdated rule in an excerpt points you toward a particular source; a correct excerpt paired with a wrong answer suggests that content alone may not explain the mistake.
In Breezaro, select the relevant agent and open Sources → Documents & websites. Find the shipping page, uploaded FAQ and any other documents covering the same topic. Open the processed text. A well-designed table on your website may look different after extraction.
Check these details:
- Does the saved text contain the current threshold?
- Are the country, currency and delivery method still attached to the price?
- Does an old PDF contain a previous €30 threshold?
- Is the exception for oversized items missing?
Reviewing documents shows what material was available. It does not prove which specific passage the agent used for that answer. If the content looks correct, keep other possibilities open: the issue may involve selecting the relevant material, interpreting the question or following instructions.
If you also use social posts as knowledge sources, check older campaigns. A free-delivery offer that has expired may still appear in a post caption.
3. Correct the rule and its conditions
“Free delivery over €50” leaves questions unanswered. Does that mean before or after discounts? Does it apply abroad? Does a basket worth exactly €50 qualify?
A clearer example source might say:
Delivery to a collection point in Ireland is free when the value of the goods after discounts is at least €50. The delivery charge does not count toward this threshold. This rule does not apply to other countries or oversized items.
Only add conditions your business actually uses. If the rules are unclear, confirm them with the person responsible for delivery before rewriting the document.
Correct the main source and remove outdated copies from the agent’s material. If the error comes from a website page, update the website too. Editing only the imported text in Breezaro does not fix the information customers still read in your store.
You can edit processed text in Documents & websites. You can also create a document there if a short explanation is missing. Give it a specific title, such as “Delivery in Ireland: prices and exceptions,” and keep it focused on one topic. The ecommerce FAQ template can help you prepare the content.
4. Finish the update and retraining
Updating a website does not automatically mean your agent is already using the new content. For a web source, recrawl the relevant page or refresh the source and check the result. For a document, save your changes. If Breezaro shows unprocessed material, use Retrain model and wait for completion.
Before testing, reopen the content and check the corrected amount. Also check the source status: a failed crawl or a capacity limit will not resolve itself because you ask the same question again. Test once the corrected material has finished processing.
5. Repeat the original question and nearby cases
Start a new conversation in the agent preview. Enter the original customer question exactly as written, including any typo. Then try a few variations in separate conversations.
| Example question | What the answer should preserve |
|---|---|
| “There’s no postage on a €40 basket, right?” | The example threshold has not been reached. |
| “My basket is exactly €50. Is collection-point delivery in Ireland free?” | It handles the boundary amount correctly. |
| “It was €60 before the discount and €45 after. What about delivery?” | It uses the amount after discounts. |
| “Does that apply to France too?” | It does not transfer Irish conditions to another country. |
| “Do you offer free delivery?” | It states the conditions or asks for missing details. |
Finally, replay the original sequence of messages in one conversation. Separate questions check the rule; follow-up questions check whether the agent handles a change in country or amount. Record the actual answers. One successful attempt does not establish that every similar case is fixed.
6. Distinguish knowledge, catalog data and live records
For an incorrect product price, check the product catalog and its source too. Review the currency, variant, availability and last sync. An updated FAQ cannot fix an old price still supplied by the catalog. Price-range product search is useful when the underlying product data matches your offer.
“Where is my order?” needs access to a particular record. A general dispatch policy cannot confirm the status of an individual parcel. A configured custom action can retrieve that information from a connected system. Availability depends on your plan and account permissions. In its setup wizard, test the API response and the fields your agent is allowed to use. Configure the necessary customer verification and access rules for private information.
See custom actions and webhooks in practice for a worked example. A webhook can support a follow-up notification or an event record. Sending it does not replace correcting a source or checking an answer.
7. Change instructions and models for a specific reason
If your material is correct and processed, review the agent’s instructions. “Always satisfy the customer” can encourage inappropriate promises. Be specific about when the agent should verify a missing detail, ask a question or offer human help. Avoid copying business rules into several places that you may later forget to update.
Breezaro also includes Comparator, where you can send the same question to multiple models side by side and inspect their answers and credit usage. Use the same set of cases and assess accuracy, handling of uncertainty and the next step offered to the customer. A longer answer is not automatically a better one.
Keep this small collection of questions after the fix. Run through it when shipping rules, catalog data or instructions change. You can add more cases from the AI chatbot testing checklist. You will be improving your AI agent against actual customer situations, with a record you can use to assess the next change.