Coach AI Agent with feedback

Now that you have reviewed the performance of AI Agent, you can improve it through coaching. Specifically, by identifying and filling gaps in knowledge (in Help Center articles, Guidance, and Actions), as well as providing feedback to improve the AI algorithm.

When reviewing a ticket handled by AI Agent, you can:

• Rate the conversation Bad, Okay, or Good
• See which sources AI Agent pulled from
• Instruct the AI Agent to pull from other knowledge for similar questions in the future
• Create additional Actions, Guidance, and Help Center content
• Report issue to the Gorgias product team

Pro Tip: We recommend setting aside at least one hour per week to review AI Agent’s responses, or more if your team has the bandwidth. Coaching is especially impactful when you get started with AI Agent, when you’ll find the most opportunities to create additional knowledge content and provide feedback.

More ways to improve AI Agent’s responses

On top of reviewing AI Agent’s conversations and rating the responses, you can improve AI Agent’s performance by refining your knowledge sources (Help Center and Guidance).

Here are some tips on how to approach refining your AI Agent:

1. Find topics you haven’t addressed in the AI Library 

In Settings, under Channels, go to Help Center and click AI Library. You will see a list of articles that AI has generated based on your customers’ most asked questions.

Don’t have these articles in your Help Center yet? It’s a good idea to publish them now. One, to help AI Agent’s accuracy when answering tickets, and two, to help customers self-serve.

Pro Tip: Address the new topics in new Guidance instructions to increase AI Agent’s helpfulness.

2. Audit the top ticket intents you receive

Another way to see what your shoppers are contacting you about is to head to Statistics and, under Ticket Insights, select Intents. 

Here, you can also see the reasons shoppers reach out to you:

One great advantage of this dashboard is that you can focus on certain time periods or channels. 

For instance, you can apply a filter to only see the top intents among Email and Contact Form messages. 

Similarly, you can apply a filter to see Intents data from November-January of the previous year, to understand how your Intents change during Black Friday-Cyber Monday to better prepare for the upcoming holidays.

Just like with your top questions from shoppers, you can cross-check this data with your current Help Center and Guidance content to see if there’s an Intent with a high volume that isn’t currently covered. Fill this gap in your knowledge content to automate those questions with AI Agent.

3. Monitor the tickets AI Agent hand over

While the above tip gives you a high-level sense of what customers are asking, it’s also helpful to see real tickets. This gives you visibility into why AI Agent may not be able to resolve a ticket even if you have Guidance or Help Center content on the ticket’s topic.

The AI Agent Handover view isolates all the tickets AI could not resolve on its own — either because it did not have adequate information, or because it falls within a Handover topic.

Spend an hour each week going through the tickets in this View. Within each ticket, look for the reason the AI Agent handed it over – likely because:

• The customer was angry (which will always be handed over)
• The message included a Handover Topic
• AI Agent could not find an answer in the Help Center or Guidance
• The customer’s request required an Action that is not set up

Pro Tip: Limit your Handover topics to four or fewer. This helps AI Agent focus on the right tickets without extending handovers unnecessarily. Keeping it simple helps maintain a strong automation rate.

Let’s take a look at this ticket:

The customer wanted to update the address on a recent order. AI Agent correctly identified the customer’s order and was able to tell the customer that because the order is unfulfilled, an address update should be possible. However, after responding, AI Agent was handed over the request. 

By looking at the righthand bar, in the Ticket Feedback tab, you can see the resources AI Agent used — in this case, 2 pieces of knowledge but no Action. 

This ticket indicates you should set up the Update Shopify Order Shipping Address Action to fully resolve issues like this moving forward. 

Let’s look at another example — this time, where the problem is missing information in a Guidance:

In this example, where the customers seemed to ask about upcoming discounts, AI Agent was able to correctly identify the topic but — as you can see in the right-hand Ticket Feedback section — it was not able to find any knowledge to use.

The agent was able to come in and let the customer know there are no current or upcoming discounts to share, and instead direct them to the brand’s Instagram page for future updates.

In this case, you could create a Guidance instructing AI Agent that when customers inquire about discounts when no active discounts are live, instead direct customers to follow the Instagram page. 

And when you do offer discounts, you can edit this Guidance to empower AI Agent to share the correct discount when appropriate.

4. Segment tickets by creating new Views

Earlier in this playbook, you set up Tags for AI Agent. Here’s a refresher:

So, for instance, let’s say your focus for the week is maximizing the number of WISMO tickets AI Agent can resolve. 

Since your AI Agent is tagging all tickets “for questions about order status” with the tag “wismo,” you can use this tag — as well as the ai_answered tag that always gets added when AI Agent responds — for a View:

You can also choose to apply the “ai_handover” tag if you’d only like the View to show tickets AI Agent was not able to resolve and instead handed over to your team. 

📺 Having trouble improving your AI Agent’s performance? Register for our next AI Agent webinar.