OK, this might be one of the dumbest statements you’ll read today… Measuring web metrics is a necessity in digital marketing. There I said it. Now that I got that out of the way, what I really want to focus on is what metrics to focus on when measuring the effectiveness of your Chatbots. As with every other aspect in digital marketing, the beauty of leveraging such immediate feedback for chatbots is launching, measuring, and making adjustments as needed – quickly.
Table of Contents:
1. Confusion Triggers
2. Goal Completion Rate (GCR)
3. User Satisfaction
4. Self-Service Rate
5. Target Audience Session Volume
Conclusion
One of the leaders in the conversational marketing channel, Drift uses a handful of metrics right out of the box to help you gain better insight into how your chatbots are performing. By measuring things like Chatbots Sent; Chatbots that lead to engagement or conversations; How many Emails are captured; and How many Meetings have been booked – there is practically enough to qualify to make any adjustments to any chatbots launched right out of the gate.
But what if you want to go deeper. Sure those metrics above help us see, at a glance, what’s working and what may need some tweaks. But do these really tell you the whole story? Here are 5 more metrics I recommend you measure to understand how effective your chatbots truly are.
1. Confusion Triggers
It’s not uncommon for chatbots to become confused. The natural language of humans frequently makes chatbots not understand or misinterpret what users really mean. A red flag may be waving if you’re seeing a lot of “I don’t understand” type responses from your bot. This definitely requires steps to take to figure out what’s causing confusion. By focusing on this metric you may uncover the opportunity to learn more about your customer’s needs.
2. Goal Completion Rates (GCR)
More simply put, capturing the percentage of successful engagement through chatbots. Users will most likely try to various information or services. For an eCommerce company, a chatbot’s goal may be to inform a user about a product’s detail or purchasing a product. Other ways to leverage this metric may be to mine data through questions by the users. This would show an overall trend in consumer preferences – providing a greater emphasis to be placed on that specific preference when developing new bots.
3. User Satisfaction
Found through exit surveys, this new metric measures the experience users have with a specific chatbot. Asking binary questions like: “Did the bot perform well? – Yes, or No” helps the strategist or developer of the bot to know immediately if this chatbot helped or hurt the overall user experience. This leads to a direct measure of how effective a chatbot may or may not be. As effective as this feedback is, it is even more valuable to get this feedback on per chatbot message basis rather than on a per chatbot basis so you can better identify where weak points are in the chatbot – instead of the bot as a whole not being effective.
4. Self-Service Rate
This rate corresponds to the number of users who were able to obtain the help they needed through the responses given by your chatbot, without subsequently having to call Customer Service. If the chatbot’s function is to answer most Frequently Asked Questions without the help of a human and does so successfully, then that bot is functioning quite well. By measuring how often a human is required to engage, and over what question or for what reason, provides strong insight as to what other questions may be necessary for the bot to handle, or help uncover a weakness within the chatbot.
5. Target Audience Session Volume
This indicator is essential for verifying that you are achieving your goals. If you are targeting a specific population, you can measure the penetration rate for this audience in order to verify that the intended people are making sufficient use of your chatbot. Otherwise, it’s imperative to rethink your change management or customer experience strategies in order to get your users on board!
Conclusion
Knowing these 5 key indicators is essential for evaluating your chatbot’s effectiveness. However, the best indicators to track aren’t always the same for every company or for every chatbot: it’s up to you to choose the most relevant ones based on your line of business, your goals and the needs of your users. If you would like some guidance to help you figure out which KPIs will be most helpful in measuring the effectiveness of your chatbot, don’t hesitate to contact Revenue River.