Christoph Börner at Cyara, spoke with CMSWire about the perks of chatbot testing and customers’ pet peeves with conversational AI.
Customers increasingly rely on conversation with bots to interact with brands – whether they have a simple question or a more complex problem. Because these conversations can vary wildly, organizations can’t rely on the same static, robotic bots to meet customers’ high expectations. People want a quick, seamless conversation that gives them the exact information they need, and a bad interaction may mean they don’t trust the brand anymore.
That’s why continuous chatbot testing is so important. Ensuring that customer interactions are always effective and always improving will help organizations keep their customers happier. Christoph Börner, senior director, digital at Cyara, spoke with CMSWire about the perks of chatbot testing, customers’ pet peeves with conversational AI and how subject matter experts can guide organizations on the right path as they seek to improve the customer experience.
The Most Effective Dialogue of a Successful Chatbot
The balance between a bot that acts either too human or too robotic can be difficult to achieve. Customers generally are at ease with something in between. This is where conversational AI steps in. A simple chatbot that is based on rules or keyword recognition will generally act more robotic. On the other hand, conversational AI allows for more natural and complex conversations with users that feel more human. The organization can decide how human the bot behaves based on what their customers prefer.
Börner gives the example of Google’s Duplex assistants as something that sounds completely human. Many technologies make this possible, including speech recognition, emotion detection, conversational intelligence and speech synthesis. Other companies may choose something less complex if their customers want a different kind of chat experience.
The language a chatbot uses can also make a big impression on customers, Börner adds.
“A chatbot has to speak the language of its customers. And that also covers jargon, slang or even dialects,” he says. “The right amount of jargon is usually defined by the chatbot’s purpose. A simple bot to do small talk will need less jargon than a technical support bot.”
“Important is that these things are tested,” he adds “Does your bot speak and understand jargon? What about slang, dialects or multiple languages in general? All those things can be answered by serious testing and automation.”“Important is that these things are tested,” he adds “Does your bot speak and understand jargon? What about slang, dialects or multiple languages in general? All those things can be answered by serious testing and automation.”
Börner says that he is optimistic about many new developments in conversational AI and that many of the technologies related to it are getting significantly better. “Large language models and prediction models are improving. Contact center providers are including conversational AI to drive their support lines. Speech synthesis, emotion detection, [and] natural language understanding and generation – all those components are improving,” he says.
Identifying Challenges and Mistakes with Chatbots
Common problems regarding chatbots are related to their accuracy and understanding of language, Börner says. Regression testing is a helpful tool to identify when a bot is not delivering accurate answers in a timely manner. Problems in language understanding of a chatbot are usually located in the training data, Börner adds, and natural language processing (NLP) analytics highlight these errors.
Börner also says that chatbot developers must be able to put themselves in their customers’ shoes in order to create a more positive chatbot experience. People’s most common frustrations with bots include when they can’t understand or answer their requests, when they’re slow and when they don’t work properly on their preferred channels. Testing is what allows organizations to try out different dialogues and see how effectively the bot responds. Is this the ideal experience for a user with a comment or question? With so many potential ways for a conversation to go, large-scale testing is the only way to ensure that the customer experience is optimized.
“Testing reveals the problems their customers will face,” Börner says. “Checking all conversation flows, analyzing the training data, testing functional and non-functional – those are the ingredients to deliver an outstanding chatbot experience. [This can be] combined with continuous production monitoring to fine tune the bot with cases it cannot handle.”
Get Guidance From the Professionals
Börner suggests that companies work with experts who know all the complexities of how chatbots function and how to address common issues. “All big conversational AI platforms also provide professional services,” he says. “Companies who think they can build up these capabilities by trial and error usually fail miserably. Instead they should do the first steps together with experts and then step by step take over.”
A traditional quality assurance engineer will be overwhelmed by the challenges of testing conversational AI, he adds. But with the help of the right tools and experts, these professionals have a much better chance at success.
Learn More About Botium
Testing is key to every step of the way as organizations design and perfect their chatbots. Companies buy bots and expect that what they get is something that works well and that has been tested, Börner says. But because conversational AI is constantly learning, it also must continuously be tested even after a company gets access to a bot. Buying a bot and not expecting to have to do further testing can lead to negative customer interactions down the line.
Solutions like Cyara Botium constantly monitor bots and make sure that they operate correctly – for all types of chatbot testing.
“Testing starts on day one,” Börner says. “It comes with huge challenges for the QA departments. Manual testing is not an option due to the nature of conversational AI and its infinitely large test sets. Test automation is the key enabler to success.”
To learn about Cyara Botium and how it utilizes chatbot testing, go here.
