Your Online Car sales Assistant: Empowering Users with an AI Chatbot
Project
AI Chatbot
Role
UX Design
UI Design
QA testing
Team
UI/UX Designer
Data scientist
QA Tester
Developers
PM
where
Smyle AutoScout24
year
2023
Introduction
Buying a car is one of the biggest purchasing decisions users make in their life. Most buyers prefer to have human contact and reassurance before making such a decision.
During a data science hackathon, our team experimented with the Claude instant model by Anthropic. This initiative grew into a user-facing product aimed at solving real user problems. The AI chatbot was designed to leverage the FAQ content to provide support, instant responses, and availability around the clock for users who require assistance and confirmation during the online car purchasing process.
Targeted User Flow
Our research findings from an exit survey that was conducted in the checkout flow, revealed that users often enter the checkout process to seek more information and understand how the online car purchasing process works. Therefore, we decided to integrate the chatbot in the checkout section to cater to these users’ needs. The chatbot would act as a solution to provide relevant information and support during the checkout process.
Requirements and Limitations
To maintain response quality and prevent AI from hallucinating or degrading response quality, we agreed with the data scientist that the model would support three questions and then reset. Additionally, we decided to only send the last question to the model, optimizing the chatbot’s performance.
We also aimed to provide multilingual support, as we identified a small percentage of users who required support in languages other than German.
Unlike other bots that provide suggestions or predefined options, we allowed users to interact with the chatbot using natural language. This approach aimed to gain more insights into the users’ primary questions and provide a more personalized experience.
User Interface Design
The UI of the chatbot followed the design of a previously tested prototype with a similar message concept from another team. I built upon that design and enriched the desktop version UX. The name, welcome message, final response after reaching the three-question limit, and error messages were provided by the UX writer to ensure a cohesive user experience.
Use Cases
When the chatbot reaches the 3 turns limit, it promotes communication between the user and Customer Advocates. The input field is replaced by a button to start new chat.
If the chatbot doesn’t reply within 10 seconds, a hardcoded message will be displayed along with a “Try again” button.
We will prevent users from sending multiple questions before receiving a reply by enabling the send button only after the chatbot has responded.
The chat would follow the user in the different checkout steps and would remember the conversation in the same session.
Setting up the Instructions File
The chatbot was relying on using the FAQ content and specific instructions on what not to do or not to reply e.g. our product did not have social media accounts but in a question about a possible social media it responded with account handles that did not exist.
The order of the content in the instructions file impacts the quality of the answers. Most common questions should be placed at the beginning or end of the file.
We were very explicit on the name of the bot and its character and tone of voice otherwise it was presenting itself as a harmless – honest and helpful AI assistant by Antropic.
We also gave instructions to provide short answers.
As a first approach, we wanted to limit the requests to our Customer Support team. So one of the instructions was in case of direct question to provide contact details was to not reveal them and try to give help via the chat. Once we had the analysis of the questions, we could revisit this approach. The contact details are still available in the actual FAQs.
Model Testing and Performance Evaluation
Me along with our QA tester and UX writer were testing the chatbot with different temperature levels. That was a crucial step in order to determine the optimal level for providing quality responses.
It was important to identify the languages the chatbot is fluent in and the languages in which it has only basic fluency. We promoted communication in german and english as well as other languages and would monitor the responses.
Creating a list of the most commonly asked questions and their correct answers helped us ensure high-quality responses and a positive user experience. This list will also be used to evaluate the performance of the chatbot after making system modifications.
Post-Release Observations
After releasing the chatbot in the checkout process, we observed the following:
- The adoption rate of the chatbot feature was very low. To increase adoption, we planned to raise awareness among users about the benefits of the chatbot.
- The response time of the chatbot was 3-4 seconds, which is considered very good. With the typing effect the full response was taking up to 8 seconds. Still under the threshold of 10 sec. In a usability test we learned that the typing effect is considered part of the AI chatbots as it gives the feeling of an instant generated response and we decided to keep it.
- The majority of questions asked were about services that were not offered by smyle, such as price negotiation, trade-in, and car pick-up.
Future Enhancements
- Collecting user feedback and conducting iterative design improvements is crucial for enhancing the chatbot’s adoption and effectiveness.
- Monitoring usage metrics and user satisfaction.
- Offer the chatbot also in the car detail page and enhance it with the possibility to reply on specific questions about the car.
- Review our content strategy for the FAQ and instruction file, based on the findings as well as other parts of the product pages.
- Monitor the impact of the chatbot on the workload of the Customer Support team
What my team says about me
Designed & built by kerry.gr