AI Chat Bot Sprint
Timeline: December 2023- January 2024
My Role: Lead UX Researcher, Interview Moderator, Project Manager
Methods: Qualitative digital interviews, heuristic evaluation, Big 5 personality framework, quantatative survey
Tools: Alchemer, Lucid, Lookback, Condens
Framework: Heuristic Evaluation of Conversational Agents. In CHI ’21: ACM CHI Conference on Human Factors in Computing Systems, May 8–13, 2021, Yokohama, Japan. ACM, New York, NY, USA, 21 pages. https://doi.org/0
*Due to NDA, this description includes adapted content. Process description reflects the original work but client details and branded terms have been altered.
Objectives
Process
Brand wants to expand the ways consumers interact with brand ecosystem to increase brand engagement and deepen relationships.
Build an ecosystem that initiates and encourages experiences that will grow a customer fan base.
Connected ecosystem has key brand qualities NDA.
Methodology
Quantitative digital survey
n=900, USA resident, 18+, M/F, Census GenPop.
Quotas around social media engagement, online brand engagement, brand purchasing behaviour, brand awareness, chatbot engagement experience.
n=300 per brand segment NDA.
Qualitative Interviews
Moderated concept test, n=7 Mobile
Conducted with internal client employees to protect confidential concept.
AI Chat Bot Prototype Heuristic Evaluation
Review chatbot experience using the modified heuristic framework “Heuristic Evaluation of Conversational Products”.
Review and complete report for a design team with 2 researchers.
Gather usability test info from internal stakeholders to identify bugs and model red flags.
Missing Information
What experiences would be fun for individual consumers?
What role might the brand play in connecting fans with their communities?
What are the big 5 personality profiles of brand fans?
Phase 1:
1. Kickoff call with client and strategy team.
2. Background: review brand audience information and conduct competitive analysis.
3. Draft research design to address objectives and missing information.
4. Create survey, mod guide and heuristic evaluation templates for client and strategy approval.
5. Field the research, recruit and gather data.
6. Analyze quant and qual data, review internal user heuristic evaluations and gather feedback.
7. Implement research into deck and incorporate data into clear and concise insights.
8. Present to internal teams, gather feedback and iterate presentation for client readout.
Impact
Led to development of a fully functional, branded AI Chat Bot.
Gained new insights into key opportunities for customers to engage more deeply with brand ecosystems.
Uncovered a new method for the team of heuristically evaluating conversational models.