For my Master’s thesis project, which has been verified and approved by the University of Victoria’s Human Research Ethics Board (HREB) for participant involvement, I’m looking into how to best test digital products to find out what makes them easy or frustrating to use. My research compares the traditional way of getting feedback (like talking to users) with a newer, automated method that uses AI to analyze user behavior. To do this, I’ve set up a simple usability test where volunteers are asked to complete a fictional sign-up task using tools I built specifically for this study.
Here’s a quick breakdown of the project:
The Main Goal: The project’s primary aim is to figure out whether traditional feedback or an AI-based analysis is faster and more accurate at identifying user experience problems.
The Technical Setup: At the core of this research are two custom WordPress plugins I developed. One plugin presents a standard web form, while the other powers a conversational chatbot using the ChatGPT API to ask the same questions.
The Participant Task: Volunteers can choose which of the two interfaces—the form or the chatbot they want to use to complete the fictional sign-up task.
Interaction Records: The plugins work together to anonymously collect detailed interaction records in the background. This includes everything from time spent and mouse movements to clicks, scrolls, and typing patterns, all while collecting no personal information.
This project is currently in progress, and the results from testing with 100 users will be analyzed to form the basis of my thesis.