Multifactor Authentication in University Settings
Overview
Our team was sponsored by Duo Security to conduct a usability study
of their authentication app across users at the University of Washington.
We conducted a survey and recruited 6 participants to take part in a task-based moderated
testing session. We utilized time-on-task andpost-task Likert scale metrics, coupled with
a think-aloud protocol to gather qualitative and quantitative data.
Timeline
January - March 2024
Methods
Surveys, Moderated Testing
Role
UX Researcher
Impact & Outcomes
6
User Testing Sessions
23
Survey Respondents
3
Implementation Priorities
Key Findings
Clear Mental Models
Users can generally anticipate what they need to do with Duo for a typical Canvas log-in process.
2FA Misconceptions
A segment of users don't understand the purpose of 2FA, and express frustration about the additional step needed to log in.
Setup Friction
Linking a new platform with Duo's 2FA was a journey with unclear steps due to context switching between platforms.
Discovery & Planning
We began with stakeholder interviews across different roles within the Cisco Duo team to align our research objectives with their business goals. Through these discussions, we identified key areas of investigation and developed our research strategy.
Research Objectives
Primary Goals
- Understand mental models and authentication workflows
- Identify pain points in setup process
- Examine user behavior differences
Methodology Design
Research Approach
- Mixed-methods: Surveys (n=23) + Usability Testing (n=6)
- Active UW students and staff participants
- Focus on Duo authentication workflows
Stakeholder Insights
User Experience
Focus on reducing friction in authentication flows
Security
Balance security with user convenience
Platform
Cross-platform consistency and integration
User Research
Our research execution phase consisted of two main components: a broad survey to gather quantitative data and in-depth usability testing sessions to observe actual user behavior.
Survey Design & Distribution
Distribution Channels
- UW student channels
- Staff communication networks
- Department mailing lists
Key Metrics
- 23 responses collected
- Two-week collection period
- Focus on authentication patterns
Usability Testing Sessions
Session Structure
- 6 moderated sessions
- Diverse user profiles
- Think-aloud protocol
Data Collection
- Mental model exploration
- Service setup scenarios
- Post-task interviews
Research Timeline
Survey Launch
Week 1
Data Collection
Week 2
Testing Sessions
Week 3
Survey Results
Respondent Demographics
- 23 Respondents
- 12 Male, 12 Female, 1 GNC
- 82.6% Students, 17.4% Employee/Staff
- 69.6% iPhone Users, 26.1% Android Users, 4.3% Multi-Phone Users
Usage Insights
- 87% interact with Duo at least once a week
- 39.1% unaware they could link personal accounts with Duo 2FA
- 82.6% have fewer than 3 linked accounts with Duo
Usability Study Results
Likert scale ratings from 1-7, where 1 represents negative attributes and 7 represents positive attributes.
Task 1
100% SuccessDescription
Think-aloud process of logging into Canvas with Duo (no device)
User Sentiment
"Intuitive, matched my mental model."
Task 2
100% SuccessDescription
Actual login to Canvas using Duo Mobile
Time on Task
1.58 mins
Perceived As
User Sentiment
"Easy, quick, no issues."
Task 3
33% SuccessDescription
Add a third-party app (e.g., Gmail) to Duo for 2FA
Time on Task
9.8 mins
Perceived As
User Sentiment
"Frustrating," "Impossible," "Confusing."
Task 3 Failure Analysis
participants failed to complete the task
minute completion time range, indicating high variability in user struggles
Reflections
Our study revealed that while Duo's 2FA system is generally effective, there's a significant opportunity to improve the onboarding experience and reduce friction during third-party service integration.
We found that users' mental models of authentication were consistent with their actual actions during the task of authentication. Successfulness of linking new accounts with Duo's authentication service varied based on their technical background, suggesting a need for more adaptive and contextual guidance during complex setup processes.
Moving forward, we recommend focusing on streamlining the third-party integration process, increasing instructional clarity, and building messaging that educates users about the advantages of two-factor authentication solutions and promotes trust.