2FA Beyond the Basics: A Usability Study of Duo Mobile at the University of Washington

Overview

Duo Security handles over one billion monthly authentications. At the University of Washington, it's the mandatory gateway for logging into Canvas, email, and every other institutional system. Nearly every student, faculty member, and staff employee uses it daily — but almost exclusively for one thing: approving the push notification when logging into a UW service.

Duo also supports a feature most users don't know exists: linking third-party personal accounts — Gmail, Instagram, Amazon — to use Duo as a general-purpose 2FA app. In HCDE 517: Usability Studies, our five-person team was sponsored by Duo Security to investigate how well UW users could discover and use that feature.

The short answer: they couldn't. Four of six participants failed to complete the task. The failure rate was predictable once we looked closely — three interlocking design problems made the flow nearly impossible for first-time users. We documented them, rated their severity, and delivered actionable recommendations directly to Duo's product team.

Problem

The research question Duo brought to us was focused on power users — people who had intentionally linked five or more personal accounts and were using Duo as a primary 2FA manager. The sponsor wanted to understand how that population navigated the app.

We couldn't find them. Survey recruitment across UW channels reached 23 respondents, and the data made clear why power users were rare: 39.1% of UW Duo users didn't know they could link third-party accounts at all. The feature existed; the awareness didn't.

This pivot became the study's first finding before testing even began. The research question shifted from how do power users navigate the app? to can regular users navigate the account-linking flow at all? — and what's stopping them.

Method

The study ran in two phases over the Winter 2024 quarter, with findings delivered to Duo stakeholders at the end of Week 11.

Phase 1 — Screening survey

A Google Forms survey ran from February 16–25, distributed through UW student and staff channels. It gathered demographic data, Duo usage patterns, and awareness of the third-party linking feature, and invited interested respondents to sign up for a usability test session. We received 23 responses; 17 expressed interest in follow-up testing.

My contributions in this phase: co-designing the survey instrument, writing question copy, and coordinating distribution logistics. The survey also served as a recruiting screener — participants needed to be UW-affiliated and actively using Duo in an academic context.

Phase 2 — Moderated usability testing

We selected 6 participants from the 17 interested respondents and conducted moderated in-person sessions between February 25 and March 1. Sessions were held in booked campus rooms with audio recording (with consent). Each participant completed three tasks with think-aloud protocol throughout:

• Task 1: Verbally walk through the process of logging into UW Canvas with Duo, step by step, without using your phone. (Mental model elicitation)
• Task 2: Actually log into UW Canvas using Duo on your mobile device.
• Task 3: Add two-factor authentication to a third-party service of your choice — Gmail, Amazon, Instagram, etc. — using Duo.

For analysis, I co-led affinity mapping of qualitative data and applied the HCDE 517 severity rating scale (1 = cosmetic, 4 = catastrophic) to classify each finding by frequency, impact, and persistence. Qualitative codes were organized into themes; quantitative metrics — time on task, post-task satisfaction ratings, success/failure — were tabulated separately and integrated into the final report and presentation delivered to Duo.

Participants

Six participants: 4 students, 1 instructor, 1 employee. Four iOS users, 2 Android. Four women, two men. All were active UW Duo users.

Findings

Tasks 1 and 2 showed no usability issues. All 6 participants completed both at 100% success rate, and their mental models of the login flow (Task 1) matched their actual behavior (Task 2) exactly — the standard authentication flow is well-understood and well-designed.

Task 3 was the opposite. Two participants completed it. Four failed. Average time on task: 9.8 minutes. Range: 6 to 18 minutes. All three issues identified were rated Severity 4 — Catastrophic — meaning they were high frequency, hard to overcome, and persistent.

Finding 1 — Participants couldn't complete the flow

The third-party linking process requires navigating between Duo and a target app, using either a QR code or an activation code to complete the connection. The interface provided no explanation of what a QR code was for in this context, no indication of where to find the activation code in the target app, and no progress signal of how many steps remained.

Participants would reach an impasse — the Duo screen showed a QR code scanner with a vague instruction to "go back to [app name]" — and have no idea where to go or what to do next. Without any path forward, most stopped.

"The task felt impossible." — P6

"It's impossible for me to complete the task and I would definitely not want to try again." — P4

Recommendations: Deep-link from within Duo directly to the 2FA settings page of the target app, eliminating the need for users to navigate through menus of an unfamiliar app. Add a clear progress indicator to the linking flow so users know how many steps remain and where they are.

Finding 2 — Instructions were inaccurate and generic

Duo's in-app instructions read "Head back to Amazon" regardless of which app the user was linking. For participants trying to link Gmail or Instagram, this instruction was not just unhelpful — it was actively misleading. The instructional copy had not been updated to reflect how different apps surface their 2FA settings, and the generic fallback created confusion about what step they were even being asked to perform.

A secondary issue: the QR code scanner button and activation code button appeared side by side with no explanation of when to use each, and were rendered in gray — a visual treatment that communicated "inactive" rather than "available." Participants consistently hesitated, unsure whether they'd already made a mistake.

"I am so lost! What is happening here?" — P6

"It's just not enough information and when I try to add it, it's just bad design. Bad interaction." — P5

Recommendations: Replace generic copy with app-specific instructions, including direct links to the relevant settings section of each supported service. Replace the gray icon treatment with instructional imagery showing exactly where in the target app users should navigate, with visual arrows. Clarify when to use QR vs. activation code with a single unambiguous decision point.

Finding 3 — Participants were unwilling to proceed due to mistrust

Three of six participants believed Duo was a University of Washington application — built and operated by UW for UW purposes. When asked to link a personal Gmail or Instagram account to what they understood as a UW-controlled system, they refused or hesitated significantly. They didn't understand why Duo needed access to their personal accounts, what Duo would be able to see, or whether this was safe.

This wasn't irrational — Duo's interface provided no onboarding, no explanation of what the app is outside of mandatory institutional authentication, and no visible privacy or security information at the point of account linking. The absence of trust-building content turned a feature into a threat.

"I don't understand the point of this app at all. Like, how does it actually add value to my data protection?" — P4

"I don't know enough about Duo outside of UW and how they treat my data." — P5

"There are just too many things that are put on my decision-making without enough information." — P5

Recommendations: Create a short onboarding sequence for new users explaining what Duo is, why it's trustworthy, and what it can and cannot access. Add inline "info" buttons at each decision point that explain what an action does before the user commits to it. Include a contextual reminder when users initiate account-linking that explains how the feature keeps their accounts safe.

Additional Findings

Several secondary observations from testing added nuance to the three main findings:

• The "Remember Me" feature felt non-functional to participants — several said they pressed it every time but still got prompted to re-authenticate repeatedly. Whether this was a bug, a misconfiguration, or a feature working as designed with low session persistence, the user experience was of a broken control.
• Participants who succeeded in Task 3 did not succeed independently — they required researcher hints, redirection, or initiated their own Google searches to find information Duo's interface didn't provide.
• The original goal of finding power users revealed a fundamental feature awareness problem: the feature's target user doesn't exist in quantity because the feature isn't discoverable. Designing for power users is premature when basic users can't complete the flow.

Reflection

The study's most interesting moment was the pivot. We designed a study for one user group and discovered mid-recruitment that the group barely existed — and that its absence was itself a finding. Forty percent of the survey respondents we reached didn't know the third-party linking feature was there. Power users require awareness before they can develop power, and Duo had failed to create that awareness. Reframing the study around first-time use rather than expert use was the right call.

The limitation I'd address first in future work is team coordination during testing. Sessions were run individually by different team members rather than together, which meant each moderator observed different user behaviors in real time without the ability to cross-check interpretations live. Earlier pilot testing would also have helped — some ambiguities in the task wording only became apparent once participants encountered them.

The sponsor relationship was useful but uneven — Duo provided access to the product and attended the findings presentation, but offered limited support for participant incentives, which made recruitment harder and may have affected sample diversity. That said, delivering evidence-based recommendations with real user data to an active product team at Cisco was the kind of outcome that makes research feel grounded in something that matters.