Three ways to intervene.
Different reasons to hesitate.
How would people understand these concepts, whom would they trust to run them, and where would they expect problems?
I built the Community Notes notification prototype. With Nina Lutz and Ben Yamron, I shared study planning, recruiting, analysis, and policy recommendations.
We compared six ideas, then took three into think-aloud sessions. All five participants saw every concept as a set of static screens. We recruited through personal contacts, with different levels of X use but limited variation in education and location.
A literature review shaped five areas of inquiry: free speech, bias, transparency, societal impact, and user effort. The mockups used misleading posts about the 2024 campus protests to make the discussion concrete.
See how we selected the concepts
A correction competes with everything else in the inbox.
Participants wanted updates that could help them avoid repeating inaccurate information. But if merely viewing a post triggered a notice, a normal scroll could leave them expecting too many updates.
“I like getting notifications like this about ongoing events, but I wouldn’t want it for everything I view or interact with.”
Study participant, final report, p. 13
Our recommendation: make likes, reposts, and replies the threshold for a notification. Set criteria for when a changed note deserves another notice.
Participants also understood Community Notes differently. We recommended an explanation of who writes and rates notes, so a correction would come with a way to understand the process behind it.
The narrower rule trades coverage for relevance. It leaves people who only viewed a post outside the notification audience. We did not test whether that tradeoff would improve attention or trust.
What we tested, and what we recommended.
Tested: my Community Notes prototype
-
Who gets notified
- Tested
- Anyone who viewed the post. A normal scroll could bring too many notices.
- Recommended
- Only people who liked, reposted, or replied.
-
Who wrote this?
- Tested
- Labeled “Readers added context,” with no word on who the readers are. Participants understood Community Notes differently.
- Recommended
- Explain who writes and rates notes.
-
A second notice
- Tested
- An updated note sent another notice.
- Recommended
- Set criteria for when a changed note deserves another notice.
Recommended: notify people who interacted
Reconstructed; not tested.
Step 1 of 5:The mockup shows an older image presented as a recent event at UT Austin.
Who gets to put a post on hold?
The quarantine concept kept a post visible to followers, but paused interactions and removed it from search and algorithmic feeds during review.
Participants saw a possible way to limit spread, and one that asked nothing of them. They also saw a tool that coordinated reporters might turn against creators, activists, or small businesses. Some expected people to bypass the restriction with screenshots.
“I don’t repost much – I’m a lurker. So I guess I want them [X] to just clean up my feed from misinfo garbage and make sure that others don’t repost it.”
Study participant, final report, p. 15
- FollowersAvailable
- Likes and repostsAvailable
- Search and recommendationsAvailable
A report requests review.
In our recommendation, reports alone would not restrict a post. Quarantine authority would sit with moderators.
Our recommendation. Not X’s policy; not tested.
Our recommendation: reports request a review; only moderators can initiate quarantine. Reserve the restriction for potential immediate harm.
That puts more control with the moderation team. The study did not establish whether this safeguard would resolve participants’ concerns about bias or abuse.
Walk through review and its outcomes
Step 1 of 5:Under the proposed policy, moderators could restrict a post during review. It would remain visible to followers, with interactions disabled.
Not implemented or tested.
-
Paused, not removed Followers still saw the post, but interactions paused and it left search and algorithmic feeds during review.
-
Who can start a review? Participants feared coordinated reporters could turn it against creators, activists, or small businesses.
-
A screenshot gets around it Some participants expected people to share screenshots of a restricted post.
The default view carries a point of view.
The Story Tracker placed a journalist’s Analysis tab beside an uncurated Latest tab. Participants described using the coverage to follow news and bring context into conversations with family and friends.
One participant worried that opening on Analysis would push the platform’s interpretation. A useful reference still raised questions about who chose the journalist and whose perspective came first.
Journalist-curated
Context gathered into a coherent account.
Uncurated posts
A live stream of what people are saying.
Our recommendation: clarify the entry text, add a way to share the coverage, and define journalist-selection criteria that address concerns about outlet bias.
“Follow story” needed work because following already meant something else on X. The default view remained an unresolved choice, rather than a settled research result.
Walk through the proposed controls
Step 1 of 5:We recommended replacing Follow story with clearer entry text. This illustration uses View coverage; that wording was not retested.
Not implemented or retested.
-
“Follow” was taken Following already meant something else on X, so the entry text needed work.
-
Who chose the journalist? Participants asked how the journalist was picked and whose perspective came first.
-
Analysis opens first One participant worried that opening on Analysis would push the platform’s interpretation.
Recommendations for the interface and its rules.
We translated the think-aloud sessions into a report with specific interface and policy recommendations.
The evaluation made the next design decisions clearer: when to send a correction, who can restrict a post, and how to explain the context people see.
| What we heard | What we recommended | Who would act |
|---|---|---|
| Notification overloadA passing view could lead to too many corrections. | Use likes, reposts, and replies as the notification threshold. | Product and notifications engineering |
| Unfamiliar processParticipants understood Community Notes differently; those unsure how it worked worried about contributor bias. | Explain who writes and rates notes alongside the correction. | Design and content |
| Misuse of reportingCoordinated reports could be used against creators. | Let reports request review; reserve the hold for moderators and potential immediate harm. | Policy and the moderation team |
| Unclear framingEntry text and journalist selection shaped how people understood the coverage. | Clarify the entry text, add sharing, and define journalist-selection criteria. | Design, with policy and news partnerships for selection |
A follow-up study would use a broader sample and an interactive feed to measure whether people notice, understand, and use the interventions while other content competes for attention.
Looking back, the people who only read were the hardest to reach.
My reading now, not part of what we delivered.
Our notification rule reaches people who like, repost, or reply. The participant who asked X to clean up their feed called themselves a lurker and wanted protection that asked nothing of them.
That divides the work by how people use X. A correction in the inbox suits people who interact with posts. People who only read can be reached in the feed, through measures like the hold, and the hold was the concept participants questioned most. Reaching them depends on the safeguards participants asked for: who can act, and when.
The cheapest recommendation may have been the most useful. Participants who doubted Community Notes did not know how notes were written and rated. Explaining a process that already exists costs less than changing it.
What I would do next: test the interventions in an interactive feed with people who mostly read. If inbox corrections rarely reach them, the next investment belongs in the feed.