AI Reviews Summary
Lead product designer.
I conceptualized & pitched the feature, taking part in
every step from definition to delivery.
1 Product manager, 2 developers, 4 data scientists.
Hypothesis
Reviews exist to reduce uncertainty, but shoppers don't have time to read through hundreds of them.
What if AI could instantly summarize hundreds of reviews into the key insights shoppers actually care about?
The challenge
At Yotpo, I worked on the Reviews product, helping eCommerce merchants to collect and utilize user generated content. Our biggerst challenge at this point was finding genuine innovation in a mature, highly competitive market.
After leading a team workshop where we mapped out merchant pain points, I found myself thinking about the problem differently, instead of asking what merchants needed, what if we asked what shoppers needed? That reframe became the north star for the project:
How might we help shoppers make faster, more confident purchase decisions?
Proof of concept
To validate the idea quickly, I built a simple proof of concept using ChatGPT, feeding a top merchant's customer reviews into it and generating a concise summary of key insights. It demonstrated the potential clearly enough to win strong support from the team and leadership, becoming the foundation for further exploration.
Discovery & research
Before designing, I validated whether AI summaries could help shoppers decide faster without compromising trust. I researched both audiences - testing with shoppers to understand review behavior and trust, and validating with merchants, who ultimately decide whether to display the summary.
KEY INSIGHT - SHOPPERS
Transparency
Shoppers wanted to understand how the summary was generated and what it was based on.
KEY INSIGHT - SHOPPERS
Reliability
Balanced summaries felt more trustworthy than overly positive ones.
KEY INSIGHT - MERCHANTS
Confidence
Merchants needed assurance the summary accurately reflected their customers' reviews.
KEY INSIGHT - merchants
Control
Merchants wanted flexibility in how the summary was presented on their storefront.
In-house language model
Testing the POC showed that relying on an external LLM wouldn't be cost-effective at the scale of thousands of product pages. Our data science team developed Yotpo's own language model, which was faster and more cost-efficient for high-scale usage. I worked closely with the team to help train and refine the model, ensuring the summaries sounded natural and followed the UX best practices I'd researched around readability and text processing.
Core UX challenges
If shoppers questioned the summary, the feature would fail regardless of how accurate the AI actually was.
How might we help shoppers verify the human source behind each summarized insight?
User-generated content is rarely black and white, and conflicting opinions are often what make reviews valuable.
How might we summarize hundreds of reviews while preserving the diversity and nuance of customer feedback?
Merchants were excited by the innovation but wary of an AI element competing with their page hierarchy.
How might we make the feature discoverable & valuable without overwhelming the product page?
Design process
I ran usability sessions with potential shoppers across multiple versions of the summary; an early ChatGPT-inspired design,
familiar star-rating patterns, different information hierarchies, to learn which information they gravitated toward and which layout was easiest to scan.
Final design
PositioningThe summary lives in two places that follows the shopper's buyer journey on the page; next to the star rating where shoppers first encounter reviews count and in the header of the reviews section.
Interaction
The summary opens in a thin drawer with a slight page darkening, keeping shoppers oriented while letting them explore the summary.
From my research I understood that a key factor in readability is making people understand the overall sentiment of each surfaced topic summary. The sentiment can be either Positive, Negative or Mixed (when we can't say for sure).
Important note: As all of Yotpo-Reviews widgets that are installed in each shopper Ecommerce store, the design inherits the font and colors of the brand, making sure it fits seamlessly within their UI. This is an area which Merchants usually have the most opinions about and we must create a version that works well with most stores.
Trust was built into every layer. Familiar anchors, like the overall rating and review count, connected the summary to elements shoppers had already seen, reinforcing continuity and credibility. I surfaced only a small set of the most mentioned topics that came up from the reviews.
Usability testing also showed a subtle sentiment icon next to each topic significantly improved comprehension, letting shoppers distinguish positive, negative and mixed insights at a glance.
Outcome
with the summary
with adoption still growing
Key learnings
- 1. Trust is everything in AI experiences.Building trust meant connecting the human element with the AI-generated output, making the technology feel transparent, reliable and aligned with merchant expectations.
- 2. New technology means balancing multiple stakeholders.Merchants were hesitant about introducing an AI feature into their product pages, so I worked closely with top merchants to build confidence and give them real control over the experience.
- 3. Designing AI features means designing the output, not just the interface.The quality, structure, and tone of the generated summary was as much a design decision as the UI presenting it.