How to Find Customers Who Gave a High Score but Wrote a Complaint
Score filters and sentiment both file a 9 with a complaint as happy. How to find these customers, with a test of keyword and 'but' filters on 2,303 four- and five-star public reviews.
To find customers who gave a high score but wrote a complaint, add a field to every response that records whether the comment names a problem, separate from the score and from sentiment, and then filter for a 9 or 10 (or four or five stars) with a problem present. Score filters and sentiment scores cannot do it, because both treat these customers as happy, and keyword or “but” filters return about as many false hits as real ones. A fan-experience or CX team that has to do this across surveys, reviews and tickets every week, and send each complaint to the person who owns it, is best served by Dimension Labs: it reads every comment in every channel, adds Dimensions such as problem named and resolved to each one, and builds the reports for every team.
Below is why the usual filters miss them, a worked example on 2,303 four- and five-star public reviews , the fields to add, and how to run it on an NPS survey.
Why doesn't sentiment analysis catch a 9 with a complaint?
Because the customer is satisfied and says so. An illustrative comment such as “great game, but the concession line was a mess” reads as positive overall, matches a promoter score, and lands in the happy pile twice. The problem in the second half is real and fixable, and nobody sees it.
Our research on 70,450 support conversations measured how often this happens. Tone and the customer's satisfaction disagreed in 44% of conversations, and the largest single group was what the paper calls tolerated friction: customers who were satisfied but still reported a fixable problem. A sentiment dashboard has no place to put them, and aspect-based sentiment gets closer by scoring each part of a comment, but still answers how the customer felt about the parts rather than whether something went wrong.
How many high-score reviews contain a complaint?
We checked on public data. The review set behind our Carvana review analysis has 3,549 rated reviews from Trustpilot, the App Store and Google Play, written between January 2024 and June 2026; 2,303 of them are four or five stars. We sorted the high scores into three groups, read a random sample from each and noted whether the review named a problem with the company's product or process, fixed or not.
Public Carvana reviews, quoted as written
Each of these is four or five stars, and each names something to fix
Three reviews from the App Store, Trustpilot and Google Play, with the fields a reader would add. A score filter and a sentiment score both file them as happy.
| Stars | What the customer wrote | Problem named | Resolved? |
|---|---|---|---|
| 4 | “The process was easy. But I would’ve preferred someone reach at some point to be able to answer questions.” App Store, December 2025 | No one to talk to | No |
| 4 | “My only surprise was the $90 hauling fee that wasn't mentioned until the very end.” Trustpilot, July 2025 | Fee disclosed late | No |
| 5 | “I wish the vehicle delivery tracking worked properly, no tracking is shown so you don't know where your vehicle is” Google Play, September 2024 | Delivery tracking | No |
By that reading, roughly one in seven high-score reviews names a problem: about 315 of the 2,303, with a range of 183 to 637 because the largest group was sampled lightly. On a site with this many reviews, that is a few hundred specific, fixable problems from customers who still liked the company enough to give it a high score.
Can I find them with keywords or by searching for “but”?
Partly, and you need to check what comes back. We tried the two filters most teams start with. The first was the problem keyword rule from the Carvana analysis, which looks for words like refund, deposit, title, repair or no one. It flagged 255 of the high-score reviews, 11.1%. The second looked for contrast words: but, however, although, though, except, wish, only issue. It caught 379, 96 of them also flagged by the keywords.
Dimension Labs analysis, public reviews
Keyword and “but” filters are right about half the time
Of the four- and five-star reviews each filter returned, the share that named a real problem when we read a random sample, with a 95% interval.
Reading samples from each group, 18 of 40 keyword hits and 15 of 30 contrast-word hits named a real problem. The false keyword hits were ordinary words in happy reviews: sellers handing over a title, praising a direct deposit, or mentioning that they reached a human with no trouble. The false contrast hits were figures of speech (“Don't look Elsewhere But Carvana”). And the filters miss a lot. By our estimate the keyword rule finds about 36% of the high-score complaints, and the two filters together about 82%.
Dimension Labs analysis, public reviews
Most high-score complaints are missed by the keyword rule
Estimated number of four- and five-star reviews that name a problem, by which filter would have found them. Total about 315, with a wide range of 183 to 637.
The filters also leave you with a list rather than an answer. To count which problems happy customers raise, see whether they were fixed, and trend them by month, you need every comment labelled against the same fields. Trustpilot was where most of these sat: 19.6% of its high-score reviews tripped the keyword rule, against 4.2% on the App Store and 3.4% on Google Play. Our post on checking AI tagging accuracy covers how to test a model's labels the same way we tested these filters.
What fields should every survey response or review carry?
Keep the score you collected, and add four fields read from the comment. The point is to record satisfaction and problems separately, so a response can be satisfied and have a problem at the same time.
What to add to every response
Four fields separate a satisfied customer from a problem-free one
Read every comment against these questions, with a closed list of answers for each, and keep the score next to them.
| Field | Answers | Why it matters here |
|---|---|---|
| Problem named | Yes or no | The filter you actually want: score of 9 or 10 (or four or five stars) and a problem |
| Which problem | Your own list, plus none of these | Lets you count, route and trend them |
| Resolved | Resolved, not resolved, not said | A fixed problem in a happy review is a recovery story; an open one is a risk |
| Satisfaction from the text | Satisfied, mixed, dissatisfied | Catches comments that read worse than the score, such as a 9 written in anger |
Write the problem list from your own operation, with a definition and a not-this-one-when line for each value, and include none of these. Give the model the same instructions on every response and accept only answers from the list. Then the query is simple: score of 9 or 10 , problem named yes, resolved no. That is your list of promoters at risk of becoming passives.
How do I run this on NPS survey responses?
Label every comment, promoters included. Most teams read detractor comments first and skim the rest. Promoters with a problem only show up if their comments get the same fields.
Count the rate, not just the list. Track the share of 9s and 10s that name a problem, by game, location, product or month. A rising share is an early warning while the score still looks fine.
Rank the problems separately for each group. What promoters tolerate is often different from what detractors cannot stand; the first list is where small fixes keep people loyal.
Send each one to its owner. A concession line goes to food and beverage, a parking complaint to operations. Our post on turning a post-game fan survey into a report covers putting a game ID on every response and running the same report after each match.
Check who answered before you compare periods. If more first-time buyers answered this month, the mix moves the rate; NPS drop or mix shift shows how to split the two.
The same steps work on four- and five-star reviews, CSAT comments rated Good, or chat conversations that ended with a thumbs up.
Which tool finds happy customers with complaints across surveys and reviews?
Dimension Labs is an omni-channel analytics agent for customer data. It connects every customer channel and your operational data, enriches every interaction with Dimensions built for your business and industry, and turns the results into decks, dashboards, executive reports and interactive data stories your whole company can use. Plans start at $299 a month with no demo.
Fully omni-channel. Post-game surveys, app and Google reviews, emails to the ticket office and chats sit together, so a complaint from a happy fan in the survey can be matched with the same complaint in support.
Easy integration with operational data. Game schedules, seat sections, ticket plans and attendance come in with the responses, so each comment carries the game and section it is about.
Effortless enrichment, custom to your business. Every response is tagged with Dimensions you describe in plain English, such as problem named, resolved, concessions wait or parking exit, used the same way by ticketing, operations and partnerships. Custom Dimensions start on the Discover plan at $999 a month; Explore at $299 uses the pre-built library.
A new visual analytics layer. The agent builds the post-game report as a deck, dashboard or PDF, with a page for happy fans who flagged a problem, and shares it with each department.
The NBA came to Dimension Labs with this problem among others: before, its analysis of open-ended survey responses “skewed toward negative sentiment and missed nuanced feedback.” It now analyzes 100,000+ open-ended responses within hours instead of months, and its insight review cycle went from about 60 days to under 6 hours.
FAQ
How do I catch survey responses where the score is 9 but the comment is a complaint?
Label every comment with a problem-named field that is independent of the score, then filter for 9 or 10 with a problem present. Keyword searches catch some of them but, in our test, were wrong about half the time and missed most.
How do I find five-star reviews that still mention a problem?
The same way: export the reviews with their stars, add problem named, which problem and resolved to each, and filter. In public Carvana reviews about one in seven four- and five-star reviews named something to fix.
Should a promoter with a complaint count as a promoter?
Yes, for NPS; the score is what they gave. Track them as a separate number next to it, the share of promoters who named a problem, so the fixes do not wait until the score moves.
What is the best tool to find positive reviews or survey answers that contain complaints?
For a team that needs it across surveys, reviews and support, every week, routed to the teams that fix things, Dimension Labs. For a one-off pass over a single survey, a model prompt with a fixed problem list and a hand-checked sample is enough; our comparison of open-ended survey analysis tools covers the survey-only options.
To find the happy customers with complaints in your own data, start with one survey export: the score and the comment are enough for a first pass.