Correlation vs Causation in Customer Experience Data
In three public-review analyses, the issue customers mentioned most was not the one doing the most damage to ratings. Here is why, and what a causal estimate does differently.
In customer experience data, a correlation tells you which issues show up next to low ratings, while a causal estimate tells you how much the rating would change if that issue stopped happening, with everything else held the same. The two often disagree: in three sets of public reviews we analyzed, the issue customers mentioned most was not the one doing the most damage to ratings.
This post walks through those three cases, explains where the gap comes from, and lists what to do about it in your own reviews, surveys or tickets. The general method, with the full vocabulary of confounders, colliders and estimators, is in Causal Inference: How to Move From Correlation to Cause. Here we stay with CX data.
Three cases where the loudest issue was not the costly one
Each analysis below started the same way. We read every public review, tagged it with every issue it named, and then measured each issue two ways: how often it comes up, and how much it changes the chance that a review is one or two stars once the location or location type and the reviewer are held fixed.
Hotels: the front desk
8,504 rated public Google reviews of 45 Marriott-family hotels in Orlando, January 2025 to July 2026.
Front desk and check-in comes up in 27.2% of comments, more than any other topic, and a standard driver analysis put it second on the list of things to fix. In the causal model its effect on the chance of a one- or two-star review is about two points, and the confidence interval includes zero. A dirty room adds 34 points and rude or unprofessional staff 44.
From the report: What your CX tools can't tell you
Most stays pass the desk, so most stories about a stay mention it, good or bad. That keeps its count high and its correlation with low scores real, while the stay usually went wrong somewhere else.
Gyms: broken equipment
9,186 public Google reviews of 33 Planet Fitness clubs in metro Chicago, January 2025 to July 2026.
Broken equipment is the failure members name most often, in 122 reviews. It raises the chance of a one- or two-star review by 4.7 points. Front desk and staff problems, named in 107 reviews, raise it by 36.3 points. We go through the gym data method by method in key driver analysis from text.
From the report: Chicago gym report
Cafés: food quality
1,995 public Google reviews of 24 Panera cafés in the Chicago metro, January 2025 to April 2026.
Food quality is the most-mentioned topic, in 343 reviews, and those reviews average 3.81 stars because food draws about as much praise as complaint. A wrong or missing order comes up in 116 reviews and averages 1.92 stars. On cause, a rude or absent employee adds about 33 bad reviews per 100 reviewed visits and a wrong order about 20, while ordering through the app or a kiosk adds 3, which the report treats as no real effect.
From the report: Chicago café diagnostic
Why mention counts and correlations point the wrong way
A count measures how common an experience is. It says nothing about how bad the experience is when it happens. A correlation with low scores gets closer, but it picks up everything that travels with the issue. In review and ticket data, three things travel with almost every complaint:
Review length. Long reviews mention more topics and are more often negative. In the gym data, a review that names a failure has a median of 56 to 79 words; the rest have a median of 12. Even among reviews that name none of the failures we tagged, the share that are one or two stars rises from 1.9% for the shortest to 33.1% for those over 100 words.
The other problems in the same review. About a fifth of gym reviews naming broken equipment also name another failure. If those reviews are angry, a simple comparison credits the anger to the equipment.
Where the review comes from. Busy urban locations get more complaints of every kind. A topic that is common at the worst locations will look important across the whole chain. Our restaurant location comparison shows how much location type moves the baseline.
Dimension Labs analysis
Long reviews are more often negative even when they name no specific failure
Share of one- and two-star reviews among the 8,742 gym reviews that named none of the six failures we tagged, by review length.
These are confounders: common causes of both whether an issue gets mentioned and how the review is rated. The remedy is to compare like with like, which is the subject of most of Hernán and Robins' free textbook Causal Inference: What If. In the gym data, reviews naming broken equipment are one or two stars 54.1% of the time, against 6.9% for the rest, a raw gap of 47 points. Hold review length, club type, review period and reviewer type fixed, and the estimated effect falls to 4.7 points.
What a causal estimate does differently
A causal estimate asks a counterfactual question: for reviews like this one, from the same kind of location, of the same length, by the same kind of reviewer, how much more likely is a bad rating when the issue is present than when it is absent? In practice that means four things the correlation skips.
It holds factors fixed. The hotel model held the hotel, tier, language, review length and reviewer type fixed; the café model compared reviews at the same store in the same year.
It reports a range. The gym staff effect is 36.3 points with a 95% interval of 26.1 to 51.9. A wide range is a reason to hold a ranking loosely.
It tries to break its own result. The hotel, gym and café analyses each swapped in a fake issue (a placebo) and checked that the effect went to zero, added a random common cause, and refit on random subsets. In the hotel data the placebo effects ranged from minus 0.4 to plus 0.2 points.
It can report nothing. The front-desk estimate is close to zero, and the honest output is to say so. A ranked list of correlations always has a first place, whether or not it means anything.
A causal model on observational data is still only as good as the factors it holds fixed. If something that drives both the issue and the rating was never measured, the estimate can be off. That is why the reports label some effects directional and why the next step is to check them against your own outcomes.
What to do with your own CX data
Tag every record with every issue it names. One review can describe a dirty room and a wrong charge. A single topic per record hides the second issue and makes the first look worse than it is. See semantic clustering for what a one-label approach misses.
Tag the specific failure. "Service" is a department. "Never called back" and "repair had to come back" are failures someone can fix.
Record the context. Location, location type, channel, period, review or ticket length, customer tenure. These are the factors you will hold fixed.
Pick an outcome you care about. A one- or two-star review, a stated intent to leave, a renewal, a refund. The ranking can change with the outcome: in the gym data, staff problems led on bad reviews and billing led on talk of leaving (why gym members say they will cancel).
Estimate each effect with an interval and test it. Run a placebo and a subset check before the number goes in a deck.
Size it. Effect per occurrence times how often it occurs gives a rough share of the damage. A rare, severe failure and a common, mild one can cost about the same.
FAQ
Is correlation useless in CX analytics?
No. A correlation is a fast way to find candidates worth testing, and it is the right tool for monitoring whether a known issue is rising. It is the wrong tool for deciding which fix comes first.
Do I need an A/B test to get a causal estimate?
An experiment is the strongest evidence when you can run one. Most CX questions are about things that already happened, so the analyses above use observational methods that hold the known context fixed and then stress-test the result.
How many records do I need?
Enough that each issue you test appears a few dozen times at least. The smallest issue tested in the gym data, crowding, appeared in 33 reviews, and its interval ran from slightly below zero to about 18 points.
Dimension Labs reads every review, ticket, chat, call and survey response, turns each one into typed fields we call Dimensions, joins them to your operational data and estimates which ones move ratings, retention and revenue, with confidence intervals on the Prove plan and up. The agent then builds the reports, decks and alerts on a schedule. See review intelligence, or start with one file.