Your Customers Already Told You What Your Next Decision Will Do
In 8,504 public hotel reviews, the most-mentioned topic barely moved ratings while rude staff raised the chance of a bad stay by 44 points. Why the next decision should be tested on a model of your own customers.
Most companies decide what to fix by counting. In 8,504 rated public reviews of 45 Orlando hotels, the front desk came up in 27.2% of comments, more than any other topic, and its measured effect on the chance of a one- or two-star stay was about two points, too small to tell from zero. Rude or unprofessional staff came up in 3.6% of comments and raised that chance by 44 points.
The loudest problem and the costly one were different problems. That gap is why I believe a company should test its next decision on a model of its own customers before it makes it, and why that model should be built from what those customers already said and did. A panel of simulated people has never stayed in your hotel.
The meeting where the decision gets made
A COO at a chain with 200 locations has a proposal on the table to rebuild check-in: new software, a kiosk in the lobby, an extra person at the desk on weekend evenings. The case for it is a dashboard of last quarter, where front desk complaints sit at the top of the list.
The dashboard answers the question it was built for, which is how often each topic comes up. The decision needs an answer to a different question: if we spend this money, what happens to the guests who would otherwise have left unhappy? Counting cannot answer that, and a dashboard that looks healthy can hide the answer completely. When we rebuilt the standard dashboard from the Orlando reviews, it showed NPS of +32 and CSAT of 68%, flat for 18 months. Strong stays and broken ones were cancelling out in the average.
What the guests already told you
The Orlando analysis used nothing but public Google reviews: 9,446 of them, each read and turned into 23 structured fields. Then we asked the same reviews a second question. For a review from the same hotel and tier, in the same language, of the same length, by the same kind of reviewer, how much more likely is a bad rating when a given issue is present?
Public reviews, 45 Orlando hotels
The topic guests mention most falls to last once you rank by what it does to a bad stay
The five issues a standard dashboard flags, ranked by share of comments (left) and by measured effect on the chance of a one- or two-star review (right).
The order reverses. The front desk is mentioned about 7.6 times as often as staff attitude , and it sits last on effect. Cleanliness, room maintenance and billing all land between 19 and 34 points, and every effect passed placebo, random-cause and subset tests. The desk looks important because most stays pass through it, so most stories about a stay mention it, and it rides along in long, unhappy reviews where the stay went wrong somewhere else.
For the COO, that is the difference between a check-in project that moves nothing and a staffing and coaching program aimed at the 44-point problem. The guests had already said which one mattered, in reviews anyone could read; the dashboard only counted them.
The four layers, on one decision
Public reviews were enough to show the reversal. They are not enough to size a decision, because they hold only two of the four kinds of data a customer world model needs: what customers say, and one thing they did, the star they left. A company's own record holds the other two, and that is where most of the value is.
One decision, four layers
What a model of your own guests needs before you fund a front desk fix
The decision: a new check-in process across 200 locations. What each layer holds, and what it adds that the dashboard does not.
| Layer | For this decision | What it adds |
|---|---|---|
| 1 What customers say | Reviews, surveys, tickets and chats, each tagged with the same fields: front desk wait, rude staff, dirty room, surprise fee | Which issues each guest raised, not only the loudest one |
| 2 What customers do | The star rating, whether the guest came back, loyalty stays, spend per stay | Which of those issues changed what guests did next |
| 3 What the business did | Dated changes: properties that already moved to mobile check-in, desk staffing changes, renovations | Earlier versions of the decision, already run on these guests |
| 4 What the world did | School holidays, conventions, storms, heat, a new hotel opening nearby | The things that moved the same weeks and would otherwise get the credit |
Start with what customers say. Every review, survey answer, ticket and chat gets the same fields, so "rude staff" means the same thing at a resort in March and a select-service hotel in October. This is the layer most companies already have, scattered across several tools.
Then what they did next. A star rating is one outcome. Whether the guest booked again, how many loyalty nights they stayed and what they spent are better ones, and they sit in systems the CX team rarely joins to the comments. Joined, they tell you which complaints cost a return visit.
Then what the business did. The decision you are weighing has usually happened before, somewhere, in a smaller form. A few properties moved to mobile check-in last year. Another region added weekend desk staff. A renovation closed a lobby for a month. Each one is a natural experiment, run on your own guests, with a date attached.
Then what the world did. Orlando hotels live by the school calendar, the convention schedule and the weather. A check-in change that went live in a slow week will look like a success on any before-and-after. The rule we use for what earns a place here: a source belongs if it is a plausible confounder for an outcome you already care about. Weather, the calendar, foot traffic, nearby competition and the makeup of the trade area usually qualify. Industry benchmarks usually do not: they tell you about the market, and the decision is about your guests.
With all four in place, the question changes from "what are guests complaining about?" to "when we changed something like this before, what did these guests do next, with everything else that moved that week held still?"
A range, and a record of where it missed
What comes back is a range, and it holds only if the relationship we measured holds. That is less satisfying than a single number and far more useful, because a range can be checked.
We check ours by backtest: freeze the data before a decision that already happened, set the range, then compare with what came next. The one we have published, a hardware company's February 2026 product launch, held on both headline ranges and missed on 3 of 14, all on the high side. The full scorecard, misses included, is in what is a customer world model?
The misses are what I would put in front of the COO first. They show where the model was thin, in that case customer tiers that were still growing, and so which part of a range to lean on and which part needs more data before a budget rides on it. A dashboard of last quarter never tells you how far to trust it. A model with a graded record does, and the record gets longer with every decision you add.
Why your own customers, and not simulated ones
Simulated panels answer quickly and can be good at the direction of a reaction. The size of your guests' reaction to your front desk is a different kind of number: it lives in what those guests did after real stays. The 44-point gap between staff attitude and check-in came from people who were there. What published studies found when simulated customers were checked against real ones is in are synthetic customers accurate enough for pricing decisions? Use a simulation to decide what to test. Use your own record to decide how much it is worth.
What to ask before the next sign-off
Before approving the next big change, ask for three things. Where has a version of this already happened in our own data, and what did those customers do afterwards? Ranked by effect rather than by mentions, where does the problem this change targets actually sit? And what is the range, with the weather, the calendar and the other changes of those weeks held still, plus the record of which past ranges held?
If the answers come back as a ranked list of complaints and a single projected number, the decision is still a guess. If they come back as a range built from your own customers, you can decide, and you can check yourself a quarter later. The method is worked through on a price increase in how to test a business decision before you make it, and the four layers are defined in what is a customer world model?.
What does Dimension Labs build for a decision like this?
Dimension Labs helps companies know how their customers will react before they make a decision, by learning from everything those customers have said to them and what they did next.
In practice that means every channel read together, from reviews and surveys to tickets, chats and calls; every interaction enriched with Dimensions written for your business, so the front desk and staff fields mean the same thing at every property; and the results turned into decks, dashboards and executive reports the operations team can share. On the Prove plan and up, the platform joins those fields to your CRM and revenue data and estimates each driver's effect with a confidence interval, so you can see which complaint is worth fixing before you fund the fix. That join is how Travelers found $3.13M in savings.
FAQ
Can I test a decision on my customers before making it?
Yes, when a version of it has happened before in your data. Measure what customers did after those earlier changes against comparable customers or locations that did not change, with weather and the calendar held fixed, and use the result as a range for the new decision.
What is the difference between the most common complaint and the most costly one?
The most common complaint is the one customers mention most. The most costly is the one that most changes what they do next. In the Orlando hotel reviews the two were at opposite ends of the list.
Do I need all four layers to start?
No. The first two layers, comments and one outcome, already rank issues by effect. Dated decisions and outside data are what turn that ranking into a range for a decision you have not made yet.
Upload one decision's worth of customer data, the reviews or tickets from before and after a change with its date, and see which complaint actually moves your rating: start with one file. Plans start at $299 a month with a 14-day free trial; the join to revenue data and measured effects are on Prove and up.