Six Ways to Analyze 8,504 Hotel Reviews, Compared
We analyzed the same 8,504 rated hotel reviews six ways. The first four describe the file; only a fixed schema and a causal estimate ranked problems by their effect on the rating.
To analyze thousands of reviews without reading them all, ask every review the same fixed set of questions with a language model, store each answer as a field, and then measure which answers change the star rating. Dimension Labs does this end to end, from Dimensions on every review to effects ranked against the rating (on the Prove plan and up) and the report your teams read. We ran six methods on the same file of 8,504 rated hotel reviews. Star averages, keyword counts, sentiment scores and topic clustering each describe the reviews; only a fixed schema joined to the rating, and a causal estimate on top of it, ranked the problems by what they cost. The topic guests mentioned most, the front desk, in 27.2% of comments, turned out to move the chance of a bad review by about two points.
The file: public Google reviews of 45 Marriott-family hotels in the Orlando metro, January 2025 to July 2026. 9,446 reviews, 8,504 with a star rating, 1,967 of those one or two stars. Public reviews only; no Marriott data was used.
How do the six ways to analyze hotel reviews compare?
Same 8,504 reviews, six methods
Only the last two methods rank the problems by what they do to the rating
What each method told us about the Orlando hotel reviews, and what it left out.
| Method | What it told a hotel operator, and what it missed |
|---|---|
| 1 Star averages | 3.84 stars, NPS proxy +32, flat for 18 months; poor-stay share by hotelWhat it missedWhy any hotel is better or worse |
| 2 Word clouds and keyword counts | Front desk and check-in in 27.2% of comments, the most common topicWhat it missedWhether a common topic hurts; paraphrases with no keyword |
| 3 Sentiment scores | Whether each review sounds positive or negativeWhat it missedWhat the guest was unhappy about |
| 4 Topic clustering (k-means, LDA) | Six of eight k-means clusters were about tone; LDA found a fees topic and a front-desk topicWhat it missedSeveral problems in one review; stable counts between runs |
| 5 Fixed LLM schema | 23 fields answered for every review, such as surprise fee and wrong charge kept apartWhat it missedAnything outside the schema; it needs checking |
| 6 Causal effect estimates | Front desk about +2 points (not significant); dirty room +34; rude staff +44What it missedIssues too rare to estimate; effects are from observed reviews, not an experiment |
Each method answers a different question. The first four tell you what the reviews look like. The last two tell you what to fix. The sections below take them in order, with what each showed on the Orlando file.
1. What do star averages tell a hotel?
Averages are the first thing everyone computes, and they are useful for spotting where to look. Across the file, the average was 3.84 stars and had sat at about 3.8 for 18 months. Converted to the usual proxies, that is an NPS of +32 and a CSAT of 68%. 23.1% of rated reviews were one or two stars, and the share by hotel ran from 7.89% to 41.18%.
What averages miss is the reason. A flat 3.8 can hide very good stays and very bad ones cancelling out, which is what the brief found: "strong stays and broken ones are cancelling out." An average also moves slowly, so a problem that started last month barely shows.
2. What do word clouds and keyword counts show?
Counting how often topics come up is the standard text analytics output, and the dashboard rebuilt in the brief ranked them this way: front desk and check-in in 27.2% of comments, cleanliness 10.0%, room maintenance 7.9%, fees and billing 5.2%, staff attitude 3.6%. Front desk and check-in appears in 2,317 reviews. A word cloud gives the same picture with less precision.
Counts tell you how common something is, not how much it hurts. Most stays pass the front desk, so it comes up in good reviews and bad ones alike. Ranked by measured effect instead, the order flips:
From the report: What your CX tools can't tell you
Keyword lists have a second problem: they only find the words you thought of. "The room was filthy" and "housekeeping never cleaned" share no keyword. And reviews often name several problems; a keyword check on the 1,967 bad reviews found 33.1% named two or more of five problems we looked for.
3. What does a sentiment score add?
Sentiment scoring labels each review positive, negative or neutral, or gives it a score. On support tickets and surveys, where there is no star, that is useful. On a review site the guest has already given a star, so a review-level sentiment score mostly restates it, and a negative label puts a wrong charge in the same pile as a cold breakfast.
Aspect-based sentiment, which scores each topic within a review, gets closer, but it still measures tone about a topic rather than the topic's effect on the rating. The brief kept sentiment as one of 23 fields, read "from the words, not the star," next to fields that say what went wrong. Our posts on text analytics vs sentiment analysis and aspect-based sentiment analysis go further.
4. Does topic clustering find the real problems?
Clustering groups similar reviews without being told what to look for, which makes it a good first look at a new file. With TF-IDF and k-means at eight clusters, six clusters were built on tone words such as "amazing", "great location" and "nice clean"; they held 59.5% of reviews, and only 2.4% of those were one or two stars. One cluster of 3,199 reviews took 1,789 of the 1,967 bad reviews, 91%: one pile for every kind of bad stay.
Clustering only the bad reviews produced usable piles: parking charges, smells and mold, card and refund disputes, the airport shuttle. The biggest, 444 reviews or 22.6% of the negatives, was front-desk stories, so a team that treats cluster size as priority would put the desk first. LDA, a topic model, separated more: it found a topic built on parking, charge, fee and resort, and another built on desk, told and check. It also gave 56% of reviews two or more topics, which shows how mixed reviews are.
The counts did not hold still. Rerunning the same k-means setup with different random seeds moved the size of the biggest complaint cluster by almost 300 bad reviews:
Dimension Labs analysis
Rerun k-means with a different seed and the biggest complaint cluster changes size
One- and two-star reviews in the largest complaint cluster, k-means with eight clusters on the same 8,504 reviews, five random seeds.
For LDA, two runs with different seeds agreed on each review's strongest topic only weakly (adjusted Rand index 0.27). That is fine for exploring and a problem for a monthly report. How clustering works step by step is in semantic clustering demystified, and the full comparison with LDA in topic modeling vs LLM classification.
5. What does a fixed LLM schema change?
A fixed schema turns the question around. You write the questions first (was a fee mentioned, and which kind; was the room clean; was a staff member named, and praised or blamed) and a language model answers every one of them for every review. The brief used 23 such fields, which we call Dimensions, extracted in one pass over each of 9,446 reviews.
Three things change for an operator. One review can carry several problems at once. The definitions stay the same from month to month, so counts can be tracked and compared between hotels. And the fields can be written in the hotel's own terms, so "a surprise or hidden fee" and "a wrong or disputed charge" are separate answers rather than one pile of words about money. That split mattered: 438 reviews mentioned a surprise fee and 98 a wrong charge, and the two turned out to do very different damage.
The schema has costs. It only finds what it asks about, so it is worth keeping a free-text field such as "the single worst thing, in the guest's words". And the labels need checking: before the hotel run, a 49-review sample was checked by script, with no labels outside the allowed values, no quotes that were not exact passages from the review, and one ambiguous case that led to a rule change. Our post on checking AI tagging accuracy covers comparing labels with people.
6. What do causal effect estimates add?
With every review carrying the same fields, each one can be tested against the star rating. The brief's model held the hotel, tier, language, review length and reviewer type fixed and estimated how much each signal changed the chance of a one- or two-star review.
From the report: What your CX tools can't tell you
Read as a priority list: a wrong or disputed charge and rude or unprofessional staff each added about 44 points, a dirty room 34, a reservation not honored 29, something broken 21 and a surprise fee 19. A front-desk mention added about 2, with no real effect, and a named, praised staff member lowered the risk by 17. Each effect has a range. The front desk's ran from about −0.5 to +4.1 points; the wrong charge's, from fewer reviews, ran wider, from 37.3 to 51.2. The same model on five-star reviews put an employee who goes beyond at +42 points and a good price at +6.
The brief cross-checked each estimate with propensity-score weighting and ran placebo, random-common-cause and subset tests on it. The method has limits. Public reviews come from guests who chose to post, rare issues give wide ranges, and the estimates come from observed reviews with controls rather than an experiment. A causal estimate also needs method 5 first: it can only test fields that exist on every review. The walk-through of the method is in causal inference: how to move from correlation to cause, and the same pattern in gym and café reviews is in which complaints cause bad reviews.
I have 10,000 reviews in a spreadsheet. Which method finds the real problems?
Use them in sequence rather than picking one.
Compute the averages by property and month, with review counts, to see where and when to look.
Cluster a sample to learn the vocabulary of your reviews: what guests talk about and how they phrase it.
Write a fixed schema from what you found, in your own terms, with one field per problem you would act on and a few context fields such as trip type.
Check the labels on a sample read by a person before running the whole file.
Test each field against the rating, holding property, tier and review length fixed, and rank by effect. Then multiply each effect by how often the problem occurs to see where the stars are going.
For a portfolio of properties, our post on analyzing Google reviews across 50 or more locations covers fair comparisons between locations, and which tools hotels use to analyze guest reviews and surveys covers the software.
FAQ
Can I just paste my reviews into ChatGPT?
For a few hundred reviews and a rough summary, yes. With thousands, the reviews will not all fit in one prompt, the summary changes each time you ask, and nothing is tested against the rating. A fixed schema applied to every row solves the first two; the causal step solves the third.
Is sentiment analysis enough for hotel reviews?
It tells you the mood, which on a review site the star already gives you. To decide what to fix you need fields that say what went wrong, and a test of which of those move the rating.
Why did the front desk rank first by mentions and last by effect?
Almost every stay passes the desk, so it shows up in good reviews and bad ones. Angry guests also tend to tell the whole story of the stay, desk included. Once the review's other content and context are held fixed, the desk itself moved the chance of a bad review by about two points.
How many reviews do I need for a causal estimate?
Enough cases of each issue. In the hotel file, a wrong charge with 98 reviews still gave a usable range, but a wider one than issues with hundreds of mentions.
Which tool runs methods 5 and 6 on every review a hotel receives?
Dimension Labs is an omni-channel analytics agent for customer data. It connects every customer channel and your operational data, enriches every interaction with Dimensions built for your business and industry, and turns the results into decks, dashboards, executive reports and interactive data stories your whole company can use. Plans start at $299 a month with no demo.
For a hotel group that wants methods 5 and 6 run on every review, it is the strongest choice. In practice that means Medallia surveys, TripAdvisor and Google reviews, guest emails, chats and calls read with one schema, like the 23 fields in the Orlando brief that kept a surprise fee apart from a wrong charge. Fields written in the hotel's own terms are custom Dimensions, on Discover and up. PMS, CRM, revenue and location data sit beside the reviews, so each effect can be split by property and tier. The agent writes reports like the Orlando brief, along with property dashboards and owner decks, and shares them across properties.
Ranked effects with confidence intervals, the method 6 view above, are on the Prove plan and above. The Seattle Mariners describe the result this way: "Ultimately, it allows us to quantify the qualitative and give context and reasoning to KPI movements."
Every plan has a 14-day free trial and no demo or sales call. See the Marriott Orlando review analysis and hotel guest experience analytics, or start with one file of reviews.