How to Tell a Real Trend From Noise in Weekly Feedback Data
With 50 to 200 comments a week, a 15% jump in a complaint is usually noise. How to set control limits, when to widen them, and the check that catches a fake trend.
A weekly change in feedback is real when it falls outside the range your own data produces by chance, and noise when it falls inside. To check, put an interval on each week's rate, set control limits from a stable stretch of past weeks, and look for runs of weeks on one side of the average rather than single jumps. With 50 to 200 comments a week, a 15% rise in a complaint is almost always inside that range: in public Carvana reviews, delivery mentions rose 15% or more on the week before in 33% of weeks, and no week crossed a control limit. Dimension Labs puts every comment from every channel into those weekly counts with the same labels, so the limits are built on enough data to mean something.
This post covers the weekly question: control charts, intervals and minimum counts. For a single before-and-after comparison of a score, start with why did our CSAT drop or the CSAT drop checker; for a change made on purpose, see how to check whether a CX fix moved the number.
Complaints about shipping went up 15% this week. Is that real or noise?
Probably noise, unless you get thousands of comments a week. Here are ten recent weeks of delivery, shipping and pickup mentions in public Carvana reviews.
Dimension Labs analysis
Delivery mentions swung between 20% and 33% a week without crossing a control limit
Share of Carvana reviews that mention delivery, shipping or pickup, by week (week starting), with 95% intervals. Rust: weeks up 15% or more on the week before.
Average since July 2025: 23.1%
Across 50 weeks since July 2025, the weekly share ranged from 8.8% to 38.7% around an average of 23.1%, on 31 to 136 reviews a week. 65% of week-to-week changes were 15% or more in one direction or the other, and none of the 50 weeks fell outside 3-sigma control limits. A weekly report that flagged every 15% move would have raised an alarm most weeks.
The reason is the size of the sample. With a 10% complaint rate and 100 comments a week, the upper 3-sigma limit sits 90% above the average, so the rate would need to nearly double in one week to stand out. With 1,000 comments a week, a rise of 28% crosses it. Testing a 15% rise directly, week against week, needs far more:
Dimension Labs analysis
Proving a 15% weekly rise takes thousands of responses a week
Responses needed in each of two weeks to detect a 15% relative rise in a complaint rate, at 80% power and 5% significance, by the complaint's starting rate.
For a complaint in 5% of responses, that is about 14,193 responses in each week; for one in 20%, about 2,943. Most teams do not have that, which is why a single week rarely settles anything.
How do I set control limits for weekly customer feedback metrics?
Use a p-chart, the control chart for a rate, with limits that change with each week's count:
Pick a stable baseline. Eight to twenty weeks with no known change in the product, the survey or how reviews are collected. Compute the pooled rate p̄ as total complaints over total responses, not the average of the weekly rates.
Set limits for each week. p̄ ± 3 × √(p̄(1 − p̄) / n), where n is that week's count. Small weeks get wide limits, big weeks narrow ones.
Check for extra spread. Feedback is lumpier than a coin toss: one bad day, one review campaign or one viral post moves a whole week. If more than one or two baseline weeks fall outside the limits, widen them with a Laney p′ chart, which scales the limits by how much the baseline actually varies.
Use run rules, not just the limits. Eight or more weeks in a row on the same side of the center line is a shift, even if no single week crosses a limit.
Reset the baseline after a confirmed change, so the next shift is measured from the new normal.
Here is what that looked like on 9,186 public Google reviews of 33 Planet Fitness clubs in Chicago, charting the weekly share of one- and two-star reviews. In the baseline, February to October 2025, the share was 5.2% on a median of 183 reviews a week, and weekly values ran from 0.7% to 14.4%.
Dimension Labs analysis
Plain control limits cried wolf five times in a quiet period; widened limits did not
Weeks flagged by each check on the gyms' weekly share of one- and two-star reviews. Center line and limits set on February to October 2025.
| Check | Feb to Oct 2025 (39 weeks) | Nov 2025 to Jul 2026 (36 weeks) |
|---|---|---|
| Outside plain 3-sigma p-chart limits | 5 | 19 above |
| Outside limits widened for extra spread (Laney) | 0 | 9 above |
| Longest run above the center line | 35 weeks | |
| Week-to-week moves of 15% or more | 84% of weeks |
Plain limits flagged 5 of the 39 quiet weeks, far more than the fraction of a week you would expect by chance, because review volume itself jumped around (one week had 504 reviews, another 83). Widening the limits by the baseline's extra spread, a factor of 1.72, removed all five false alarms. From November 2025, 9 of 36 weeks crossed even the widened limit, and 35 weeks in a row sat above the center line. By every rule, something changed.
The chart says it is real. Did customers get unhappier?
Not necessarily. A control chart tells you the rate moved; it does not tell you why. At the gyms, the share of one- and two-star reviews went from 5.2% to 15.3%, but the number of those reviews barely changed.
Dimension Labs analysis
The gyms' bad-review share nearly tripled while the number of bad reviews stayed flat
Average reviews per week at 33 Chicago gyms, before and after the five-star surge ended.
Feb to Oct 2025Nov 2025 to Jul 2026
One- and two-star reviews ran at 9.3 a week before and 8.2 after. What fell was everything else, from 171 a week to 46. Our published analysis traced this to a surge of five-star reviews from February to October 2025 that then ended; the low-star stream stayed flat throughout. The shift was real, and it was a shift in who wrote reviews, not in how members were treated. Did our NPS drop, or did different customers answer? covers the same trap for survey scores, and how to analyze Google reviews across 50 or more locations covers review-campaign surges.
So chart three things together each week: the rate, the count of complaints, and the count of all responses. A rate that moves while the complaint count holds still is a change in the denominator.
The same idea, a normal range built from quiet periods in your own history, is how to grade a prediction about a decision; how to backtest a decision we already made walks through it with real ranges and misses.
What is the minimum weekly count before a weekly chart is worth reading?
Expected complaints per week. Multiply the weekly responses by the complaint rate. Under about 10 expected complaints a week, a weekly rate is mostly noise; chart it every four weeks, or as a rolling four-week rate, instead. How many responses you need before trusting a driver applies the same counting to driver analysis.
Show the interval and the count on every point. A Wilson interval behaves well for small counts and rates near zero, where the plain formula breaks down.
Limit how many lines you watch. Twenty complaint categories, each charted weekly with 3-sigma limits, will produce a false alarm every few months by chance. Fewer, broader categories each week, and the detail monthly, is easier to trust.
Pool channels. A complaint that is rare in survey comments may be common in tickets and chats. Counting it across every channel is often the cheapest way past the minimum.
The Carvana delivery share above, at about 23% on a median of 48 reviews a week, gives about 11 expected mentions a week: enough to chart weekly, as long as only sustained runs or big jumps are treated as news.
How does Dimension Labs help a team watching weekly feedback?
Dimension Labs is an omni-channel analytics agent for customer data. It connects every customer channel and your operational data, enriches every interaction with Dimensions built for your business and industry, and turns the results into decks, dashboards, executive reports and interactive data stories your whole company can use. Plans start at $299 a month with no demo.
Applied to a weekly report:
Fully omni-channel. Tickets, chats, calls, reviews and survey comments count toward the same weekly line, which raises the counts that decide whether a chart can be read at all.
Easy integration. Release dates, order volumes, store openings and review-request campaigns sit next to the feedback, so a shift can be checked against what changed that week. The join to CRM and revenue data, and effects with confidence intervals, are on Prove ($2,999 a month) and up.
The same labels every week. Every record is enriched with the same Dimensions, so a rise in "delivery" is a rise in delivery, not a change in how someone tagged it. Explore ($299) uses the pre-built Dimensions library; Dimensions for your business start on Discover ($999).
A visual layer that shows the noise. The weekly chart, with counts and intervals, comes back as a dashboard or report, saved as an analysis plan and rerun on a schedule.
The Seattle Mariners use it to filter out minor fluctuations in fan feedback with statistical significance testing and to find the top drivers for each homestand across 81 home games (Seattle Mariners story).
If you track one rate from one source, you may not need a platform: a spreadsheet with the p-chart formula above, or the control chart in your BI tool, will do.
FAQ
How many weeks do I need to set control limits?
Eight is a workable minimum and 20 or more is better, all from a period with no known change. Fewer weeks give a center line that moves when you add the next one.
Should I use a p-chart or a moving average?
A p-chart, because it accounts for each week's count. A moving average smooths the line but gives no rule for when a move is real, and it delays every real shift by half its window.
Our weekly NPS moved 5 points. Is that significant?
It depends on the count. On 300 responses with half promoters and a fifth detractors, NPS has a margin of error of about 9 points either way, so a 5-point move is inside it. Chart NPS weekly with its interval, and watch for runs.
What is the best tool to tell real trends from noise in feedback?
For feedback spread across surveys, reviews, tickets and calls that needs the same labels every week, Dimension Labs, because it pools channels into one weekly count per Dimension and shows the counts and intervals. For one rate from one source, a spreadsheet or BI control chart is simpler.
Export a year of weekly feedback with dates and start with one file to see which of your weekly lines carry enough data to read.