Did Our NPS Drop, or Did Different Customers Answer?
NPS can fall when no group of customers got less happy, because more answers came from a group that always scores low. Here is a worked example, the Python to split the change, and a checklist for your own survey.
Yes, NPS can fall without any group of customers getting less happy, if the mix of who answered the survey changes. NPS is one percentage taken across everyone who responded, so when more answers come from a group that always scores low, such as customers who just contacted support, the total drops even if every group's own score holds steady.
Below is a worked example on a clearly labelled illustrative dataset, the Python to split a change into "who answered" and "how they felt", and a checklist for your own survey data.
How a change in respondents moves the score
On the standard 0 to 10 question, promoters answer 9 or 10, passives 7 or 8 and detractors 0 to 6, and NPS is the percentage of promoters minus the percentage of detractors. Because it is computed over the whole pool, the overall NPS equals each segment's NPS weighted by that segment's share of responses. Change the weights and the total moves.
Segments that score very differently are common: customers surveyed after a support ticket against customers surveyed in the product, one region against another, new accounts against long-standing ones. When one of those groups grows in the response pool, because a survey trigger changed, a channel was added, or one region was emailed and another was not, the headline number shifts. In its extreme form, where every group moves one way and the total moves the other, this is Simpson's paradox.
A worked example (illustrative data)
The numbers in this section are made up to show the method. They do not come from a customer. A software company runs NPS through three channels. In Q3 it starts sending the survey after every closed ticket instead of a sample and, to avoid over-surveying, shows the in-app prompt less often. Each quarter collects 5,000 responses.
| Channel | Q2 responses | Q2 share | Q2 NPS | Q3 responses | Q3 share | Q3 NPS |
|---|---|---|---|---|---|---|
| In-app survey | 3,000 | 60% | 50 | 1,800 | 36% | 50 |
| Email after renewal | 1,500 | 30% | 30 | 1,500 | 30% | 28 |
| After a support ticket | 500 | 10% | -10 | 1,700 | 34% | -10 |
| All responses | 5,000 | 100% | 38 | 5,000 | 100% | 23 |
The headline fell 15 points, from 38 to 23. Two of the three channels did not move at all, and the third slipped 2 points. The drop comes from the support channel growing from a tenth of responses to about a third.
Illustrative data
The support channel went from a tenth of responses to about a third
Share of the 5,000 responses each quarter, by survey channel. Channel NPS beside each label. Made-up data for this post.
Q2 shareQ3 share
Splitting the change: who answered vs how they felt
A shift-share split divides the change into two parts that add up exactly. The mix part is each segment's change in share times its average NPS across the two quarters. The within part is each segment's average share times its change in NPS. Using the average of both quarters keeps the split symmetric, so it does not matter which quarter you start from.
Here, the change in who answered accounts for -14.4 points and the change within groups for -0.6. Holding the Q2 mix fixed, Q3's NPS would have been 37.4, almost exactly where it started.
Illustrative data
Of the 15-point drop, 14.4 points came from who answered
NPS from Q2 to Q3, split into the change in response mix and the change in scores within each channel. Made-up data for this post.
NPS
answered
groups
NPS
The whole calculation is a few lines of pandas:
import pandas as pd
# one row per segment and quarter: promoters, passives, detractors
df["n"] = df.prom + df.pas + df.det
df["nps"] = 100 * (df.prom - df.det) / df.n
df["share"] = df.n / df.groupby("quarter").n.transform("sum")
a = df[df.quarter == "Q2"].set_index("segment")
b = df[df.quarter == "Q3"].set_index("segment")
mix = ((b.share - a.share) * (a.nps + b.nps) / 2).sum() # -14.4
within = (((a.share + b.share) / 2) * (b.nps - a.nps)).sum() # -0.6
q3_at_q2_mix = (a.share * b.nps).sum() # 37.4A mix shift is not always a measurement artifact. If more customers answered from the support channel because more customers needed support, the mix shift is itself the news, and the next question is why ticket volume rose. The split tells you which question to ask: "what changed in the survey program?" or "what changed for customers?"
Product loop
Ask the question in plain English and get the chart
The same kind of question, asked of survey data joined to customer records.
How to check your own data
Export responses with the fields that define who answered. Survey trigger or channel, region, plan, tenure and whether the customer had a recent ticket. Keep the date the survey was sent as well as the date it was answered.
Compute NPS and response share per segment per period. Look for any segment whose share moved by more than a few points.
Run the split for each segmentation you care about. Channel, then region, then plan. A drop that is mostly mix on one cut and mostly within on another tells you where it lives.
Report a fixed-mix NPS next to the headline. Reweight each period to a reference mix (last year's, or your customer base's) so a change in the survey program does not read as a change in loyalty.
Check segment sizes. A segment NPS on a few dozen responses can move several points by chance. Show the count next to every segment score.
Read the comments in the segment that moved. If within-segment change is real, the open-text answers in that segment say why. Our post on survey text analysis covers how to tag them at scale.
FAQ
What is a mix shift in NPS?
A mix shift is a change in the share of responses coming from each group of customers. Because NPS is computed across all responses, a larger share from a low-scoring group lowers the total even if no group changed its scores.
Should I reweight NPS?
Report both. The raw number is what customers who answered said. A fixed-mix number tells you whether loyalty within each group changed. When they disagree, the survey program or the customer mix changed.
Which segments should I check first?
Start with anything that changes how or when the survey is sent: trigger, channel and region. Then customer traits that usually score differently, like tenure, plan and recent support contact.
Dimension Labs joins every survey response, with its comments tagged into Dimensions, to your CRM, billing and support data, so a question like "did NPS drop, or did different customers answer?" gets a chart and a split in one step, and the agent can rerun it every time the survey closes. See customer intelligence and why dashboards don't answer why, or start with one file of survey responses.