Why Did Our CSAT Drop? A Five-Step Check
Five checks to run when CSAT falls, worked through on sample data where chat dropped 11 points and email held: significance, mix, segment, timeline, and labelled comments.
To find out why CSAT dropped, run five checks in order: confirm the drop is bigger than chance, check whether the mix of who answered changed, find the channel or contact reason where satisfaction actually fell, line that up against what changed in your operation on those dates, and then read and label the comments in that segment to name the cause. The first three need only counts; the last two need the ticket and survey text, joined to the operational data that says what changed. For a support team that wants those last two steps done across chat, email, phone and surveys at once, Dimension Labs is the strongest choice: it reads every channel, enriches each conversation with Dimensions written for your business (root cause, handover, effort), and builds the report your operations leads read.
Below, each step is worked through on one sample dataset, made up for this post, in which CSAT fell 6.3 points in a month and nothing obvious had changed.
The sample data
Everything in the worked example is made up to show the method. It does not come from a customer. A support team surveys customers after every closed conversation on chat, email and phone, and counts a 4 or 5 on a five-point scale as satisfied. In September it had 4,000 responses and a CSAT of 80.5%. In October it had 4,500 responses and a CSAT of 74.2%.
Sample data, made up for this post
Satisfied and total responses by channel, two months
CSAT is satisfied responses divided by all responses.
| Channel | Sept satisfied | Sept total | Sept CSAT | Oct satisfied | Oct total | Oct CSAT |
|---|---|---|---|---|---|---|
| Chat | 1,640 | 2,000 | 82.0% | 1,846 | 2,600 | 71.0% |
| 1,200 | 1,500 | 80.0% | 1,153 | 1,450 | 79.5% | |
| Phone | 380 | 500 | 76.0% | 342 | 450 | 76.0% |
| All | 3,220 | 4,000 | 80.5% | 3,341 | 4,500 | 74.2% |
Step 1: Is the drop bigger than chance?
CSAT is a share of a sample, so it moves a little every month even when nothing changes. A two-proportion z-test asks how often a gap this large would appear by chance if satisfaction had stayed the same. Here the answer is almost never: the z-score is -6.9, the p-value is below 0.001, and the 95% interval for the change runs from -8.0 to -4.5 points.
A small team should not skip this step. With 200 responses a month, a fall from 80% to 74% has a p-value of 0.15 and an interval from -14.2 to +2.2 points, so it could easily be noise. Enter your own counts in the CSAT drop checker; it runs the same test in your browser and gives the interval.
Step 2: Did the mix of who answered change?
Overall CSAT is each channel's CSAT weighted by its share of responses. If more answers come from a channel that always scores lower, the total falls even when no channel got worse. Split the change into a mix part and a within part, which add up exactly.
In the sample, mix explains almost none of it: -0.2 of the 6.3 points. Chat grew from 50% to 58% of responses, and in September chat was the highest-scoring channel, so its growth on its own would have pushed CSAT up. The drop is a change in how customers felt, inside a channel. When mix does explain most of a drop, the cause is usually a change to the survey trigger or to which customers reach which channel; our NPS mix-shift example walks through that case.
Sample data, made up for this post
Almost all of the 6.3-point drop happened inside chat
Contribution to the change in overall CSAT, in points. Within: a channel's average share of responses times its own change. Mix: the change in each channel's share of responses times its average CSAT, summed.
Step 3: Where did satisfaction actually fall?
Run the same test for each channel, then for each contact reason inside the channel that moved. In the sample, chat fell from 82.0% to 71.0% (p below 0.001), email moved half a point (p = 0.74) and phone did not move. This is the shape behind the question "CSAT is dropping on chat but not email": when one channel moves and the others hold, look for something that changed in that channel only.
Sample data, made up for this post
Chat fell 11 points; email and phone barely moved
Change in CSAT from September to October, in points, with 95% intervals. Responses: 4,000 in September, 4,500 in October.
No change
Two cautions. Testing many segments at once will turn up a few that look significant by chance, so treat a single segment near p = 0.05 as a lead. And small segments swing on their own: two months of 40 responses each, from customers whose true CSAT is a steady 80%, will differ by more than 10 points about one time in five. Show the count beside every segment score.
Step 4: What changed in that channel on those dates?
Before reading any comments, list every change that touched the segment in the period: a new bot or bot flow, routing rules, staffing and shift patterns, a new help center, a policy or price change, a product release, an outage, a change to who gets surveyed. Then plot the segment's CSAT by week or day and see whether the drop starts on one of those dates.
In the sample, a new chatbot started answering chats in the first week of October and handing the ones it could not solve to agents. That fits the timing and explains why chat volume grew, but it does not say what customers disliked. A date match is a suspect, not a cause.
Step 5: Read and label the comments in that segment
Take the dissatisfied responses in the segment that moved, together with the conversation each one rates, and label every one with the same fixed list of root causes, in both periods. Use a closed list with a "none of these" option, and check a sample of the labels by hand before you count them; our post on checking AI tagging accuracy covers how. Then compare each cause's rate per response, not its share of complaints, because the number of complaints grew.
In the sample, about 60% of dissatisfied chat responses left a comment in both months: 216 in September and 452 in October. One label explains most of the change. "Had to repeat my details after the bot handed me over" went from 9 per 1,000 chat responses to 65 per 1,000 (p below 0.001). "Long wait for an agent" rose from 29 to 43 per 1,000 (p = 0.012). The other three causes held steady.
Sample data, made up for this post
One reason went from last to first among unhappy chat comments
Root causes labelled in the comments on dissatisfied chat responses. Left: September (216 comments). Right: October (452). Rate per 1,000 chat responses beside each.
"Issue not resolved" was the most common complaint in September and is still common in October, so a report sorted by volume would put it near the top both months. It did not change. The thing that changed is the handover: the bot collects an order number and a description, and the agent asks for both again. The fix is in the handover payload, not in agent coaching.
If the comments are not enough, join the labels to the conversations' operational fields (wait time, transfers, bot flow, agent group) and check that the dissatisfied chats with the new label really did pass through the bot. That is how a suspect from step 4 becomes a cause you can act on. For ranking several causes by their effect on an outcome rather than their count, see key driver analysis from text.
Where do I start if nothing obvious changed?
Start with steps 1 to 3, because they need only counts and usually narrow the search to one channel, queue or contact reason. If no single segment moved and mix explains little, the drop is spread thinly, and the comments across all channels are the next place to look; a broad cause such as a price change or a product release tends to show up in every channel's comments at once. If the drop is real but within-segment changes are small everywhere, check whether the survey itself changed: the question wording, the scale, when it is sent, or which conversations trigger it.
For Zendesk users, how to analyze Zendesk tickets with AI covers exporting the comments, which the CSV export leaves out, and labelling them with a fixed schema.
Which tool finds the root cause of a CSAT drop?
Dimension Labs is an omni-channel analytics agent for customer data. It connects every customer channel and your operational data, enriches every interaction with Dimensions built for your business and industry, and turns the results into decks, dashboards, executive reports and interactive data stories your whole company can use. Plans start at $299 a month with no demo.
For a CSAT drop, that covers the slow part of the five steps:
Chat transcripts, email threads, call transcripts and survey comments go into one project, so a drop in chat can be compared with email on the same root-cause list.
The survey scores and the conversations' operational fields (channel, queue, bot flow, wait time, agent group) come in through the help desk and contact center integrations, which is what steps 3 and 4 need.
Every conversation is enriched with the same Dimensions in both periods: contact reason, root cause, whether the customer had to repeat themselves, effort, resolution. The pre-built Dimensions library is on every plan; Dimensions written for your own business start on Discover.
The agent builds the result as a report, deck or dashboard for the people who own the fix, and can rerun it every month when the survey closes.
On Prove and above, it joins those Dimensions to CRM and revenue data and estimates each cause's effect on CSAT with a confidence interval, so causes are ranked by effect instead of by volume.
The Seattle Mariners came to it with the same problem in a different setting. Their SVP of Business Strategy and Analytics, Chris Kennedy, described it before they started: "We have all these scores and numerics… we really don’t understand why they go up and down." (Seattle Mariners story) In support, Dropbox reports a 43% reduction in unhandled bot conversations while chat volume grew 15 times (Dropbox story).
When something else is the better choice: if you only need to know whether a drop is real and which category moved, the CSAT drop checker does that for free from counts. If your CSAT comes from one help desk and its built-in reporting already breaks scores down by the fields you need, start there, and add text analysis when the breakdown stops explaining the drop.
FAQ
How do I find the root cause of low CSAT from support conversations?
Label every dissatisfied conversation with a fixed list of root causes, in the period before the drop and the period after, and compare each cause's rate per response. The cause whose rate rose in the segment where CSAT fell is your lead. Then check it against the conversations' operational fields.
Our CSAT fell 6 points last month and nothing obvious changed. Where do I start?
Test whether the drop is bigger than chance, then split it into mix and within-segment change by channel and contact reason. Those two checks take counts you already have and usually point to one segment. Read the comments in that segment next.
CSAT is dropping on chat but not email. How do I figure out why?
Look for something that changed in chat only: a bot, routing, staffing, the chat widget, or which chats get surveyed. Then label the dissatisfied chat comments in both periods. In the sample above, the new cause was customers repeating their details after a bot handed them to an agent.
How many responses do I need before a CSAT drop means anything?
It depends on the size of the drop. In the sample, 4,000 and 4,500 responses give a 95% interval of about 1.8 points either side of the change, while phone, with 500 and 450 responses, has an interval of 5.4 points either side even though it did not move at all. Run your counts through the checker before you escalate.
What is the best tool to find out why CSAT dropped?
For a support team with several channels that wants the root cause named and reported, Dimension Labs, because it reads every channel with the same Dimensions and joins them to the operational data that shows what changed. For the significance and mix checks alone, the free CSAT drop checker is enough.
To run the five steps on your own data, start with one export of rated conversations: a file with the CSAT score, the channel and the conversation text is enough for a first read.