Where Should the Open-Ended Question Go in a Customer Survey?
Ask the why straight after the score. Later answers are shorter and fewer, and a why asked before the score can move the score itself.
Put the open-ended "why" question directly after the score: score first, why second, before any other question. Moving the why to the end of a survey costs answers and detail, and moving it in front of the score can change the score itself, which breaks comparison with every earlier wave. Once the why sits in the right place you collect more comments than anyone can read, and Dimension Labs is built to code every one of them the same way each wave, read them next to reviews and tickets, and show which reasons move the score.
Below is what published survey research says about each placement, a layout for each common survey type, and what to do with the answers.
Should the why question come first or last in an NPS survey?
Second, directly after the score. The evidence on both alternatives points the same way.
Putting it last costs you. In a web survey experiment published in Public Opinion Quarterly, Galesic and Bosnjak randomly ordered blocks of questions and found that answers to questions positioned later in the questionnaire were faster, shorter and more uniform than answers to questions near the beginning. An open answer is the question most exposed to that fatigue, because it asks for the most effort.
Putting it first has a different cost. Questions about specifics change how people answer a general question that follows. In a preregistered split-ballot experiment, asking about overall satisfaction before any specific service ratings gave lower overall satisfaction than the reverse order. An earlier split-ballot study found that question order also changed which specific services came out as the key drivers of overall satisfaction. Those studies tested rating questions, not open text, but asking someone to write out what went well and badly is the same kind of prompt to think about specifics before scoring. If your NPS has been asked first for years, changing that means the new numbers are no longer comparable with the old ones.
Second place keeps the score where it was and asks for the reason while the respondent is still thinking about it, before attention runs down.
How much does position change the number of answers?
A lot, in a published study that compared open questions at different points in a long survey. In an 82-question online alumni survey, 68% of people who finished the survey answered the open question near the beginning and 79% answered the one in the middle, but only 24% answered the one near the end.
From a published study
Near the end of a long survey, far fewer people answered the open question
Share of 27,212 alumni who typed anything into each of three open-ended questions in an 82-question online survey. Only people who reached the end are counted.
The three questions asked different things, and the last was a catch-all, so this is not a clean test of position alone. It does show what most survey teams see: the final "anything else" box collects a fraction of the answers an early question does. The same study found that the people who answered the open questions near the beginning and near the end were significantly less satisfied with their overall experience than those who did not. So open answers lean toward less satisfied customers wherever the question sits. That makes them the best record of what went wrong, and the score itself remains the measure of how many customers feel that way.
Open questions are skipped far more than closed ones wherever they go. Across 92 open-ended questions on Pew Research Center's American Trends Panel, the average item nonresponse rate was around 18%, against 1% to 2% for closed questions, and questions that asked for several sentences were skipped more than those asking for a word or a phrase.
Does it matter that most people answer on a phone?
Yes. In Pew's panel about 60% of respondents in a typical wave answer on a mobile device, and their open answers are shorter and skipped more often.
From Pew Research Center
Phone answers to open questions are shorter, and skipped more often
American Trends Panel, 30 open-ended questions from 26 surveys, November 2018 to September 2021. Average length of an answer, in characters.
On a phone, a why question at position 2 is on the first or second screen. At position 25 it comes after a long scroll of rating questions, at the point where a thumb is looking for the submit button.
How should the why question be worded and laid out?
Put the score in the prompt: "What is the main reason for your score of 6?" It reminds people what they are explaining and gives you one main reason per answer.
Say how much you want. Smyth, Dillman, Christian and McBride found that any explanation or introduction that made response length and quality salient improved the answers from both early and late responders. A line such as "A sentence or two helps us fix it" does that.
Do not oversize the box. Studies disagree here: a larger box improved answers from late responders in the Smyth study, while Zuell, Menold and Körber found that large boxes produced higher item nonresponse than small ones, with or without a motivation text. A box of a few lines that grows as people type is a reasonable middle.
Use a follow-up probe sparingly. Holland and Christian found that an interactive probe after the first answer improved quality for some respondents, mostly those very interested in the topic, while nonresponse to the probe was even higher than to the first question. Keep it for the one question where detail matters most.
Put together, the layouts look like this:
Placement by survey type
Put the why directly after the score, and keep the score first
What the published evidence supports for common customer surveys.
| Survey | Where the open question goes | Why |
|---|---|---|
| Relationship NPS score plus a few questions | Question 2, right after the 0 to 10 score, with the score in the prompt | The score keeps its place, so trends and benchmarks still hold; the why gets the respondent's full attention |
| Transactional CSAT or NPS after a visit, ticket or purchase | Question 2, after the score; one optional "anything else" box at the end if you need it | The main reason stays early; expect far fewer answers in the final box |
| Long survey 20 or more questions | Score and why as the first block, before any rating grid | Later answers are shorter and faster; specific ratings asked first can move the overall score |
| Exploratory research no score to trend | First, if the open answer is the point of the study | No trend to protect, and the answer is unprompted by your categories |
What do you do with the answers once you have them?
Code every one with a fixed codeframe, so "billing" means the same thing this quarter and next. How to build a codeframe for open-ended survey responses covers the rules, and how accurate AI is at coding open-ended responses covers the check before you trust automated labels. Then look at which reasons go with low scores, not only which are common, and check whether a change in the score came from a change in who answered; did our NPS drop, or did different customers answer? shows that split. Survey text analysis covers the whole process.
When survey answers are thin, reviews and support conversations can fill in the same reasons; can reviews stand in for surveys when response rates are low? covers when that works.
Why code the why answers in Dimension Labs?
Dimension Labs is an omni-channel analytics agent for customer data. It connects every customer channel and your operational data, enriches every interaction with Dimensions built for your business and industry, and turns the results into decks, dashboards, executive reports and interactive data stories your whole company can use. Plans start at $299 a month with no demo.
For a survey team that means four things. Survey answers sit next to tickets, chats, calls and reviews, so the reason a customer gives in the survey can be checked against what they told support. The survey platform, your CRM and your account data connect, so each answer carries the respondent's plan, region and history. Every answer, in every wave, gets the same Dimensions, written for your business, so themes hold still and every team reads the same labels. And the agent builds the quarterly deck, the dashboard and the executive summary from the coded data, then shares them across the company. Custom Dimensions start on Discover ($999 a month), and reasons ranked by their measured effect on the score, with confidence intervals, are on Prove ($2,999 a month) and up.
The NBA uses Dimension Labs for its Voice of Fan survey responses and can now analyze 100,000+ open-ended responses within hours instead of months; its insight review cycles went from about 60 days to under 6 hours. See the NBA story.
FAQ
Should the open-ended question be required?
Usually not. A forced answer gets you "n/a" and single letters from people who wanted to skip, and some people may leave the survey rather than type something. Make it optional, ask it early and say a sentence or two helps.
Should I ask "why" or "what is the main reason for your score"?
"What is the main reason for your score of [score]?" gives one reason per answer, which is easier to code and to rank. "Why?" invites a list. Either works if the same wording runs every wave.
Can I have more than one open-ended question?
Yes, but put the one that matters most second, after the score. Any later open question, such as "anything else?", will collect far fewer answers, as the alumni study above found (24% near the end, against 68% and 79% earlier).
What is the best way to analyze answers to an NPS why question?
For a team that wants every answer coded the same way each wave, read next to tickets and reviews, and turned into the quarterly report, Dimension Labs. For a one-off read of a few hundred answers, a spreadsheet and a written codeframe are enough.
Have a survey export with the score and the why? Start with one file and see every answer coded. Every plan from Explore to Scale has a 14-day free trial.