Do Customers Do What They Say? Checking Survey Intentions Against Behavior
Stated intentions run ahead of behavior: in one survey about half as many households bought a car as said they would. When surveys predict well, and a five-step check against what customers did.
Customers do what they say in surveys only in part, and the gap runs in a known direction: they say they will buy more often than they do. In one published comparison, 9.87% of households said they intended to buy a car within a year and 5.01% bought one, about half as many. The answers are still useful once they are checked against what the same customers did next, and for a CX or insights lead whose customers also talk in reviews, chats and calls, Dimension Labs runs that check across every channel: it joins what customers said in every channel to the orders, renewals and cancellations that followed.
Below: what sixty years of research says about stated intentions, when they predict well, why asking the question changes the answer, and a five-step check you can run on your own survey. For judging any customer prediction, not only surveys, see how accurate are customer behavior predictions?
Do customers do what they say in surveys?
Less often than they say, and more for some questions than others. Vicki Morwitz's 2014 review of sixty years of research on purchase intentions concludes that intentions are correlated with sales and predict them, but imperfectly. The household surveys in the chart below show the usual shape: more people state an intention than act on it within the period they named.
Published survey data
About half as many households bought a car as said they would
Share of households saying they intended to buy, and share that bought in the same period, from two U.S. intention surveys with purchases tracked afterwards.
Said they intended to buyBought
The computer rows show where the gap shrinks. For a purchase within six months, the share who bought came close to the share who said they would, 89 per 100; for seven to twelve months out, it was 39 per 100. Near-term intentions held up far better than distant ones.
How accurate are purchase intent surveys?
Accurate enough to beat the obvious alternative, and not accurate enough to read at face value. Armstrong, Morwitz and Kumar tested four ways of forecasting sales from intentions on four data sets, French and U.S. car sales and U.S. wireless services. Every intentions-based method was more accurate than extrapolating past sales, and combining the methods gave error rates about a third lower than the extrapolation. So a survey carries real information about what customers will do. A company with its own purchase records can go a step further and adjust the answers with what similar customers did before.
When do purchase intentions predict sales?
Morwitz, Steckel and Gupta compared intention studies across many real-world settings and found six conditions under which intentions and purchases line up more closely.
When intentions predict
Intentions track purchases better in six conditions
Findings of Morwitz, Steckel and Gupta from intention studies across a wide range of real-world settings.
| Intentions match purchases more closely | than |
|---|---|
| for existing products | for new ones |
| for durable goods | for non-durable goods |
| over short time horizons | over long ones |
| when people name a specific brand or model | when they answer for a whole category |
| when purchase is measured as trial | when it is measured as total market sales |
| when options are asked about side by side | when each is asked about alone |
Read against a typical pre-decision survey, the list is a warning label. "Would you still visit if we raised prices?" asks about an existing product, which helps, but over an open time horizon and about a whole category of behavior, which hurts. Asking about a named option, side by side with the alternative, and over a defined period gets you closer to what customers will do.
Does asking customers about their plans change what they do?
Yes, and that complicates any check run only on survey responders. Morwitz, Johnson and Schmittlein found that simply asking people once about their intention to buy raised their later purchase rate, while asking low-intent people repeatedly lowered it. Chandon, Morwitz and Reinartz then measured how much the survey itself flatters the result: across three large field studies, the link between intentions and purchases was 58% stronger among surveyed customers than among similar customers who were never asked.
The practical consequence: a survey validated only on the people who answered it will look more predictive than it is for the whole customer base. The fix is to compare responders with customers who were not surveyed, and to read what those customers said without being asked, in reviews, support chats and calls. Unprompted words were not shaped by any question, which is why they make a good second source next to the survey.
How do I check survey answers against what customers actually did?
Join each answer to the outcome it was about, count the share who followed through at each answer level, and use that table to adjust the next survey. The five steps:
The check, step by step
Checking survey answers against behavior takes one join and one table
What you need at each step to compare what customers said with what they did, and what comes out.
| Step | What you need | What comes out |
|---|---|---|
| 1 Link answers to customers | Survey responses with a customer ID or email | Each answer attached to a person |
| 2 Fix the window | A date range that matches the question, such as 'in the next 6 months' | One yes or no outcome per person |
| 3 Count by answer level | Orders, renewals or cancellations from your own systems | For each answer, the share who did it: your calibration table |
| 4 Add who never answered | The same outcome for customers who were not surveyed | Whether responders behave like everyone else |
| 5 Read the gaps | Comments from people whose answer and behavior differ | The reasons intent did not turn into action |
Link answers to customers. Use the customer ID, email or loyalty number the survey already collects. Anonymous surveys cannot be checked this way, which is a reason to keep at least one identified wave.
Match the window to the question. If the question said "in the next six months", count behavior in those six months only. A loose window flatters the survey.
Build the calibration table. For each answer, from "definitely will" to "definitely will not", the share who bought, renewed or cancelled. That table converts the next survey's answers into an expected count, and it corrects for the upward bias in stated intentions that the research keeps finding.
Add the customers you did not survey. Their behavior over the same window shows whether responders are typical. If they are not, weight the survey to the whole base before using it.
Read why answers and behavior differ. Customers who said they would leave and stayed, or said they would stay and left, often explain themselves in later tickets, chats or reviews. Those comments tell you which survey questions to rewrite.
Run the table once per question type and keep it. Each survey after that comes with its own correction, and differences between waves become easier to trust; did our NPS drop, or did different customers answer? covers the related check on who responded.
What does Dimension Labs add when survey answers have to match behavior?
Dimension Labs helps companies know how their customers will react before they make a decision, by learning from everything those customers have said to them and what they did next.
For an insights lead, the check above depends on four things the platform does together. It is fully omni-channel: survey answers, reviews, tickets, chats, calls and emails are read in one place, so the unprompted comments sit next to the survey. It integrates those channels with operational data, so each answer is joined to the order, renewal or cancellation it predicted; the join to CRM and revenue data and effects with confidence intervals are on Prove ($2,999 a month) and up. It enriches every response and conversation with Dimensions custom to your business, such as stated intent, reason and timing, so "thinking about cancelling" means the same thing in a survey comment and a chat; custom Dimensions start on Discover ($999 a month). And its visual analytics layer turns the calibration table and the gaps behind it into dashboards, decks, PDF and executive reports, or an interactive data story the rest of the company can open.
The NBA uses Dimension Labs to enrich fan survey responses with predictive dimensions tied to fan behavior and value, and to find churn signals and upsell indicators in fans' own words; its insight review cycles went from about 60 days to under 6 hours. See the NBA story.
When your survey tool is enough: if you only need the answers to one survey and have no way to link them to customer records, the survey platform's own reports will do. Dimension Labs earns its place when the survey is one of several channels and the question that matters is what customers did next. Kept over time, that record of what customers said and did is a customer world model, and it is also the test any synthetic panel has to pass; are synthetic customers accurate? shows why.
FAQ
Do customers do what they say in surveys?
Partly. In published household surveys, about half as many people bought a car within a year as said they would. Intentions still predict behavior, imperfectly, and better over short horizons and for specific products.
How accurate are purchase intent surveys?
More accurate than extrapolating past sales, in a test across four data sets, and best when adjusted and combined: combined intentions forecasts had errors about a third lower than extrapolation. Raw answers overstate purchases, so calibrate them on past behavior.
How do I check survey answers against what customers actually did?
Link each response to a customer ID, count the share who acted within the period the question named at each answer level, compare with customers you did not survey, and read the comments of people whose answers and actions differ.
Why does asking customers change their behavior?
The question itself acts on the customer. Asking once raised later purchase rates in Morwitz, Johnson and Schmittlein's study, and the intention-purchase link was 58% stronger among surveyed customers than among similar ones who were not asked.
What is the best way to check whether customers did what they said?
For a company with surveys plus reviews, tickets and chats, Dimension Labs: it reads every channel with the same labels and, on Prove, joins answers to orders, renewals and cancellations with ranges. For one identified survey and one outcome, a spreadsheet join will do.
Have a past survey with customer IDs? Export it with the outcome that followed, such as renewals or orders, and start with one file to see how closely your customers' answers matched what they did. Each self-serve plan starts with a 14-day free trial.