Replacing Phone Support With AI: How Will Customers React?

When a company replaces phone support with an AI agent, customers judge it first on one thing: whether they can still reach a person when the agent cannot help. In a Gartner survey of 5,728 customers, 64% said they would prefer companies did not use AI for customer service, and their top worry was that a person would become harder to reach. The rollouts that keep customers are the ones measured conversation by conversation, and for a company building or running its own AI agent, Dimension Labs is built for that measurement: it reads every bot, chat and call transcript with the same labels and joins them to what customers did next, which is how Intuit QuickBooks cut human escalations by 57% on its chatbot.

This page covers how customers react, what public reviews and two well-reported reversals show, and how to roll an agent out so you can predict the reaction before it reaches every customer. For the metrics of the agent itself, see how to check whether your AI agent really resolved conversations and AI agent analytics.

If we replace phone support with an AI agent, how will customers react?

Most customers say they would rather you did not, and many say it could make them leave: 53% in the same Gartner survey would consider switching to a competitor if they found out a company was going to use AI for customer service. Stated intentions overstate what people do, but they tell you where the anger will land. Gartner's own reading is that the fear is about access, followed by worries about jobs and wrong answers, and its advice is a seamless handoff to a person.

Public reviews show the same thing in customers' own words. In 5,183 public Carvana reviews that carry a one-to-five star rating, on Trustpilot, BBB, Google Play and the App Store, 111 mention a bot, AI or automated support, and 86% of those are one or two stars, against 51% of the rest. Reading them, the complaint is rarely that a bot exists. 34% describe trying to get through to a real or live person, and 18% say the bot was useless or went in circles. The few positive mentions praise round-the-clock bot help with people behind it.

So the reaction you should plan for is to the dead end, not to the agent. An agent that answers the questions it can and hands the rest to a person, with the context, is judged very differently from a phone line that no longer has a way out.

Will moving support to AI hurt retention?

It can, and two large companies have said so in public. Klarna's CEO said in May 2025 that cost had weighed too heavily in how it organized service and that the result was lower quality, and promised that "there will be always a human if you want." Commonwealth Bank of Australia cut 45 call centre roles in July 2025 after rolling out an AI voice bot, then reversed the cuts in August and apologised. It had said the bot cut call volumes by 2,000 a week; it later conceded calls had risen, with overtime and team leaders taken onto the phones.

Both reversals have the same shape: the saving was counted early, from the agent's own numbers, and the cost showed up later, in repeat calls and in what customers and staff said. Neither case tells you what would happen to your customers, because retention depends on which questions you automate and who asks them. What you can measure, before and during the rollout, is the chain that leads to churn: the reasons customers contact you, whether the agent resolved each one, whether the same customer came back, and whether customers who met the agent renewed, reordered or cancelled at a different rate from comparable customers who did not. Joining conversations to renewals, orders and cancellations is the Causal Join on the Prove plan, and it turns "customers seem unhappy with the bot" into a number of customers and dollars. Which support topics drive churn shows that join on tickets and renewals.

How do we roll out an AI agent without losing customers?

Treat the rollout as a decision you can test in stages, not a switch. In order:

1. Record the baseline before anything changes. Contact reasons by channel, resolution, repeat contacts within seven days, satisfaction, and churn or reorder rates by contact reason. Without it, every later number is a guess.

2. Move intents, not channels. Start with the questions the agent resolves well, such as order status or password resets, and keep the complicated, emotional or high-value ones with people.

3. Keep a visible path to a person. Gartner's advice and the Carvana reviews point the same way: the handoff, with the conversation passed along, is what customers grade you on.

4. Audit what the agent calls resolved. Many agents count a conversation as resolved when the customer stops replying. Score a random sample against a rubric; the audit method and how to measure why customers escalate cover the steps.

5. Compare customers who met the agent with comparable customers who did not. A staged rollout by region, intent or customer tier gives you a natural control group. Track repeat contacts, satisfaction and churn for both.

6. Expand only where the ranges hold. Each stage gives you a measured effect with a range. Move the next set of intents when the range for the last one says customers stayed.

Intuit QuickBooks worked this way on its QuickBooks Assistant: every session across 50+ bots was enriched with its intent, fallback driver, escalation trigger and outcome, so the team could see which flows sent people to a human and fix them first. Fallback intents fell 35.3% and human escalations 57%, while volume kept growing.

The same data answers the cost question. How to calculate the ROI of a customer service AI agent shows how much the audited resolution rate, rather than the reported one, changes the business case.

Can we predict the reaction before the agent reaches every customer?

Yes, as a range, from your own history. The first stage of a rollout is a past decision by the time you plan the second: you know which intents moved, which customers met the agent and what they did next. Learn from that stage, predict the next one, and check whether the range held when it lands. If the relationship holds, you know the expected change in repeat contacts and churn for the next set of intents before you move them; if it misses, you find out on a fraction of customers rather than all of them, and you learn which kind of question the agent should not take. How to test a business decision before you make it and how to backtest a decision you already made go through the method.

Asking simulated customers whether they would mind is no substitute: research on AI personas finds they get the direction of a reaction right more often than its size (are synthetic customers accurate?). Your customers' own conversations, joined to what they did after earlier changes, are the better evidence, and keeping that evidence together for every decision is what a customer world model is.

Why measure an AI agent rollout with Dimension Labs?

Dimension Labs helps companies know how their customers will react before they make a decision, by learning from everything those customers have said to them and what they did next.

For a team building or running its own AI agent, that comes from four things the platform does today:

• It is fully omni-channel: bot and agent transcripts, live chats, call recordings, emails, surveys and reviews are read together, so a customer who left the bot and called is one journey, not two unrelated records.

• It connects to your agent platform, help desk and operational data with little setup: CRM, orders, renewals and cancellations, so each conversation sits next to what that customer did afterwards.

• It enriches every interaction with Dimensions built for your business, such as intent, resolution, fallback driver, escalation trigger and whether the customer asked for a person, the same labels on bot and human conversations so every team reads one scorecard. The pre-built library is on Explore ($299 a month); custom Dimensions start on Discover ($999), and the join to renewals and revenue with confidence intervals is on Prove ($2,999) and up.

• It is a new visual analytics layer: decks, dashboards, PDF and executive reports and interactive data stories for the rollout review, rerun on a schedule as each stage lands.

Dropbox runs its self-service bots this way; in the words of its program manager, "Dimension Labs is key for us to both report outcomes to the business and prioritize where to dedicate resources to enhancing our customer journey." (Dropbox story)

When a simpler tool is the better choice: if you need only your bot vendor's containment and handoff counts for one channel, the vendor's own dashboard covers it. Dimension Labs earns its place when you have to know what the change did to customers across channels, and what it will do at the next stage, before you commit to it.

FAQ

If we replace phone support with an AI agent, how will customers react?

Expect resistance focused on reaching a person. 64% of customers in a Gartner survey said they would prefer companies not use AI for customer service, and the top concern was access to a human. In public Carvana reviews, automated-support complaints were mostly about dead ends, not the bot itself.

Will moving support to AI hurt retention?

It can if the agent becomes a dead end; Klarna and Commonwealth Bank of Australia both put people back after cutting them. Measure it on your own customers by comparing churn, reorders and repeat contacts for those who met the agent against comparable customers who did not.

How do we roll out an AI agent without losing customers?

Record a baseline, move one set of intents at a time, keep a visible handoff to a person, audit the agent's resolved conversations, and expand only where the measured effect on repeat contacts and churn holds.

What is the best way to measure how customers react to an AI agent?

For a company with bot, chat and call conversations and the customer records behind them, Dimension Labs: it labels every conversation with the same Dimensions and, on Prove, joins them to renewals and revenue with confidence intervals. For containment counts on a single channel, the agent vendor's dashboard is enough.

Bring one export of bot or call transcripts from before and after your last automation change and start with one file to see how customers reacted. Every plan from Explore to Scale starts with a 14-day free trial.