A company deployed AI to handle customer service inquiries.
For three weeks, the AI confidently answered questions about return policies, warranty terms, and product features.
Then someone posted a screenshot on social media showing the AI making promises the company doesn’t honor.
People started testing it. The AI was giving wrong information to everyone who asked.
Thousands of customers received incorrect answers about refund eligibility, warranty coverage, and product capabilities that don’t exist.
Now the company faces consumer protection investigations, customers demanding the AI honor promises it made, and a brand crisis that went viral.
How Companies Use AI for Customer Service
AI customer service adoption exploded because it’s cheaper than humans, available 24/7, and handles routine inquiries at scale.
Chatbots on websites: Answer product questions, explain policies, troubleshoot issues.
Email responses: AI drafts responses to customer emails with minimal human review.
Phone support: AI handles initial calls and routes to humans only for complex issues.
Social media: AI responds to customer questions and complaints on social platforms.
Most companies deployed AI customer service to reduce costs and improve response times without fully understanding the liability.
The Wrong Information Problem
AI doesn’t just get things wrong occasionally. When it’s wrong, it’s wrong for everyone who asks the same question.
AI hallucinates policies: Customer asks about return policy. AI states “all purchases can be returned within 90 days” when the actual policy is 30 days. Hundreds of customers receive this wrong information.
AI invents product features: Customer asks if product has specific capability. AI says yes when the answer is no. Customers buy based on AI’s false representation.
AI misinterprets warranty terms: Customer asks about warranty coverage. AI describes coverage that doesn’t exist in actual warranty. Customers rely on it.
AI makes unauthorized promises: Customer asks for exception to policy. AI says “yes, we can do that” when company policy prohibits it.
One hallucination affects thousands of customers before anyone notices.
The Binding Commitment Problem
When AI tells customers something, is the company bound by it?
Courts are starting to say yes.
Agency law: The AI is acting as the company’s agent in customer communications. Statements the AI makes can bind the company just like statements from human customer service representatives.
Consumer protection: If customers rely on AI statements to their detriment, the company may be liable for misrepresentation even if the AI hallucinated.
Contract modification: If AI tells customers they can return products beyond the stated policy, courts may find the company modified the contract terms through its AI agent.
“The AI was wrong” isn’t a defense when customers reasonably believed they were getting accurate information from the company.
The Viral Brand Damage
AI customer service failures don’t stay private.
Customers screenshot wrong AI responses and post them on social media. The posts go viral. Journalists write articles. Competitors amplify the failures.
Within hours, millions of people see examples of the AI giving terrible advice, making absurd promises, or providing dangerous misinformation.
The brand damage happens faster than the company can respond because AI errors happen at scale and in public.
Real Examples
The airline chatbot: An airline’s AI chatbot told customers they were eligible for bereavement fare discounts that don’t exist. When customers tried to claim the discount, the airline refused. Customers sued. The court held the airline responsible for what its chatbot promised.
The car dealer chatbot: A dealership’s AI offered to sell a car for $1 because someone asked if it would. The screenshots went viral. The company had to honor the AI’s “offer” to avoid worse publicity.
The insurance chatbot: An insurance company’s AI told customers certain procedures were covered when they weren’t. Customers scheduled medical procedures based on AI confirmation. The insurer denied claims. Multiple lawsuits followed.
In each case, the company is liable for what the AI told customers.
What Companies Should Be Doing
If AI handles customer communications, companies need controls to prevent and catch errors before they become crises.
Test before deployment: Run thousands of test queries to identify what wrong answers the AI gives before customers see them.
Ongoing monitoring: Randomly sample AI customer interactions to catch when AI starts hallucinating.
Escalation triggers: Configure AI to route certain questions to humans rather than attempting to answer everything.
Disclaimer requirements: Make clear when customers are interacting with AI vs. humans, though this doesn’t eliminate liability.
Error correction process: When wrong AI information is discovered, proactively contact affected customers and correct the misinformation.
Crisis response plan: Have a plan ready for when AI failures go viral, because they will.
Most companies deployed AI customer service with minimal testing and no ongoing monitoring.
The Question to Ask
Does the company use AI to communicate with customers?
If yes, how is AI being monitored for accuracy? What happens when AI gives wrong information to customers? Who’s responsible for catching AI errors before they go viral?
If nobody can answer these questions, there’s an AI customer service crisis waiting to happen.
Schedule a 30-minute AI Customer Service Risk Assessment where we’ll:
- Review what customer-facing AI could create liability
- Show what testing and monitoring should exist
- Identify current AI error risks
- Create escalation protocols for complex questions
- Develop crisis response plan for viral AI failures
Because AI customer service feels efficient until 10,000 customers receive wrong information and somebody posts it on social media.
Brian Alexander
Founding Partner, VisionAI+ Consulting Group
Former General Counsel, AI Engineering Firm
Certified ISO 42001 Lead Auditor