A lawyer needs case law supporting a client’s motion.
They use AI to research precedents. The AI generates what looks like perfect cases directly on point.
The lawyer doesn’t verify they’re real. The cases look legitimate with proper citations, judicial reasoning, and favorable holdings.
The lawyer cites them in a court filing.
The judge asks for copies of the cases. They don’t exist. The AI invented them.
Now the lawyer is sanctioned for citing fake cases. The client’s motion is denied. There’s a malpractice claim.
And the professional liability insurance carrier is arguing that AI use voids coverage.
Why Professionals Are Using AI
Lawyers, accountants, consultants, and engineers face the same pressure as everyone else: do more work in less time with fewer resources.
AI promises to help by researching legal issues faster than manual research, drafting documents in minutes instead of hours, analyzing financial data and identifying issues, reviewing contracts for problems, and generating technical specifications and reports.
For professionals billing by the hour, AI creates a dilemma. It makes work faster, which is good for clients but bad for billable hours. But if competitors use AI and offer faster turnaround, professionals feel pressure to adopt it too.
Most professionals started using AI without understanding the professional liability risks.
What AI Hallucinations Actually Are
AI hallucinations occur when AI generates content that sounds authoritative but is completely fabricated.
The AI doesn’t know it’s wrong. It’s generating text based on patterns in training data, creating content that looks plausible.
In legal research, AI invents case citations that don’t exist. It fabricates judicial holdings and reasoning. It creates statutes that were never enacted.
In accounting, AI generates financial analysis based on incorrect assumptions. It applies tax rules that don’t exist. It creates audit findings without proper support.
In consulting, AI fabricates industry data and statistics. It invents best practices that aren’t real. It generates recommendations based on false premises.
In engineering, AI creates specifications that violate physical laws. It generates designs with hidden flaws. It fabricates testing results.
The professional receives AI output that looks correct and professional. They assume it’s accurate because it’s detailed and well-written. They use it in client work without verification.
Real Cases of Professional AI Failures
Multiple lawyers have now been sanctioned for citing AI-generated fake cases in court filings.
In Mata v. Avianca, a lawyer used ChatGPT for legal research. The AI generated six cases supporting the client’s position. All six were fake. The lawyer submitted them to the federal court. The judge sanctioned the lawyer and required payment of opposing counsel’s fees.
This has now happened in multiple jurisdictions with multiple lawyers. The pattern is always the same: AI generates fake cases, the lawyer doesn’t verify, the court discovers the fraud, and sanctions follow.
The Professional Duty Problem
Professionals have duties that AI use can violate.
Duty of competence: Professionals must provide competent service. This includes understanding the work performed. If AI generates work the professional doesn’t understand or can’t verify, they’re violating the duty of competence.
Duty to supervise: Professionals are responsible for supervising their work product. AI-generated content requires the same supervision as work by junior staff. Using AI without verification is inadequate supervision.
Duty to verify: Professionals can’t rely on unverified information. They must confirm facts, check sources, and ensure accuracy. AI outputs require verification just like information from any other source.
Duty of candor: Lawyers have a duty of candor to tribunals. Submitting AI-generated fake cases violates this duty. Similar duties exist in other professions when submitting information to regulators or courts.
When professionals use AI without adequate verification, they breach these professional duties.
The Malpractice Liability Question
When AI hallucinates, and professionals rely on it, who’s liable for the resulting harm?
The professional is liable: They provided the service. They’re responsible for accuracy. “AI made a mistake” isn’t a defense.
The firm is liable: Under partnership agreements and vicarious liability principles, firms are liable for professional errors by their members.
The AI vendor probably isn’t liable: Most AI vendor contracts disclaim liability for accuracy. They provide AI “as is” for “informational purposes only.” The vendor isn’t practicing law, accounting, or engineering.
So liability falls on the professional and the firm, even though the AI generated the error.
The Insurance Coverage Problem
Professional liability insurance covers malpractice, but does it cover AI-related errors?
Some insurance carriers are arguing that using AI without adequate verification is negligence or intentional misconduct not covered by insurance.
If the professional knew AI could hallucinate but used it anyway without verification, that might be considered reckless or intentional conduct excluded from coverage.
Policy language typically requires professionals to exercise “reasonable care.” Using AI that generates fake information without checking might violate the duty of reasonable care, potentially voiding coverage.
This is still being litigated, but professionals shouldn’t assume their malpractice insurance covers AI-related errors.
What Professional AI Use Policies Should Require
If professionals use AI for client work, they need strict protocols.
Verification requirement: All AI-generated factual claims must be verified against primary sources before use in client work.
Disclosure to clients: Clients should be informed when AI is used in their matters and have the right to object.
Supervision: AI-generated work requires the same review and supervision as work by the most junior team member.
Documentation: Document what AI was used, what it generated, how it was verified, and what changes were made.
Training: Professionals must be trained on AI limitations, hallucination risks, and verification requirements before using AI for client work.
Prohibited uses: Some uses of AI should be prohibited entirely, such as citing AI-generated legal research without verifying every citation or submitting AI-generated analysis to regulators without independent verification.
Most professionals using AI have none of these safeguards.
Ask Yourself This
Are professionals in the organization using AI for client work?
If yes, do they verify every factual claim AI generates before relying on it?
Do clients know AI is being used in their matters?
Does professional liability insurance cover AI-related malpractice?
If the answers are unclear, there’s an AI professional liability risk that needs immediate attention.
Schedule a 30-minute Professional AI Use Assessment where we’ll:
- Review how professionals are actually using AI
- Show what verification and supervision requirements should be
- Create AI use policies for professional services
- Provide training on AI hallucination risks
- Help assess professional liability insurance coverage
Because using AI for legal research feels efficient until you’re sanctioned for citing cases that don’t exist.
Brian Alexander
Founding Partner, VisionAI+ Consulting Group
Former General Counsel, AI Engineering Firm
Certified ISO 42001 Lead Auditor