A company received notice that regulators were reviewing their AI hiring system.
Legal scrambled to pull documentation that didn’t exist. IT looked for bias testing records that were never created. HR searched for human oversight logs that nobody kept.
Three months of chaos followed, trying to reconstruct evidence of governance that was never actually implemented.
The regulator found exactly what the company feared – an AI system making consequential decisions about people with no documentation, no testing history, and no meaningful oversight.
What regulators look for during AI audits isn’t complicated. Most companies just never built it.
What Regulators Actually Examine
Regulatory AI audits focus on five core areas consistently across agencies and jurisdictions.
Documentation is always first. Regulators want to understand what the AI does, what data it uses, how decisions are made, and what risks were identified before deployment. Companies without technical documentation of their AI systems can’t demonstrate they understood what they deployed.
Bias testing history comes next. When was the AI tested for discriminatory impact? What methodology was used? What were the results? What changed when problems were found? Companies that tested once at deployment and never since struggle to explain why they stopped monitoring for discrimination.
Human oversight records matter significantly. Regulators distinguish between genuine oversight and performative approval. They want evidence that humans actually evaluated AI recommendations rather than rubber-stamping them. This means logs, review records, and documentation of decisions where humans overrode the AI.
Incident response history reveals whether companies treated AI problems seriously. When internal concerns were raised about AI performance or fairness, what happened? Companies that can show investigation records and remediation steps demonstrate functioning governance. Companies that can’t suggest problems were ignored.
Vendor management documentation shows whether companies understood what they deployed. Contracts, due diligence records, and ongoing vendor communication demonstrate whether the company treated AI procurement as a governance matter or just a technology purchase.
The Documentation Gap
Most companies have significant documentation gaps they don’t discover until audits force the issue.
AI gets deployed quickly to meet business objectives. Technical teams focus on making the AI work correctly, not on documenting how it works for future regulatory review. Legal gets involved after deployment when problems emerge rather than before when documentation requirements could be built into the process.
By the time regulators ask for documentation, the people who built the system may have moved on, institutional knowledge is scattered, and reconstructing accurate records is difficult or impossible.
The documentation regulators want isn’t complicated to create when AI is being built. It’s extremely difficult to reconstruct after the fact under regulatory pressure.
The Bias Testing Problem
Regulators have become sophisticated about bias testing and can identify inadequate approaches quickly.
Testing only at deployment is insufficient. Regulators ask about ongoing monitoring because AI systems change over time and bias that didn’t exist initially can develop as input data shifts.
Testing only aggregate metrics misses disparate impact on specific populations. An AI with good overall accuracy can still discriminate against protected groups if performance varies significantly across demographics.
Testing without documentation of methodology is effectively no testing at all from a regulatory perspective. Companies need records showing what was tested, how, when, by whom, and what the results were.
What Genuine Oversight Looks Like
Human oversight is one of the most contested areas in AI audits because companies and regulators often disagree about what counts.
From a regulatory perspective, oversight requires that humans understand what factors the AI considered, have enough time to genuinely evaluate recommendations rather than process them in bulk, have clear authority to override AI decisions, and actually exercise that authority when recommendations seem wrong.
Companies whose oversight consists of humans approving AI recommendations at high volume without genuine evaluation are creating the appearance of oversight without its substance. Regulators are increasingly able to identify this pattern through review volume data and override rate analysis.
Building Audit Readiness
Companies that fare well in AI audits built documentation and testing into their deployment processes rather than trying to create it retrospectively.
This means technical documentation gets created when AI is built, not when regulators ask for it. Bias testing happens before deployment and continues on a defined schedule. Human oversight processes create records demonstrating genuine review. Vendor contracts get reviewed against governance requirements before signing.
The companies most vulnerable to regulatory AI audits are those that treated AI deployment as purely a technology decision and compliance as something to address later. The companies best positioned are those that treated documentation, testing, and oversight as operational requirements from day one.
Brian Alexander
Founding Partner, VisionAI+ Consulting Group
Former General Counsel, AI Engineering Firm
Certified ISO 42001 Lead Auditor