Healthcare AI Regulation: What Actually Governs It Right Now

Ask five people what regulates healthcare AI and you'll get five different answers, and none of them will be wrong exactly, just partial. There is no single federal 'AI in healthcare' statute. What actually applies depends on what the tool does, whether it touches protected health information, and whether it's embedded in certified health IT. That patchwork is the actual governance landscape, so it's worth walking through each piece separately instead of treating 'healthcare AI regulation' as one thing.

If It Diagnoses, Predicts, or Recommends: FDA and Software as a Medical Device

The FDA regulates software that meets the legal definition of a medical device, a category it calls Software as a Medical Device (SaMD). If an AI tool analyzes a mammogram and flags likely malignancy, or predicts sepsis risk from vitals and recommends a clinical action, it can fall under FDA's device authorities, most commonly cleared through the 510(k) pathway by showing substantial equivalence to a predicate device. Administrative tools, scheduling software, and most clinical documentation assistants generally fall outside this because they don't diagnose or direct treatment on their own; the line is about clinical decision-making function, not the presence of AI itself.

The harder problem FDA has had to solve is that most medical devices are cleared once and stay static, while machine learning models can keep learning and changing after clearance. FDA's answer is the predetermined change control plan (PCCP): a manufacturer specifies in advance what kinds of updates it intends to make and how it will validate them, and FDA reviews that plan up front rather than requiring a new submission for every retrain. That is the specific mechanism that lets an AI/ML-based device evolve without regulatory limbo, and it's worth knowing by name if you're evaluating a vendor's clearance.

If It Touches Patient Data: HIPAA Still Applies, AI or Not

HIPAA does not have an AI-specific chapter, and it doesn't need one. If an AI tool creates, receives, maintains, or transmits protected health information, HIPAA's Privacy and Security Rules apply exactly as they would to any other system handling that data, including the requirement for a Business Associate Agreement with any vendor processing PHI on a covered entity's behalf. The practical wrinkle AI introduces is what happens to that data once it's inside a model. An ambient documentation tool that listens to a patient visit is processing PHI in real time; the questions worth asking a vendor are whether that audio and transcript are retained, whether they're used to train models beyond the current patient's care, and where the processing actually happens.

If It's Inside Certified Health IT: ONC's HTI-1 Transparency Rule

In late 2023, the Office of the National Coordinator for Health IT (ONC) finalized the HTI-1 rule, which added new certification criteria specifically for AI and predictive decision support tools embedded in certified health IT, most relevantly EHR systems. The core requirement is transparency: developers of certified health IT have to disclose source attributes about how a predictive model was developed, validated, and how it performs, structured so clinicians and health systems can evaluate a model before trusting its output. This is narrower than it sounds, since it applies to certified health IT specifically, but it's the most direct federal transparency requirement healthcare AI has right now.

The Voluntary Layer: NIST's AI Risk Management Framework and CHAI

Underneath the binding rules sits a layer of voluntary frameworks that health systems increasingly use to fill the gaps FDA, HIPAA, and HTI-1 leave open, particularly for AI tools that never touch FDA's device definition. NIST's AI Risk Management Framework provides a general structure, organized around Govern, Map, Measure, and Manage functions, for identifying and managing AI risk that health systems have adapted for clinical AI governance committees. The Coalition for Health AI (CHAI), a nonprofit formed by health systems, academic medical centers, and technology companies, has published governance playbooks aimed specifically at helping hospitals evaluate and monitor AI tools post-deployment, an area federal rules mostly don't reach once a tool is already in clinical use.

What This Means When You're Evaluating a Tool

In practice, this means the right regulatory question is never just 'is this FDA approved.' It's a short sequence: does the tool make or influence a clinical decision (FDA), does it touch PHI and how (HIPAA and the BAA), is it embedded in certified health IT (HTI-1 transparency data), and does the vendor have a real answer for ongoing monitoring after deployment (NIST AI RMF, CHAI, or an internal equivalent). A vendor that can answer all four clearly is telling you something real about how seriously they've thought about this. A vendor whose answer to all four is 'it's just AI, it doesn't need that' is telling you something too.