Artificial intelligence is an increasingly common presence in America’s exam rooms and hospitals. This includes use cases like AI scribes which listen to a conversation and create draft doctor’s notes, and clinicians asking AI clinical questions about a patient.

Yet what is unsettled these days is who gets to write the rules about these tools are regulated. The federal government and the states are currently moving in opposite directions on this issue. The outcome of this regulatory tug of war will shape how much evidence stands behind the AI tools doctors utilize.

As an ER doc who spends a lot of my time thinking about, developing and deploying these AI tools, I see this as one of the most consequential health policy questions we face today.

Here is where things stand. And how it all could play out.

The FDA Has Been Loosening Regulations

The Food and Drug Administration (FDA) regulates AI as a medical device. In January, the agency loosened its guidance on clinical decision support software. AI tools that, for example, recommend a single diagnosis or treatment where the clinician can review the reasoning can now avoid FDA device requirements.

But even for the AI tools the FDA does review, this is a modest bar. Most devices are cleared through what’s called the 510(k) pathway. The requirement is to show similarity to a product already on the market, not proof that it works. A census of all 1,357 AI devices FDA had cleared or approved through early December 2025 found that only 34 (2.5%) had been tested in prospective trials, and only three had been studied with patient-centered outcomes such as death, complications or readmissions.

In August, the FDA released a discussion paper on how it might regulate generative AI devices in the future. Comments are due October 19 (the day of this article’s release). Importantly, the paper is not guidance, nor does it propose any policy. But it is a preview where the agency may go. I see four possible paths:

  1. Hands off for clinician-facing tools. The FDA keeps expanding exemptions, treating most AI that assists a licensed clinician as outside its jurisdiction. It only reserves review for tools that act autonomously or are very high risk. The upside is that this approach would result in rapid movement into the market and into patient care. The downside is that AI scribes, chart summarizers and many diagnostic assistants would reach patients with little independent validation.
  2. Test AI the way we credential doctors. The discussion paper floats a competency-based premarket review. This has been likened to regulating AI like a doctor. Instead of testing every possible output, which is impossible, the FDA would benchmark a device’s proficiency in areas such as safety and clinical knowledge. Then it would require clinical confirmation scaled to risk. Public leaderboards that rank AI models on real physician consultation cases show this kind of testing is feasible.
  3. Lighter upfront review, but heavier monitoring. The discussion paper also raises accepting more uncertainty at approval in exchange for stronger post-market monitoring. This might include periodic benchmarking, spot checks by clinicians and tracking of performance drift. But it only works if someone actually funds and runs the monitoring. The downside of this approach is that for tools that influence time-sensitive ER diagnoses like stroke or sepsis, harm could theoretically occur before monitoring catches a problem.
  4. Congress steps in to regulate clinical AI. Lawmakers could create a new legal framework for clinical AI, possibly with explicit federal preemption of state laws. Yet Congress has so far declined, even going so far as to strip a proposed 10-year moratorium on state AI laws from last year’s budget bill. That would likely require either a high-profile AI-related harm that galvanizes bipartisan action, or industry lobbying strong enough to overcome the states’-rights objections that sank the moratorium.

In my opinion, the most likely near-term outcome is a mix of Paths 1 and 3: fewer premarket requirements for clinician-facing tools, with more of the monitoring burden shifted to the hospitals and clinicians who use them.

States Are Tightening Regulations On How AI Is Used, Mostly

The FDA regulates products. By contrast, states license clinicians and regulates the practice of medicine. Their laws target how AI is used, at the bedside.

California recently passed the most stringent regulation. On Sept. 30, Gov. Gavin Newsom signed a package of health AI laws . The biggest, AB 1979, requires clinicians to retain independent judgment over AI-informed decisions and takes effect Jan. 1, 2027.

Its author, Assembly member Mia Bonta, summed up its premise: “Technology should not replace human decision-making in healthcare.”

It requires health facilities, clinics and physician offices to take reasonable steps to ensure a licensed clinician exercises independent judgment whenever AI output informs care. It also bars AI from independently performing functions reserved for licensed professionals. Violations will count as unprofessional conduct, enforced by the Medical Board of California and other licensing boards.

A companion law, SB 503, requires developers of clinical decision support tools to document training data, testing and bias risks, and requires the hospitals and practices that deploy them to monitor their effects and fix problems.

Importantly, Newsom also vetoed two bills. One would have restricted AI’s use in psychotherapy. The other would have protected clinicians from retaliation when they override an AI recommendation. Yet in both veto messages, he endorsed the underlying goals and urged lawmakers to revisit them.

Sandy Reding, president of the California Nurses Association, criticized the anti-retaliation veto: “The governor knows A.I. should never replace a nurse’s clinical judgment, but he vetoed the bill that would have protected us for using our judgment. That makes no sense to any nurse at the bedside.” Notably, California had already banned AI from implying it is a licensed clinician, effective this year.

Other states are taking their own approaches. Texas now requires practitioners who use AI for diagnosis to review AI-generated records and disclose the use to patients.

By contrast, Utah’s regulation is actually a counter example where it is innovating rather than restricting. Through a regulatory sandbox, the state is letting an AI system called Doctronic gradually take over renewing prescriptions for chronic conditions, with physician review phased out over time. Yet, there is pushback: Utah’s own Medical Licensing Board has asked to suspend the program.

Meanwhile, Pennsylvania is advancing bi-partisan legislation sponsored by state reps Arvind Venkat and Joe Hogan which would require that hospitals, clinicians and health insurers to disclose when and how AI tools are being used for clinical decision-making or coverage determinations. If it passes, it will also require insurers, hospitals, and healthcare facilities to submit formal compliance statements proving that their internal AI governance minimizes algorithmic bias and respects existing privacy and anti-discrimination laws.

A Collision Course of Product and Practice Regulation

The two above tracks are heading in opposite directions. A December 2025 executive order declared: “United States AI companies must be free to innovate without cumbersome regulation.” It also directed the attorney general to create an AI Litigation Task Force to challenge state AI laws. But executive orders cannot rewrite state laws. Unless Congress acts or courts strike them down, state AI laws will remain in force.

This suggests three possible endpoints. First, a federal-state split might harden with the FDA governing what AI products can be sold while the states govern how clinicians use them. In the second, Congress passes preemption and California-style rules disappear.

In the third, a patchwork persists, with some states legislating new AI-specific rules and national companies build to the strictest state standard. This is similar to what happened when automakers built their cars following the most stringent emission rules.

In my view, the first and third seem most likely in the near term, since Congress has already shown it won’t pass preemption. Additionally, the executive order’s challenge to state law is on shaky constitutional footing.

What Coming Regulation Could Mean For AI Tools In Use Today

Nevertheless, clinicians are using a variety of AI tools. And they use them every day. Along with AI scribes, they are using tools like OpenEvidence and Doximity Ask, AI search tools for recommendations. They are also using tools embedded in electronic health records like Epic. Many also are using tools Claude, ChatGPT, and others. What is likely is that the new regulation will affect different tools in different ways.

For example, when it comes to clinical evidence tools, because they show their sources and leave decisions to the clinician, they are the best positioned under the FDA’s January guidance and Path 1. Yet under state law, tools that aid diagnosis or treatment decisions may fit California’s definition of clinical decision support. That would mean bias and training-data documentation for hospital customers. This matters as OpenEvidence moves from individual doctors toward formal contracts with health systems.

Regarding AI scribes, companies like Abridge, Ambience, Cleo and others have largely been treated as documentation software, outside FDA review. But that line is blurring. Some of the tools can suggest orders or differential diagnoses. Once a scribe does this, it starts looking like clinical decision support under state laws, and possibly even to the FDA.

For electronic health record vendors like Epic, the strategy will likely be to make compliance a built-in feature, showing that a clinician reviewed and could override each AI output. Patient disclosure prompts could adjust to each state’s rules in the scaled electronic health record company. And monitoring dashboards could help hospitals meet obligations like SB 503. That could turn regulation into a reason to choose Epic’s AI over third-party tools.

When it comes to doctors using commonly use tools like ChatGPT and Claude for queries, both explicitly disclaim use for diagnosis or treatment. This matters because the FDA regulates products based on intended use. Yet those disclaimers may not hold up indefinitely: personalized answers to clinical queries clearly resembles clinical decision support, falling under the jurisdiction of state medical boards.

Ultimately, the rules for clinical AI are being written now, in Washington and in state capitals. Which path wins will determine whether medical AI is held to a standard closer to a new drug or a new app. But here’s what’s clear is today’s legal environment: the clinician who signs the note is the one who owns the final decision, whether AI was involved or not.