You can no longer go a single day without hearing about artificial intelligence (AI), and certainly, there have been many discussions on its role in healthcare – like whether it’s helping address provider burnout, if it’s increasing overall costs, and whether and how insurers should be allowed to use the technology on claims.
While everyone is generally in alignment that AI is good to take over administrative tasks like notetaking, states are starting to tease out the safety and ethical implications of AI conducting one of the most core provider duties there is: prescribing medications.
We all know that prescribing medications is limited to a select set of medical professionals; this has been in law across the U.S. for over a century. But earlier this year, Utah launched a pilot program whereby a narrow portion of residents within the state can get their prescriptions refilled online by an AI chatbot (called Doctronic), rather than having to get their provider to do it.
It is overseen by a five-member board of AI specialists – that’s AI specialists, not medical specialists – none of whom have medical degrees.
Here’s how it works: you visit the Doctronic website built specially for the program. You input your information, the program confirms you have a valid prescription through the national pharmacy database, and if there are no issues or flags, it sends your refill to your pharmacy of choice. For the initial phase of the program, human doctors are indeed reviewing the refill orders, but not for long.
The benefits are obvious. Patients would not have to wait for a provider’s office to get back to them for refills, which would reduce medication lapses. Provider offices wouldn’t have to then call these patients back, reducing administrative burden. It could also increase access to healthcare in areas where there are physician shortages, and those rural physicians could focus more on acute cases, rather than routine refill requests.
Utah allows this through a regulatory sandbox that allows officials to waive various state laws for AI companies.
But the questions this raises are numerous, and many providers, lawyers, and public health experts are concerned about patient safety and the precedent this sets: simply this acting as if AI has a medical license.
States have been contemplating the “AI as doctor” concept for some time now, and thus far generally the answer has been that it’s a no-go, at least on its own. This is particularly the case when it comes to mental and behavioral health. Illinois was one of the first states to explicitly ban AI from diagnosing or treating such conditions, and was joined by New York, Nevada, Tennessee, and Colorado in passing similar types of legislation. Oregon and Washington banned nonhuman entities from using the title of “nurse” and other related titles.
Many, many other states have passed legislation requiring keeping a human provider in the loop when it comes to duties like diagnosing and prescribing.
And in Utah, the medical board states that they found out about the program at the same time the public did. They’ve protested the program, citing danger to patients. This could come from something like missing the subtle appearance of side effects or drug interactions, or the fact that a medication may no longer be proper just because it was once prescribed.
The American Medical Association (AMA) commented that renewing prescriptions is just not the “administrative transactions” that AI is good at.
Doctronic has expressed interest in expanding beyond Utah in the future, and with the approach the Trump Administration has taken with regard to AI, there’s a bit of uncertainty about how much states might be allowed to strictly regulate something like this. But admittedly, Congress hasn’t made much progress there, so for now, states will likely continue to weigh those ethical and safety considerations as technology continues to advance.
