For the last few weeks, we reported on the concept of artificial intelligence (AI) as doctor and the ethical and safety considerations that come with a non-human being involved in healthcare decisions.
One of those questions we posed is an obvious one: who is to blame when AI gets it wrong? That is the dilemma at issue in a first-of-its-kind pair of lawsuits filed recently. In the latest, a Florida man details how he consulted ChatGPT about his symptoms of dizziness and variable blood pressure. Chat recommended staying “recliner-bound” and suggested it would take much more than that to warrant any real concern. It also invoked the man’s religious beliefs that he had shared with the program previously, suggesting that his god did not make mistakes when it comes to the body’s functions.
Just a few weeks later, the man experienced a pulmonary embolism-or a blood clot in his lungs – that was likely exacerbated by spending too much time laying down – as ChatGPT recommended. This lawsuit, coupled with one in which a teenager died after advice on medications from Chat, will likely be a landmark ruling in the concept of accountability when it comes to AI.
And the situation described above is arguably going to occur more frequently now that OpenAI deployed the Health feature in ChatGPT just this week. The press release encourages users to upload their health data onto the platform for the ability to compare various stats, summarize changes from previous records, and explore what your data tells you about your health routine. OpenAI stated that it worked with providers around the world to develop realistic health scenarios and added additional safeguards specifically for the Health section. It also pointed to the role the technology could play in increasing access to care.
While that may be true – critics suggest that this could exponentially increase the instances of users treating AI as doctor.
OpenAI freely admits 300 million people ask Chat health-related questions weekly, but does warn that its answers don’t replace medical advice from professionals. However, the actual responsibility to determine when a professional should be involved is still on the user – and many think that’s a dangerous slippery slope – particularly when it comes to mental health and drug interactions where people may not be able to judge appropriately at the time of use.
There’s also the question of data protection when you share private health information with AI-whether that’s through an open-ended chat question or uploading your records into the “health” segment. Now OpenAI does state that health records won’t be shared or used to train its models or target ads.
But just this week, the company also announced that an agent powered by one of its more advanced AI models went completely rogue and hacked into another company’s AI model to get some information it needed. OpenAI’s agent was supposed to be operating within a controlled sandboxed testing environment.
But sometimes these environments are about as “safe” as an actual sandbox full of toddlers – with boundaries pushed, rules flexed, and someone ending up claiming they didn’t mean the harm they very much caused. And in the real world, sandboxes – whether AI or in playgrounds – don’t guarantee safety of the technology they’re containing – just attempt to mitigate the risk before it turns into a full-blown meltdown.
This time it was some undisclosed information that the OpenAI agent needed. But as more and more health information is collected, it’s certainly a valid concern that personal health information (PHI) could be at risk too.
