Imagine sitting in a doctor’s office, pouring out your concerns, and having all your words summarised by an AI transcription tool. Sounds efficient, right? But what if that tool starts making things up entirely? That’s the issue researchers found with OpenAI’s Whisper, a tool powering medical transcription services used by many hospitals.
Here are the key points:
- Nabla, the company behind the medical transcription tool estimating 7 million transcribed medical conversations, uses Whisper. Over 30,000 clinicians and 40 health systems are said to rely on this technology.
- Researchers discovered that Whisper hallucinated in about 1 percent of transcriptions, creating entire sentences with violent sentiments or nonsensical phrases, especially during silences in recordings.
- Examples include made-up medical conditions or phrases that you’d expect from a YouTube video like “Thank you for watching!”
The research conducted by a group from Cornell University and the University of Washington indicated that Whisper’s hallucinations are particularly common in audio samples from TalkBank’s AphasiaBank, where silence is prevalent when individuals with aphasia speak.
An OpenAI spokesperson stated:
“We take this issue seriously and are continually working to improve, including reducing hallucinations. For Whisper use on our API platform, our usage policies prohibit use in certain high-stakes decision-making contexts.”
In conclusion, the use of AI in transcription tools, especially in critical domains like healthcare, raises concerns that require immediate attention. Researchers’ findings shed light on the potential risks associated with relying on technology that may not always provide accurate and reliable information. It is crucial for developers and users to be vigilant and ensure the ethical and responsible use of AI tools to prevent any unintended consequences.
Leave feedback about this