Nearly All 26,000 Patient Visit Records Contained Fabricated Content: The Structural Risks of Whisper Hallucinations and Business Countermeasures

Introduction: Fabricated Patient Visit Records

In October 2024, the Associated Press (AP) published an investigation that brought a quietly acknowledged problem in the Speech Recognition industry into the open. Researchers reviewed approximately 26,000 patient visit records generated by Nabla, a medical transcription tool built on OpenAI Whisper, and found that nearly every record contained fabricated content absent from the audio. This is a typical Whisper hallucination: words appear in the transcript that were never present in the recording. Another study identified 187 hallucinations in more than 13,000 clearly spoken audio snippets.

This was not an incident involving a niche tool. According to the report, more than 30,000 clinicians and 40 health systems—including Mankato Clinic in Minnesota and Children's Hospital Los Angeles—were using the tool, which had transcribed approximately seven million patient visits. A University of Michigan researcher found hallucinations in roughly 80% of the transcription samples studied. The fabricated content included medication instructions that were never given, violent statements that were never made, and words placed in records that neither doctors nor patients had said.

If these fabricated words enter medical records, quality review reports, or compliance recording archives, no one will notice—because they “sound completely like something a person would say.”

This is not a bug. It is an architectural problem. This article explains three things: why Whisper fabricates content, what that costs in business settings, and how you, as a technical decision-maker, can verify whether a transcription engine hallucinates.

1. Why Whisper “Fabricates”: Structural Risks of the LLM Decoder

The conclusion first: Whisper hallucinates because its decoder is fundamentally a language model.

Whisper's architecture has two parts. The encoder converts audio into acoustic features, and the decoder “translates” those features into text. The key is that the decoder is an autoregressive language model. During training, it learns extensive text-continuation capabilities; at each step of generation, it predicts “which word is most likely to come next.”

This design works well with clear speech. Whisper's recognition accuracy is widely respected in the industry, and academic papers explicitly acknowledge it. The problem arises when the audio contains no meaningful speech. With silence, long pauses, background noise, or coughing, the correct behavior for a purely acoustic system should be to “return empty text”—nothing is being said. But a language model's training pushes it to “complete the most plausible human-sounding content.” It has seen so many examples of “this is usually what comes next” that it speaks when it should stay silent.

Engineers generally consider silent segments and background noise important triggers for hallucinations. To be frank, however, the precise cause has not yet been fully identified. This is not a defect that a patch to a particular version can solve; it is a behavior inherent in probabilistic language generation.

Quantified Evidence

Cornell University's study “Careless Whisper: Speech-to-Text Hallucination Harms”, by Koenecke and colleagues (arXiv:2402.08021), provides some of the most informative quantitative evidence available:

• Approximately 1% of transcriptions contain entire hallucinated sentences completely absent from the audio. These are whole sentences, not recognition errors in individual words.
• 38% of hallucinated content is explicitly harmful: violent statements, false associations—such as combining different people's remarks into one sentence—and fabricated authoritative statements.
• Speakers with longer non-vocal intervals have significantly higher hallucination rates. The study highlights people with speech impairments, including aphasia, as particularly affected. The very people who most need transcription assistance face the greatest risk of fabricated content.

OpenAI's own position also reflects the seriousness of the problem: Whisper's usage policies prohibit its use in high-risk decision-making contexts, and the open-source model card recommends against use in high-risk domains. Yet many medical institutions are still integrating it into critical workflows. Major Japanese technology outlets, including ITmedia, have also covered the issue. If you are responsible for selecting technology at a Japanese company, this belongs on the agenda.

Key Insight: Hallucinations Are the “Most Credible” Errors

One fact is easy to underestimate: hallucinations are not random errors; the model errs in the direction of what sounds most human. Recognition errors are often obvious garbled text or misspellings that are easy to spot. Hallucinations, however, produce complete sentences with correct grammar, plausible meaning, and a natural tone. They look particularly credible, making them particularly dangerous. Downstream reviewers, retrieval systems, and LLM analysis pipelines all assume that “transcripts are trustworthy primary records.” Hallucinations exploit precisely that trust.

2. The Business Cost of Hallucinations: Healthcare, Call Centers, Finance, and Accessible Captions

Discussing a “1% hallucination rate” in the abstract is meaningless. What matters is what it becomes in your business.

The fourth setting deserves particular emphasis. Whisper is widely used to generate captions for people with hearing impairments. Hearing users can compare captions with the audio and notice that “nobody is speaking here”; users with hearing impairments cannot. Every fabricated passage they receive becomes a “fact” they cannot verify.

One further point about Call Centers: customer service conversations contain substantial silence while customers think, look things up, or amid background noise. These are precisely the conditions where hallucinations are common. If fabricated “customer statements” enter quality-control training data or agent training materials, every operational decision based on that incorrect data will be systematically skewed—and no one will know the source was wrong.

3. Why “Post-Processing Filters” Cannot Solve Hallucinations

Many teams' first reaction is to add an LLM after transcription to review the output and filter out hallucinations. This approach does not work, for three reasons:

1. Checking hallucinations with hallucinations. The reviewing LLM is itself a language model and has the same hallucination problem. Worse, when it sees hallucinated text with “correct grammar and plausible meaning,” it has no original audio ground truth to compare against. It cannot determine whether a sentence was spoken; it can only judge whether it “sounds like something a person would say.” Hallucinations are precisely what sounds most human.
2. Costs double. On top of the transcription cost per hour of audio comes the token cost of another LLM call. For businesses processing thousands of audio hours daily, this directly doubles the inference bill.
3. Latency accumulates. Real-time settings, such as live customer service quality reviews and Live Subtitles, require Low Latency. Adding an LLM review stage in sequence significantly lengthens the processing chain.

The conclusion: hallucinations must be addressed at the decoder level, not patched downstream. An architectural problem requires an architectural answer. This is also why DolphinVoice chose a non-LLM decoder from the outset: we are not better at filtering hallucinations; we do not generate them in the first place.

4. Zero-Hallucination Architecture: No Fabrication at the Source

DolphinVoice's transcription engine uses a purely acoustic model architecture without an LLM decoder. The fundamental differences regarding hallucinations are as follows:

Zero hallucinations has a specific meaning here: speech absent from the audio will not appear in the output; silent segments return empty text. This is an architectural property, independent of any post-processing safeguard.

That does not mean solving hallucinations is all we do. As a commercial engine, DolphinVoice currently handles more than 15,000 audio hours daily, with approximately 5% WER on 8kHz Call Center audio, about one second to the first real-time text, and roughly 58 seconds to transcribe an hour of recorded audio, including speaker diarization. Real-time speaker diarization, custom Hotwords, ITN, filler-word and sensitive-word filtering are all built in. See the developer documentation for interface details and the API pricing page for pricing. The CPU inference architecture is also worth highlighting: one core supports one concurrent stream, with no GPU required. This brings the full zero-hallucination capability to on-premises deployments without depending on cloud GPU availability.

5. How to Verify “Zero Hallucinations” When Choosing a Vendor: A Buyer's Checklist

Whichever vendor you ultimately choose, we recommend including hallucination testing in your POC acceptance criteria. The method is simple and requires no special equipment:

Step 1: Prepare four types of “difficult audio,” with around 10 examples of each:

1. Silent segments, including at least 3 seconds of continuous silence between two normal utterances.
2. Short speech against white-noise or pink-noise backgrounds.
3. Non-speech vocal sounds: coughing, throat-clearing, laughter, and prolonged fillers such as “ええと…”.
4. Atypical speech rates and long pauses: deliberately slowed speech and monologues with pauses of at least 5 seconds between sentences.

Step 2: Transcribe and count whether any text appears at the positions of silent and noisy segments. You can compare against a waveform manually or ask the vendor for timestamped output and check it segment by segment.

Step 3: Check timestamp alignment. Text timestamps from a zero-hallucination engine should fall strictly within intervals containing meaningful speech. If a timestamped interval has text attached but contains no audible speech, that is a hallucination.

Recommended criterion: not “is the hallucination rate below X%?” but “do silent segments return empty text?” The former can be made to look better through luck and sample bias; the latter is architectural behavior, and the test shows exactly what it is.

This test applies to every vendor, including us. A vendor confident in its architecture should welcome tests using audio with silence. You can also upload a recording containing silent intervals directly to DolphinVoice Playground. Both real-time and offline modes immediately show how the engine behaves during silence.

Conclusion

The article in one sentence: Whisper hallucinations are a structural risk of its LLM decoder—approximately 1% of transcriptions contain entire hallucinated sentences, 38% of hallucinations are explicitly harmful, and post-processing cannot solve the problem; the answer must be architectural. Verification is equally simple: test audio containing silent segments, and a zero-hallucination engine returns empty text during silence.

Closing Thoughts: A Transcription Tool's First Responsibility Is Fidelity

Return to the 26,000 patient visit records at the beginning. A Speech Recognition tool's first responsibility is faithful recording. On this fundamental requirement, fabrication is far more dangerous than omission. Omission reflects the limits of capability; fabrication destroys credibility. A system that inserts false content that “looks completely true” into quality reports, compliance recordings, or medical records should not be part of a critical workflow, no matter how high its accuracy is.

If you are evaluating or already using a Whisper-based solution, start with the self-test checklist in Section 5 and see what your silent audio segments become. Upload audio directly to Playground or connect for testing through the quick registration page.

We, the DolphinVoice team, can also provide a set of silent and noisy audio samples for hallucination self-testing. Please contact us for a testing plan.

Frequently Asked Questions (FAQ)

What is a Whisper hallucination?

Words appear in the transcript that do not exist in the audio. Typical examples include silent or noisy segments, coughing, or long pauses being “completed” into grammatically fluent sentences. In AP's investigation of approximately 26,000 Whisper-based patient visit records, nearly every record contained this kind of fabricated content.

When are Whisper hallucinations most likely to occur?

When the audio contains no meaningful speech: continuous silence, background noise, non-speech vocal sounds such as coughing, laughter, or fillers, and long pauses. A purely acoustic system should return empty text, while an LLM decoder follows its language-model tendencies to “complete the most plausible human-sounding content.”

Can LLM post-processing filter out hallucinations?

Not reliably. The reviewing LLM is itself a language model: hallucinations checking hallucinations. Without the original audio for comparison, it can only judge whether the text sounds human—and hallucinations are precisely what sounds most human. This also doubles costs and adds latency. Hallucinations must be addressed at the decoder level.

How can you test whether a transcription engine hallucinates?

Prepare four types of difficult audio: silent segments, short speech with background noise, non-speech vocal sounds such as coughing and laughter, and slow speech with long pauses. After transcription, check whether any text appears at silent and noisy positions. The criterion is not “a hallucination rate below X%,” but “whether silent segments return empty text.” The latter is architectural behavior and cannot be made to look better by manipulating samples.

Why does DolphinVoice not have these hallucinations?

DolphinVoice uses a purely acoustic model architecture without an LLM decoder: it outputs only what is in the audio and returns empty text during silence. This is an architectural property that does not depend on post-processing safeguards. The engine currently processes more than 15,000 hours daily in real commercial deployments in Japan, covering multiple sectors and applications including Call Centers and Meeting Minutes, and meeting the demands for high concurrency and high recognition accuracy in actual use.

Data Sources

• AP investigation, October 2024: review of approximately 26,000 Nabla patient visit records based on Whisper.
• Cornell University study “Careless Whisper: Speech-to-Text Hallucination Harms,” by Koenecke and colleagues: arXiv:2402.08021.
• Restrictions on high-risk uses in OpenAI Whisper's usage policies and open-source model card.
• DolphinVoice performance data: dolphinvoice.ai website (August 2026).