AI Translation Hallucinations: Real Examples and How to Catch Them
AI translation can sound fluent and still be wrong. See real cases, the main hallucination types, and a 6-step checklist to catch errors before you publish.
Modern AI translation rarely fails by producing gibberish, it’s that accurate presently, and it’s evolving. It fails by producing fluent, confident text that is quietly wrong. Researchers who analysed more than 100 language pairs found that "hallucinations" cluster in lower-resource language directions and when translating out of English, and simply using a bigger model does not reliably fix them. (Guerreiro, Alves, Waldendorf, Haddow, Birch, Colombo and Martins, "Hallucinations in Large Multilingual Translation Models," Transactions of the Association for Computational Linguistics, vol. 11, 2023; presented at EMNLP 2023.) This article explains what a translation hallucination is, walks through documented real-world cases, and gives a practical checklist for catching errors before they reach customers, courts or patients.
What Is an AI Translation "Hallucination"?
An AI translation hallucination is a translation that reads fluently but is "unfaithful" to the source text: it invents, distorts or drops meaning, or bears almost no relation to the original. Researchers describe two classic patterns: detached output, unrelated to the source, and oscillatory output, which repeats words or phrases. Early research on neural machine translation traced both patterns to noise in the training data. Later work found that large language models (LLMs) produce qualitatively different hallucinations from older systems, so both kinds of tool need checking, not just the newest ones. The danger is - "fluency hides the fault": a reader who does not know the source language has no obvious warning signs.
We are regularly facing such issues with AI translation, typically incase of marketing content, creative transcreations and Pharma-ICF content. AI translation needs an almost reworking, to the extent that some translators refuse to edit the AI translation.
Real Examples: When "Fluent" and "Wrong" had consequences
These are not hypothetical risks. Documented cases show fluent machine output contributing to an arrest, a rejected asylum claim and public apologies from major platforms.
- Facebook, October 2017: "good morning" became "attack them." A Palestinian construction worker posted a photo beside a bulldozer with an Arabic "good morning" caption. Facebook’s translation rendered it as "attack them" in Hebrew and "hurt them" in English. Police arrested him; no Arabic-speaking officer had read the post, and he was released a few hours later once the real meaning was confirmed. Facebook apologised. Arabic’s many dialects are widely cited as a reason it is hard for machine translation.
- Afghan asylum claims, 2020 to 2023: "I" became "we." Rest of World reported that machine translation of Dari and Pashto had led to confusion and at least one rejected asylum claim. In the case it described, an automated tool had swapped the singular "I" in a refugee’s statement for "we"; a U.S. court denied her asylum bid because her written application did not match the account she gave in her interviews. (Rest of World, "AI translation is jeopardizing Afghan asylum claims," 19 April 2023.)
- WeChat, October 2017: a neutral phrase became a slur. Around the same time as the Facebook case, WeChat apologised after its translation system rendered a neutral Chinese phrase meaning "black foreigner" as a racial slur in English.
our translation systems made an error last week
Why Hallucinations Happen, and the Types to Watch For
Hallucinations tend to appear where the model has the least reliable evidence: noisy training data, rarer language pairs, and text unlike anything it has seen. Research has linked natural hallucinations to specific noise patterns in the training corpus, and found that models rely less on the source sentence when translating into or out of low-resource languages. The patterns to watch for:
- Detached output: a fluent sentence that has little to do with the source, with the meaning going haywire. More frequent in low-resource language pairs.
- Oscillatory output: repeated words or phrases, a classic sign of a model losing track of the source.
- Toxic insertions: offensive or harmful text can surface, especially when translating out of English into low-resource languages.
- Subtle distortions: swapped pronouns, numbers, negations or terms that read perfectly well. Strictly these are accuracy errors rather than hallucinations, but the asylum case shows they can be just as damaging.
- LLM-specific patterns: studies find that LLMs hallucinate differently from older neural systems, so checks tuned to one may miss the other.
Neural machine translation vs LLM translation
| Feature | Neural Machine Translation (NMT) | LLM Translation (GenAI) |
|---|---|---|
| Primary Focus | Direct, task-specific text conversion | Flexible, general-purpose text generation |
| Best Used For | Standard documentation, high-volume catalogs, strict terminology | Creative copy, marketing, localized nuance, conversational text |
| Risk / Downside | Can feel robotic or overly literal; can also hallucinate, especially in low-resource pairs | Prone to hallucinations, verbosity, or term drift |
| Customization | Requires large datasets, translation memories, and formal glossaries | Uses natural-language prompts, style guides, and RAG (Retrieval-Augmented Generation) |
Why Indian-Language Content Deserves Extra Scrutiny
The research pattern matters for India: many Indian languages are lower-resource than English or French, and translating out of English is the higher-risk direction. That does not mean every Hindi, Bengali or Tamil output is unreliable; engine quality varies by language and content type. It means you should test on your own content, such as legal text, product copy or medical information, instead of assuming. Dialects and colloquial registers add risk: the Facebook case involved Arabic dialects, and reporting on the Afghan cases noted that Dari is treated as a variety of Persian by at least one major tool. Code-mixed text such as Hinglish is likely to be similarly hard, so it is worth testing explicitly.
We’ve regularly noticed with any subject matter, Tamil invariably has the worst AI translated output, needing minimum 25-30% revision, and at times Tamil translators tend to completely reject the AI output insisting on a fresh translation. This is not the case with other languages for the same subject. For instance, we recently had a 13,000 words Coke survey into 11 Indian languages (Hindi, Punjabi, Bengali, Assamese, Telugu, Tamil, Kannada, Malayalam, Urdu, Gujarati, Marathi), all of them except "Tamil" had upto 90% accuracy, but Tamil had to be retranslated.
A 6-Step Checklist to Catch AI Translation Hallucinations
- Triage by risk first. Tag content before translating. Legal, medical, safety, pricing and immigration text should always get human translation or review; low-risk content can use AI with spot checks.
- Run automatic sanity checks. Flag output whose length is wildly different from the source, repeated phrases, untranslated source-language chunks, and empty or truncated segments.
- Verify the details that matter. Compare numbers, dates, names, units, negations and pronouns between source and output with a script or glossary check.
- Cross-check with a second, different system. Disagreement between two engines is a useful flag. Research found that fallback models trained on different data can virtually eliminate certain hallucination types. If you use back translation, use a different engine or a human, and compare meaning rather than wording.
- Score and route, but do not trust scores alone. Quality-estimation scores or an LLM-based checker can flag risky segments, but studies found quality estimation did not reliably separate hallucinations from milder errors. Use scores to decide where humans look first.
- Finish with in-context native review. A subject-qualified native speaker should review on the staging page or final document, and corrections should feed your glossary and translation memory so the same error does not return.
Frequently Asked Questions
No. The 2017 Facebook case involved a neural machine translation system, before today’s LLMs. Research shows both older neural systems and LLMs hallucinate, though in qualitatively different ways.
Studies find hallucinations mainly in low-resource language directions and when translating out of English. Because quality varies by language and content, test your own language pairs rather than relying on general rankings.
Partly. Translating back with the same engine can reproduce or hide the error, so use a different engine or a human back translator and compare meaning, not exact wording.
It can help. Recent research found LLM-based detectors can match or beat earlier detection methods, but quality-estimation scores alone have not reliably separated hallucinations from milder errors, so combine automated flags with human review.
TYPICALLY legal, medical, regulatory, patents, pharma/ICF content, advertising content needing creativity.
Ready for a Precise Quote?
Tell us your languages and content type, and we'll scope the right approach for your budget and timeline.