AI Models Often Give the Right Answers but Point to the Wrong Sources
1 min read
AI Security, Privacy & Model/Prompt Risk Management
-/5
In short
- Leading AI models like GPT and Gemini routinely cite text passages in document analyses that do not actually support their answers.
- Even when the answer is correct, the cited evidence is often incorrect.
- Researchers at Peking University refer to this phenomenon as "attribution hallucination," which poses risks in regulated fields such as law and medicine.
Leading AI models like GPT and Gemini routinely cite text passages in document analyses that do not actually support their answers. Even when the answer is correct, the cited evidence is often incorrect. Researchers at Peking University refer to this phenomenon as "attribution hallucination," which poses risks in regulated fields such as law and medicine. Their new CiteVQA benchmark is the first to systematically test for this issue. This development is noteworthy, but it should be assessed in the context of previous challenges in AI development. A final assessment would be premature at this point.
Source:
-
AI models often give the right answers but point to the wrong sources — The Decoder (EN-US)