[TOC]
- Title: Hallucination Is Inevitable an Innate Limitation Llm 2024
- Author: Ziwei Xu et. al.
- Publish Year: 22 Jan 2024
- Review Date: Sun, Jan 28, 2024
- url: arXiv:2401.11817v1
Summary of paper
Contribution
The paper formalizes the issue of hallucination in large language models (LLMs) and argues that it is impossible to completely eliminate hallucination. It defines hallucination as inconsistencies between a computable LLM and a computable ground truth function. By drawing from learning theory, the paper demonstrates that LLMs cannot learn all computable functions, thus always prone to hallucination. The formal world is deemed a simplified representation of the real world, implying that hallucination is inevitable for real-world LLMs. Additionally, for real-world LLMs with provable time complexity constraints, the paper identifies tasks prone to hallucination and provides empirical validation. Finally, the paper evaluates existing hallucination mitigators using the formal world framework and discusses practical implications for the safe deployment of LLMs.
Some key terms
hallucination
- the model generate plausible but factually incorrect or nonsensical information
- Up to now, research on LLM hallucination remains largely empirical. Useful as they are, empirical studies cannot answer the fundamental question: can hallucination be completely eliminated? The answer to this question is fundamental as it indicates a possible upper limit of LLMs’ abilities.
Formal definition of hallucination is difficult
- In the real world, formally defining hallucination, a factual or logical error of LLM, turns out to be extremely difficult. This is because a formal definition of semantics in the real world is still an open problem [12, 58].
- To address this, the paper establishes a formal world of computable functions where precise discussions on hallucination become feasible. Hallucination is defined as the failure of an LLM to reproduce the output of a computable function exactly.
Results
In defence of LLMs and Hallucination
LLMs are continuously evolving, with advancements in model architecture and error correction strategies expected to mitigate the severity of hallucinations over time. While complete elimination is improbable, researchers aim to better understand and control hallucination for various applications.
Moreover, hallucination is not entirely negative. In creative fields like art and literature, the unintended outputs from LLMs can inspire human creators, offering unique perspectives and fostering innovation. Thus, the hallucinatory aspect of LLMs can be viewed positively as a source of creativity and inspiration.
Practical implications
Guardrails and Fences are Essential: Without proper guardrails and fences, LLMs cannot be relied upon for critical decision-making. These mechanisms are designed to ensure that LLMs operate within expected boundaries and do not deviate into unethical, disturbing, or destructive content. Given the inevitability of hallucination, guardrails and fences are deemed essential safeguards.
Summary
Possible hallucination mitigators
- larger models and more training data
- Prompting LLMs with Chain of Thoughts/Reflections/Verification
- Prompting is effective in mitigating hallucination by guiding them towards solutions more preferred by humans, which are possibly the ones with lower complexities, and within LLMs’ capabilities. However, this approach will only work for specific tasks.
- Ensemble of LLMs
- This approach uses multiple instances of LLMs to solve a single problem. The solution is usually produced through majority votes
- Guardrails and Fences
- The guardrails are principles that align LLMs’ output with human values, ethics, and legal requirements. The fences is a list of critical tasks that should never be fully automated using LLMs