Classification and solution of large language model hallucination

College of Electronic and Information Engineering, Tongji University, China

^* This email address is being protected from spambots. You need JavaScript enabled to view it.

Abstract

Large language models (LLMs) offer significant benefits across various fields, enhancing research efficiency and innovation. However, a critical challenge is their tendency to produce “hallucinations”—content that appears fluent and reasonable but is factually inaccurate or nonsensical. This phenomenon can undermine content credibility and lead to erroneous decisions in critical areas like academic research, medical diagnosis, and news dissemination. This article provides a comprehensive overview of LLM hallucinations. It begins by defining LLMs as deep learning models trained on massive text datasets, typically using a Transformer architecture through stages like pre-training, supervised fine-tuning (SFT), and Reinforcement Learning from Human Feedback (RLHF). The paper then defines and classifies hallucinations into two main types: internal (factual) hallucinations, where the output contradicts objective facts , and external (context-inconsistent) hallucinations, where the output is logically inconsistent with the given prompt or conversational context. Furthermore, the article explores detection methods based on output content analysis, which check for features like semantic consistency and factual ambiguity. Finally, it reviews mitigation mechanisms, discussing strategies such as training on high-quality data and SFT to address factual hallucinations , and using targeted instruction fine-tuning and context-aware decoding for context-inconsistent hallucinations to improve the overall reliability of LLMs.