收藏切换
Survey of Hallucinations in Large Vision-Language Models: Causes, Evaluations and Mitigations
收藏切换
Xu Li1, Rui Zhu1, Xiaolei Chen1, Jinxuan Wu2, Yi Zheng2, Chenghang Lai1, Yuxuan Liang2, Bin Li2, Xiangyang Xue1, 2, *
Journal of Computer Research and Development | 2025, 62(12) : 2929 - 2950
Less
收藏切换
Journal of Computer Research and Development | 2025, 62(12): 2929-2950
• Artificial Intelligence •
Survey of Hallucinations in Large Vision-Language Models: Causes, Evaluations and Mitigations
Full
Xu Li1, Rui Zhu1, Xiaolei Chen1, Jinxuan Wu2, Yi Zheng2, Chenghang Lai1, Yuxuan Liang2, Bin Li2, Xiangyang Xue1, 2, *
Affiliations
  • 1Institute of Big Data, Fudan University, Shanghai 200433
  • 2School of Computer Science, Fudan University, Shanghai 200438
  • born in 1998. PhD candidate. His main research interests include multimodal information processing, multimodal large language models, and large vision-language models

    born in 2002. PhD candidate. Her main research interests include computer vision and large vision-language models

    born in 1999. PhD candidate. His main research interests include computer vision and machine learning

    born in 2001. Master candidate. Her main research interests include large multimodal models, and algorithmic fairness and safety

    born in 1998. Master candidate. His main research interests include computer vision and large vision language models

    born in 1994. PhD candidate. His main research interests include deep learning, machine learning, and multimodal knowledge extraction and reasoning

    born in 2001. Master candidate. His main research interests include artificial intelligence and large vision language models

    born in 1982. PhD, professor, PhD supervisor. His main research interests include machine learning and vision intelligence

    born in 1968. PhD, professor, PhD supervisor. His main research interests include computer vision, multimedia, and machine learning

About Author:

born in 1998. PhD candidate. His main research interests include multimodal information processing, multimodal large language models, and large vision-language models

born in 2002. PhD candidate. Her main research interests include computer vision and large vision-language models

born in 1999. PhD candidate. His main research interests include computer vision and machine learning

born in 2001. Master candidate. Her main research interests include large multimodal models, and algorithmic fairness and safety

born in 1998. Master candidate. His main research interests include computer vision and large vision language models

born in 1994. PhD candidate. His main research interests include deep learning, machine learning, and multimodal knowledge extraction and reasoning

born in 2001. Master candidate. His main research interests include artificial intelligence and large vision language models

born in 1982. PhD, professor, PhD supervisor. His main research interests include machine learning and vision intelligence

born in 1968. PhD, professor, PhD supervisor. His main research interests include computer vision, multimedia, and machine learning

Published: 2025-12-01 doi: 10.7544/issn1000-1239.202440444
Outline
收藏切换

LVLMs (large vision-language models) represent a significant advancement in the intersection of natural language processing and computer vision. By integrating pre-trained visual encoders, vision-language adapters, and large language models, LVLMs can understand both visual and textual information, and generate responses in natural language, making them suitable for a range of downstream vision-language tasks such as image captioning and visual question answering. However, these models commonly exhibit hallucinations — generating inaccurate perceptions of image contents. Such hallucinations significantly limit the application of LVLMs in high-stakes domains like medical image diagnosis and autonomous driving. This survey aims to systematically organize and analyze the causes, evaluations, and mitigation strategies of hallucinations to guide research in the field and enhance the safety and reliability of LVLMs in practical applications. It begins with an introduction to the basic concepts of LVLMs and the definition and classification of hallucinations within them. It then explores the causes of hallucinations from four perspectives: training data, training task, visual encoding, and text generation, while also discussing the interactions among these factors. Following this, it discusses mainstream benchmarks for assessing LVLM hallucinations in terms of task setting, data construction, and assessment metrics. Additionally, it examines hallucination mitigating techniques across five aspects: training data, visual perception, training strategy, model inference, and post-hoc corrections. Finally, the review provides directions for future research in the areas of cause analysis, evaluation, and mitigation of hallucinations in LVLMs.

natural language processing  /  computer vision  /  large vision-language models  /  multimodal large language models  /  hallucinations
李煦, 朱睿, 陈小磊, 伍瑾轩, 郑毅, 赖承杭, 梁宇轩, 李斌, 薛向阳. 视觉语言大模型的幻觉综述:成因、评估与治理. 计算机研究与发展, 2025 , 62 (12) : 2929 -2950 . DOI: 10.7544/issn1000-1239.202440444
Xu Li, Rui Zhu, Xiaolei Chen, Jinxuan Wu, Yi Zheng, Chenghang Lai, Yuxuan Liang, Bin Li, Xiangyang Xue. Survey of Hallucinations in Large Vision-Language Models: Causes, Evaluations and Mitigations[J]. Journal of Computer Research and Development, 2025 , 62 (12) : 2929 -2950 . DOI: 10.7544/issn1000-1239.202440444
Year 2025 volume 62 Issue 12
144
68
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202440444
  • Receive Date:2024-06-05
  • Online Date:2026-11-10
  • Published:2025-12-01
Article Data
Affiliations
History
  • Received:2024-06-05
  • Revised:2025-04-09
Affiliations
    1Institute of Big Data, Fudan University, Shanghai 200433
    2School of Computer Science, Fudan University, Shanghai 200438
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202440444
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT