收藏切换
Large Language Model-Based Multimodal Recommendation Algorithm with Adaptive Modality Fusion
收藏切换
Guangyi Xiao1, Min Hou1, *, Yuxin Liao1, Han Wu1, Chenxi Bai1, Si Wei2, Le Wu1
Journal of Computer Research and Development | 2026, 63(8) : 2169 - 2181
Less
收藏切换
Journal of Computer Research and Development | 2026, 63(8): 2169-2181
• Artificial Intelligence •
Large Language Model-Based Multimodal Recommendation Algorithm with Adaptive Modality Fusion
Full
Guangyi Xiao1, Min Hou1, *, Yuxin Liao1, Han Wu1, Chenxi Bai1, Si Wei2, Le Wu1
Affiliations
  • 1Intelligent Interconnected Systems Laboratory of Anhui Province (Hefei University of Technology), Hefei 230009
  • 2Artificial Intelligence Research Institute, iFLYTEK Co., Ltd., Hefei 230088
  • born in 1994. PhD candidate. His main research interests include large language model and recommender system

    born in 1995. PhD, lecturer. Her main research interests include user modeling, recommender system, and data mining

    born in 2001. PhD candidate. Her main research interests include user modeling and personalized recommendation

    born in 1993. Postdoc. Her main research interests include knowledge evolution analysis, recommender system, and data mining

    born in 2001. Master candidate. His main research interest includes recommender system based on large language models

    born in 1982. PhD. Senior Member of CCF. His main research interest includes key core technologies of artificial intelligence such as intelligent voice, natural language processing, and general large language models

    born in 1988. PhD, professor, PhD supervisor. Her main research interests include data mining, personalized recommendation, and causal reasoning and decision making

About Author:

born in 1994. PhD candidate. His main research interests include large language model and recommender system

born in 1995. PhD, lecturer. Her main research interests include user modeling, recommender system, and data mining

born in 2001. PhD candidate. Her main research interests include user modeling and personalized recommendation

born in 1993. Postdoc. Her main research interests include knowledge evolution analysis, recommender system, and data mining

born in 2001. Master candidate. His main research interest includes recommender system based on large language models

born in 1982. PhD. Senior Member of CCF. His main research interest includes key core technologies of artificial intelligence such as intelligent voice, natural language processing, and general large language models

born in 1988. PhD, professor, PhD supervisor. Her main research interests include data mining, personalized recommendation, and causal reasoning and decision making

Published: 2026-08-01 doi: 10.7544/issn1000-1239.202550836
Outline
收藏切换

Multimodal recommendation methods have introduced features such as text and images to enrich item representations, significantly enhancing recommender system performance. In recent years, large language models (LLMs), with their extensive world knowledge and powerful reasoning capabilities, have shown great potential in the field of recommendation. However, effectively integrating LLMs with multimodal information to construct LLM-based multimodal recommender systems still faces several challenges. Firstly, there is a significant semantic gap between the textual input format of LLMs and the multimodal information, which limits the model’s ability to fully understand multimodal features. Secondly, there exists an information imbalance between different modalities, where naive feature fusion can lead to performance degradation. To address these challenges, we propose an LLM-based multimodal recommendation algorithm with adaptive modality fusion (AMFRec). The framework introduces multimodal hybrid prompt templates and projection modules, aligning non-textual features with the LLM token space and recommendation objectives through instruction tuning. To mitigate modality imbalance, a multimodal weight dynamic allocation strategy based on gating mechanism is designed, enabling adaptive adjustment of modality weights by combining mixture-of-experts projection and gating network. In addition, the algorithm requires training only the projection module and gating network, without updating the LLM itself, thereby significantly reducing training costs. Extensive experiments on multiple datasets demonstrate the effectiveness of the proposed algorithm.

large language model  /  multimodal recommendation  /  mixture-of-experts  /  adaptive modality fusion  /  modality imbalance
肖光义, 侯旻, 廖宇欣, 武晗, 白晨希, 魏思, 吴乐. 基于大语言模型的多模态自适应融合推荐算法. 计算机研究与发展, 2026 , 63 (8) : 2169 -2181 . DOI: 10.7544/issn1000-1239.202550836
Guangyi Xiao, Min Hou, Yuxin Liao, Han Wu, Chenxi Bai, Si Wei, Le Wu. Large Language Model-Based Multimodal Recommendation Algorithm with Adaptive Modality Fusion[J]. Journal of Computer Research and Development, 2026 , 63 (8) : 2169 -2181 . DOI: 10.7544/issn1000-1239.202550836
Year 2026 volume 63 Issue 8
88
43
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202550836
  • Receive Date:2025-11-24
  • Online Date:2026-11-17
  • Published:2026-08-01
Article Data
Affiliations
History
  • Received:2025-11-24
  • Revised:2026-04-22
  • Accepted:2026-04-30
Affiliations
    1Intelligent Interconnected Systems Laboratory of Anhui Province (Hefei University of Technology), Hefei 230009
    2Artificial Intelligence Research Institute, iFLYTEK Co., Ltd., Hefei 230088
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202550836
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT