收藏切换
Dual-Perspective Multi-Level Cross-Modal Recommendation Based on Large Language Models
收藏切换
Yiming Li, Yaxin Yu*, Zhisheng Yu, Yiting Si, Yusong Ye
Journal of Computer Research and Development | 2026, 63(1) : 147 - 161
Less
收藏切换
Journal of Computer Research and Development | 2026, 63(1): 147-161
• Artificial Intelligence •
Dual-Perspective Multi-Level Cross-Modal Recommendation Based on Large Language Models
Full
Yiming Li, Yaxin Yu*, Zhisheng Yu, Yiting Si, Yusong Ye
Affiliations
  • School of Computer Science and Engineering, Northeastern University, Shenyang 110169
  • Key Laboratory of Intelligent Computing in Medical Image (Northeastern University), Ministry of Education, Shenyang 110169
  • born in 2000. Master candidate. His main research interests include knowledge graph and recommendation system

    born in 1971. PhD, associate professor, master supervisor. Member of IEEE, ACM and CCF. Her main research interests include data mining and social network

    born in 2001. Master candidate. His main research interests include recommendation system and causal relationship inference

    born in 2001. Master candidate. His main research interests include recommendation system and large language model

    born in 2001. Master candidate. His main research interests include data mining and recommendation system

About Author:

born in 2000. Master candidate. His main research interests include knowledge graph and recommendation system

born in 1971. PhD, associate professor, master supervisor. Member of IEEE, ACM and CCF. Her main research interests include data mining and social network

born in 2001. Master candidate. His main research interests include recommendation system and causal relationship inference

born in 2001. Master candidate. His main research interests include recommendation system and large language model

born in 2001. Master candidate. His main research interests include data mining and recommendation system

Published: 2026-01-01 doi: 10.7544/issn1000-1239.202550040
Outline
收藏切换

Multimodal recommendation systems aim to deliver more accurate and personalized recommendations by leveraging diverse data modalities such as text and images. Despite their potential, existing approaches face several critical challenges. 1) Feature distortion. The input embeddings are processed by relatively small pre-trained language models and deep convolutional neural networks, which may lead to inaccurate and incomplete feature representations. 2) Single encoding perspective. The multimodal encoding layers of current models only consider encoding from a single perspective that is either memory-based or expansion-based, which limits representational capacity and leads to information loss. 3) Poor multimodal alignment. Embeddings from different modalities are distributed in distinct and heterogeneous semantic spaces, and need to be effectively mapped into a shared semantic space to achieve accurate alignment. However, existing methods typically rely on simple operations such as direct product of behavioral information, which cannot adequately capture the complex and high-order relationships among modalities. As a result, precise alignment across multiple modalities is difficult to achieve. To address these issues, we propose a novel model named DPRec. This model considers encoding from both memory and expansion perspectives and introduces hypergraphs for multi-level precise cross-modal alignment. The model is evaluated through extensive experiments on three real-world datasets. Experimental results confirm the effectiveness of DPRec, demonstrating its superiority over state-of-the-art approaches in improving recommendation accuracy.

multimodal recommendation  /  feature representations  /  encoding perspective  /  hypergraph  /  cross-modal alignment
李佚名, 于亚新, 于之晟, 司一廷, 叶育松. 基于大语言模型的双视角多级跨模态推荐. 计算机研究与发展, 2026 , 63 (1) : 147 -161 . DOI: 10.7544/issn1000-1239.202550040
Yiming Li, Yaxin Yu, Zhisheng Yu, Yiting Si, Yusong Ye. Dual-Perspective Multi-Level Cross-Modal Recommendation Based on Large Language Models[J]. Journal of Computer Research and Development, 2026 , 63 (1) : 147 -161 . DOI: 10.7544/issn1000-1239.202550040
Year 2026 volume 63 Issue 1
316
155
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202550040
  • Receive Date:2025-01-15
  • Online Date:2026-11-06
  • Published:2026-01-01
Article Data
Affiliations
History
  • Received:2025-01-15
  • Revised:2025-08-27
  • Accepted:2025-09-02
Affiliations
    School of Computer Science and Engineering, Northeastern University, Shenyang 110169
    Key Laboratory of Intelligent Computing in Medical Image (Northeastern University), Ministry of Education, Shenyang 110169
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202550040
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT