收藏切换
Multimodal Sentiment Analysis Method for Sentimental Semantic Inconsistency
收藏切换
Yuanyi Luo, Rui Wu*, Jiafeng Liu, Xianglong Tang
Journal of Computer Research and Development | 2025, 62(2) : 374 - 382
Less
收藏切换
Journal of Computer Research and Development | 2025, 62(2): 374-382
• Artificial Intelligence •
Multimodal Sentiment Analysis Method for Sentimental Semantic Inconsistency
Full
Yuanyi Luo, Rui Wu*, Jiafeng Liu, Xianglong Tang
Affiliations
  • College of Computer Science and Technology, Harbin Institute of Technology, Harbin 150006
  • born in 1996. PhD candidate. His main research interest includes multimodal learning

    born in 1976. PhD, associate professor, PhD supervisor. His main research interests include pattern recognition and multimodal learning

    born in 1968. PhD, associate professor. His main research interests include pattern recognition and machine learning

    born in 1960. PhD, professor, PhD supervisor. His main research interests include pattern recognition and computer vision

About Author:

born in 1996. PhD candidate. His main research interest includes multimodal learning

born in 1976. PhD, associate professor, PhD supervisor. His main research interests include pattern recognition and multimodal learning

born in 1968. PhD, associate professor. His main research interests include pattern recognition and machine learning

born in 1960. PhD, professor, PhD supervisor. His main research interests include pattern recognition and computer vision

Published: 2025-02-01 doi: 10.7544/issn1000-1239.202330199
Outline
收藏切换

Multimodal sentiment analysis is a multimodal task that uses multiple modalities of subjective information to analyze sentiment. In some scenarios, the sentimental expression in different modalities is inconsistent, even contradictory, which will weaken the effect of multimodal collaborative decision-making. In this paper, a multimodal learning method is proposed to learn the modal feature representations with consistent sentimental semantics. In order to improve the common feature representation of different modalities and learn the dynamic interaction between modalities without affecting the original information, we first learn the common feature representation of each modality, and then use cross attention to enable one modality to effectively obtain auxiliary information from the common feature representations of other modalities. In multimodal fusion, we propose a multimodal attention, which is used to weighted concatenate modal feature representations, in order to increase the expression of contributed modalities and suppress the influence of weak modalities. The experimental results of the proposed method on the sentiment analysis datasets MOSI, MOSEI, CH-SIMS are better than those of the compared models, indicating the necessity and rationality of considering the problem of sentimental semantic inconsistency in multimodal sentiment analysis.

multimodal sentiment analysis  /  common features  /  sentimental semantic inconsistency  /  attention mechanism
罗渊贻, 吴锐, 刘家锋, 唐降龙. 面向情感语义不一致的多模态情感分析方法. 计算机研究与发展, 2025 , 62 (2) : 374 -382 . DOI: 10.7544/issn1000-1239.202330199
Yuanyi Luo, Rui Wu, Jiafeng Liu, Xianglong Tang. Multimodal Sentiment Analysis Method for Sentimental Semantic Inconsistency[J]. Journal of Computer Research and Development, 2025 , 62 (2) : 374 -382 . DOI: 10.7544/issn1000-1239.202330199
Year 2025 volume 62 Issue 2
116
51
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202330199
  • Receive Date:2023-03-27
  • Online Date:2026-11-10
  • Published:2025-02-01
Article Data
Affiliations
History
  • Received:2023-03-27
  • Revised:2024-01-23
Affiliations
    College of Computer Science and Technology, Harbin Institute of Technology, Harbin 150006
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202330199
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT