收藏切换
Adversarial Soft-Contrast Modulated Dynamic Graph Learning for Multimodal Emotion Recognition in Conversations
收藏切换
Shunjie Wang, Guoyong Cai*, Guangrui Lü
Journal of Computer Research and Development | 2026, 63(2) : 305 - 323
Less
收藏切换
Journal of Computer Research and Development | 2026, 63(2): 305-323
• Artificial Intelligence •
Adversarial Soft-Contrast Modulated Dynamic Graph Learning for Multimodal Emotion Recognition in Conversations
Full
Shunjie Wang, Guoyong Cai*, Guangrui Lü
Affiliations
  • School of Computer and Information Security, Guilin University of Electronic Technology, Guilin, Guangxi 541004
  • Guangxi Key Laboratory of Trusted Software (Guilin University of Electronic Technology), Guilin, Guangxi 541004
  • born in 1996. PhD candidate. Her main research interests include deep learning and multi-modal affective computing

    born in 1971, PhD, professor, PhD supervisor. Distinguished member of CCF. His main research interests include multi-modal affective computing, trustable AI theory and techniques

    born in 1989. PhD, master supervisor. Member of CCF. His main research interests include computer vision and multi-modal information processing

About Author:

born in 1996. PhD candidate. Her main research interests include deep learning and multi-modal affective computing

born in 1971, PhD, professor, PhD supervisor. Distinguished member of CCF. His main research interests include multi-modal affective computing, trustable AI theory and techniques

born in 1989. PhD, master supervisor. Member of CCF. His main research interests include computer vision and multi-modal information processing

Published: 2026-02-01 doi: 10.7544/issn1000-1239.202440896
Outline
收藏切换

Multimodal emotion recognition in conversations (MERC) has emerged as a prominent research focus in the domain of human-computer intelligent interaction. It demonstrates extensive applicability across diverse scenarios, including affective dialogue systems and conversational recommendation systems, thereby substantially improving user’s experience through enhanced emotional engagement. Identifying abstract emotional semantics in multimodal conversation scenarios poses a significant challenge. Current research predominantly relies on contrastive learning frameworks to extract discriminative feature. Although these approaches effectively enforce intra-class feature consistency, they inherently constrain fine-grained feature diversity. This limitation consequently impairs the model’s generalization capacity, particularly when processing samples from minority classes. In addition, current dialogue context graph learning only captures the dependence with a fixed window, neglecting the dynamic cross-correlation of the conversation flow, which can lead to either redundancy or insufficient of extracted context information. To address the aforementioned limitations, the adversarial soft-contrast modulated dynamic graph learning is proposed for MERC. Specifically, a dynamic discourse graph is firstly constructed according to the number of utterances emitted by the speaker in the conversation, which allows for dynamically modeling the conversational dependency range of the utterance in each modality to extract rich contextual information more precisely. Secondly, the adversarial soft-contrast training mechanism is designed to enhance the discrimination and robustness of the network learning. Among them, the class sample space is expanded by adding perturbations to the hidden layers of different modal feature extractors to generate adversarial samples, and the soft-contrast learning is exploited to maximize label semantic consistency of the original samples and the adversarial samples. Finally, a bi-stream graph learning strategy for different modalities is constructed, which not only explores the cross-modal consistent feature interaction guided by text semantics, but also learns the context dynamic dependencies within multimodal dialogue flow to collaboratively facilitates the complementary fusion of multimodal conversations data. Extensive experiments conducted on the IEMOCAP and MELD benchmark datasets have demonstrated that our proposed model achieves competitive results in terms of effectiveness for MERC, compared with current state-of-the-art methods.

multimodal emotion recognition in conversations (MERC)  /  dynamic discourse graph  /  adversarial soft-contrast  /  bi-stream graph learning  /  semantic consistency
王顺杰, 蔡国永, 吕光瑞. 对抗软对比调制动态图学习的多模态会话情绪识别方法. 计算机研究与发展, 2026 , 63 (2) : 305 -323 . DOI: 10.7544/issn1000-1239.202440896
Shunjie Wang, Guoyong Cai, Guangrui Lü. Adversarial Soft-Contrast Modulated Dynamic Graph Learning for Multimodal Emotion Recognition in Conversations[J]. Journal of Computer Research and Development, 2026 , 63 (2) : 305 -323 . DOI: 10.7544/issn1000-1239.202440896
Year 2026 volume 63 Issue 2
350
128
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202440896
  • Receive Date:2024-11-18
  • Online Date:2026-11-06
  • Published:2026-02-01
Article Data
Affiliations
History
  • Received:2024-11-18
  • Revised:2025-06-30
  • Accepted:2025-07-17
Affiliations
    School of Computer and Information Security, Guilin University of Electronic Technology, Guilin, Guangxi 541004
    Guangxi Key Laboratory of Trusted Software (Guilin University of Electronic Technology), Guilin, Guangxi 541004
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202440896
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT