Article(id=1262079398585483542, tenantId=1146029695717560320, journalId=1260987750510510108, issueId=1262079396291141802, articleNumber=null, orderNo=null, doi=10.19651/j.cnki.emt.2519542, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=null, receivedDate=1754323200000, receivedDateStr=2025-08-05, revisedDate=null, revisedDateStr=null, acceptedDate=null, acceptedDateStr=null, onlineDate=1778832776923, onlineDateStr=2026-05-15, pubDate=null, pubDateStr=null, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1778832776923, onlineIssueDateStr=2026-05-15, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1778832776923, creator=13701087609, updateTime=1778832776923, updator=13701087609, issue=Issue{id=1262079396291141802, tenantId=1146029695717560320, journalId=1260987750510510108, year='2026', volume='49', issue='6', pageStart='1', pageEnd='256', issueExtLink='null', onlineDate='null', pubDate='null', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=1, issueType=1, specialIssue=null, createTime=1778832776377, creator=13701087609, updateTime=1778832914473, updator=13701087609, preIssue=null, nextIssue=null, ext={EN=IssueExt(id=1262079975688122904, tenantId=1146029695717560320, journalId=1260987750510510108, issueId=1262079396291141802, language=EN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=), CN=IssueExt(id=1262079975688122905, tenantId=1146029695717560320, journalId=1260987750510510108, issueId=1262079396291141802, language=CN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=)}, issueFiles=null}, startPage=20, endPage=28, ext={EN=ArticleExt(id=1262079400833630501, articleId=1262079398585483542, tenantId=1146029695717560320, journalId=1260987750510510108, language=EN, title=Research on multimodal sentiment analysis technology based on conversations, columnId=1262079399520755889, journalTitle=Electronic Measurement Technology, columnName=Research and Design, runingTitle=null, highlight=null, articleAbstract=

Focused on the issue that multimodal emotion recognition in conversation (MERC) is difficult to effectively capture cross-modal semantic associations in conversation rounds and has limited discrimination ability for minority classes and semantically confusing classes of emotions, a new multimodal sentiment analysis model (FuseNet) is proposed. This model adopts the bidirectional attention dialogue encoder (BiDRN) to capture the context dependency of the dialogue, effectively integrates audio and visual cues from different speakers, and realizes dynamic multimodal fusion through the Hi-gated fusion module based on the hierarchical gated mechanism. Meanwhile, class-aware multimodal contrastive (CAMC) loss is introduced to enhance the inter-class discriminability and improve the discrimination ability of minority classes and semantically similar sentiment categories. Experimental results on the two benchmark ERC datasets of IEMOCAP and MELD show that compared with the current advanced model CORECT, the F1 score of the proposed framework has improved by 2.91% and 2.00%, respectively, which are better than the existing baseline model in terms of classification performance in most emotions, especially in identifying a few classes and semantic similar categories of emotions.

, correspAuthors=null, authorNote=null, correspAuthorsNote=null, copyrightStatement=null, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=null, magXml=null, pdfUrl=null, pdf=null, pdfFileSize=null, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=null, mapNumber=null, authorCompany=null, fund=null, authors=null, authorsList=Yafang Zhao, Zhijian Liang), CN=ArticleExt(id=1262079402876256564, articleId=1262079398585483542, tenantId=1146029695717560320, journalId=1260987750510510108, language=CN, title=基于对话的多模态情感分析技术研究, columnId=1262079402104447170, journalTitle=电子测量技术, columnName=研究与设计, runingTitle=null, highlight=null, articleAbstract=

针对多模态对话情感识别(MERC)中难以有效捕捉对话中跨模态语义关联以及对少数类和语义易混淆类情感的区分能力有限的问题,提出了一种新的多模态情感分析模型(FuseNet)。该模型采用双向注意力对话编码器(BiDRN)以捕捉对话上下文依赖,有效整合来自不同说话人的音频与视觉线索,并通过基于分层门控机制的融合模块实现动态多模态融合,同时引入类感知多模态对比(CAMC)损失以增强类间判别性,提升对少数类以及语义相近情感类别的区分能力。在IEMOCAP和MELD两个基准ERC数据集上的实验结果表明,与当前先进模型CORECT相比,FuseNet的F1分数分别提升了2.91%和2.00%,在多数情感类别的分类性能上均优于现有基线模型,尤其在识别少数类和语义相近类情感上改进显著。

, correspAuthors=null, authorNote=null, correspAuthorsNote=
梁志剑(通信作者),博士,教授,主要研究方向为人工智能、自然语言处理。E-mail:
, copyrightStatement=null, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=6YEWTJ+c/Rc0mElKIFIN5A==, magXml=5f8k4ZTuJLqrsexJieFYLQ==, pdfUrl=null, pdf=VbWP/MupCubPhBuTU6Dj/A==, pdfFileSize=3348490, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=LgqTMGDr5orwGdTvP0Vcvw==, mapNumber=null, authorCompany=null, fund=null, authors=

赵亚芳,硕士研究生,主要研究方向为人工智能、自然语言处理。E-mail:

, authorsList=赵亚芳, 梁志剑)}, authors=[Author(id=1262079404797247806, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=1508753785@qq.com, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1262079405053100357, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, authorId=1262079404797247806, language=EN, stringName=Yafang Zhao, firstName=Yafang, middleName=null, lastName=Zhao, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=School of Computer Science and Technology, North University of China, Taiyuan 030051, China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1262079405472530761, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, authorId=1262079404797247806, language=CN, stringName=赵亚芳, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=中北大学计算机科学与技术学院 太原 030051, bio={"content":"

赵亚芳,硕士研究生,主要研究方向为人工智能、自然语言处理。E-mail:

"}, bioImg=null, bioContent=

赵亚芳,硕士研究生,主要研究方向为人工智能、自然语言处理。E-mail:

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1262079404407177527, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, xref=null, ext=[AuthorCompanyExt(id=1262079404449120568, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, companyId=1262079404407177527, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=School of Computer Science and Technology, North University of China, Taiyuan 030051, China), AuthorCompanyExt(id=1262079404461703481, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, companyId=1262079404407177527, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=中北大学计算机科学与技术学院 太原 030051)])]), Author(id=1262079405816463692, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=116585916@qq.com, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1262079406013595987, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, authorId=1262079405816463692, language=EN, stringName=Zhijian Liang, firstName=Zhijian, middleName=null, lastName=Liang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=School of Computer Science and Technology, North University of China, Taiyuan 030051, China, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null), CN=AuthorExt(id=1262079406651130201, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, authorId=1262079405816463692, language=CN, stringName=梁志剑, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=中北大学计算机科学与技术学院 太原 030051, bio=null, bioImg=null, bioContent=null, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1262079404407177527, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, xref=null, ext=[AuthorCompanyExt(id=1262079404449120568, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, companyId=1262079404407177527, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=School of Computer Science and Technology, North University of China, Taiyuan 030051, China), AuthorCompanyExt(id=1262079404461703481, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, companyId=1262079404407177527, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=中北大学计算机科学与技术学院 太原 030051)])])], keywords=[Keyword(id=1262079407091532128, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, language=EN, orderNo=1, keyword=multimodal emotion recognition in conversation), Keyword(id=1262079408723116400, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, language=EN, orderNo=2, keyword=bidirectional attention), Keyword(id=1262079409234821504, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, language=EN, orderNo=3, keyword=hierarchical gated mechanism), Keyword(id=1262079409696194955, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, language=EN, orderNo=4, keyword=dynamic multimodal fusion), Keyword(id=1262079410434392481, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, language=EN, orderNo=5, keyword=contrastive loss), Keyword(id=1262079411264864688, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, language=CN, orderNo=1, keyword=多模态对话情感识别), Keyword(id=1262079413492040123, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, language=CN, orderNo=2, keyword=双向注意力), Keyword(id=1262079414762914256, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, language=CN, orderNo=3, keyword=分层门控机制), Keyword(id=1262079415622746585, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, language=CN, orderNo=4, keyword=动态多模态融合), Keyword(id=1262079419410203110, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, language=CN, orderNo=5, keyword=对比损失)], refs=null, funds=[Fund(id=1262079421431857654, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, awardId=2024SZ23, language=CN, fundingSource=山西省2024年度研究生教育创新计划项目(2024SZ23), fundOrder=null, country=null), Fund(id=1262079422614651390, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, awardId=J20240839, language=CN, fundingSource=2024年山西省高等学校教学改革创新项目(J20240839), fundOrder=null, country=null)], companyList=[AuthorCompany(id=1262079404407177527, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, xref=null, ext=[AuthorCompanyExt(id=1262079404449120568, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, companyId=1262079404407177527, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=School of Computer Science and Technology, North University of China, Taiyuan 030051, China), AuthorCompanyExt(id=1262079404461703481, tenantId=1146029695717560320, journalId=1260987750510510108, articleId=1262079398585483542, companyId=1262079404407177527, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=中北大学计算机科学与技术学院 太原 030051)])], figs=null, attaches=null, journal=Journal(id=1260987432859090969, delFlag=0, nameCn=电子测量技术, nameEn=Electronic Measurement Technology, nameHistory1=null, nameHistory2=null, issn=1002-7300, eissn=null, cn=11-2175/TN, coden=null, periodic=3, language=CN, oaType=null, ccby=null, superviseOffice=null, ownerOffice=null, pubOffice=null, editorOffice=null, officeType=null, aims=null, clcCode=null, officeProv=null, officeCity=null, officeAddr=null, officeZip=null, officeEmail=null, officePhone=null, editDirector=null, officeDirector=null, officeDirectorPhone=null, officeStaffNum=null, officeEmpNum=null, coverPicUrl=pKFQJafOL5ABLx3QxMRT/g==, journalPrice=null, startedYear=null, abbrevIsoEn=Electronic Measurement Technology, journalRemark=null, publicationField=null, createdTime=1778572432012, updatedTime=1778572709318, createdBy=18614031015, updatedBy=13701087609, firstLetterCn=E, firstLetterEn=E, subjectCode=Engineering, subjectName=null, subjectCodeEn=Engineering, subjectNameEn=null, picCn=pKFQJafOL5ABLx3QxMRT/g==, picEn=VvWY+aS5ZQKZvXV5uqOm0Q==, jcr=null, cjcr=null, exts=[JournalExt(id=1260988596186694515, language=CN, name=电子测量技术, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1778572709366, updatedTime=1778572709366, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://emt.etmchina.com/emt/author/login, submissionEditorUrl=https://emt.etmchina.com/emt/editor/login, submissionReviewUrl=https://emt.etmchina.com/emt/reviewer/login, submissionCeEditorUrl=, submissionAeEditorUrl=, option={"copyright":""}), JournalExt(id=1260988596245414772, language=EN, name=Electronic Measurement Technology, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1778572709380, updatedTime=1778572709380, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://emt.etmchina.com/emt/author/login, submissionEditorUrl=https://emt.etmchina.com/emt/editor/login, submissionReviewUrl=https://emt.etmchina.com/emt/reviewer/login, submissionCeEditorUrl=, submissionAeEditorUrl=, option={"copyright":""})], databaseList=null, tenantJournalId=1260987750510510108, websiteList=[Website(id=1260990269097390900, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1260987750510510108, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/dzcljs/CN, language=CN, createTime=1778573108219, createBy=18614031015, updateTime=1778576584079, updateBy=18614031015, name=电子测量技术-中文, tplId=1146099689490845704, title=电子测量技术, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1261004970938216792, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1260990269097390900, code=articleTextType, value=kx, createTime=1778576613411, updateTime=1778576613411, creator=18614031015, updator=18614031015), WebsiteProps(id=1261004970875302229, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1260990269097390900, code=banner, value=null, createTime=1778576613396, updateTime=1778576613396, creator=18614031015, updator=18614031015), WebsiteProps(id=1261004970959188315, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1260990269097390900, code=grayFlag, value=0, createTime=1778576613416, updateTime=1778576613416, creator=18614031015, updator=18614031015), WebsiteProps(id=1261004970866913620, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1260990269097390900, code=logo, value=https://castjournals.cast.org.cn/joweb/dzcljs/CN/file/pic?fileId=sKVVjY9hwKgYWVksbpAtUw==, createTime=1778576613394, updateTime=1778576613394, creator=18614031015, updator=18614031015), WebsiteProps(id=1261004970975965533, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1260990269097390900, code=minRunFlag, value=0, createTime=1778576613420, updateTime=1778576613420, creator=18614031015, updator=18614031015), WebsiteProps(id=1261004970892079447, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1260990269097390900, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/dzcljs/CN/file/pic, createTime=1778576613400, updateTime=1778576613400, creator=18614031015, updator=18614031015), WebsiteProps(id=1261004970967576924, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1260990269097390900, code=silenceFlag, value=0, createTime=1778576613418, updateTime=1778576613418, creator=18614031015, updator=18614031015), WebsiteProps(id=1261004970883690838, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1260990269097390900, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_cn_619/, createTime=1778576613398, updateTime=1778576613398, creator=18614031015, updator=18614031015), WebsiteProps(id=1261004970946605401, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1260990269097390900, code=themeColor, value=null, createTime=1778576613413, updateTime=1778576613413, creator=18614031015, updator=18614031015), WebsiteProps(id=1261004970950799706, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1260990269097390900, code=themeStyle, value=null, createTime=1778576613414, updateTime=1778576613414, creator=18614031015, updator=18614031015)]), Website(id=1260990269269357367, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1260987750510510108, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/dzcljs/EN, language=EN, createTime=1778573108260, createBy=18614031015, updateTime=1778576580634, updateBy=18614031015, name=电子测量技术-英文, tplId=1146101810881728533, title=Electronic Measurement Technology, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1261004931830526286, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1260990269269357367, code=articleTextType, value=kx, createTime=1778576604087, updateTime=1778576604087, creator=18614031015, updator=18614031015), WebsiteProps(id=1261004931809554763, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1260990269269357367, code=banner, value=null, createTime=1778576604082, updateTime=1778576604082, creator=18614031015, updator=18614031015), WebsiteProps(id=1261004931855692113, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1260990269269357367, code=grayFlag, value=0, createTime=1778576604093, updateTime=1778576604093, creator=18614031015, updator=18614031015), WebsiteProps(id=1261004931801166154, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1260990269269357367, code=logo, value=https://castjournals.cast.org.cn/joweb/dzcljs/EN/file/pic?fileId=sKVVjY9hwKgYWVksbpAtUw==, createTime=1778576604080, updateTime=1778576604080, creator=18614031015, updator=18614031015), WebsiteProps(id=1261004931893440851, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1260990269269357367, code=minRunFlag, value=0, createTime=1778576604102, updateTime=1778576604102, creator=18614031015, updator=18614031015), WebsiteProps(id=1261004931822137677, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1260990269269357367, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/dzcljs/EN/file/pic, createTime=1778576604085, updateTime=1778576604085, creator=18614031015, updator=18614031015), WebsiteProps(id=1261004931885052242, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1260990269269357367, code=silenceFlag, value=0, createTime=1778576604100, updateTime=1778576604100, creator=18614031015, updator=18614031015), WebsiteProps(id=1261004931813749068, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1260990269269357367, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_en_623/, createTime=1778576604083, updateTime=1778576604083, creator=18614031015, updator=18614031015), WebsiteProps(id=1261004931838914895, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1260990269269357367, code=themeColor, value=null, createTime=1778576604089, updateTime=1778576604089, creator=18614031015, updator=18614031015), WebsiteProps(id=1261004931847303504, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1260990269269357367, code=themeStyle, value=null, createTime=1778576604091, updateTime=1778576604091, creator=18614031015, updator=18614031015)])], journalTitle=电子测量技术, weixinUrl=null, journalUrl=null, iacademicId=null, status=1, seqNo=null, journalTitleEn=Electronic Measurement Technology, journalPhotoCn=pKFQJafOL5ABLx3QxMRT/g==, journalPhotoEn=VvWY+aS5ZQKZvXV5uqOm0Q==, journalFirstLetter=E, journalRecommend=null, journalNew=null, journalCollection=null, jcrJf=null, cjcrJf=null, jcrJfStr=null, cjcrJfStr=null, submissionFirstDecision=null, sciSubjectClassification=null, casSubjectClassification=null, citeScore=null, totalCitationFrequency=null, icpCode=null, psCode=null, advertisingLicenseCode=null, copyrightInformation=null, country=null, option=, provinceCode=null, provinceName=null, collectFlag=false), detailUrlCn=https://castjournals.cast.org.cn/joweb/dzcljs/CN/10.19651/j.cnki.emt.2519542, detailUrlEn=https://castjournals.cast.org.cn/joweb/dzcljs/EN/10.19651/j.cnki.emt.2519542, pdfUrlCn=https://castjournals.cast.org.cn/joweb/dzcljs/CN/PDF/10.19651/j.cnki.emt.2519542, pdfUrlEn=https://castjournals.cast.org.cn/joweb/dzcljs/EN/PDF/10.19651/j.cnki.emt.2519542, aliStartDate=null, aliEndDate=null, collectionFlag=false, citedCount=null, citedUrl=null, reference=null)
收藏切换
基于对话的多模态情感分析技术研究
收藏切换
PDF下载
赵亚芳 , 梁志剑
电子测量技术 | 研究与设计 2026,49(6): 20-28
收起
收藏切换
电子测量技术 | 研究与设计 2026, 49(6): 20-28
基于对话的多模态情感分析技术研究
全屏
赵亚芳 , 梁志剑
作者信息
  • 中北大学计算机科学与技术学院 太原 030051
  • 赵亚芳,硕士研究生,主要研究方向为人工智能、自然语言处理。E-mail:

通讯作者:

梁志剑(通信作者),博士,教授,主要研究方向为人工智能、自然语言处理。E-mail:
Research on multimodal sentiment analysis technology based on conversations
Yafang Zhao , Zhijian Liang
Affiliations
  • School of Computer Science and Technology, North University of China, Taiyuan 030051, China
doi: 10.19651/j.cnki.emt.2519542
文章导航
收藏切换

针对多模态对话情感识别(MERC)中难以有效捕捉对话中跨模态语义关联以及对少数类和语义易混淆类情感的区分能力有限的问题,提出了一种新的多模态情感分析模型(FuseNet)。该模型采用双向注意力对话编码器(BiDRN)以捕捉对话上下文依赖,有效整合来自不同说话人的音频与视觉线索,并通过基于分层门控机制的融合模块实现动态多模态融合,同时引入类感知多模态对比(CAMC)损失以增强类间判别性,提升对少数类以及语义相近情感类别的区分能力。在IEMOCAP和MELD两个基准ERC数据集上的实验结果表明,与当前先进模型CORECT相比,FuseNet的F1分数分别提升了2.91%和2.00%,在多数情感类别的分类性能上均优于现有基线模型,尤其在识别少数类和语义相近类情感上改进显著。

多模态对话情感识别  /  双向注意力  /  分层门控机制  /  动态多模态融合  /  对比损失

Focused on the issue that multimodal emotion recognition in conversation (MERC) is difficult to effectively capture cross-modal semantic associations in conversation rounds and has limited discrimination ability for minority classes and semantically confusing classes of emotions, a new multimodal sentiment analysis model (FuseNet) is proposed. This model adopts the bidirectional attention dialogue encoder (BiDRN) to capture the context dependency of the dialogue, effectively integrates audio and visual cues from different speakers, and realizes dynamic multimodal fusion through the Hi-gated fusion module based on the hierarchical gated mechanism. Meanwhile, class-aware multimodal contrastive (CAMC) loss is introduced to enhance the inter-class discriminability and improve the discrimination ability of minority classes and semantically similar sentiment categories. Experimental results on the two benchmark ERC datasets of IEMOCAP and MELD show that compared with the current advanced model CORECT, the F1 score of the proposed framework has improved by 2.91% and 2.00%, respectively, which are better than the existing baseline model in terms of classification performance in most emotions, especially in identifying a few classes and semantic similar categories of emotions.

multimodal emotion recognition in conversation  /  bidirectional attention  /  hierarchical gated mechanism  /  dynamic multimodal fusion  /  contrastive loss
赵亚芳, 梁志剑. 基于对话的多模态情感分析技术研究. 电子测量技术, 2026 , 49 (6) : 20 -28 . DOI: 10.19651/j.cnki.emt.2519542
Yafang Zhao, Zhijian Liang. Research on multimodal sentiment analysis technology based on conversations[J]. Electronic Measurement Technology, 2026 , 49 (6) : 20 -28 . DOI: 10.19651/j.cnki.emt.2519542
  • 山西省2024年度研究生教育创新计划项目(2024SZ23)
  • 2024年山西省高等学校教学改革创新项目(J20240839)
2026年第49卷第6期
PDF下载
28
4
引用本文
BibTeX
文章信息
doi: 10.19651/j.cnki.emt.2519542
  • 接收时间:2025-08-05
  • 首发时间:2026-05-15
补充材料
相关文章
文章信息
作者
出版历史
  • 收稿日期:2025-08-05
基金
山西省2024年度研究生教育创新计划项目(2024SZ23)
2024年山西省高等学校教学改革创新项目(J20240839)
作者信息
    中北大学计算机科学与技术学院 太原 030051

通讯作者:

梁志剑(通信作者),博士,教授,主要研究方向为人工智能、自然语言处理。E-mail:
参考文献
分享链接
https://castjournals.cast.org.cn/joweb/dzcljs/CN/10.19651/j.cnki.emt.2519542
分享至
全文二维码

扫描看全文

引用本文
BibTeX
本文的引用情况
2种不同金属材料的力学参数

Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)

Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏