Article(id=1327017677962457226, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1327017642604475226, articleNumber=null, orderNo=null, doi=10.7544/issn1000-1239.202440624, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=research-article, receivedDate=1721404800000, receivedDateStr=2024-07-20, revisedDate=1736870400000, revisedDateStr=2025-01-15, acceptedDate=1740931200000, acceptedDateStr=2025-03-03, onlineDate=1794315269221, onlineDateStr=2026-11-10, pubDate=1761926400000, pubDateStr=2025-11-01, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1794315269221, onlineIssueDateStr=2026-11-10, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1794315269221, creator=13701087609, updateTime=1794315269221, updator=13701087609, issue=Issue{id=1327017642604475226, tenantId=1146029695717560320, journalId=1324759914146529478, year='2025', volume='62', issue='11', pageStart='2651', pageEnd='2887', issueExtLink='null', onlineDate='null', pubDate='1761926400000', pubDateStr='2025-11-01', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=1, issueType=-1, specialIssue=null, createTime=1794315260792, creator='13701087609', updateTime=1794376986370, updator='13701087609', preIssue=null, nextIssue=null, articleTotal=null, ext={EN=IssueExt(id=1327276538975859352, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1327017642604475226, language=EN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=), CN=IssueExt(id=1327276538980053657, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1327017642604475226, language=CN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=)}, issueFiles=null, downloadFileDto=null}, startPage=2697, endPage=2709, ext={EN=ArticleExt(id=1327017678155395211, articleId=1327017677962457226, tenantId=1146029695717560320, journalId=1324759914146529478, language=EN, title=A Multimodal Cross-Domain Sentiment Analysis Algorithm Based on Feature Disentanglement Meta-Optimization, columnId=1325489891789910943, journalTitle=Journal of Computer Research and Development, columnName=Artificial Intelligence, runingTitle=null, highlight=null, articleAbstract=

Multimodal sentiment analysis aims to utilize the multimodal customer comments and other data to identify users’ sentimental tendencies. To realize cross-domain application with the domain bias, commonly used solutions are unsupervised domain adaptation methods. Nevertheless, this type of solutions focuses on the extraction of domain-invariant features, and it neglects the significance of domain-specific features at the target domain. Thus, a meta-optimization based domain-invariant and domain-specific feature disentanglement network is proposed. First, by embedding adapters into the pre-trained large model with fine-tuning fitting, the image-text fused sentiment feature encoder is accordingly constructed. Then, a feature disentanglement module is constructed on the basis of the factorization operation, which utilizes domain adversary and domain classification, together with collaborative independence constraints, respectively, to achieve knowledge-transferable domain-invariant feature embedding while extracting the domain-specific features to enhance the performance of sentiment classification at the target domain. To ensure the consistency of the overall optimization tendency for feature disentanglement and sentiment classification, a meta-learning-based meta-optimization training strategy is put forward to synergistically optimize the sentiment analysis network. Comparative experiments on bidirectional sentiment transfer tasks constructed by MVSA and Yelp datasets demonstrate that compared with other advanced image-text sentiment transfer algorithms, the proposed algorithm achieves superior performance on bidirectional sentiment transfer tasks in terms of three consensus metrics: Precision, Recall and F1 score.

, authors=Xibin Jia1, 2, Chen Li1, Luo Wang1, Muchen Zhang1, Xiaojian Liu1, Yangyang Zhang3, Jiakai Wen4, authorsList=Xibin Jia, Chen Li, Luo Wang, Muchen Zhang, Xiaojian Liu, Yangyang Zhang, Jiakai Wen, authorCompany=null, correspAuthors=null, authorNote=

born in 1969. Professor, PhD supervisor. Distinguished member of CCF. Her main research interests include visual computing, multimodal deep learning, affection computing, intelligent medical image, and behaviour understanding and computing

born in 1998. Master. Her main research interests include multimodal sentiment and transfer learning

born in 1990. PhD, lecturer. Member of CCF. His main research interests include computer vision and deep learning

born in 2002. Master candidate. Her main research interests include multimodal deep learning and computer vision

born in 1989. PhD, senior engineer. Member of CCF. His main research interests include software engineering and cybersecurity

born in 1976. Master. Professorial senior engineer. Member of CCF. Her main research interests include software and systems engineering, and standardization of IT

born in 1979. Bachelor, senior engineer. Member of CCF. His main research interests include machine translation and big data

, correspAuthorsNote=null, copyrightStatement=Copyright © 2025 Journal of Computer Research and Development. All rights reserved., copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=null, magXml=null, pdfUrl=null, pdf=null, pdfFileSize=null, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=null, mapNumber=null, fund=null), CN=ArticleExt(id=1327017680072192160, articleId=1327017677962457226, tenantId=1146029695717560320, journalId=1324759914146529478, language=CN, title=基于元优化特征解耦的多模态跨域情感分析算法, columnId=1325489892104483745, journalTitle=计算机研究与发展, columnName=人工智能, runingTitle=null, highlight=null, articleAbstract=

多模态情感分析旨在利用多模态点评等数据识别用户情感倾向. 为实现存在域偏移的跨域应用,常用无监督领域自适应方法. 然而,该类方法着重于领域不变特征提取,忽略了目标领域特定特征的重要作用. 为此,提出基于元优化的领域不变及领域特定特征解耦网络. 首先,通过嵌入情感适配器对预训练大模型微调,建立图文融合情感特征编码器. 进而,构建基于因子分解的特征解耦模块,分别利用领域对抗及领域分类、协同独立性约束,实现知识可传递的领域不变特征编码的同时,提取领域特定特征以增强目标域情感分类性能. 为保证特征解耦与情感分类的总体优化方向一致性,提出基于元学习的元优化训练策略,实现情感分析网络的协同优化. 基于MVSA和Yelp数据集构建的双向情感迁移任务的对比实验表明,较之其他先进的图文情感迁移算法,所提算法于双向情感迁移任务的精确率、召回率和F1值3项评价指标均取得了优异的性能.

, authors=贾熹滨1, 2, 李宸1, 王珞1, 张沐晨1, 刘潇健1, 张旸旸3, 温家凯4, authorsList=贾熹滨, 李宸, 王珞, 张沐晨, 刘潇健, 张旸旸, 温家凯, authorCompany=null, correspAuthors=null, authorNote=

1969年生. 教授,博士生导师. CCF杰出会员. 主要研究方向为视觉计算、多模态深度学习、情感计算、智能医学影像、行为理解与计算

1998年生. 硕士. 主要研究方向为多模态情感、迁移学习

1990年生. 博士,讲师. CCF会员. 主要研究方向为计算机视觉、深度学习

2002年生. 硕士研究生. 主要研究方向为多模态深度学习、计算机视觉

1989年生. 博士,高级工程师. CCF会员. 主要研究方向为软件工程、网络安全

1976年生. 硕士,教授级高工. CCF会员. 主要研究方向为软件与系统工程、信息技术标准化

1979年生. 学士,高级工程师. CCF会员. 主要研究方向为机器翻译、大数据

, correspAuthorsNote=null, copyrightStatement=版权所有 © 2025 计算机研究与发展编辑部, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=n68m6lzjOPpnML+QtLNLCA==, magXml=wUcFmeos4SsHAPwdyCu6cA==, pdfUrl=null, pdf=h77Y0pqKuzVNvzodvEOI0g==, pdfFileSize=1272749, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=BkTma+3Fo508mqScqKBCKw==, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=oBsa8+jZ6EvkBRSJ9yPJ4w==, mapNumber=null, fund=null)}, authors=[Author(id=1327017682215481518, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=jiaxibin@bjut.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017682299367601, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017682215481518, language=EN, stringName=Xibin Jia, firstName=Xibin, middleName=null, lastName=Jia, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1College of Computer, Beijing University of Technology, Beijing 100124
2Beijing Institute of Artificial Intelligence (Beijing University of Technology), Beijing 100124, bio={"content":"

born in 1969. Professor, PhD supervisor. Distinguished member of CCF. Her main research interests include visual computing, multimodal deep learning, affection computing, intelligent medical image, and behaviour understanding and computing

"}, bioImg=null, bioContent=

born in 1969. Professor, PhD supervisor. Distinguished member of CCF. Her main research interests include visual computing, multimodal deep learning, affection computing, intelligent medical image, and behaviour understanding and computing

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017682400030898, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017682215481518, language=CN, stringName=贾熹滨, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1北京工业大学计算机学院 北京 100124
2北京人工智能研究院(北京工业大学) 北京 100124, bio={"img":"TIEcl09yccjbwRwXOJgOew==","content":"

1969年生. 教授,博士生导师. CCF杰出会员. 主要研究方向为视觉计算、多模态深度学习、情感计算、智能医学影像、行为理解与计算

"}, bioImg=TIEcl09yccjbwRwXOJgOew==, bioContent=

1969年生. 教授,博士生导师. CCF杰出会员. 主要研究方向为视觉计算、多模态深度学习、情感计算、智能医学影像、行为理解与计算

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017680281907361, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, xref=1, ext=[AuthorCompanyExt(id=1327017680290295970, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Computer, Beijing University of Technology, Beijing 100124), AuthorCompanyExt(id=1327017680298684579, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1北京工业大学计算机学院 北京 100124)]), AuthorCompany(id=1327017680361599140, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, xref=2, ext=[AuthorCompanyExt(id=1327017680369987749, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680361599140, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2Beijing Institute of Artificial Intelligence (Beijing University of Technology), Beijing 100124), AuthorCompanyExt(id=1327017680374182054, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680361599140, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2北京人工智能研究院(北京工业大学) 北京 100124)])]), Author(id=1327017682471334068, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017682555220150, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017682471334068, language=EN, stringName=Chen Li, firstName=Chen, middleName=null, lastName=Li, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Computer, Beijing University of Technology, Beijing 100124, bio={"content":"

born in 1998. Master. Her main research interests include multimodal sentiment and transfer learning

"}, bioImg=null, bioContent=

born in 1998. Master. Her main research interests include multimodal sentiment and transfer learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017682630717623, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017682471334068, language=CN, stringName=李宸, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1北京工业大学计算机学院 北京 100124, bio={"img":"FQHInM+ds8Jf7jmHTtak1A==","content":"

1998年生. 硕士. 主要研究方向为多模态情感、迁移学习

"}, bioImg=FQHInM+ds8Jf7jmHTtak1A==, bioContent=

1998年生. 硕士. 主要研究方向为多模态情感、迁移学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017680281907361, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, xref=1, ext=[AuthorCompanyExt(id=1327017680290295970, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Computer, Beijing University of Technology, Beijing 100124), AuthorCompanyExt(id=1327017680298684579, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1北京工业大学计算机学院 北京 100124)])]), Author(id=1327017682697826489, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017682815267003, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017682697826489, language=EN, stringName=Luo Wang, firstName=Luo, middleName=null, lastName=Wang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Computer, Beijing University of Technology, Beijing 100124, bio={"content":"

born in 1990. PhD, lecturer. Member of CCF. His main research interests include computer vision and deep learning

"}, bioImg=null, bioContent=

born in 1990. PhD, lecturer. Member of CCF. His main research interests include computer vision and deep learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017682882375868, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017682697826489, language=CN, stringName=王珞, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1北京工业大学计算机学院 北京 100124, bio={"img":"27Og6Tjm3oX+L5YKUASW1Q==","content":"

1990年生. 博士,讲师. CCF会员. 主要研究方向为计算机视觉、深度学习

"}, bioImg=27Og6Tjm3oX+L5YKUASW1Q==, bioContent=

1990年生. 博士,讲师. CCF会员. 主要研究方向为计算机视觉、深度学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017680281907361, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, xref=1, ext=[AuthorCompanyExt(id=1327017680290295970, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Computer, Beijing University of Technology, Beijing 100124), AuthorCompanyExt(id=1327017680298684579, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1北京工业大学计算机学院 北京 100124)])]), Author(id=1327017682966261950, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017683054342336, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017682966261950, language=EN, stringName=Muchen Zhang, firstName=Muchen, middleName=null, lastName=Zhang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Computer, Beijing University of Technology, Beijing 100124, bio={"content":"

born in 2002. Master candidate. Her main research interests include multimodal deep learning and computer vision

"}, bioImg=null, bioContent=

born in 2002. Master candidate. Her main research interests include multimodal deep learning and computer vision

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017683121451201, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017682966261950, language=CN, stringName=张沐晨, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1北京工业大学计算机学院 北京 100124, bio={"img":"zlliAjrmwfEUvyIzZurJfw==","content":"

2002年生. 硕士研究生. 主要研究方向为多模态深度学习、计算机视觉

"}, bioImg=zlliAjrmwfEUvyIzZurJfw==, bioContent=

2002年生. 硕士研究生. 主要研究方向为多模态深度学习、计算机视觉

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017680281907361, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, xref=1, ext=[AuthorCompanyExt(id=1327017680290295970, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Computer, Beijing University of Technology, Beijing 100124), AuthorCompanyExt(id=1327017680298684579, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1北京工业大学计算机学院 北京 100124)])]), Author(id=1327017683184365763, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017683259863237, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017683184365763, language=EN, stringName=Xiaojian Liu, firstName=Xiaojian, middleName=null, lastName=Liu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Computer, Beijing University of Technology, Beijing 100124, bio={"content":"

born in 1989. PhD, senior engineer. Member of CCF. His main research interests include software engineering and cybersecurity

"}, bioImg=null, bioContent=

born in 1989. PhD, senior engineer. Member of CCF. His main research interests include software engineering and cybersecurity

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017683331166406, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017683184365763, language=CN, stringName=刘潇健, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1北京工业大学计算机学院 北京 100124, bio={"img":"HQRa3rME15up6GdI6fV8+w==","content":"

1989年生. 博士,高级工程师. CCF会员. 主要研究方向为软件工程、网络安全

"}, bioImg=HQRa3rME15up6GdI6fV8+w==, bioContent=

1989年生. 博士,高级工程师. CCF会员. 主要研究方向为软件工程、网络安全

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017680281907361, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, xref=1, ext=[AuthorCompanyExt(id=1327017680290295970, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Computer, Beijing University of Technology, Beijing 100124), AuthorCompanyExt(id=1327017680298684579, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1北京工业大学计算机学院 北京 100124)])]), Author(id=1327017683402469576, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, orderNo=5, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017683473772746, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017683402469576, language=EN, stringName=Yangyang Zhang, firstName=Yangyang, middleName=null, lastName=Zhang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=3, address=3China Electronics Standardization Institute, Beijing 100007, bio={"content":"

born in 1976. Master. Professorial senior engineer. Member of CCF. Her main research interests include software and systems engineering, and standardization of IT

"}, bioImg=null, bioContent=

born in 1976. Master. Professorial senior engineer. Member of CCF. Her main research interests include software and systems engineering, and standardization of IT

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017683540881611, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017683402469576, language=CN, stringName=张旸旸, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=3, address=3中国电子技术标准化研究院 北京 100007, bio={"img":"+H5jwYnZyfZEFj3eFyLepw==","content":"

1976年生. 硕士,教授级高工. CCF会员. 主要研究方向为软件与系统工程、信息技术标准化

"}, bioImg=+H5jwYnZyfZEFj3eFyLepw==, bioContent=

1976年生. 硕士,教授级高工. CCF会员. 主要研究方向为软件与系统工程、信息技术标准化

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017682022543527, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, xref=3, ext=[AuthorCompanyExt(id=1327017682030932136, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017682022543527, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3China Electronics Standardization Institute, Beijing 100007), AuthorCompanyExt(id=1327017682039320745, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017682022543527, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3中国电子技术标准化研究院 北京 100007)])]), Author(id=1327017683612184781, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, orderNo=6, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017683683487951, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017683612184781, language=EN, stringName=Jiakai Wen, firstName=Jiakai, middleName=null, lastName=Wen, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=4, address=4Guangxi Daring Technology Co., Ltd., Nanning 530006, bio={"content":"

born in 1979. Bachelor, senior engineer. Member of CCF. His main research interests include machine translation and big data

"}, bioImg=null, bioContent=

born in 1979. Bachelor, senior engineer. Member of CCF. His main research interests include machine translation and big data

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017683746402512, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017683612184781, language=CN, stringName=温家凯, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=4, address=4广西达译科技有限公司 南宁 530006, bio={"img":"f/WmoCwRJfpHEChdChh2XQ==","content":"

1979年生. 学士,高级工程师. CCF会员. 主要研究方向为机器翻译、大数据

"}, bioImg=f/WmoCwRJfpHEChdChh2XQ==, bioContent=

1979年生. 学士,高级工程师. CCF会员. 主要研究方向为机器翻译、大数据

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017682123206826, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, xref=4, ext=[AuthorCompanyExt(id=1327017682131595435, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017682123206826, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=4Guangxi Daring Technology Co., Ltd., Nanning 530006), AuthorCompanyExt(id=1327017682139984044, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017682123206826, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=4广西达译科技有限公司 南宁 530006)])])], keywords=[Keyword(id=1327017683842871505, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=EN, orderNo=1, keyword=multimodal sentiment analysis), Keyword(id=1327017683901591762, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=EN, orderNo=2, keyword=unsupervised domain adaptation), Keyword(id=1327017683977089235, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=EN, orderNo=3, keyword=cross-domain sentiment classification), Keyword(id=1327017684035809492, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=EN, orderNo=4, keyword=feature disentanglement), Keyword(id=1327017684102918357, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=EN, orderNo=5, keyword=meta-optimization), Keyword(id=1327017684182610134, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=CN, orderNo=1, keyword=多模态情感分析), Keyword(id=1327017684258107607, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=CN, orderNo=2, keyword=无监督领域自适应), Keyword(id=1327017684316827864, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=CN, orderNo=3, keyword=跨领域情感分类), Keyword(id=1327017684379742425, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=CN, orderNo=4, keyword=特征解耦), Keyword(id=1327017684451045594, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=CN, orderNo=5, keyword=元优化)], refs=[Reference(id=1327017687546441962, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=1, rfOrder=0, authorNames=null, journalName=null, refType=null, unstructuredReference=Das R, Singh T D. Multimodal sentiment analysis: A survey of methods, trends, and challenges[J]. ACM Computing Surveys, 2023, 55(13s): 1−38, articleTitle=null, refAbstract=null), Reference(id=1327017687609356523, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=2, rfOrder=1, authorNames=null, journalName=null, refType=null, unstructuredReference=Chan J Y L, Bea K T, Leow S M H, et al. State of the art: A review of sentiment analysis based on sequential transfer learning[J]. Artificial Intelligence Review, 2023, 56(1): 749−780, articleTitle=null, refAbstract=null), Reference(id=1327017687760351468, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=3, rfOrder=2, authorNames=null, journalName=null, refType=null, unstructuredReference=Singhal P, Walambe R, Ramanna S, et al. Domain adaptation: Challenges, methods, datasets, and applications[J]. IEEE Access, 2023, 11: 6973−7020, articleTitle=null, refAbstract=null), Reference(id=1327017687840043245, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=4, rfOrder=3, authorNames=null, journalName=null, refType=null, unstructuredReference=Azuma C, Ito T, Shimobaba T. Adversarial domain adaptation using contrastive learning[J]. Engineering Applications of Artificial Intelligence, 2023, 123: 106394, articleTitle=null, refAbstract=null), Reference(id=1327017687902957806, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=5, rfOrder=4, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhou Qianyu, Gu Qiqi, Pang Jiangmiao, et al. Self-adversarial disentangling for specific domain adaptation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(7): 8954−8968, articleTitle=null, refAbstract=null), Reference(id=1327017687982649583, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=6, rfOrder=5, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Jingjing, Chen Erpeng, Ding Zhengming, et al. Maximum density divergence for domain adaptation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43(11): 3918−3930, articleTitle=null, refAbstract=null), Reference(id=1327017688062341360, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=7, rfOrder=6, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhu Yongchun, Zhuang Fuzhen, Wang Jindong, et al. Deep subdomain adaptation network for image classification[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 32(4): 1713−1722, articleTitle=null, refAbstract=null), Reference(id=1327017688129450225, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=8, rfOrder=7, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhao Han, Des Combes R T, Zhang Kun, et al. On learning invariant representations for domain adaptation[C]//Proc of the 36th Int Conf on Machine Learning. New York: PMLR, 2019: 7523−7532, articleTitle=null, refAbstract=null), Reference(id=1327017688196559090, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=9, rfOrder=8, authorNames=null, journalName=null, refType=null, unstructuredReference=Johansson F D, Sontag D, Ranganath R. Support and invertibility in domain-invariant representations[C]//Proc of the 22nd Int Conf on Artificial Intelligence and Statistics. New York: PMLR, 2019: 527−536, articleTitle=null, refAbstract=null), Reference(id=1327017688263667955, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=10, rfOrder=9, authorNames=null, journalName=null, refType=null, unstructuredReference=Zadeh A, Chen Minghai, Poria S, et al. Tensor fusion network for multimodal sentiment analysis[C]//Proc of the 2017 Conf on Empirical Methods in Natural Language Processing. New York: PMLR, 2017: 1103−1114, articleTitle=null, refAbstract=null), Reference(id=1327017688334971124, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=11, rfOrder=10, authorNames=null, journalName=null, refType=null, unstructuredReference=Truong Q T, Lauw H W. VistaNet: Visual aspect attention network for multimodal sentiment analysis[C]//Proc of the 33rd Association for the Advancement of Artificial Intelligence. Palo Alto, CA: AAAI, 2019: 305−312, articleTitle=null, refAbstract=null), Reference(id=1327017688406274293, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=12, rfOrder=11, authorNames=null, journalName=null, refType=null, unstructuredReference=Gui Tao, Zhu Liang, Zhang Qi, et al. Cooperative multimodal approach to depression detection in Twitter[C]//Proc of the 33rd Association for the Advancement of Artificial Intelligence. Palo Alto, CA: AAAI, 2019: 110−117, articleTitle=null, refAbstract=null), Reference(id=1327017688473383158, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=13, rfOrder=12, authorNames=null, journalName=null, refType=null, unstructuredReference=Ling Yan, Yu Jianfei, Xia Rui. Vision-language pre-training for multimodal aspect-based sentiment analysis[C]//Proc of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg, PA: ACL, 2022: 2149−2159, articleTitle=null, refAbstract=null), Reference(id=1327017688540492023, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=14, rfOrder=13, authorNames=null, journalName=null, refType=null, unstructuredReference=Ganin Y, Ustinova E, Ajakan H, et al. Domain-adversarial training of neural networks[J]. Journal of Machine Learning Research, 2016, 17(59): 1−35, articleTitle=null, refAbstract=null), Reference(id=1327017688611795192, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=15, rfOrder=14, authorNames=null, journalName=null, refType=null, unstructuredReference=Jia Xibin, Li Chen, Zeng Meng, et al. An improved unified domain adversarial category-wise alignment network for unsupervised cross-domain sentiment classification[J]. Engineering Applications of Artificial Intelligence, 2023, 126: 107108, articleTitle=null, refAbstract=null), Reference(id=1327017688674709753, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=16, rfOrder=15, authorNames=null, journalName=null, refType=null, unstructuredReference=黄学坚, 马廷淮, 王根生. 基于样本内外协同表示和自适应融合的多模态学习方法[J]. 计算机研究与发展, 2024, 61(5): 1310−1324, articleTitle=null, refAbstract=null), Reference(id=1327017688733430010, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=16, rfOrder=16, authorNames=null, journalName=null, refType=null, unstructuredReference=Huang Xuejian, Ma Tinghuai, Wang Gensheng. Multimodal learning method based on intra- and inter-sample cooperative representation and adaptive fusion[J]. Journal of Computer Research and Development, 2024, 61(5): 1310−1324 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327017688800538875, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=17, rfOrder=17, authorNames=null, journalName=null, refType=null, unstructuredReference=Qi Fan, Yang Xiaoshan, Xu Changsheng. A unified framework for multimodal domain adaptation[C]//Proc of the 26th ACM Int Conf on Multimedia. New York: ACM, 2018: 429−437, articleTitle=null, refAbstract=null), Reference(id=1327017688863453436, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=18, rfOrder=18, authorNames=null, journalName=null, refType=null, unstructuredReference=Ma Xinhong, Zhang Tianzhu, Xu Changsheng. Deep multi-modality adversarial networks for unsupervised domain adaptation[J]. IEEE Transactions on Multimedia, 2019, 21(9): 2419−2431, articleTitle=null, refAbstract=null), Reference(id=1327017688922173693, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=19, rfOrder=19, authorNames=null, journalName=null, refType=null, unstructuredReference=Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Proc of the 31st Int Conf on Neural Information Processing Systems. New York: ACM, 2017: 6000−6010, articleTitle=null, refAbstract=null), Reference(id=1327017688989282558, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=20, rfOrder=20, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Junnan, Selvaraju R, Gotmare A, et al. Align before fuse: Vision and language representation learning with momentum distillation[J]. Advances in Neural Information Processing Systems, 2021, 34: 9694−9705, articleTitle=null, refAbstract=null), Reference(id=1327017689052197119, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=21, rfOrder=21, authorNames=null, journalName=null, refType=null, unstructuredReference=Chen Shoufa, Ge Chongjian, Tong Zhan, et al. Adaptformer: Adapting vision transformers for scalable visual recognition[J]. Advances in Neural Information Processing Systems, 2022, 35: 16664−16678, articleTitle=null, refAbstract=null), Reference(id=1327017689131888896, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=22, rfOrder=22, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Ya, Tian Xinmei, Gong Mingming, et al. Deep domain generalization via conditional invariant adversarial networks[C]//Proc of the 2018 European Conf on Computer Vision (ECCV). Berlin: Springer, 2018: 624−639, articleTitle=null, refAbstract=null), Reference(id=1327017689207386369, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=23, rfOrder=23, authorNames=null, journalName=null, refType=null, unstructuredReference=Bui M H, Tran T, Tran A, et al. Exploiting domain-specific features to enhance domain generalization[J]. Advances in Neural Information Processing Systems, 2021, 34: 21189−21201, articleTitle=null, refAbstract=null), Reference(id=1327017689274495234, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=24, rfOrder=24, authorNames=null, journalName=null, refType=null, unstructuredReference=Wei Guoqiang, Lan Cuiling, Zeng Wenjun, et al. MetaAlign: Coordinating domain alignment and classification for unsupervised domain adaptation[C]//Proc of the IEEE/CVF Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ: IEEE, 2021: 16643−16653, articleTitle=null, refAbstract=null), Reference(id=1327017690562146563, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=25, rfOrder=25, authorNames=null, journalName=null, refType=null, unstructuredReference=Niu Teng, Zhu Shiai, Pang Lei, et al. Sentiment analysis on multi-view social data[C]//Proc of the 22nd Int Conf on Multimedia Modeling. Berlin: Springer, 2016: 15−27, articleTitle=null, refAbstract=null), Reference(id=1327017690654421252, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=26, rfOrder=26, authorNames=null, journalName=null, refType=null, unstructuredReference=刘琦玮, 李俊, 顾蓓蓓, 等. TSAIE: 图像增强文本的多模态情感分析模型[J]. 数据与计算发展前沿, 2022, 4(3): 131−140, articleTitle=null, refAbstract=null), Reference(id=1327017690729918725, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=26, rfOrder=27, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Qiwei, Li Jun, Gu Beibei, et al. TSAIE: Text sentiment analysis model based on image enhancement[J]. Frontiers of Data & Computing, 2022, 4(3): 131−140 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327017690822193414, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=27, rfOrder=28, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Yuhao, Zhang Ying, Guo Wenya, et al. Learning disentangled representation for multimodal cross-domain sentiment analysis[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 34(10): 7956−7966, articleTitle=null, refAbstract=null), Reference(id=1327017690906079495, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=28, rfOrder=29, authorNames=null, journalName=null, refType=null, unstructuredReference=Baltrušaitis T, Ahuja C, Morency L P. Multimodal machine learning: A survey and taxonomy[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 41(2): 423−443, articleTitle=null, refAbstract=null), Reference(id=1327017690977382664, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=29, rfOrder=30, authorNames=null, journalName=null, refType=null, unstructuredReference=Dosovitskiy A. An image is worth 16x16 words: Transformers for image recognition at scale[J]. arXiv preprint, arXiv: 2010.11929, 2020, articleTitle=null, refAbstract=null), Reference(id=1327017691052880137, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=30, rfOrder=31, authorNames=null, journalName=null, refType=null, unstructuredReference=Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint, arXiv: 1810.04805, 2018, articleTitle=null, refAbstract=null), Reference(id=1327017691111600394, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=31, rfOrder=32, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Junnan, Li Dongxu, Xiong Caiming, et al. BLIP: Bootstrapping language-image pre-training for unified vision-language understanding and generation[C]//Proc of the 39th Int Conf on Machine Learning. New York: PMLR, 2022: 12888−12900, articleTitle=null, refAbstract=null), Reference(id=1327017691191292171, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=32, rfOrder=33, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhu Tong, Li Leida, Yang Jufeng, et al. Multimodal sentiment analysis with image-text interaction network[J]. IEEE Transactions on Multimedia, 2022, 25: 3375−3385, articleTitle=null, refAbstract=null), Reference(id=1327017691254206732, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=33, rfOrder=34, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Jingzhe, Wang Chengji, Luo Zhiming, et al. Modality-dependent sentiments exploring for multi-modal sentiment classification[C]//Proc of the 2024 IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2024: 7930−7934, articleTitle=null, refAbstract=null)], funds=null, companyList=[AuthorCompany(id=1327017680281907361, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, xref=1, ext=[AuthorCompanyExt(id=1327017680290295970, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Computer, Beijing University of Technology, Beijing 100124), AuthorCompanyExt(id=1327017680298684579, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1北京工业大学计算机学院 北京 100124)]), AuthorCompany(id=1327017680361599140, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, xref=2, ext=[AuthorCompanyExt(id=1327017680369987749, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680361599140, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2Beijing Institute of Artificial Intelligence (Beijing University of Technology), Beijing 100124), AuthorCompanyExt(id=1327017680374182054, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680361599140, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2北京人工智能研究院(北京工业大学) 北京 100124)]), AuthorCompany(id=1327017682022543527, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, xref=3, ext=[AuthorCompanyExt(id=1327017682030932136, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017682022543527, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3China Electronics Standardization Institute, Beijing 100007), AuthorCompanyExt(id=1327017682039320745, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017682022543527, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3中国电子技术标准化研究院 北京 100007)]), AuthorCompany(id=1327017682123206826, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, xref=4, ext=[AuthorCompanyExt(id=1327017682131595435, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017682123206826, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=4Guangxi Daring Technology Co., Ltd., Nanning 530006), AuthorCompanyExt(id=1327017682139984044, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017682123206826, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=4广西达译科技有限公司 南宁 530006)])], figs=[ArticleFig(id=1327017684602040539, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=EN, label=Figure 1, caption=Meta-optimization-based framework for domain-invariant and domain-specific feature disentanglement networks, figureFileSmall=wjdx3YwS99toxqN+4TbwHw==, figureFileBig=BkTma+3Fo508mqScqKBCKw==, tableContent=null), ArticleFig(id=1327017684673343708, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=CN, label=图1, caption=基于元优化的领域不变及领域特定特征解耦网络框架, figureFileSmall=wjdx3YwS99toxqN+4TbwHw==, figureFileBig=BkTma+3Fo508mqScqKBCKw==, tableContent=null), ArticleFig(id=1327017684870475997, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=EN, label=Figure 2, caption=SenA-Transformer block, figureFileSmall=a540wRllSFgQ/b9xuhAthw==, figureFileBig=3AW6e2VOmoWjUwaBGVaPpw==, tableContent=null), ArticleFig(id=1327017686552391903, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=CN, label=图2, caption=SenA-Transformer块, figureFileSmall=a540wRllSFgQ/b9xuhAthw==, figureFileBig=3AW6e2VOmoWjUwaBGVaPpw==, tableContent=null), ArticleFig(id=1327017686619500768, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=EN, label=Figure 3, caption=Interactive optimization training process for feature decoupling task and sentiment classification task, figureFileSmall=LDmpiGCdLGQRmNscBUNnjA==, figureFileBig=MDaYkHxUXuhC1f3ZkOUc7g==, tableContent=null), ArticleFig(id=1327017686682415329, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=CN, label=图3, caption=特征解耦任务和情感分类任务的交互优化训练过程, figureFileSmall=LDmpiGCdLGQRmNscBUNnjA==, figureFileBig=MDaYkHxUXuhC1f3ZkOUc7g==, tableContent=null), ArticleFig(id=1327017686774690018, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=EN, label=Table 1, caption=Comparative Experiments for the Multimodal Cross-Domain Sentiment Classification Tasks %, figureFileSmall=null, figureFileBig=null, tableContent=
方法MVSA→YelpYelp→MVSA
PreRecF1PreRecF1
注:黑体数值表示最优值;下划线数值表示次优值.
VisualSent[27]22.133.225.821.331.122.2
TextualSent[27]27.841.532.422.134.224.8
ViT[29]35.434.728.422.633.326.9
BERT[30]38.240.332.728.237.227.0
EearlySent[27]13.233.316.819.134.118.9
LateSent[27]26.236.127.620.333.124.5
TFN[10,27]33.033.131.132.231.616.4
VistaNet[11,27]40.341.333.140.440.335.1
ALBEF[20]40.842.135.041.141.735.4
BLIP[31]57.941.225.455.143.445.2
ITIN[32]47.539.631.3
MDSE[33]56.640.526.261.845.748.7
MMAN[18,27]42.945.340.143.141.436.0
MDANN[17,27]46.946.343.940.140.336.9
DACAN[15]51.747.744.643.744.539.6
DiSRAN[27]53.851.049.246.745.642.4
MDISN(本文)54.050.447.446.744.941.7
MDISN-A(本文)51.542.030.256.250.650.5
MDISN-B(本文)41.440.026.055.165.858.4
MDISN-C(本文)51.753.145.8
), ArticleFig(id=1327017686946656483, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=CN, label=表1, caption=

在多模态跨域情感分类任务中的对比实验

, figureFileSmall=null, figureFileBig=null, tableContent=
方法MVSA→YelpYelp→MVSA
PreRecF1PreRecF1
注:黑体数值表示最优值;下划线数值表示次优值.
VisualSent[27]22.133.225.821.331.122.2
TextualSent[27]27.841.532.422.134.224.8
ViT[29]35.434.728.422.633.326.9
BERT[30]38.240.332.728.237.227.0
EearlySent[27]13.233.316.819.134.118.9
LateSent[27]26.236.127.620.333.124.5
TFN[10,27]33.033.131.132.231.616.4
VistaNet[11,27]40.341.333.140.440.335.1
ALBEF[20]40.842.135.041.141.735.4
BLIP[31]57.941.225.455.143.445.2
ITIN[32]47.539.631.3
MDSE[33]56.640.526.261.845.748.7
MMAN[18,27]42.945.340.143.141.436.0
MDANN[17,27]46.946.343.940.140.336.9
DACAN[15]51.747.744.643.744.539.6
DiSRAN[27]53.851.049.246.745.642.4
MDISN(本文)54.050.447.446.744.941.7
MDISN-A(本文)51.542.030.256.250.650.5
MDISN-B(本文)41.440.026.055.165.858.4
MDISN-C(本文)51.753.145.8
), ArticleFig(id=1327017687030542564, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=EN, label=Table 2, caption=Sentiment Classification Performance Experiment on the Same Domain %, figureFileSmall=null, figureFileBig=null, tableContent=
方法MVSA数据集Yelp数据集
PreRecF1PreRecF1
注:黑体数值表示最优值;下划线数值表示次优值.
EearlySent[27]40.138.336.439.736.132.4
LateSent[27]40.742.832.741.641.539.5
TFN[10,27]43.345.141.745.644.843.9
VistaNet[11,27]41.748.537.662.262.961.5
DiSRAN[27]47.653.246.669.869.969.1
MDSE[33]66.869.167.676.777.577.0
ALBEF[20]65.369.865.256.559.054.1
BLIP[31]65.858.661.147.935.434.3
MDISN(本文)66.370.166.457.860.055.7
MDISN-A(本文)63.868.564.445.743.143.9
), ArticleFig(id=1327017687160565989, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=CN, label=表2, caption=

在同域上情感分类性能实验

, figureFileSmall=null, figureFileBig=null, tableContent=
方法MVSA数据集Yelp数据集
PreRecF1PreRecF1
注:黑体数值表示最优值;下划线数值表示次优值.
EearlySent[27]40.138.336.439.736.132.4
LateSent[27]40.742.832.741.641.539.5
TFN[10,27]43.345.141.745.644.843.9
VistaNet[11,27]41.748.537.662.262.961.5
DiSRAN[27]47.653.246.669.869.969.1
MDSE[33]66.869.167.676.777.577.0
ALBEF[20]65.369.865.256.559.054.1
BLIP[31]65.858.661.147.935.434.3
MDISN(本文)66.370.166.457.860.055.7
MDISN-A(本文)63.868.564.445.743.143.9
), ArticleFig(id=1327017687236063462, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=EN, label=Table 3, caption=Ablation Study on MDISN for Multimodal Cross-Domain Sentiment Classification Tasks %, figureFileSmall=null, figureFileBig=null, tableContent=
方法MVSA→YelpYelp→MVSA
PreRecF1PreRecF1
注:黑体数值表示最优值.
MDISN_SA-51.449.146.739.744.739.3
MDISN_DI-52.644.740.144.638.031.9
MDISN_DS-50.447.944.740.540.436.5
MDISN_MO-52.047.242.841.044.037.0
MDISN(本文)54.050.447.446.744.941.7
), ArticleFig(id=1327017687298978023, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=CN, label=表3, caption=

MDISN在多模态跨域情感分类任务上的消融实验

, figureFileSmall=null, figureFileBig=null, tableContent=
方法MVSA→YelpYelp→MVSA
PreRecF1PreRecF1
注:黑体数值表示最优值.
MDISN_SA-51.449.146.739.744.739.3
MDISN_DI-52.644.740.144.638.031.9
MDISN_DS-50.447.944.740.540.436.5
MDISN_MO-52.047.242.841.044.037.0
MDISN(本文)54.050.447.446.744.941.7
), ArticleFig(id=1327017687361892584, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=EN, label=Table 4, caption=Comparative Experiments of Four Optimization Methods for Multimodal Cross-Domain Sentiment Classification Tasks, figureFileSmall=null, figureFileBig=null, tableContent=
方法参数量MVSA→YelpYelp→MVSA
Pre/%Rec/%F1/%Pre/%Rec/%F1/%
注:黑体数值表示最优值,下划线数值表示次优值.
Fine-tune196.57×10653.650.147.147.545.642.1
SenA-T#L1.26×10652.049.245.444.442.540.3
SenA-T#H1.26×10653.450.146.945.943.841.1
SenA-T2.52×10654.050.447.446.744.941.7
), ArticleFig(id=1327017687441584361, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, language=CN, label=表4, caption=

4种调优方法在多模态跨域情感分类任务上的实验对比

, figureFileSmall=null, figureFileBig=null, tableContent=
方法参数量MVSA→YelpYelp→MVSA
Pre/%Rec/%F1/%Pre/%Rec/%F1/%
注:黑体数值表示最优值,下划线数值表示次优值.
Fine-tune196.57×10653.650.147.147.545.642.1
SenA-T#L1.26×10652.049.245.444.442.540.3
SenA-T#H1.26×10653.450.146.945.943.841.1
SenA-T2.52×10654.050.447.446.744.941.7
)], attaches=null, journal=Journal(id=1324759606947315908, delFlag=0, nameCn=计算机研究与发展, nameEn=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, issn=1000-1239, eissn=null, cn=11-1777/TP, coden=null, periodic=0, language=CN, oaType=null, ccby=null, superviseOffice=null, ownerOffice=null, pubOffice=null, editorOffice=null, officeType=null, aims=null, clcCode=null, officeProv=null, officeCity=null, officeAddr=null, officeZip=null, officeEmail=null, officePhone=null, editDirector=null, officeDirector=null, officeDirectorPhone=null, officeStaffNum=null, officeEmpNum=null, coverPicUrl=k2l8zgqwstuxxevFX2qxVA==, journalPrice=null, startedYear=null, abbrevIsoEn=Journal of Computer Research and Development, journalRemark=null, publicationField=null, createdTime=1793776903141, updatedTime=1793850698612, createdBy=18614031015, updatedBy=13701087609, firstLetterCn=J, firstLetterEn=J, subjectCode=Electronic Information, subjectName=null, subjectCodeEn=Electronic Information, subjectNameEn=null, picCn=k2l8zgqwstuxxevFX2qxVA==, picEn=r6k0Ry+RD+c/VXj5QarmDw==, jcr=null, cjcr=null, exts=[JournalExt(id=1325069127710286688, language=CN, name=计算机研究与发展, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698640, updatedTime=1793850698640, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""}), JournalExt(id=1325069127777395553, language=EN, name=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698656, updatedTime=1793850698656, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""})], databaseList=null, tenantJournalId=1324759914146529478, websiteList=[Website(id=1325070300139892771, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN, language=CN, createTime=1793850978169, createBy=18614031015, updateTime=1793851040259, updateBy=18614031015, name=计算机研究与发展-中文, tplId=1146099689490845704, title=计算机研究与发展, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070893751349360, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=articleTextType, value=kx, createTime=1793851119697, updateTime=1793851119697, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893730377837, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=banner, value=null, createTime=1793851119692, updateTime=1793851119692, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893772320883, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=grayFlag, value=0, createTime=1793851119702, updateTime=1793851119702, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893721989228, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851119690, updateTime=1793851119690, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893784903797, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=minRunFlag, value=0, createTime=1793851119705, updateTime=1793851119705, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893742960751, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic, createTime=1793851119695, updateTime=1793851119695, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893780709492, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=silenceFlag, value=0, createTime=1793851119704, updateTime=1793851119704, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893738766446, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_cn_619/, createTime=1793851119694, updateTime=1793851119694, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893755543665, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeColor, value=null, createTime=1793851119698, updateTime=1793851119698, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893763932274, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeStyle, value=null, createTime=1793851119700, updateTime=1793851119700, creator=18614031015, updator=18614031015)]), Website(id=1325070300580294698, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN, language=EN, createTime=1793850978274, createBy=18614031015, updateTime=1793851036638, updateBy=18614031015, name=计算机研究与发展-英文, tplId=1146101810881728533, title=Journal of Computer Research and Development, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070925288321148, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=articleTextType, value=kx, createTime=1793851127216, updateTime=1793851127216, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925263155321, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=banner, value=null, createTime=1793851127210, updateTime=1793851127210, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925309292671, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=grayFlag, value=0, createTime=1793851127221, updateTime=1793851127221, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925254766712, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851127208, updateTime=1793851127208, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925326069889, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=minRunFlag, value=0, createTime=1793851127225, updateTime=1793851127225, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925279932539, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic, createTime=1793851127214, updateTime=1793851127214, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925317681280, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=silenceFlag, value=0, createTime=1793851127223, updateTime=1793851127223, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925271543930, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_en_623/, createTime=1793851127212, updateTime=1793851127212, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925292515453, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeColor, value=null, createTime=1793851127217, updateTime=1793851127217, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925300904062, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeStyle, value=null, createTime=1793851127219, updateTime=1793851127219, creator=18614031015, updator=18614031015)])], journalTitle=计算机研究与发展, weixinUrl=null, journalUrl=https://crad.ict.ac.cn/, iacademicId=null, status=1, seqNo=null, journalTitleEn=Journal of Computer Research and Development, journalPhotoCn=k2l8zgqwstuxxevFX2qxVA==, journalPhotoEn=r6k0Ry+RD+c/VXj5QarmDw==, journalFirstLetter=J, journalRecommend=null, journalNew=null, journalCollection=null, jcrJf=null, cjcrJf=null, jcrJfStr=null, cjcrJfStr=null, submissionFirstDecision=null, sciSubjectClassification=null, casSubjectClassification=null, citeScore=null, totalCitationFrequency=null, icpCode=null, psCode=null, advertisingLicenseCode=null, copyrightInformation=null, country=null, option=, provinceCode=null, provinceName=null, collectFlag=false, interPubPlatform=, interPubPlatformUrl=null), detailUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202440624, detailUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202440624, pdfUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/PDF/10.7544/issn1000-1239.202440624, pdfUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/PDF/10.7544/issn1000-1239.202440624, aliStartDate=0, aliEndDate=0, collectionFlag=false, citedCount=null, citedUrl=null, previewStatus=0, delFlag=0, hasFullText=1, orderTime=1761926400000, fullTextJson=null, articleText=null, reference=null)
收藏切换
基于元优化特征解耦的多模态跨域情感分析算法
贾熹滨 1, 2 , 李宸 1 , 王珞 1 , 张沐晨 1 , 刘潇健 1 , 张旸旸 3 , 温家凯 4
计算机研究与发展 | 人工智能 2025,62(11): 2697-2709
收起
收藏切换
计算机研究与发展 |人工智能 2025 , 62 (11) : 2697 -2709
基于元优化特征解耦的多模态跨域情感分析算法
全屏
[Author(id=1327017682215481518, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=jiaxibin@bjut.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017682299367601, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017682215481518, language=EN, stringName=Xibin Jia, firstName=Xibin, middleName=null, lastName=Jia, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1College of Computer, Beijing University of Technology, Beijing 100124
2Beijing Institute of Artificial Intelligence (Beijing University of Technology), Beijing 100124, bio={"content":"

born in 1969. Professor, PhD supervisor. Distinguished member of CCF. Her main research interests include visual computing, multimodal deep learning, affection computing, intelligent medical image, and behaviour understanding and computing

"}, bioImg=null, bioContent=

born in 1969. Professor, PhD supervisor. Distinguished member of CCF. Her main research interests include visual computing, multimodal deep learning, affection computing, intelligent medical image, and behaviour understanding and computing

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017682400030898, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017682215481518, language=CN, stringName=贾熹滨, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, address=1北京工业大学计算机学院 北京 100124
2北京人工智能研究院(北京工业大学) 北京 100124, bio={"img":"TIEcl09yccjbwRwXOJgOew==","content":"

1969年生. 教授,博士生导师. CCF杰出会员. 主要研究方向为视觉计算、多模态深度学习、情感计算、智能医学影像、行为理解与计算

"}, bioImg=TIEcl09yccjbwRwXOJgOew==, bioContent=

1969年生. 教授,博士生导师. CCF杰出会员. 主要研究方向为视觉计算、多模态深度学习、情感计算、智能医学影像、行为理解与计算

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017680281907361, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, xref=1, ext=[AuthorCompanyExt(id=1327017680290295970, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Computer, Beijing University of Technology, Beijing 100124), AuthorCompanyExt(id=1327017680298684579, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1北京工业大学计算机学院 北京 100124)]), AuthorCompany(id=1327017680361599140, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, xref=2, ext=[AuthorCompanyExt(id=1327017680369987749, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680361599140, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2Beijing Institute of Artificial Intelligence (Beijing University of Technology), Beijing 100124), AuthorCompanyExt(id=1327017680374182054, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680361599140, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2北京人工智能研究院(北京工业大学) 北京 100124)])]), Author(id=1327017682471334068, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017682555220150, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017682471334068, language=EN, stringName=Chen Li, firstName=Chen, middleName=null, lastName=Li, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Computer, Beijing University of Technology, Beijing 100124, bio={"content":"

born in 1998. Master. Her main research interests include multimodal sentiment and transfer learning

"}, bioImg=null, bioContent=

born in 1998. Master. Her main research interests include multimodal sentiment and transfer learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017682630717623, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017682471334068, language=CN, stringName=李宸, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1北京工业大学计算机学院 北京 100124, bio={"img":"FQHInM+ds8Jf7jmHTtak1A==","content":"

1998年生. 硕士. 主要研究方向为多模态情感、迁移学习

"}, bioImg=FQHInM+ds8Jf7jmHTtak1A==, bioContent=

1998年生. 硕士. 主要研究方向为多模态情感、迁移学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017680281907361, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, xref=1, ext=[AuthorCompanyExt(id=1327017680290295970, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Computer, Beijing University of Technology, Beijing 100124), AuthorCompanyExt(id=1327017680298684579, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1北京工业大学计算机学院 北京 100124)])]), Author(id=1327017682697826489, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017682815267003, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017682697826489, language=EN, stringName=Luo Wang, firstName=Luo, middleName=null, lastName=Wang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Computer, Beijing University of Technology, Beijing 100124, bio={"content":"

born in 1990. PhD, lecturer. Member of CCF. His main research interests include computer vision and deep learning

"}, bioImg=null, bioContent=

born in 1990. PhD, lecturer. Member of CCF. His main research interests include computer vision and deep learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017682882375868, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017682697826489, language=CN, stringName=王珞, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1北京工业大学计算机学院 北京 100124, bio={"img":"27Og6Tjm3oX+L5YKUASW1Q==","content":"

1990年生. 博士,讲师. CCF会员. 主要研究方向为计算机视觉、深度学习

"}, bioImg=27Og6Tjm3oX+L5YKUASW1Q==, bioContent=

1990年生. 博士,讲师. CCF会员. 主要研究方向为计算机视觉、深度学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017680281907361, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, xref=1, ext=[AuthorCompanyExt(id=1327017680290295970, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Computer, Beijing University of Technology, Beijing 100124), AuthorCompanyExt(id=1327017680298684579, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1北京工业大学计算机学院 北京 100124)])]), Author(id=1327017682966261950, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017683054342336, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017682966261950, language=EN, stringName=Muchen Zhang, firstName=Muchen, middleName=null, lastName=Zhang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Computer, Beijing University of Technology, Beijing 100124, bio={"content":"

born in 2002. Master candidate. Her main research interests include multimodal deep learning and computer vision

"}, bioImg=null, bioContent=

born in 2002. Master candidate. Her main research interests include multimodal deep learning and computer vision

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017683121451201, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017682966261950, language=CN, stringName=张沐晨, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1北京工业大学计算机学院 北京 100124, bio={"img":"zlliAjrmwfEUvyIzZurJfw==","content":"

2002年生. 硕士研究生. 主要研究方向为多模态深度学习、计算机视觉

"}, bioImg=zlliAjrmwfEUvyIzZurJfw==, bioContent=

2002年生. 硕士研究生. 主要研究方向为多模态深度学习、计算机视觉

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017680281907361, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, xref=1, ext=[AuthorCompanyExt(id=1327017680290295970, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Computer, Beijing University of Technology, Beijing 100124), AuthorCompanyExt(id=1327017680298684579, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1北京工业大学计算机学院 北京 100124)])]), Author(id=1327017683184365763, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017683259863237, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017683184365763, language=EN, stringName=Xiaojian Liu, firstName=Xiaojian, middleName=null, lastName=Liu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Computer, Beijing University of Technology, Beijing 100124, bio={"content":"

born in 1989. PhD, senior engineer. Member of CCF. His main research interests include software engineering and cybersecurity

"}, bioImg=null, bioContent=

born in 1989. PhD, senior engineer. Member of CCF. His main research interests include software engineering and cybersecurity

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017683331166406, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017683184365763, language=CN, stringName=刘潇健, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1北京工业大学计算机学院 北京 100124, bio={"img":"HQRa3rME15up6GdI6fV8+w==","content":"

1989年生. 博士,高级工程师. CCF会员. 主要研究方向为软件工程、网络安全

"}, bioImg=HQRa3rME15up6GdI6fV8+w==, bioContent=

1989年生. 博士,高级工程师. CCF会员. 主要研究方向为软件工程、网络安全

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017680281907361, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, xref=1, ext=[AuthorCompanyExt(id=1327017680290295970, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Computer, Beijing University of Technology, Beijing 100124), AuthorCompanyExt(id=1327017680298684579, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017680281907361, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1北京工业大学计算机学院 北京 100124)])]), Author(id=1327017683402469576, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, orderNo=5, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017683473772746, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017683402469576, language=EN, stringName=Yangyang Zhang, firstName=Yangyang, middleName=null, lastName=Zhang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=3, address=3China Electronics Standardization Institute, Beijing 100007, bio={"content":"

born in 1976. Master. Professorial senior engineer. Member of CCF. Her main research interests include software and systems engineering, and standardization of IT

"}, bioImg=null, bioContent=

born in 1976. Master. Professorial senior engineer. Member of CCF. Her main research interests include software and systems engineering, and standardization of IT

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017683540881611, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017683402469576, language=CN, stringName=张旸旸, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=3, address=3中国电子技术标准化研究院 北京 100007, bio={"img":"+H5jwYnZyfZEFj3eFyLepw==","content":"

1976年生. 硕士,教授级高工. CCF会员. 主要研究方向为软件与系统工程、信息技术标准化

"}, bioImg=+H5jwYnZyfZEFj3eFyLepw==, bioContent=

1976年生. 硕士,教授级高工. CCF会员. 主要研究方向为软件与系统工程、信息技术标准化

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017682022543527, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, xref=3, ext=[AuthorCompanyExt(id=1327017682030932136, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017682022543527, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3China Electronics Standardization Institute, Beijing 100007), AuthorCompanyExt(id=1327017682039320745, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017682022543527, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3中国电子技术标准化研究院 北京 100007)])]), Author(id=1327017683612184781, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, orderNo=6, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327017683683487951, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017683612184781, language=EN, stringName=Jiakai Wen, firstName=Jiakai, middleName=null, lastName=Wen, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=4, address=4Guangxi Daring Technology Co., Ltd., Nanning 530006, bio={"content":"

born in 1979. Bachelor, senior engineer. Member of CCF. His main research interests include machine translation and big data

"}, bioImg=null, bioContent=

born in 1979. Bachelor, senior engineer. Member of CCF. His main research interests include machine translation and big data

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327017683746402512, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, authorId=1327017683612184781, language=CN, stringName=温家凯, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=4, address=4广西达译科技有限公司 南宁 530006, bio={"img":"f/WmoCwRJfpHEChdChh2XQ==","content":"

1979年生. 学士,高级工程师. CCF会员. 主要研究方向为机器翻译、大数据

"}, bioImg=f/WmoCwRJfpHEChdChh2XQ==, bioContent=

1979年生. 学士,高级工程师. CCF会员. 主要研究方向为机器翻译、大数据

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327017682123206826, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, xref=4, ext=[AuthorCompanyExt(id=1327017682131595435, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017682123206826, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=4Guangxi Daring Technology Co., Ltd., Nanning 530006), AuthorCompanyExt(id=1327017682139984044, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327017677962457226, companyId=1327017682123206826, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=4广西达译科技有限公司 南宁 530006)])])]
贾熹滨1, 2 , 李宸1, 王珞1, 张沐晨1, 刘潇健1, 张旸旸3, 温家凯4
作者信息
  • 1北京工业大学计算机学院 北京 100124
  • 2北京人工智能研究院(北京工业大学) 北京 100124
  • 3中国电子技术标准化研究院 北京 100007
  • 4广西达译科技有限公司 南宁 530006
作者简介:

1969年生. 教授,博士生导师. CCF杰出会员. 主要研究方向为视觉计算、多模态深度学习、情感计算、智能医学影像、行为理解与计算

1998年生. 硕士. 主要研究方向为多模态情感、迁移学习

1990年生. 博士,讲师. CCF会员. 主要研究方向为计算机视觉、深度学习

2002年生. 硕士研究生. 主要研究方向为多模态深度学习、计算机视觉

1989年生. 博士,高级工程师. CCF会员. 主要研究方向为软件工程、网络安全

1976年生. 硕士,教授级高工. CCF会员. 主要研究方向为软件与系统工程、信息技术标准化

1979年生. 学士,高级工程师. CCF会员. 主要研究方向为机器翻译、大数据

A Multimodal Cross-Domain Sentiment Analysis Algorithm Based on Feature Disentanglement Meta-Optimization
Xibin Jia1, 2 , Chen Li1, Luo Wang1, Muchen Zhang1, Xiaojian Liu1, Yangyang Zhang3, Jiakai Wen4
Affiliations
  • 1College of Computer, Beijing University of Technology, Beijing 100124
  • 2Beijing Institute of Artificial Intelligence (Beijing University of Technology), Beijing 100124
  • 3China Electronics Standardization Institute, Beijing 100007
  • 4Guangxi Daring Technology Co., Ltd., Nanning 530006
  • born in 1969. Professor, PhD supervisor. Distinguished member of CCF. Her main research interests include visual computing, multimodal deep learning, affection computing, intelligent medical image, and behaviour understanding and computing

    born in 1998. Master. Her main research interests include multimodal sentiment and transfer learning

    born in 1990. PhD, lecturer. Member of CCF. His main research interests include computer vision and deep learning

    born in 2002. Master candidate. Her main research interests include multimodal deep learning and computer vision

    born in 1989. PhD, senior engineer. Member of CCF. His main research interests include software engineering and cybersecurity

    born in 1976. Master. Professorial senior engineer. Member of CCF. Her main research interests include software and systems engineering, and standardization of IT

    born in 1979. Bachelor, senior engineer. Member of CCF. His main research interests include machine translation and big data

出版时间: 2025-11-01 doi: 10.7544/issn1000-1239.202440624
文章导航
收藏切换

多模态情感分析旨在利用多模态点评等数据识别用户情感倾向. 为实现存在域偏移的跨域应用,常用无监督领域自适应方法. 然而,该类方法着重于领域不变特征提取,忽略了目标领域特定特征的重要作用. 为此,提出基于元优化的领域不变及领域特定特征解耦网络. 首先,通过嵌入情感适配器对预训练大模型微调,建立图文融合情感特征编码器. 进而,构建基于因子分解的特征解耦模块,分别利用领域对抗及领域分类、协同独立性约束,实现知识可传递的领域不变特征编码的同时,提取领域特定特征以增强目标域情感分类性能. 为保证特征解耦与情感分类的总体优化方向一致性,提出基于元学习的元优化训练策略,实现情感分析网络的协同优化. 基于MVSA和Yelp数据集构建的双向情感迁移任务的对比实验表明,较之其他先进的图文情感迁移算法,所提算法于双向情感迁移任务的精确率、召回率和F1值3项评价指标均取得了优异的性能.

多模态情感分析  /  无监督领域自适应  /  跨领域情感分类  /  特征解耦  /  元优化

Multimodal sentiment analysis aims to utilize the multimodal customer comments and other data to identify users’ sentimental tendencies. To realize cross-domain application with the domain bias, commonly used solutions are unsupervised domain adaptation methods. Nevertheless, this type of solutions focuses on the extraction of domain-invariant features, and it neglects the significance of domain-specific features at the target domain. Thus, a meta-optimization based domain-invariant and domain-specific feature disentanglement network is proposed. First, by embedding adapters into the pre-trained large model with fine-tuning fitting, the image-text fused sentiment feature encoder is accordingly constructed. Then, a feature disentanglement module is constructed on the basis of the factorization operation, which utilizes domain adversary and domain classification, together with collaborative independence constraints, respectively, to achieve knowledge-transferable domain-invariant feature embedding while extracting the domain-specific features to enhance the performance of sentiment classification at the target domain. To ensure the consistency of the overall optimization tendency for feature disentanglement and sentiment classification, a meta-learning-based meta-optimization training strategy is put forward to synergistically optimize the sentiment analysis network. Comparative experiments on bidirectional sentiment transfer tasks constructed by MVSA and Yelp datasets demonstrate that compared with other advanced image-text sentiment transfer algorithms, the proposed algorithm achieves superior performance on bidirectional sentiment transfer tasks in terms of three consensus metrics: Precision, Recall and F1 score.

multimodal sentiment analysis  /  unsupervised domain adaptation  /  cross-domain sentiment classification  /  feature disentanglement  /  meta-optimization
贾熹滨, 李宸, 王珞, 张沐晨, 刘潇健, 张旸旸, 温家凯. 基于元优化特征解耦的多模态跨域情感分析算法. 计算机研究与发展, 2025 , 62 (11) : 2697 -2709 . DOI: 10.7544/issn1000-1239.202440624
Xibin Jia, Chen Li, Luo Wang, Muchen Zhang, Xiaojian Liu, Yangyang Zhang, Jiakai Wen. A Multimodal Cross-Domain Sentiment Analysis Algorithm Based on Feature Disentanglement Meta-Optimization[J]. Journal of Computer Research and Development, 2025 , 62 (11) : 2697 -2709 . DOI: 10.7544/issn1000-1239.202440624
情感分析[1]是一种语境挖掘技术,用来识别人们对某种事物的情感倾向,如对用户点评态度分类,为商家决策、商品推荐等提供参考等. 随着富媒体形式的广泛存在,加之图文多模态大模型的快速发展,有效利用图文等多模态数据实现多模态情感分析有助于克服单模态歧义性,成为情感分析的主流. 然而,大量有标签图文数据对的获取费时费力. 此外,由于不同应用领域的情感态度表达方式亦存在差异性,在实际应用中存在域偏移现象,阻碍了源域训练获得的多模态情感分析算法在目标领域的应用性能. 为提高不同场景下的应用性能和降低目标域标注需求,跨领域情感分析[2]被提出,其目的是研究如何利用来自具有良好标注的源域可迁移知识来指导无标注或少标注目标域的高性能情感分类模型优化.
其中,无监督领域自适应[3]因其无目标域数据标注要求,为跨领域情感分析模型学习带来了更为高效的解决思路. 目前,基于领域对抗[4-5]和基于分布度量[6-7]的2种主流无监督领域自适应方法被广泛应用于跨领域情感分析任务中,它们的共同原理是尽可能去除领域特定特征以充分提取领域不变的共有特征,降低领域分布的差异,实现源域标签监督信息指导下目标域情感分析算法的优化学习,获得更为精准的目标域情感极性分类. 然而,情感表达与情感态度判别的界定与特定领域密切相关,忽略目标域类别可判别的领域特定特征不利于网络更全面理解和表达本领域情感信息[8-9],因此,仅依赖领域不变的共有特征不足以实现从源域到目标域的最佳迁移,尤其是当目标域样本多样性较大时此问题更为明显.
为了解决上述问题,本文面向图文多模态跨领域情感分析任务,提出了一种基于元优化的领域不变及领域特定特征解耦网络(meta-optimization-based domain-invariant and domain-specific feature disentanglement network,MDISN). 从基于预训练模型的情感适配调优、基于解耦的领域不变特征和领域特定特征编码器构建以及基于元优化策略的模型训练3个方面进行了模型改进和优化. 主要工作和贡献包括4个方面:
1)为实现图文多模态情感表示的适配,建立更为轻量化的多模态特征提取网络以减少所需的训练数据量;利用适配调优机制,即在基于Transformer预训练模型的特征编码器中引入情感适配器,在情感数据集上进行训练优化,提取适用于下游分类的有效情感特征.
2)利用因子解耦构建特征解耦网络,将融合情感特征解耦为领域不变特征和领域特定特征;利用领域不变特征实现源域到目标域的共有情感分类知识的迁移,同时利用领域特定特征实现专有情感分类知识表达,通过领域不变特征及领域特定特征的协同学习增强目标域的情感分类性能.
3)在模型学习中,为了缓解解耦目标和分类目标的优化方向不一致问题,提出利用基于元学习的元优化训练策略,提升图文情感分类模型的域适应迁移学习总体性能.
4)在MVSA和Yelp这2个图文情感分类数据集上构建的双向迁移实验数据集进行的实验证明,所提出的算法在图文多模态情感迁移任务上取得了更具竞争力的迁移和情感极性分类性能.
随着社交媒体的飞速发展,用户发表的内容数据也更多呈现图片与短文本并用的多模态表达形式. 利用数据样本中不同模态相互补充,可以为基于文本的语义理解和语义消歧提供额外的线索. 因此,多模态情感分析逐渐成为研究热点.
近年来,基于深度学习的多模态情感分析方法围绕情感分类目标,重点研究多模态特征融合算法. 例如,Zadeh等人[10]提出了一种端到端的张量融合网络TFN,以动态建模在线视频中的模态内和模态间的交互. 受图像-文本情感相关性的内在特征启发,Truong等人[11]提出了一种视觉方面注意力网络VistaNet,将图像作为与文本内容一致的注意力来源. Gui等人[12]针对社交媒体中抑郁症的检测问题,提出了一种融合文本和视觉信息的多智能体强化学习方法. 近年来,随着基于Transformer预训练模型方法的巨大成功,越来越多的研究人员尝试将基于Transformer预训练模型的方法引入多模态情感分析中. Ling等人[13]针对多模态方面级情感分析问题提出了一种特定于任务的视觉语言预训练框架MABSA,在预训练过程中设计特定的预训练任务,以细粒度方面助力图文的跨模态对齐.
传统的情感分析方法只讨论了模型于单一领域的性能,但情感表达用于的领域差异性,导致情感分析模型跨域应用性能受限. 领域自适应,特别是无监督领域自适应方法的提出使跨领域情感分类的模型迁移成为可能. 无监督领域自适应是一种源域数据有标签、目标域数据无标签以及源任务和目标任务相同,但源域和目标域数据分布不同的迁移学习方法. 主流的无监督领域自适应方法目前主要分为两大类,即基于领域对抗的方法和基于分布度量的方法.
Ganin等人[14]构建了领域对抗性神经网络DANN,通过领域判别器混淆领域来源以学习尽可能多的领域不变特征. Zhu等人[7]提出了一种基于分布度量的领域自适应方法,该方法设计了一个局部最大平均差异LMMD度量,通过对齐不同领域间领域特定层的相关子域分布来减小领域差异. 结合以上2种方法的优势,Jia等人[15]从全局和局部2个方面对齐了领域分布,提出了一种领域对抗类别对齐网络DACAN.
广泛的领域自适应研究在计算机视觉和自然语言处理方面取得了巨大的成功,因而被逐渐引入到多模态情感分析领域中. 黄学坚等人[16]提出了一种基于样本内外多模态协同的表示方法,分别学习模态特有与共享特征并实现协同优化,设计了基于注意力机制和门控神经网络的自适应多模态融合方法,在情感数据集上的实验结果优于基线方法. Qi等人[17]提出了多模态领域自适应神经网络MDANN,通过构建混合域约束来提高模型在跨领域情感识别任务中的分类性能. Ma等人[18]引入了多模态对抗网络MMAN来解决基于图像-文本数据对中存在的领域自适应问题,所提出的MMAN在多模态社会事件识别任务中表现最佳.
综上,多模态情感自适应为解决跨域的高性能情感识别提供了解决路径,但不同模态间的差异使多模态情感跨领域知识迁移变得更为复杂. 近年来,多模态大模型预训练不断完善,能够为下游任务提供大量图文对齐先验知识. 因此,本文利用预训练大模型作为迁移网络的骨干网络,并在其中嵌入情感适配器以实现有效的下游情感任务鲁棒的多模态融合特征提取. 以此模型为基础,本文探讨有效的域适应技术,实现多模态融合特征的跨域迁移学习,进一步挖掘前述工作中忽略的领域特有特征对于情感分类的重要作用,建立同时利用领域不变与领域特有特征的多模态情感分析算法.
针对跨领域多模态情感分类问题,在将预训练大模型通用知识与下游情感知识相结合,提取图文融合特征的基础上,本文提出显式地解耦领域不变特征和领域特定特征,同时引入元学习的思想协调解耦任务和分类任务的优化一致性,实现源域标签监督下的目标域的多模态情感分类模型的领域适应迁移学习. 具体网络设计及优化训练方法阐述如下.
本文所提的MDISN框架如图1所示. 总体网络包括特征提取模块、特征解耦模块和分类模块.
特征提取模块以基于Transformer块的图文大模型预训练特征编码器为基础,在每层Transformer块添加情感适配器. 也就是说,利用情感数据微调的适配层构建图文融合的多模态情感特征编码器. 特征解耦模块包括2个并行分支,其中一个分支由领域不变特征编码器和领域判别器构成,另一个分支由领域特定特征编码器和领域分类器构成,实现图文融合特征分解,分别输出解耦后的领域不变特征和领域特定特征. 分类模块为一个情感分类头,利用组合后的图文多模态解耦特征进行分类,实现情感极性预测.
在网络训练过程中,训练数据集由有标注源域数据与无标注目标域数据构成,以所构建网络为基础,采用领域不变特征编码分支的领域对抗约束、领域特定编码分支的领域分类约束以及2分支解耦得到的特征间的特征独立性约束和源域分类标签的一致性监督约束,实现目标域多模态情感分析网络的迁移学习. 此外,为保证特征解耦任务和情感分类任务的网络训练优化整体性能,制定元优化任务无关训练策略,支持多模态情感分析算法的优化学习.
如图1框架左侧所示,此模块用于提取多模态融合特征X. 为克服情感图文对多模态标注数据不足下,应对多模态下领域适应与图文对齐混叠带来的复杂度增加的挑战,充分利用基于Transformer的视觉-语言预训练大模型[19]对多模态特征的提取能力与对齐能力,本文以公认的融合前对齐即ALBEF[20]模型作为特征提取模块的骨干模型. 考虑到预训练大模型的特征表示通识性强,但缺少对特定领域的鉴别特征建模,经典的全量网络调优需要大规模标注数据,现实应用场景下难以满足. 因此,借鉴Adaptformer[21]的适配调优方法,本文在原始Transformer块中嵌入一种自适应、可拓展的适配器,建立SenA-Transformer块. 本文将所采用的骨干网络ALBEF[20]模型中的所有原始Transformer块替换为SenA-Transformer块,构建基于情感适配Transformer块的多模态特征提取模块.
1)SenA-Transformer块结构. 如图2所示,左侧表示原始的Transformer块结构,右侧表示本文所引入的轻量级情感适配器. 为每个原始Transformer块嵌入情感适配器,对SenA-Transformer块的情感适配优化时,冻结预训练Transformer块的参数,仅对适配器利用情感数据调优,建立情感适配Transformer块SenA-Transformer. 该方法有利于在仅添加有限可学习参数的情况下,结合预训练大模型通用知识和任务相关情感知识,提取情感图文多模态融合特征.
多模态特征编码器由多个SenA-Transformer块组成,每层适配后Transformer对特征进行变换处理,具体而言,图2所示为第l层SenA-Transformer块的计算过程. 其输入为第$ l-1 $层SenA-Transformer块的输出$ {\boldsymbol{x}}_{l-1} $. 在经由原始Transformer块的多头注意力(multi-head attention)与层归一化(layer normalization)处理后,得到中间特征$ \boldsymbol{x}'_{l}$. 此时,将$ \boldsymbol{x}'_{l} $输入情感适配器,得到情感分支输出$ {\tilde{\boldsymbol{x}}}_{l} $:
$ {\tilde{\boldsymbol{x}}}_{l}=ReLU\left(\boldsymbol{x}'_{l}\cdot {\boldsymbol{W}}_{\mathrm{D}\mathrm{o}\mathrm{w}\mathrm{n}}\right)\cdot {\boldsymbol{W}}_{\mathrm{U}\mathrm{p}} . $
情感适配器的网络设计如图2右侧所示. 适配器采用限制参数数量的瓶颈结构变换网络,其中包括1个由参数$ {\boldsymbol{W}}_{\mathrm{D}\mathrm{o}\mathrm{w}\mathrm{n}} $构成的下线性投影层、1个由参数$ {\boldsymbol{W}}_{\mathrm{U}\mathrm{p}} $构成的上线性投影层和2个投影层之间的$ ReLU $激活层. 适配器加在Transformer块的前馈层. 将$ {\tilde{\boldsymbol{x}}}_{l} $与缩放因子$ s $相乘,再将$ \boldsymbol{x}'_{l} $输入前馈层$ \mathrm{M}\mathrm{L}\mathrm{P} $,将2项输出与$ \boldsymbol{x}'_{l} $计算求和即得到第l层SenA-Transformer块的输出$ {\boldsymbol{x}}_{l} $. 最终的SenA-Transformer块输出特征$ {\boldsymbol{x}}_{l} $如式(2)所示.
$ {\boldsymbol{x}}_{l}=LN\left(MLP\left(\boldsymbol{x}'_{l}\right)+\boldsymbol{x}'_{l}+s\times {\tilde{\boldsymbol{x}}}_{l}\right) . $
2)SenA-Transformer块调优. 在网络训练阶段,对SenA-Transformer块进行训练调优,如图2所示仅对情感适配器网络参数$ {\boldsymbol{\theta }}_{G}={(\boldsymbol{W}}_{\mathrm{D}\mathrm{o}\mathrm{w}\mathrm{n}},{\boldsymbol{W}}_{\mathrm{U}\mathrm{p}}) $进行训练优化,同时冻结此块其余部分的网络参数. 也就是说,情感适配器的网络参数利用情感数据集进行优化更新,基础Transformer块的参数从预训练模型中加载权重,在训练中保持这些参数冻结. 通过对并行的情感适配器调优训练,提取适用于特定领域下游任务的多模态融合情感特征,同时利用ALBEF[20]预训练模型对图文情感数据对齐,为情感极性分类任务的领域自适应学习提供有效的图文情感特征.
为有效利用源域的标签监督,实现无标注目标域的高性能多模态情感分析,一方面,本文利用领域对抗约束学习源域与目标域共有的领域不变特征,支持跨域知识迁移;另一方面,采用领域分类约束挖掘目标域情感极性判别所需的目标领域专有的领域特定特征. 在双分支特征解耦编码器模块构建基础上,设计多个损失约束,实现跨域情感分析算法的优化学习.
1)领域不变特征损失约束$ {L}_{\mathrm{I}} $. 将特征提取模块得到的多模态融合特征X,输入领域不变特征编码器$ Q $,通过对抗性训练框架[22]学习得到领域不变特征$ {\boldsymbol{Z}}_{\mathrm{I}} $. 领域判别器$ {D}_{\mathrm{I}} $用于最小化领域标签的预测损失,通过在$ {D}_{\mathrm{I}} $前设置梯度反转层GRL使领域不变特征编码器$ Q $的优化目标转变为最大化领域标签预测损失,通过最小-最大化的对抗关系混淆领域来源,提取领域不变特征. 领域不变特征损失约束$ {L}_{\mathrm{I}} $的计算方法以及对应的特征编码器参数${\boldsymbol{\theta }}_{Q} $和领域分类器参数$ {\boldsymbol{\theta }}_{{D}_{\mathrm{I}}} $的优化目标函数如式(3)所示.
$ \underset{{\boldsymbol{\theta }}_{Q}}{\mathrm{m}\mathrm{i}\mathrm{n}}\underset{{\boldsymbol{\theta }}_{{D}_{\mathrm{I}}}}{\mathrm{m}\mathrm{a}\mathrm{x}}\left\{{L}_{\mathrm{I}}:=-{E}_{\boldsymbol{x},\boldsymbol{d}\sim \boldsymbol{X},{D}}\left[\boldsymbol{d}\mathrm{ln}{D}_{\mathrm{I}}\left({\boldsymbol{Z}}_{\mathrm{I}}\right)\right]\right\} , $
其中,${\boldsymbol{x}} $为某一融合特征,${\boldsymbol{X}} $为整体融合特征,${\boldsymbol{d}} $为某一特征对应的领域标签,$D $为整体领域标签集合.
2)领域特定特征损失约束$ {L}_{\mathrm{S}} $. 将特征提取模块得到的多模态融合特征X,输入领域特定特征编码器R,得到领域特定特征$ {\boldsymbol{Z}}_{\mathrm{S}} $. 与领域不变特征相反,为训练优化$ {\boldsymbol{Z}}_{\mathrm{S}} $,采用领域分类器$ {D}_{\mathrm{S}} $,通过预测领域标签的一致性约束,鼓励领域分类器$ {D}_{\mathrm{S}} $区分领域来源. 领域特定特征损失约束$ {L}_{\mathrm{S}} $的计算方法以及对应的特征编码器参数$ {\boldsymbol{\theta }}_{R} $和领域分类器参数$ {\boldsymbol{\theta }}_{{D}_{\mathrm{S}}} $的优化目标函数如式(4)所示.
$ \underset{{\boldsymbol{\theta }}_{R},{\boldsymbol{\theta }}_{{D}_{\mathrm{S}}}}{\mathrm{m}\mathrm{i}\mathrm{n}}\left\{{L}_{\mathrm{S}}:=-{E}_{{\boldsymbol{x}},{\boldsymbol{d}}\sim {\boldsymbol{X}},{D}}\left[\boldsymbol{d}\mathrm{ln}{D}_{\mathrm{S}}\left({\boldsymbol{Z}}_{\mathrm{S}}\right)\right]\right\} . $
3)特征独立性约束$ {L}_{\mathrm{D}}. $为保证学习得到的2个解耦特征间的独立性,考虑到最小化2个随机变量之间的协方差相当于最小化它们之间的互信息[23],因此可将2个特征$ {\boldsymbol{Z}}_{\mathrm{I}} $和$ {\boldsymbol{Z}}_{\mathrm{S}} $之间的解耦约束条件设置为计算协方差矩阵$ \boldsymbol{C}\boldsymbol{o}\boldsymbol{v}({\boldsymbol{Z}}_{\mathrm{I}},{\boldsymbol{Z}}_{\mathrm{S}}) $,并$ \mathrm{使} $其趋近于0. 基于此,领域不变特征编码器$ Q $的参数$ {\boldsymbol{\theta }}_{Q} $和领域特定特征编码器$ R $的参数$ {\boldsymbol{\theta }}_{R} $的优化目标函数如式(5)所示.
$ \underset{{\boldsymbol{\theta }}_{Q},{\boldsymbol{\theta }}_{R}}{\mathrm{m}\mathrm{i}\mathrm{n}}\left\{{L}_{\mathrm{D}}:={E}_{\boldsymbol{x}\sim \boldsymbol{X}}\left[{\|\boldsymbol{C}\boldsymbol{o}\boldsymbol{v}({\boldsymbol{Z}}_{\mathrm{I}},{\boldsymbol{Z}}_{\mathrm{S}})\|}_{2}\right]\right\} \text{,} $
其中$ {\|\cdot \|}_{2} $为$ {L}_{2} $范数.
4)特征解耦损失. 对上述优化目标函数项加权求和,则特征解耦模块的总损失函数如式(6)所示.
$ {L}_{\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{e}} = {{\lambda }_{\mathrm{D}}L}_{\mathrm{D}}+{\lambda }_{\mathrm{I}}{L}_{\mathrm{I}}+{{\lambda }_{\mathrm{S}}L}_{\mathrm{S}} \text{,} $
其中$ {\lambda }_{\mathrm{D}} $,$ {\lambda }_{\mathrm{I}} $,$ {\lambda }_{\mathrm{S}} $为超参数,表示特征独立性约束、领域不变特征损失约束和领域特定特征损失约束的权重.
在得到领域不变特征$ {\boldsymbol{Z}}_{\mathrm{I}} $和领域特定特征$ {\boldsymbol{Z}}_{\mathrm{S}}\mathrm{的} $基础上,令$ {\boldsymbol{Z}}_{\mathrm{I}}\oplus {\boldsymbol{Z}}_{\mathrm{S}} $为输入,构建情感分类器. 在网络训练阶段,利用源域的情感标签与情感极性分类预测一致性为约束进行模型训练. 令$ \boldsymbol{y} $为真实标签,情感分类器$ C $的优化损失函数$ {L}_{C} $如式(7)所示.
$ \underset{{\boldsymbol{\theta }}_{Q},{\boldsymbol{\theta }}_{R},{\boldsymbol{\theta }}_{C}}{\mathrm{m}\mathrm{i}\mathrm{n}}\left\{{L}_{C}:=-{E}_{\boldsymbol{x},\boldsymbol{y}\sim \boldsymbol{X},\boldsymbol{Y}}\left[\boldsymbol{y}\mathrm{ln}C({\boldsymbol{Z}}_{\mathrm{I}},{\boldsymbol{Z}}_{\mathrm{S}})\right]\right\} . $
依据典型训练流程,特征解耦任务和情感分类任务的优化各自独立. 一方面,模型利用源域有标注数据和无标注目标域数据,在领域对抗损失、领域判别损失和特征独立性约束下进行特征解耦任务训练,获得图文融合数据的领域不变特征及领域特定特征的解耦表示. 另一方面,模型利用源域有标注数据进行情感分类任务的优化学习,实现情感极性分类. 然而,如图3(a)所示,在该过程中,受任务差异影响,2项任务易出现优化方向不一致问题,即共享网络参数$ \phi $的梯度下降方向不一致,导致特征编码器优化过程无法向最优目标点收敛,难以学习到可判别情感特征,制约域适应迁移学习下多模态情感分类性能.
受任务无关的MetaAlign[24]元学习方法启发,本文在模型优化过程中引入元学习策略,提出一种基于元优化的训练方法,以缓解特征解耦和情感分类任务的优化不一致问题. 具体地,如图3(b)所示,本文将特征解耦任务和情感分类任务视为元学习方案中的2个任务,利用元训练确保域适应策略下领域不变特征提取性能的同时,结合元测试任务验证同一组样本的优化效果,深入发掘领域特定特征与情感判别特征的内在联系,鼓励模型以协调的方式对2项任务同时进行优化,从而在目标域时能快速精准地提取适用于目标域情感分类性能的特征.
通过该元优化训练方法,实现对本文提出的情感极性分类网络进行特征解耦和情感分类的联合优化训练. 在元训练过程,模型主要对特征解耦网络进行学习,优化领域判别器参数$ {\boldsymbol{\theta }}_{{D}_{\mathrm{I}}} $和领域分类器参数$ {\boldsymbol{\theta }}_{{D}_{\mathrm{S}}} $. 在元测试阶段,通过使用情感分类来评估模型优化对元训练任务(特征解耦)的影响,对情感分类器参数$ {\boldsymbol{\theta }}_{C} $进行优化. 在整个元优化过程中,特征提取模块和领域不变、领域特定特征编码器同时参与了2项任务,实现模型参数$ \phi ={\{\boldsymbol{\theta }}_{G},{\boldsymbol{\theta }}_{Q},{\boldsymbol{\theta }}_{R}\} $的优化. 据此,总体元优化目标如式(8)所示.
$\begin{split} & \underset{\phi ,{\boldsymbol{\theta }}_{C}}{\mathrm{m}\mathrm{i}\mathrm{n}}\underset{{\boldsymbol{\theta }}_{{D}_{\mathrm{I}}},{\boldsymbol{\theta }}_{{D}_{\mathrm{S}}}}{\mathrm{m}\mathrm{a}\mathrm{x}}{L}_{\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{e}}\left(\phi ,{\boldsymbol{\theta }}_{{D}_{\mathrm{I}}},{\boldsymbol{\theta }}_{{D}_{\mathrm{S}}}\right)+{L}_{C}\left(\phi ,{\boldsymbol{\theta }}_{C}\right)- \\& \alpha {\nabla }_{\phi }{{L}_{C}\left(\phi ,{\boldsymbol{\theta }}_{C}\right)\nabla }_{\phi }{L}_{\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{e}}\left(\phi ,{\boldsymbol{\theta }}_{{D}_{\mathrm{I}}},{\boldsymbol{\theta }}_{{D}_{\mathrm{S}}}\right) \text{,} \end{split}$
其中$ \alpha $表示元学习率. 式(8)的前2项为2.3节和2.4节提出的基于领域不变及领域特定特征解耦模块与情感分类模块的损失项,而最后1项为任务无关的元优化损失项,使得$ {\nabla }_{\phi }{L}_{C} $和$ {\nabla }_{\phi }{L}_{\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{e}} $的内积最大化,增强2项任务优化时的梯度下降方向的一致性,保证总体目标向最优点收敛,由此促进特征解耦和情感分类协同优化.
协同优化过程中,作为元测试的分类任务,可以验证元训练的特征解耦效果,并指导模型将对解耦模块的优化调整至更有利于情感分类的方向. 进而,模型在泛化到目标域时,将能够根据优化过程中积累的经验,调整特征解耦模块的参数. 因此,在目标域进行应用测试阶段,分别用2个解码器获得类可判别的领域不变特征和领域特定特征,融合后分类,有助于提升跨域情感分类性能.
基于MDISN的域适应训练和情感预测过程的具体描述如算法1所示.
算法1. 网络的训练和推理过程.
1)训练阶段
输入:源域样本$ S $和目标域样本$ T $,可学习参数$ \varPsi =\left\{\phi ,{\boldsymbol{\theta }}_{{D}_{\mathrm{I}}},{\boldsymbol{\theta }}_{{D}_{\mathrm{S}}},{\boldsymbol{\theta }}_{C}\right\} $,$ \phi =\left\{{\boldsymbol{\theta }}_{G},{\boldsymbol{\theta }}_{Q},{\boldsymbol{\theta }}_{R}\right\} $,学习率$ \eta ,\alpha $;
输出:优化后的网络参数$ {\varPsi }^{\boldsymbol{*}}=\left\{{\phi }^{\boldsymbol{*}},{\boldsymbol{\theta }}_{{D}_{\mathrm{I}}}^{\boldsymbol{*}},{\boldsymbol{\theta }}_{{D}_{\mathrm{S}}}^{\boldsymbol{*}},{\boldsymbol{\theta }}_{C}^{\boldsymbol{*}}\right\} $,$ {\phi }^{\boldsymbol{*}}=\left\{{\boldsymbol{\theta }}_{G}^{\boldsymbol{*}},{\boldsymbol{\theta }}_{Q}^{\boldsymbol{*}},{\boldsymbol{\theta }}_{R}^{\boldsymbol{*}}\right\} $.
① for $ t=1\to iterations $ do
②  元训练:
③   计算特征解耦损失$ {L}_{\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{e}} $,如式(6);
④   通过$ {L}_{\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{e}} $优化更新$ \phi $:
⑤  元测试:
⑥   计算分类损失$ {L}_{C}({\phi }^{t+1},{\boldsymbol{\theta }}_{C}^{t}) $,如式(7);
⑦  元优化:
⑧   计算网络总体损失,如式(8);
⑨   优化更新参数:$ {\varPsi }^{t+1}\leftarrow{\varPsi }^{t}-\eta {\nabla }_{{\varPsi }^{t}}{L}_{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}} $;
⑩ end for
2)推理阶段
输入:目标域样本$ T $,优化后的网络参数$ {\varPsi }^{\boldsymbol{*}}= \left\{{\phi }^{\boldsymbol{*}},{\boldsymbol{\theta }}_{C}^{\boldsymbol{*}}\right\} $,$ {\phi }^{\boldsymbol{*}}=\left\{{\boldsymbol{\theta }}_{G}^{\boldsymbol{*}},{\boldsymbol{\theta }}_{Q}^{\boldsymbol{*}},{\boldsymbol{\theta }}_{R}^{\boldsymbol{*}}\right\} $;
输出:情感极性分类预测结果$ y' $.
本文采用了2个常用的公共多模态情感数据集:MVSA[25]和Yelp[11]. 在MVSA数据集中,积极、中性和消极样本分别为11 899个、4 170个和1 500个. 在Yelp数据集中,积极、中性和消极样本分别为17 722个、8 861个和17 722个. 对数据集进行随机划分,80%的数据用于训练,5%的用于验证,15%的用于测试.
MVSA数据集的每幅图片或文本样本由3个志愿者打上标签. 遵循文献[26]的主流做法,当2个及以上志愿者为某图片给出相同的标签时,该标签视作该图片(或文本)的情感标签,同时筛选具有相同图片和文本情感标签的实例作为实验数据样本.
Yelp数据集的每个图文样本对包含了5个等级的情感量级评分. 本文将原始的Yelp数据集评分标签合并为(1,2),3,(4,5)三组,并将它们重新分配为消极、中性和积极,以保持标签空间的一致性.
利用MVSA和Yelp数据集交替作为源域和目标域,构建了2个多模态跨领域情感分类任务,即MVSA→Yelp和Yelp→MVSA.
为了验证算法的有效性,本文将MDISN与3类情感分类的多个主流先进方法进行了比较. 具体如下:
1)单模态深度分类对比方法. 视觉模态情感分析模型中,VisualSent[27]是在预训练的VGG-16网络[28]基础上建立的逻辑回归模型;ViT[29]是一种基于Transformer架构的经典视觉处理模型. 文本模态情感分析模型中,TextualSent[27]是在带有LSTM单元的BiRNN基础上建立的逻辑回归模型;BERT[30]使用双向的Transformer编码器捕捉文本的上下文信息.
2)多模态融合分类对比方法. EarlySent[27]是一种采用经典早期特征级融合策略方法,整合了VGG-16和BiRNN分别提取的图像和文本特征,并将联合表征输入逻辑回归模型进行情感分类. LateSent[27]采用经典晚期决策级融合策略方法,综合并平均VisualSent[27]和TextualSent[27]对图像和文本的预测结果,生成最终预测的情感标签. TFN[10]采用了张量融合层,将图片和文本特征结合起来进行情感分类. VistaNet[11]应用图像方面注意力来模拟编码后的图像和文本特征之间的相互作用. ALBEF[20],BLIP[31]为图文预训练大模型. ITIN[32]利用图文交互网络探究情感图像区域和文本关联关系,利用自适应跨模态门控实现多模态融合表示,而MDSE[33]采用模态无关对比学习方法建立统一的多模态融合表示,是具有先进性能的多模态情感表示方法.
3)多模态领域自适应对比方法. MMAN[18]采用堆叠注意力模块获取多模态表征,并通过对抗训练减少领域偏移. MDANN[17]由多模态注意力、融合模块和混合域约束组成,通过对抗学习来提取领域不变特征. DACAN[15]构建领域对抗模块和类别对齐模块,从全局和局部2方面对齐领域分布. 为了适应多模态图文分类任务,本文中使用ALBEF[20]作为DACAN[15]的骨干模型. DiSRAN[27]在采用模态交叉注意力计算图文联合表示的基础上,利用解耦去除领域风格,并通过对抗训练获得领域不变特征.
实验中本文提出的基于元优化的领域不变及领域特定特征解耦网络MDISN采用增加情感适配的预训练大模型ALBEF[20]作为特征提取模块的骨干模型.
训练初始阶段,使用Xavier对模型中的权重进行初始化. 训练期间采用AdamW优化器,元学习率$ \alpha $设置为$ 2\times {10}^{-3} $,是学习率$ \eta $的10倍. 损失函数中的超参数设置情况为$ {\lambda }_{\mathrm{D}}=10.0 $,$ {\lambda }_{\mathrm{I}}=1.0 $,$ {\lambda }_{\mathrm{S}}=1.0 $.
所提出的框架使用PyTorch实现,并在NVIDIA TITAN RTX显卡上进行训练和测试.
在实验中,使用精确率、召回率和F1值来评估模型性能,并为这3项指标计算宏观平均值. 考虑到数据集存在标签空间不平衡的影响,未利用准确率作为性能评估指标.
精确率反映了预测为正例的样本中预测正确的样本的比例. 精确率的计算如式(9)所示.
$ Pre=TP/(TP+FP), $
其中TP表示真阳性,FP表示假阳性.
召回率反映了正例样本中预测正确的样本数所占的比例. 召回率的计算如式(10)所示.
$ Rec=TP/(TP+FN), $
其中,FN表示假阳性.
F1值是精确率和召回率的调和平均数,其取值范围为[0, 1]. F1值越接近于1,分类模型的预测性能越好. F1值的计算如式(11)所示.
$ F1=(2\times Pre\times Rec)/(Pre+Rec) . $
表1给出了本文所提出方法MDISN和对比方法在无监督跨领域情感分类任务中的实验结果. 实验中,为了进一步验证MDISN的有效性,将多模态特征解码器替换为预训练大模型BLIP[31]、先进图文融合模型MDSE[33]和ITIN[32]进行了改进,记为MDISN-A,MDISN-B,MDISN-C.
对于多模态领域自适应方法,即本文的MDISN及改进的MDISN,DACAN[15]均遵循无监督领域自适应的惯例,使用有标签源域数据和无标签目标域数据进行训练,目标域训练集和测试集无交集. 对于其他方法,包括单模态方法,即ViT[29],BERT[30];多模态融合方法,即ALBEF[20],BLIP[31],MDSE[33],在源域应用标准的有监督训练,然后将学习到的模型直接用到目标域进行测试. ITIN[32]为论文提供的在MVSA数据集训练的开源测试代码,直接用于目标域进行测试. 多模态领域自适应方法DiSRAN[27]及对比方法VisualSent[27],TextualSent[27],EearlySent[27],LateSent[27],TFN[10],VistaNet[11],MDANN[17],MMAN[18]的结果来自文献[27].
从表1可以观察到,MDISN模型及其改进模型在MVSA→Yelp和Yelp→MVSA这2个跨域多模态情感分析任务上均取得了较对比方法更有竞争性的指标值,证明了MDISN在多模态跨领域情感分类任务上的优越性能. 对表1实验结果的详细讨论与分析如下.
对于单模态深度分类方法,由于Transformer编码器可以通过自注意力机制捕获上下文依赖关系,有利于图像和文本的全局语义理解,因而ViT[29]模型和BERT[30]模型的性能优于VisualSent[27]模型和TextualSent[27]模型. 对于多模态融合分类方法,可以发现EearlySent[27]方法和LateSent[27]方法在目标域上的精确率甚至不如单模态方法. 这一观察结果表明仅将多模态特征简单拼接会由于模态间存在不一致性与领域差异混叠,导致有噪迁移并加剧情感极性分类性能的退化. 而TFN[10],VistaNet[11],ALBEF [20],BLIP[31],ITIN[32],MDSE[33]模型通过跨模态对齐融合多模态特征,有利于文本模态与图像模态间的互补信息的利用,其中图文大模型ALBEF [20]和BLIP[31]在多个公共数据集进行预训练,具备了多模态对齐融合先验知识;ITIN[32]和MDSE[33] 采用了克服模态异质性影响等先进学习方法,在单领域上具有相对更佳的多模态融合性能,在直接跨域应用仍获得了非常优秀的分类性能.
在多模态领域自适应方法中,无标签的目标域数据也参与训练,从而极大提高了自适应性能. DACAN[15]在利用对抗学习对齐全局分布的同时,通过类别加权方式对齐了类别相关的细粒度领域特定信息,从而获得了比MDANN[17]和MMAN[18]更好的迁移性能. 同时,不同于DACAN[15]隐式对齐领域特定信息的方式,MDISN利用元优化训练策略显式地将情感特征解耦为领域不变特征和领域特定特征,通过利用领域特定信息,促进目标领域的情感分类性能,并取得了更优的自适应性能. 从利用多模态融合先进方法及预训练大模型的直接跨域结果,与MDISN采用了领域适应策略的实验结果进行对比可以发现,MDISN在MVSA上训练,跨域应用到Yelp上的召回率和F1值更高,改进的MDISN-C相较原多模态图文融合模型直接跨域3项指标更高,且取得了最高召回率. 改进的MDISN-A和MDISN-B在Yelp上训练,跨域应用到MVSA上的3项指标略低于最高的次优性能. 该结果验证了相对跨域直接应用,本文所提域适应方法有助于克服领域偏差的影响. 进一步分析本文提出的3种模型,MDISN加入情感适配,有助于融合特征适应源域多模态融合特征学习,后续可以在MDISN-A和MDISN-B加入情感适配进一步优化训练,提升跨域迁移的多模态情感分类整体性能.
与主流先进方法DiSRAN[27]相比,融合领域特定特征与领域不变特征,采用与有标注训练样本规模接近的MVSA数据集上训练,在Yelp上的识别性能精确度更高;在Yelp到MVSA的领域适应情感识别任务上,MDISN在Yelp上的有标注训练样本数更少,且由于Yelp数据对关联性弱、领域不变特征表征学习难的条件下,MDISN仍取得了相同的情感分类精确度,MDISN-A和MDISN-B的3项评价指标都远高于DiSRAN[27]. 与DiSRAN[27]在Yelp数据集上使用的积极、中性和消极样本各21261个不同,本文仅选用约70%的Yelp子集(见3.1节所列),训练集按各类标注样本数的80%划分,积极、中性和消极样本分别为14178个、7089个和14178个. 与先进的多模态情感融合方法MDSE[33]相比,在其上加入本文提出的域适应网络后,MDISN-B在召回率和F1值上取得了10%左右提升. 究其原因,与DiSRAN[27]采用领域风格解耦及在统一框架下提升领域不变特征学习相比,本文结合元优化策略学习和解耦思路,更有助于目标域类判别领域特定特征和融合跨域迁移的领域不变特征,对目标域具有更好的领域适应性. 因而,在图文融合领域适应情感识别实验取得了具有竞争力的结果. 后续可考虑采用数据增强方法和样本均衡策略克服数据规模局限性,在保证预测精确度下进一步提高总体分类性能.
表2给出了MDISN在同域,即无跨域性能对比的实验结果. 实验在MVSA和Yelp这2个独立数据集上(无跨域)分别进行了全监督训练和测试,训练样本和测试样本来自同一个数据集,按照3.1节比例划分.
同域性能分析实验中,对比了MDISN(即ALBEF[20]+情感适配为骨干)、MDISN-A(即BLIP[31]为骨干)与多模态融合SOTA方法和基准方法. 其中,方法EearlySent[27]、LateSent[27]、TFN[10]、VistaNet[11]和DiSRAN[27]的结果来自文献[27]. MDISN去除了域解耦处理,仅保留分类损失进行模型学习,训练中预训练大模型骨干的参数冻结,只训练其余参数.
在同域上的实验结果表明,MDISN在同域的多模态情感分类任务上,能够取得具有竞争力的性能. 与基准ALBEF[20]相比,在MVSA数据集上,ALBEF情感适配器模型的F1值达到了66.4%,相较于基础ALBEF模型的65.2%有所提升;在Yelp数据集上,F1值提升至55.7%,相较于基准模型ALBEF的54.1%也有提升.
对比VistaNet[11],DiSRAN[27],MDSE[33]模型,在MVSA数据集上,MDISN-A取得了最优和次优结果,特别是加上适配微调,证明了利用了预训练大模型的知识在下游任务具有很好的适应性. 但在YELP数据集上的分类性能略低. 剖析原因,MVSA数据集中的图文对通常围绕同一对象或场景展开,因而图文之间的情感语义关联性较强;而Yelp数据集的图文对主要是用户对餐厅的评价,其中图文内容更偏向于主题描述,情感语义关联性相对较弱. 因此,与采用了先进的图文对齐融合方法相比,仅利用预训练大模型构建的多模态融合特征编码器,虽然未达到最优,但其同域情感性能具有竞争力.
考虑到本文的元优化领域不变和领域特有解耦方法更多面向目标域任务,进一步结合跨域实验,对本文所提方法有效性进行了深入分析. 结合表1中单域训练直接跨域的实验结果,可以发现在MVSA数据集上,单域的3项指标远高于无域适应跨域Yelp→MVSA的结果. Yelp数据集与MVSA→Yelp的结果也如此. 引入MDISN模型,即通过在情感适配的ALBEF[20]的基础上加入领域特征解耦模块,实验结果表明MDISN在跨域任务中的F1值分别提升至47.4%(MVSA→Yelp)和41.7%(Yelp→MVSA),显著优于无域适应的跨域ALBEF[20]模型. 对比表1与表2中本文利用BLIP[31]作为图文融合解码器的实验结果,在同域实验中整体性能更高,但在Yelp数据集上其同域F1值仅为34.3%,反而低于跨域实验中的45.2%. 而MDISN-A模型在跨域实验中取得了更高的性能,Yelp数据集上的F1值达到50.5%,显著优于原始BLIP模型. 究其原因,Yelp数据集中的图文对数据关联性较弱,导致BLIP[31]在学习领域不变特征时表现出较低的表征能力,这在同域情感分类任务中成为限制因素. 与之相比,MDISN-A提取了更高质量图文对数据的目标域领域特定特征,克服了BLIP[31]在领域不变特征学习中的局限性,如Yelp→MVSA跨域任务中,在源域Yelp图文对数据关联性较弱的情况下,联合所提取的目标域MVSA的领域特定特征,在跨域自适应的情感性能取得了最优结果.
总之,实验表明跨域域适应学习仍是解决无标注目标域情感分类的有效方法. 通过同域与跨域性能分析实验,验证了本文利用预训练大模型在同域任务中的有效性,证明了MDISN提升了模型跨域应用的有效性.
为验证所提出MDISN方法中各模块和策略的有效性,对未采用相应模块的网络进行消融实验,结果如表3所示. 其中,MDISN_SA-代表从MDISN中去除情感适配器;MDISN_DI-代表只学习领域特定特征及从MDISN中去除领域不变编码器和领域判别器;MDISN_DS-代表只学习领域不变特征及从MDISN中去除领域特定编码器和领域分类器;MDISN_MO-代表MDISN未采用元优化训练.
从表3可以看出,与MDISN_SA-和MDISN_MO-相比,加入情感适配器及任务无关元学习的MDISN性能更优,证明了情感适配器和元优化训练策略在进行跨领域情感分类时发挥了不可或缺的作用,增强了网络对多模态情感特征的表征能力,并在特征解耦任务与情感分类任务的协同优化下,保证了总体优化目标. 此外,无论是在MVSA→Yelp迁移任务还是Yelp→MVSA迁移任务上,MDISN_DI-均获得了最低的F1值,这表明领域不变特征的提取对自适应性能的提升起到了重要的作用. 而相较于MDISN,仅采用领域特定特征的MDISN_DI-和领域共有特征的MDISN_DS-的多模态情感分类性能均有不同程度的降低,证明了同时利用领域不变特征与领域特定特征有助于提升跨领域自适应情感分类性能.
为探讨情感适配器在基于ALBEF[20]的特征提取模型中采用不同插入位置和不同参数时的调优性能,确定适配器嵌入调优方案,仍采用参数量、精确率、召回率和F1值作为性能指标,使用完全相同的训练配置进行对比分析实验.
ALBEF[20]的图像编码器使用ViT-B/16进行初始化,文本编码器则使用BERTbase模型的前6层进行初始化,而多模态编码器使用BERTbase模型的最后6层进行初始化. 针对该设计,表4对比了4种嵌入调优方法,其中Fine-tune表示对MDISN的特征提取模块的骨干网络(即ALBEF[20]图文特征提取模型)进行全量微调;SenA-T#L表示只在ALBEF[20]低层,也即ViT前6层与BERT的编码器部分的Transformer块插入情感适配器,并冻结其余部分;类似地,SenA-T#H表示只在ALBEF[20]高层,也即ViT后6层与多模态融合特征模块部分的Transformer块插入情感适配器,并冻结其余部分;SenA-T表示在所有Transformer块的所有层插入情感适配器.
在基于ALBEF[20]的特征提取模型对比情感适配器的不同插入位置的分类性能后,可以发现SenA-T的性能最好,SenA-T#H次之,SenA-T#L最差. 这一观察结果表明,特征提取器较低层表征通用特征,在数据集之间相似度差异较小的情况下,仅训练部分低层Transformer块对于下游任务的适应效果提升不大;而较高层表征特定特征与下游情感数据的分类直接相关,因此具有较高的训练价值. 对ALBEF[20]模型的所有层均插入情感适配器进行调优可以在较低的计算资源消耗的基础上,兼顾通用表征特征和类判别特征,促进域不变迁移学习和域特有特征挖掘,从而获得下游分类任务的性能提升.
从表4还可以发现,全量微调Fine-tune的调优方法在MVSA→Yelp迁移任务上的性能甚至不如SenA-T. 其原因在于由于MVSA数据集规模相对较小,且类别不平衡问题严重,在需要训练大规模参数的条件下,模型容易拟合到具有高占比的积极类别中,从而导致总体性能不佳. 这也进一步证明了利用适配器调优的SenA-Transformer块可以在少量目标域数据下获得使目标任务判别性能更优的特征表示.
利用预训练图文大模型对图文跨模态数据的优秀表征性能,本文首先以图文预训练大模型为基础,利用情感适配器获得良好跨模态情感融合特征. 在此基础上,重点针对跨领域情感分类任务,研究基于解耦的特征提取网络,在提取领域不变特征的同时提取并利用领域特定特征. 为应对特征解耦任务与情感分类任务的优化不一致问题,提出利用任务无关元训练策略,实现特征解耦与情感分类网络的元优化域适应迁移学习.
所提出的基于元优化特征解耦的跨领域情感分析方法面向多模态图文情感数据,利用情感适配Transformer块改进预训练大模型,避免了同时进行跨模态对齐与域偏移去除学习带来的复杂性,在少量训练数据下实现下游情感多模态特征提取. 进一步地,将多模态融合情感特征显式解耦为有助于跨领域迁移的领域不变特征和有助于情感分类的领域特定特征,并采用协同特征解耦和情感分类的任务无关元优化策略,实现跨领域知识迁移的性能提升. 于MVSA和Yelp这2个图文情感数据集上构建的双向迁移任务上进行的实验表明,所提出的模型在精确率、召回率和F1值上均获得了有竞争力的性能,证明了所提出模型的有效性. 消融实验分析进一步说明,情感表达依赖于特定领域,同时利用领域对抗所提取的领域不变特征和领域分类引导的领域特定特征使模型理解和提取更具判别性的该领域的情感分类特征,提高无标注目标域的多模态情感分析性能.
此外,所提出的元优化域适应学习方法亦可应用于其他的多模态跨域任务,特别是目标域特征空间与源域存在更大属性差异以及领域不变特征对目标域任务表征能力有限的任务场景. 为此,在下一步工作中,从提升多模态融合性能、利用数据增强等角度加强多模态特征表征能力,并在其他多模态任务开展元优化域适应方法研究.
作者贡献声明:贾熹滨指导算法研发和论文撰写;李宸提出算法、完成实验并撰写论文;王珞参与算法研讨并修订论文;张沐晨参与实验并修改论文;刘潇健提供算法优化建议并参与论文修订;张旸旸提出论文修改建议;温家凯对算法优化提供建议.

参考文献 引证文献
排序方式:
1
Das R, Singh T D. Multimodal sentiment analysis: A survey of methods, trends, and challenges[J]. ACM Computing Surveys, 2023, 55(13s): 1−38
2
Chan J Y L, Bea K T, Leow S M H, et al. State of the art: A review of sentiment analysis based on sequential transfer learning[J]. Artificial Intelligence Review, 2023, 56(1): 749−780
3
Singhal P, Walambe R, Ramanna S, et al. Domain adaptation: Challenges, methods, datasets, and applications[J]. IEEE Access, 2023, 11: 6973−7020
4
Azuma C, Ito T, Shimobaba T. Adversarial domain adaptation using contrastive learning[J]. Engineering Applications of Artificial Intelligence, 2023, 123: 106394
5
Zhou Qianyu, Gu Qiqi, Pang Jiangmiao, et al. Self-adversarial disentangling for specific domain adaptation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(7): 8954−8968
6
Li Jingjing, Chen Erpeng, Ding Zhengming, et al. Maximum density divergence for domain adaptation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43(11): 3918−3930
7
Zhu Yongchun, Zhuang Fuzhen, Wang Jindong, et al. Deep subdomain adaptation network for image classification[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 32(4): 1713−1722
8
Zhao Han, Des Combes R T, Zhang Kun, et al. On learning invariant representations for domain adaptation[C]//Proc of the 36th Int Conf on Machine Learning. New York: PMLR, 2019: 7523−7532
9
Johansson F D, Sontag D, Ranganath R. Support and invertibility in domain-invariant representations[C]//Proc of the 22nd Int Conf on Artificial Intelligence and Statistics. New York: PMLR, 2019: 527−536
10
Zadeh A, Chen Minghai, Poria S, et al. Tensor fusion network for multimodal sentiment analysis[C]//Proc of the 2017 Conf on Empirical Methods in Natural Language Processing. New York: PMLR, 2017: 1103−1114
11
Truong Q T, Lauw H W. VistaNet: Visual aspect attention network for multimodal sentiment analysis[C]//Proc of the 33rd Association for the Advancement of Artificial Intelligence. Palo Alto, CA: AAAI, 2019: 305−312
12
Gui Tao, Zhu Liang, Zhang Qi, et al. Cooperative multimodal approach to depression detection in Twitter[C]//Proc of the 33rd Association for the Advancement of Artificial Intelligence. Palo Alto, CA: AAAI, 2019: 110−117
13
Ling Yan, Yu Jianfei, Xia Rui. Vision-language pre-training for multimodal aspect-based sentiment analysis[C]//Proc of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg, PA: ACL, 2022: 2149−2159
14
Ganin Y, Ustinova E, Ajakan H, et al. Domain-adversarial training of neural networks[J]. Journal of Machine Learning Research, 2016, 17(59): 1−35
15
Jia Xibin, Li Chen, Zeng Meng, et al. An improved unified domain adversarial category-wise alignment network for unsupervised cross-domain sentiment classification[J]. Engineering Applications of Artificial Intelligence, 2023, 126: 107108
16
黄学坚, 马廷淮, 王根生. 基于样本内外协同表示和自适应融合的多模态学习方法[J]. 计算机研究与发展, 2024, 61(5): 1310−1324
Huang Xuejian, Ma Tinghuai, Wang Gensheng. Multimodal learning method based on intra- and inter-sample cooperative representation and adaptive fusion[J]. Journal of Computer Research and Development, 2024, 61(5): 1310−1324 (in Chinese)
17
Qi Fan, Yang Xiaoshan, Xu Changsheng. A unified framework for multimodal domain adaptation[C]//Proc of the 26th ACM Int Conf on Multimedia. New York: ACM, 2018: 429−437
18
Ma Xinhong, Zhang Tianzhu, Xu Changsheng. Deep multi-modality adversarial networks for unsupervised domain adaptation[J]. IEEE Transactions on Multimedia, 2019, 21(9): 2419−2431
19
Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Proc of the 31st Int Conf on Neural Information Processing Systems. New York: ACM, 2017: 6000−6010
20
Li Junnan, Selvaraju R, Gotmare A, et al. Align before fuse: Vision and language representation learning with momentum distillation[J]. Advances in Neural Information Processing Systems, 2021, 34: 9694−9705
21
Chen Shoufa, Ge Chongjian, Tong Zhan, et al. Adaptformer: Adapting vision transformers for scalable visual recognition[J]. Advances in Neural Information Processing Systems, 2022, 35: 16664−16678
22
Li Ya, Tian Xinmei, Gong Mingming, et al. Deep domain generalization via conditional invariant adversarial networks[C]//Proc of the 2018 European Conf on Computer Vision (ECCV). Berlin: Springer, 2018: 624−639
23
Bui M H, Tran T, Tran A, et al. Exploiting domain-specific features to enhance domain generalization[J]. Advances in Neural Information Processing Systems, 2021, 34: 21189−21201
24
Wei Guoqiang, Lan Cuiling, Zeng Wenjun, et al. MetaAlign: Coordinating domain alignment and classification for unsupervised domain adaptation[C]//Proc of the IEEE/CVF Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ: IEEE, 2021: 16643−16653
25
Niu Teng, Zhu Shiai, Pang Lei, et al. Sentiment analysis on multi-view social data[C]//Proc of the 22nd Int Conf on Multimedia Modeling. Berlin: Springer, 2016: 15−27
26
刘琦玮, 李俊, 顾蓓蓓, 等. TSAIE: 图像增强文本的多模态情感分析模型[J]. 数据与计算发展前沿, 2022, 4(3): 131−140
Liu Qiwei, Li Jun, Gu Beibei, et al. TSAIE: Text sentiment analysis model based on image enhancement[J]. Frontiers of Data & Computing, 2022, 4(3): 131−140 (in Chinese)
27
Zhang Yuhao, Zhang Ying, Guo Wenya, et al. Learning disentangled representation for multimodal cross-domain sentiment analysis[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 34(10): 7956−7966
28
Baltrušaitis T, Ahuja C, Morency L P. Multimodal machine learning: A survey and taxonomy[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 41(2): 423−443
29
Dosovitskiy A. An image is worth 16x16 words: Transformers for image recognition at scale[J]. arXiv preprint, arXiv: 2010.11929, 2020
30
Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint, arXiv: 1810.04805, 2018
31
Li Junnan, Li Dongxu, Xiong Caiming, et al. BLIP: Bootstrapping language-image pre-training for unified vision-language understanding and generation[C]//Proc of the 39th Int Conf on Machine Learning. New York: PMLR, 2022: 12888−12900
32
Zhu Tong, Li Leida, Yang Jufeng, et al. Multimodal sentiment analysis with image-text interaction network[J]. IEEE Transactions on Multimedia, 2022, 25: 3375−3385
33
Li Jingzhe, Wang Chengji, Luo Zhiming, et al. Modality-dependent sentiments exploring for multi-modal sentiment classification[C]//Proc of the 2024 IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2024: 7930−7934
2025年第62卷第11期
103
46
0
引用本文
BibTeX
文章信息
doi: 10.7544/issn1000-1239.202440624
  • 接收时间:2024-07-20
  • 首发时间:2026-11-10
  • 出版时间:2025-11-01
补充材料
相关文章
文章信息
作者
出版历史
  • 收稿日期:2024-07-20
  • 修回日期:2025-01-15
  • 录用日期:2025-03-03
基金
作者信息
    1北京工业大学计算机学院 北京 100124
    2北京人工智能研究院(北京工业大学) 北京 100124
    3中国电子技术标准化研究院 北京 100007
    4广西达译科技有限公司 南宁 530006
参考文献
分享链接
https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202440624
分享至
全文二维码

扫描看全文

引用本文
BibTeX
本文的引用情况
2种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏