Article(id=1329435702628037129, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1329435571803501430, articleNumber=null, orderNo=null, doi=10.7544/issn1000-1239.202550936, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=null, receivedDate=1766505600000, receivedDateStr=2025-12-24, revisedDate=1784736000000, revisedDateStr=2026-07-23, acceptedDate=null, acceptedDateStr=null, onlineDate=1794891771227, onlineDateStr=2026-11-17, pubDate=1793462400000, pubDateStr=2026-11-01, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1794891771227, onlineIssueDateStr=2026-11-17, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1794891771227, creator=13701087609, updateTime=1794891771227, updator=13701087609, issue=Issue{id=1329435571803501430, tenantId=1146029695717560320, journalId=1324759914146529478, year='2026', volume='63', issue='11', pageStart='2767', pageEnd='3031', issueExtLink='null', onlineDate='null', pubDate='1793462400000', pubDateStr='2026-11-01', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=1, issueType=1, specialIssue=null, createTime=1794891740036, creator='13701087609', updateTime=1794891740036, updator='13701087609', preIssue=null, nextIssue=null, articleTotal=null, ext=null, issueFiles=null, downloadFileDto=null}, startPage=3002, endPage=3016, ext={EN=ArticleExt(id=1329435702812586506, articleId=1329435702628037129, tenantId=1146029695717560320, journalId=1324759914146529478, language=EN, title=A Multi-Feature Image Captioning Model with Common-Specific Collaborative Modeling and Mixture-of-Experts Fusion, columnId=1325489891789910943, journalTitle=Journal of Computer Research and Development, columnName=Artificial Intelligence, runingTitle=null, highlight=null, articleAbstract=
Existing image captioning methods often struggle to simultaneously capture global semantics and fine-grained details, leading to overly generic descriptions or missing key information. To address this issue, we propose EAFCC, short for expert-aware fusion with common-specific collaboration, a multi-feature image captioning model with mixture-of-experts fusion. The model takes grid features, region features, and segmentation features as parallel inputs. In the encoding stage, a pre-fusion module first performs implicit alignment of heterogeneous visual features and models cross-feature common information within a shared semantic space, while preserving the basis for feature-specific representations. A shared Transformer encoder is then employed to learn the common representations, whose consistency and discriminability are further enhanced by a contrastive learning constraint. Meanwhile, three independent encoding branches are used to model feature-specific representations, so as to retain fine-grained complementary information associated with different feature types. A gated routing mechanism inspired by mixture-of-experts is further introduced to dynamically weight and fuse the specific information. Finally, the enhanced common features and the fused specific features are collaboratively integrated, processed by a skip-connected encoder, and fed into a Transformer decoder to generate image captions. Experimental results on MS-COCO dataset show that EAFCC achieves competitive performance on the CIDEr-D metric and outperforms existing single-feature and dual-feature methods, demonstrating the effectiveness of common-specific collaborative modeling and dynamic multi-feature fusion for image captioning.
, authors=Jianfeng Yang
1, Dongpo Zhu
2, Rui Cao
1, *, Xuanhao Cui
1, Ping Wang
1, Jie Zheng
1, authorsList=Jianfeng Yang, Dongpo Zhu, Rui Cao, Xuanhao Cui, Ping Wang, Jie Zheng, authorCompany=null, correspAuthors=Rui Cao, authorNote=
born in 1981. Master, senior engineer, master supervisor. His main research interests include software development methodologies and data science, with a focus on applications in higher education
born in 1999. Master, engineer. His main research interest includes multi-modal fusion and application
born in 1991. PhD, lecturer, master supervisor. Member of CCF. His main research interests include text representation learning, applications and optimization of generative large language models, and art generation using diffusion models
born in 2003. Master candidate. His main research interest includes large model development and application
born in 2002. Master candidate. His main research interest includes large model visual reasoning
born in 1985. PhD, professor, master supervisor. His main research interest includes edge general intelligence
, correspAuthorsNote=null, copyrightStatement=Copyright © 2026 Journal of Computer Research and Development. All rights reserved., copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=null, magXml=null, pdfUrl=null, pdf=null, pdfFileSize=null, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=null, mapNumber=null, fund=null), CN=ArticleExt(id=1329435706017034797, articleId=1329435702628037129, tenantId=1146029695717560320, journalId=1324759914146529478, language=CN, title=基于共性-特性协同建模与多专家融合的多特征图像描述模型, columnId=1325489892104483745, journalTitle=计算机研究与发展, columnName=人工智能, runingTitle=null, highlight=null, articleAbstract=
针对现有图像描述方法难以同时兼顾全局语义与细粒度细节、生成结果易偏泛化或关键信息遗漏的问题,提出一种基于共性-特性协同建模与多专家融合的多特征图像描述生成模型EAFCC(expert-aware fusion with common-specific collaboration)。该模型并行接收网格特征、区域特征与分割特征,在编码阶段首先通过预融合模块对多源异构特征进行隐式对齐。在共享语义空间中提取跨特征一致的共性信息,同时为各特征流保留差异化表达基础。随后利用共享Transformer编码器建模共性表征,并通过对比学习约束进一步增强其一致性与判别性。同时,采用3条独立编码分支对不同特征流进行特性建模,以保留与特征类型相关的细粒度补充信息,并借助多专家门控路由机制实现特性信息的动态加权融合。最后,将增强后的共性特征与融合后的特性特征进行协同整合,经跳跃连接编码器后送入Transformer解码器生成图像描述。在MS-COCO数据集上的实验表明,EAFCC在CIDEr-D指标上取得了较优性能,并优于现有单特征或双特征方法,验证了共性与特性联合建模的有效性。
, authors=杨建锋
1, 朱东坡
2, 曹瑞
1, *, 崔选皓
1, 王平
1, 郑杰
1, authorsList=杨建锋, 朱东坡, 曹瑞, 崔选皓, 王平, 郑杰, authorCompany=null, correspAuthors=曹瑞, authorNote=
作者贡献声明
杨建锋提出研究构思并指导研究开展,审阅和修改论文;朱东坡负责方法实现、模型搭建、实验与数据分析,并撰写论文;曹瑞审阅和修改论文,统筹审稿意见回复并负责最终定稿;崔选皓参与论文审阅与修改;王平参与论文审阅与修改;郑杰提出改进思路,并提供方法优化建议与分析支持。
 |
1981年生。硕士,高级工程师,硕士生导师。主要研究方向为软件开发方法与数据科学及其在高校中的应用 |
 |
1999年生。硕士,工程师。主要研究方向为多模态融合应用 |
 |
1991年生。博士,讲师,硕士生导师。CCF会员。主要研究方向为文本表示学习、生成式大语言模型应用、优化及扩散模型艺术生成 |
 |
2003年生。硕士研究生。主要研究方向为大模型开发应用 |
 |
2002年生。硕士研究生。主要研究方向为大模型视觉推理 |
 |
1985年生。博士,教授,硕士生导师。主要研究方向为边缘通用智能 |
, correspAuthorsNote=
, copyrightStatement=版权所有 © 2026 计算机研究与发展编辑部, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=t0VOgGI1AYfGSHRtwYAHFg==, magXml=k0okzEd90rsU7Kw3ZDSt0w==, pdfUrl=null, pdf=DmIv8o5O6F23TSnQKzhCrA==, pdfFileSize=8662548, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=NbfDvL4pNIPbfAKkLw50og==, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=El29xaM36g1elpa5Ah2Jjw==, mapNumber=null, fund=null)}, authors=[Author(id=1329435706369356341, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=yjf@nwu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329435706449048119, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, authorId=1329435706369356341, language=EN, stringName=Jianfeng Yang, firstName=Jianfeng, middleName=null, lastName=Yang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=
1, address=
1School of Computer Science, Northwest University, Xi’an 710127, bio={"content":"
born in 1981. Master, senior engineer, master supervisor. His main research interests include software development methodologies and data science, with a focus on applications in higher education
"}, bioImg=null, bioContent=
born in 1981. Master, senior engineer, master supervisor. His main research interests include software development methodologies and data science, with a focus on applications in higher education
, aboutCorrespAuthor=null), CN=AuthorExt(id=1329435706511962680, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, authorId=1329435706369356341, language=CN, stringName=杨建锋, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=
1, address=
1西北大学计算机学院 西安 710127, bio={"img":"rl+Yix6xaTprWHjG8wlINg==","content":"
1981年生。硕士,高级工程师,硕士生导师。主要研究方向为软件开发方法与数据科学及其在高校中的应用
"}, bioImg=rl+Yix6xaTprWHjG8wlINg==, bioContent=
1981年生。硕士,高级工程师,硕士生导师。主要研究方向为软件开发方法与数据科学及其在高校中的应用
, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329435706209972782, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, xref=1, ext=[AuthorCompanyExt(id=1329435706222555695, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, companyId=1329435706209972782, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
1School of Computer Science, Northwest University, Xi’an 710127), AuthorCompanyExt(id=1329435706230944304, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, companyId=1329435706209972782, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
1西北大学计算机学院 西安 710127)])]), Author(id=1329435706570682938, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329435706641986108, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, authorId=1329435706570682938, language=EN, stringName=Dongpo Zhu, firstName=Dongpo, middleName=null, lastName=Zhu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=
2, address=
2Beijing Sankuai Technology Co., Ltd., Beijing 100190, bio={"content":"
born in 1999. Master, engineer. His main research interest includes multi-modal fusion and application
"}, bioImg=null, bioContent=
born in 1999. Master, engineer. His main research interest includes multi-modal fusion and application
, aboutCorrespAuthor=null), CN=AuthorExt(id=1329435706717483581, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, authorId=1329435706570682938, language=CN, stringName=朱东坡, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=
2, address=
2北京三快科技有限公司 北京 100190, bio={"img":"x2S1WMCgYVTZqsPil0V8Dw==","content":"
1999年生。硕士,工程师。主要研究方向为多模态融合应用
"}, bioImg=x2S1WMCgYVTZqsPil0V8Dw==, bioContent=
1999年生。硕士,工程师。主要研究方向为多模态融合应用
, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329435706293858865, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, xref=2, ext=[AuthorCompanyExt(id=1329435706302247474, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, companyId=1329435706293858865, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
2Beijing Sankuai Technology Co., Ltd., Beijing 100190), AuthorCompanyExt(id=1329435706310636083, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, companyId=1329435706293858865, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
2北京三快科技有限公司 北京 100190)])]), Author(id=1329435706776203839, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=cr@nwu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1329435706851701313, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, authorId=1329435706776203839, language=EN, stringName=Rui Cao, firstName=Rui, middleName=null, lastName=Cao, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=
1, *, address=
1School of Computer Science, Northwest University, Xi’an 710127, bio={"content":"
born in 1991. PhD, lecturer, master supervisor. Member of CCF. His main research interests include text representation learning, applications and optimization of generative large language models, and art generation using diffusion models
"}, bioImg=null, bioContent=
born in 1991. PhD, lecturer, master supervisor. Member of CCF. His main research interests include text representation learning, applications and optimization of generative large language models, and art generation using diffusion models
, aboutCorrespAuthor=null), CN=AuthorExt(id=1329435706918810178, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, authorId=1329435706776203839, language=CN, stringName=曹瑞, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=
1, *, address=
1西北大学计算机学院 西安 710127, bio={"img":"3mjX5y/1tZNO/WqqXyMI4g==","content":"
1991年生。博士,讲师,硕士生导师。CCF会员。主要研究方向为文本表示学习、生成式大语言模型应用、优化及扩散模型艺术生成
"}, bioImg=3mjX5y/1tZNO/WqqXyMI4g==, bioContent=
1991年生。博士,讲师,硕士生导师。CCF会员。主要研究方向为文本表示学习、生成式大语言模型应用、优化及扩散模型艺术生成
, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329435706209972782, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, xref=1, ext=[AuthorCompanyExt(id=1329435706222555695, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, companyId=1329435706209972782, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
1School of Computer Science, Northwest University, Xi’an 710127), AuthorCompanyExt(id=1329435706230944304, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, companyId=1329435706209972782, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
1西北大学计算机学院 西安 710127)])]), Author(id=1329435706985919044, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329435707061416518, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, authorId=1329435706985919044, language=EN, stringName=Xuanhao Cui, firstName=Xuanhao, middleName=null, lastName=Cui, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=
1, address=
1School of Computer Science, Northwest University, Xi’an 710127, bio={"content":"
born in 2003. Master candidate. His main research interest includes large model development and application
"}, bioImg=null, bioContent=
born in 2003. Master candidate. His main research interest includes large model development and application
, aboutCorrespAuthor=null), CN=AuthorExt(id=1329435707162079815, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, authorId=1329435706985919044, language=CN, stringName=崔选皓, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=
1, address=
1西北大学计算机学院 西安 710127, bio={"img":"KCF21Vtpk9buAWULZ5uTzA==","content":"
2003年生。硕士研究生。主要研究方向为大模型开发应用
"}, bioImg=KCF21Vtpk9buAWULZ5uTzA==, bioContent=
2003年生。硕士研究生。主要研究方向为大模型开发应用
, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329435706209972782, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, xref=1, ext=[AuthorCompanyExt(id=1329435706222555695, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, companyId=1329435706209972782, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
1School of Computer Science, Northwest University, Xi’an 710127), AuthorCompanyExt(id=1329435706230944304, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, companyId=1329435706209972782, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
1西北大学计算机学院 西安 710127)])]), Author(id=1329435707233382985, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329435707308880459, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, authorId=1329435707233382985, language=EN, stringName=Ping Wang, firstName=Ping, middleName=null, lastName=Wang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=
1, address=
1School of Computer Science, Northwest University, Xi’an 710127, bio={"content":"
born in 2002. Master candidate. His main research interest includes large model visual reasoning
"}, bioImg=null, bioContent=
born in 2002. Master candidate. His main research interest includes large model visual reasoning
, aboutCorrespAuthor=null), CN=AuthorExt(id=1329435707380183628, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, authorId=1329435707233382985, language=CN, stringName=王平, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=
1, address=
1西北大学计算机学院 西安 710127, bio={"img":"1jrykixFy//FWewzY0cOTw==","content":"
2002年生。硕士研究生。主要研究方向为大模型视觉推理
"}, bioImg=1jrykixFy//FWewzY0cOTw==, bioContent=
2002年生。硕士研究生。主要研究方向为大模型视觉推理
, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329435706209972782, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, xref=1, ext=[AuthorCompanyExt(id=1329435706222555695, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, companyId=1329435706209972782, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
1School of Computer Science, Northwest University, Xi’an 710127), AuthorCompanyExt(id=1329435706230944304, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, companyId=1329435706209972782, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
1西北大学计算机学院 西安 710127)])]), Author(id=1329435707468264014, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, orderNo=5, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329435707568927312, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, authorId=1329435707468264014, language=EN, stringName=Jie Zheng, firstName=Jie, middleName=null, lastName=Zheng, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=
1, address=
1School of Computer Science, Northwest University, Xi’an 710127, bio={"content":"
born in 1985. PhD, professor, master supervisor. His main research interest includes edge general intelligence
"}, bioImg=null, bioContent=
born in 1985. PhD, professor, master supervisor. His main research interest includes edge general intelligence
, aboutCorrespAuthor=null), CN=AuthorExt(id=1329435707648619089, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, authorId=1329435707468264014, language=CN, stringName=郑杰, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=
1, address=
1西北大学计算机学院 西安 710127, bio={"img":"9Vnp7bpLM1DolD2CFV6lXQ==","content":"
1985年生。博士,教授,硕士生导师。主要研究方向为边缘通用智能
"}, bioImg=9Vnp7bpLM1DolD2CFV6lXQ==, bioContent=
1985年生。博士,教授,硕士生导师。主要研究方向为边缘通用智能
, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329435706209972782, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, xref=1, ext=[AuthorCompanyExt(id=1329435706222555695, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, companyId=1329435706209972782, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
1School of Computer Science, Northwest University, Xi’an 710127), AuthorCompanyExt(id=1329435706230944304, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, companyId=1329435706209972782, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
1西北大学计算机学院 西安 710127)])])], keywords=[Keyword(id=1329435707740893778, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, orderNo=1, keyword=image captioning), Keyword(id=1329435707799614035, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, orderNo=2, keyword=multi-feature fusion), Keyword(id=1329435707858334292, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, orderNo=3, keyword=common-specific collaborative modeling), Keyword(id=1329435707921248853, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, orderNo=4, keyword=contrastive learning), Keyword(id=1329435707996746326, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, orderNo=5, keyword=Transformer), Keyword(id=1329435708055466583, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, orderNo=1, keyword=图像描述), Keyword(id=1329435708118381144, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, orderNo=2, keyword=多特征融合), Keyword(id=1329435708185490009, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, orderNo=3, keyword=共性-特性协同建模), Keyword(id=1329435708240015962, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, orderNo=4, keyword=对比学习), Keyword(id=1329435708302930523, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, orderNo=5, keyword=Transformer)], refs=[Reference(id=1329435711129891452, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=1, rfOrder=0, authorNames=null, journalName=null, refType=null, unstructuredReference=Gurari D, Li Qing, Stangl A J, et al. VizWiz grand challenge: Answering visual questions from blind people[C]//Proc of the 31st IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2018: 3608−3617, articleTitle=null, refAbstract=null), Reference(id=1329435711209583229, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=2, rfOrder=1, authorNames=null, journalName=null, refType=null, unstructuredReference=Aneja J, Deshpande A, Schwing A G. Convolutional image captioning[C]//Proc of the 31st IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2018: 5561−5570, articleTitle=null, refAbstract=null), Reference(id=1329435711272497790, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=3, rfOrder=2, authorNames=null, journalName=null, refType=null, unstructuredReference=Yu Jun, Li Jing, Yu Zhou, et al. Multimodal Transformer with multi-view visual representation for image captioning[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2019, 30(12): 4467−4480, articleTitle=null, refAbstract=null), Reference(id=1329435711339606655, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=4, rfOrder=3, authorNames=null, journalName=null, refType=null, unstructuredReference=Lou Liangshan, Lu Ke, Xue Jian. A novel cross-fusion method of different types of features for image captioning[C]//Proc of the 33rd Int Joint Conf on Neural Networks. Piscataway, NJ: IEEE, 2023: 1−8, articleTitle=null, refAbstract=null), Reference(id=1329435711410909824, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=5, rfOrder=4, authorNames=null, journalName=null, refType=null, unstructuredReference=Mathur N, Baldwin T, Cohn T. Tangled up in BLEU: Reevaluating the evaluation of automatic machine translation evaluation metrics[C]//Proc of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2020: 4984−4997, articleTitle=null, refAbstract=null), Reference(id=1329435711486407297, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=6, rfOrder=5, authorNames=null, journalName=null, refType=null, unstructuredReference=Cui Yin, Yang Guandao, Veit A, et al. Learning to evaluate image captioning[C]//Proc of the 31st IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2018: 5804−5812, articleTitle=null, refAbstract=null), Reference(id=1329435711553516162, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=7, rfOrder=6, authorNames=null, journalName=null, refType=null, unstructuredReference=Bernardi R, Cakici R, Elliott D, et al. Automatic description generation from images: A survey of models, datasets, and evaluation measures[J]. Journal of Artificial Intelligence Research, 2016, 55: 409−442, articleTitle=null, refAbstract=null), Reference(id=1329435711633207939, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=8, rfOrder=7, authorNames=null, journalName=null, refType=null, unstructuredReference=Al-Shamayleh A S, Adwan O, Alsharaiah M A, et al. A comprehensive literature review on image captioning methods and metrics based on deep learning technique[J]. Multimedia Tools and Applications, 2024, 83(12): 34219−34268, articleTitle=null, refAbstract=null), Reference(id=1329435711700316804, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=9, rfOrder=8, authorNames=null, journalName=null, refType=null, unstructuredReference=Ghandi T, Pourreza H, Mahyar H. Deep learning approaches on image captioning: A review[J]. ACM Computing Surveys, 2023, 56(3): 1−39, articleTitle=null, refAbstract=null), Reference(id=1329435711771619973, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=10, rfOrder=9, authorNames=null, journalName=null, refType=null, unstructuredReference=Huang Lun, Wang Wenmin, Chen Jie, et al. Attention on attention for image captioning[C]//Proc of the 17th IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2019: 4634−4643, articleTitle=null, refAbstract=null), Reference(id=1329435711872283270, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=11, rfOrder=10, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Guang, Zhu Linchao, Liu Ping, et al. Entangled Transformer for image captioning[C]//Proc of the 17th IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2019: 8928−8937, articleTitle=null, refAbstract=null), Reference(id=1329435711939392135, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=12, rfOrder=11, authorNames=null, journalName=null, refType=null, unstructuredReference=Cornia M, Stefanini M, Baraldi L, et al. Meshed-memory Transformer for image captioning[C]//Proc of the 33rd IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2020: 10578−10587, articleTitle=null, refAbstract=null), Reference(id=1329435712002306696, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=13, rfOrder=12, authorNames=null, journalName=null, refType=null, unstructuredReference=Pan Yingwei, Yao Ting, Li Yehao, et al. X-linear attention networks for image captioning[C]//Proc of the 33rd IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2020: 10971−10980, articleTitle=null, refAbstract=null), Reference(id=1329435712069415561, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=14, rfOrder=13, authorNames=null, journalName=null, refType=null, unstructuredReference=Luo Yunpeng, Ji Jiayi, Sun Xiaoshuai, et al. Dual-level collaborative Transformer for image captioning[C]//Proc of the 35th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2021: 2286−2293, articleTitle=null, refAbstract=null), Reference(id=1329435712136524426, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=15, rfOrder=14, authorNames=null, journalName=null, refType=null, unstructuredReference=Nguyen V Q, Suganuma M, Okatani T. GRIT: Faster and better image captioning transformer using dual visual features[G]//LNCS 13696: Proc of the 17th European Conf on Computer Vision. Berlin: Springer, 2022: 167−184, articleTitle=null, refAbstract=null), Reference(id=1329435712195244683, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=16, rfOrder=15, authorNames=null, journalName=null, refType=null, unstructuredReference=Wu Mingrui, Zhang Xuying, Sun Xiaoshuai, et al. DIFNet: Boosting visual information flow for image captioning[C]//Proc of the 35th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 18020−18029, articleTitle=null, refAbstract=null), Reference(id=1329435712249770636, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=17, rfOrder=16, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Junnan, Li Dongxu, Xiong Caiming, et al. BLIP: Bootstrapping language-image pre-training for unified vision-language understanding and generation[C]//Proc of the 39th Int Conf on Machine Learning. New York: PMLR, 2022: 12888−12900, articleTitle=null, refAbstract=null), Reference(id=1329435712321073805, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=18, rfOrder=17, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Peng, Yang An, Men Rui, et al. OFA: Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework[C]//Proc of the 39th Int Conf on Machine Learning. New York: PMLR, 2022: 23318−23340, articleTitle=null, refAbstract=null), Reference(id=1329435712383988366, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=19, rfOrder=18, authorNames=null, journalName=null, refType=null, unstructuredReference=Bai Jinze, Men Rui, Yang Hao, et al. OFASys: A multi-modal multi-task learning system for building generalist models[J]. arXiv preprint, arXiv: 2212.04408, 2022, articleTitle=null, refAbstract=null), Reference(id=1329435712451097231, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=20, rfOrder=19, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Zirui, Yu Jiahui, Yu Adams Wei, et al. SimVLM: Simple visual language model pretraining with weak supervision[J]. arXiv preprint, arXiv: 2108.10904, 2021, articleTitle=null, refAbstract=null), Reference(id=1329435712526594704, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=21, rfOrder=20, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Wei, Gao Can, Niu Guocheng, et al. UNIMO: Towards unified-modal understanding and generation via cross-modal contrastive learning[J]. arXiv preprint, arXiv: 2012.15409, 2020, articleTitle=null, refAbstract=null), Reference(id=1329435712585314961, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=22, rfOrder=21, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhou Luowei, Palangi H, Zhang Lei, et al. Unified vision-language pre-training for image captioning and VQA[C]//Proc of the 34th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2020: 13041−13049, articleTitle=null, refAbstract=null), Reference(id=1329435712644035218, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=23, rfOrder=22, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Jiaxuan, Vo D M, Sugimoto A, et al. EVCap: Retrieval-augmented image captioning with external visual-name memory for open-world comprehension[C]//Proc of the 37th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2024: 13733−13742, articleTitle=null, refAbstract=null), Reference(id=1329435712715338387, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=24, rfOrder=23, authorNames=null, journalName=null, refType=null, unstructuredReference=Hua Hang, Liu Qing, Zhang Lingzhi, et al. FineCaption: Compositional image captioning focusing on wherever you want at any granularity[C]//Proc of the 38th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2025: 24763−24773, articleTitle=null, refAbstract=null), Reference(id=1329435712774058644, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=25, rfOrder=24, authorNames=null, journalName=null, refType=null, unstructuredReference=Anderson P, He Xiaodong, Buehler C, et al. Bottom-up and top-down attention for image captioning and visual question answering[C]//Proc of the 31st IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2018: 6077−6086, articleTitle=null, refAbstract=null), Reference(id=1329435712832778901, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=26, rfOrder=25, authorNames=null, journalName=null, refType=null, unstructuredReference=Ren Shaoqing, He Kaiming, Girshick R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137−1149, articleTitle=null, refAbstract=null), Reference(id=1329435712887304854, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=27, rfOrder=26, authorNames=null, journalName=null, refType=null, unstructuredReference=Xiong Yuwen, Liao Renjie, Zhao Hengshuang, et al. UPSNet: A unified panoptic segmentation network[C]//Proc of the 32nd IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2019: 8818−8826, articleTitle=null, refAbstract=null), Reference(id=1329435712946025111, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=28, rfOrder=27, authorNames=null, journalName=null, refType=null, unstructuredReference=Yang Xu, Tang Kaihua, Zhang Hanwang, et al. Auto-encoding scene graphs for image captioning[C]//Proc of the 32nd IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2019: 10685−10694, articleTitle=null, refAbstract=null), Reference(id=1329435713008939672, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=29, rfOrder=28, authorNames=null, journalName=null, refType=null, unstructuredReference=Yao Ting, Pan Yingwei, Li Yehao, et al. Exploring visual relationship for image captioning[G]//LNCS 11218: Proc of the 15th European Conf on Computer Vision. Berlin: Springer, 2018: 711−727, articleTitle=null, refAbstract=null), Reference(id=1329435713080242841, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=30, rfOrder=29, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Jing, Xie Yingshuai, Ding Weichao, et al. Cross on cross attention: Deep fusion transformer for image captioning[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(8): 4257−4268, articleTitle=null, refAbstract=null), Reference(id=1329435713143157402, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=31, rfOrder=30, authorNames=null, journalName=null, refType=null, unstructuredReference=Jiang Wenhao, Ma Lin, Jiang Yugang, et al. Recurrent fusion network for image captioning[G]//LNCS 11206: Proc of the 15th European Conf on Computer Vision. Berlin: Springer, 2018: 499−515, articleTitle=null, refAbstract=null), Reference(id=1329435713206071963, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=32, rfOrder=31, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Weixuan, Chen Zhihong, Hu Haifeng. Hierarchical attention network for image captioning[C]//Proc of the 33rd AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2019: 8957−8964, articleTitle=null, refAbstract=null), Reference(id=1329435713264792220, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=33, rfOrder=32, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhao Shanshan, Li Lixiang, Peng Haipeng. Incorporating retrieval-based method for feature enhanced image captioning[J]. Applied Intelligence, 2023, 53(8): 9731−9743, articleTitle=null, refAbstract=null), Reference(id=1329435713331901085, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=34, rfOrder=33, authorNames=null, journalName=null, refType=null, unstructuredReference=Ramos R, Martins B, Elliott D, et al. SmallCap: Lightweight image captioning prompted with retrieval augmentation[C]//Proc of the 36th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2023: 2840−2849, articleTitle=null, refAbstract=null), Reference(id=1329435713390621342, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=35, rfOrder=34, authorNames=null, journalName=null, refType=null, unstructuredReference=Song Jiayan, Pan Renjie, Zhou Jun, et al. M-RAT: A multi-grained retrieval augmentation Transformer for image captioning[C]//Proc of the 17th Asian Conf on Computer Vision. Berlin: Springer, 2024: 185−203, articleTitle=null, refAbstract=null), Reference(id=1329435713457730207, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=36, rfOrder=35, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Ning, Xie Jiahao, Wu Jihao, et al. Controllable image captioning via prompting[C]//Proc of the 37th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2023: 2617−2625, articleTitle=null, refAbstract=null), Reference(id=1329435713520644768, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=37, rfOrder=36, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Kun, Mao Zhendong, Wang Quan, et al. Negative-aware attention framework for image-text matching[C]//Proc of the 35th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 15661−15670, articleTitle=null, refAbstract=null), Reference(id=1329435713579365025, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=38, rfOrder=37, authorNames=null, journalName=null, refType=null, unstructuredReference=Chen Junyi, Guo Longteng, Sun Jia, et al. EVE: Efficient vision-language pre-training with masked prediction and modality-aware MoE[C]//Proc of the 38th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2024: 1110−1119, articleTitle=null, refAbstract=null), Reference(id=1329435713646473890, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=39, rfOrder=38, authorNames=null, journalName=null, refType=null, unstructuredReference=Rennie S J, Marcheret E, Mroueh Y, et al. Self-critical sequence training for image captioning[C]//Proc of the 30th IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2017: 7008−7024, articleTitle=null, refAbstract=null), Reference(id=1329435713713582755, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=40, rfOrder=39, authorNames=null, journalName=null, refType=null, unstructuredReference=Lin C Y. ROUGE: A package for automatic evaluation of summaries[C]//Proc of the Text Summarization Branches Out. Stroudsburg, PA: ACL, 2004: 74−81, articleTitle=null, refAbstract=null), Reference(id=1329435713776497316, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=41, rfOrder=40, authorNames=null, journalName=null, refType=null, unstructuredReference=Vedantam R, Lawrence Z C, Parikh D. CIDEr: Consensus-based image description evaluation[C]//Proc of the 28th IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2015: 4566−4575, articleTitle=null, refAbstract=null), Reference(id=1329435713843606181, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=42, rfOrder=41, authorNames=null, journalName=null, refType=null, unstructuredReference=Anderson P, Fernando B, Johnson M, et al. SPICE: Semantic propositional image caption evaluation[G]//LNCS 9909: Proc of the 14th European Conf on Computer Vision. Berlin: Springer, 2016: 382−398, articleTitle=null, refAbstract=null), Reference(id=1329435713906520742, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=43, rfOrder=42, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Bing, Wang Dong, Yang Xu, et al. Show, deconfound and tell: Image captioning with causal inference[C]//Proc of the 35th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 18041−18050, articleTitle=null, refAbstract=null), Reference(id=1329435713973629607, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=44, rfOrder=43, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Xuying, Sun Xiaoshuai, Luo Yunpeng, et al. RSTNet: Captioning with adaptive attention on visual and non-visual words[C]//Proc of the 34th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2021: 15465−15474, articleTitle=null, refAbstract=null)], funds=null, companyList=[AuthorCompany(id=1329435706209972782, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, xref=1, ext=[AuthorCompanyExt(id=1329435706222555695, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, companyId=1329435706209972782, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
1School of Computer Science, Northwest University, Xi’an 710127), AuthorCompanyExt(id=1329435706230944304, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, companyId=1329435706209972782, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
1西北大学计算机学院 西安 710127)]), AuthorCompany(id=1329435706293858865, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, xref=2, ext=[AuthorCompanyExt(id=1329435706302247474, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, companyId=1329435706293858865, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
2Beijing Sankuai Technology Co., Ltd., Beijing 100190), AuthorCompanyExt(id=1329435706310636083, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, companyId=1329435706293858865, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=
2北京三快科技有限公司 北京 100190)])], figs=[ArticleFig(id=1329435708437148252, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, label=Figure 1, caption=
The solutions for key issues in multi-feature image captioning, figureFileSmall=pkkUxlE3dF1p7l/OSNolOA==, figureFileBig=NbfDvL4pNIPbfAKkLw50og==, tableContent=null), ArticleFig(id=1329435708512645725, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, label=图1, caption=
针对多特征图像描述关键问题的解决思路, figureFileSmall=pkkUxlE3dF1p7l/OSNolOA==, figureFileBig=NbfDvL4pNIPbfAKkLw50og==, tableContent=null), ArticleFig(id=1329435708693000798, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, label=Figure 2, caption=
Structure diagram of EAFCC model, figureFileSmall=istkOz3vLSkTrqrPCGbIcQ==, figureFileBig=a316XBzibxAQw6E4qqiq5w==, tableContent=null), ArticleFig(id=1329435708751721055, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, label=图2, caption=
EAFCC模型结构图, figureFileSmall=istkOz3vLSkTrqrPCGbIcQ==, figureFileBig=a316XBzibxAQw6E4qqiq5w==, tableContent=null), ArticleFig(id=1329435708823024224, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, label=Figure 3, caption=
Structure diagram of pre-fusion module network, figureFileSmall=buJtpX58LJjM697CSZpnYQ==, figureFileBig=fQfeD509NeRoY3Bu1KBb8g==, tableContent=null), ArticleFig(id=1329435708890133089, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, label=图3, caption=
预融合模块网络结构图, figureFileSmall=buJtpX58LJjM697CSZpnYQ==, figureFileBig=fQfeD509NeRoY3Bu1KBb8g==, tableContent=null), ArticleFig(id=1329435708948853346, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, label=Figure 4, caption=
Skip connection encoder module, figureFileSmall=nROAy+MKUfD2pd8Hy3Es5Q==, figureFileBig=LKxn+HiLiHXWDoZzmw3odw==, tableContent=null), ArticleFig(id=1329435709032739427, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, label=图4, caption=
跳跃连接编码器模块, figureFileSmall=nROAy+MKUfD2pd8Hy3Es5Q==, figureFileBig=LKxn+HiLiHXWDoZzmw3odw==, tableContent=null), ArticleFig(id=1329435709091459684, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, label=Figure 5, caption=
Attention maps for generating descriptions in different scenarios, figureFileSmall=9FHpoNX6kg7TsGw+yHIrvA==, figureFileBig=W+gTSKYJNcB+aeEuxHV9RA==, tableContent=null), ArticleFig(id=1329435709158568549, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, label=图5, caption=
不同场景下生成描述的注意力图, figureFileSmall=9FHpoNX6kg7TsGw+yHIrvA==, figureFileBig=W+gTSKYJNcB+aeEuxHV9RA==, tableContent=null), ArticleFig(id=1329435709225677414, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, label=Figure 6, caption=
Attention visualization comparative diagrams generated by different models, figureFileSmall=QN+DRtwIAklYeUXrmbMsYg==, figureFileBig=DpzUaQ0XhZ3jVI4hP15w2g==, tableContent=null), ArticleFig(id=1329435709322146407, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, label=图6, caption=
不同模型生成的注意力可视化对比图, figureFileSmall=QN+DRtwIAklYeUXrmbMsYg==, figureFileBig=DpzUaQ0XhZ3jVI4hP15w2g==, tableContent=null), ArticleFig(id=1329435709393449576, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, label=Figure 7, caption=
Average cosine similarity heatmap of common branch and specific branch output representations, figureFileSmall=jSaQidImjRdqsGC9DuEIZA==, figureFileBig=n+ZzWwsfUspholq7/5Fkog==, tableContent=null), ArticleFig(id=1329435709485724265, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, label=图7, caption=
共性分支与特性分支输出表示的平均余弦相似度热力图, figureFileSmall=jSaQidImjRdqsGC9DuEIZA==, figureFileBig=n+ZzWwsfUspholq7/5Fkog==, tableContent=null), ArticleFig(id=1329435709561221738, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, label=Table 1, caption=
Experimental Environment Settings
, figureFileSmall=null, figureFileBig=null, tableContent=
| 类别 | 参数项 | 参数 |
|---|
实验 环境 | PyTorch | 1.11.0 |
| CUDA | 11.3 |
| Python | 3.8 |
| GPU | RTX 3090(24 GB) |
| 输入 | 预处理尺寸 | 短边600px(保持长宽比) 再随机水平翻转、标准化 |
| 网格特征图尺寸 | 7×7=49个网格,每个网格2048维 (ResNet-101conv5_x) |
| 分割特征图尺寸 | 7×7=49个空间位置,每个位置133维 (COCO133类,UPSNet语义头) |
| 区域特征数量 | 36个RoI/图(Faster R-CNN, IoU>0.7取前36个) |
| 文本词汇量 | 10201词,最大句子长度20 |
| 优化器 | 类型 | Adam |
| 批次大小 | 50 |
), ArticleFig(id=1329435709649302123, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, label=表1, caption=
实验环境设置
, figureFileSmall=null, figureFileBig=null, tableContent=
| 类别 | 参数项 | 参数 |
|---|
实验 环境 | PyTorch | 1.11.0 |
| CUDA | 11.3 |
| Python | 3.8 |
| GPU | RTX 3090(24 GB) |
| 输入 | 预处理尺寸 | 短边600px(保持长宽比) 再随机水平翻转、标准化 |
| 网格特征图尺寸 | 7×7=49个网格,每个网格2048维 (ResNet-101conv5_x) |
| 分割特征图尺寸 | 7×7=49个空间位置,每个位置133维 (COCO133类,UPSNet语义头) |
| 区域特征数量 | 36个RoI/图(Faster R-CNN, IoU>0.7取前36个) |
| 文本词汇量 | 10201词,最大句子长度20 |
| 优化器 | 类型 | Adam |
| 批次大小 | 50 |
), ArticleFig(id=1329435709724799596, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, label=Table 2, caption=
Comparative Experimental Results of EAFCC Model on COCO2014 Dataset
, figureFileSmall=null, figureFileBig=null, tableContent=
| 模型 | B4↑ | M↑ | R↑ | C↑ | S↑ |
|---|
| SCST [39] | 34.3 | 26.7 | 55.7 | 114.0 | |
| Up-Down[25] | 36.3 | 27.7 | 56.9 | 120.1 | 21.4 |
| GCN-LSTM[29] | 38.2 | 28.5 | 58.5 | 128.3 | 22.0 |
| AoANet[10] | 38.9 | 29.2 | 58.8 | 129.8 | 22.4 |
| EAT[11] | 39.9 | 28.9 | 59.0 | 127.6 | 22.6 |
| $ {\text{M}}^{2} $Transformer[12] | 39.1 | 29.2 | 58.6 | 131.2 | 22.6 |
| X-Transformer[13] | 39.7 | 29.5 | 59.1 | 132.8 | 23.4 |
| DLCT[14] | 39.8 | 29.5 | 59.1 | 133.8 | 23.0 |
| CIIC[43] | 40.2 | 29.5 | 59.4 | 133.1 | 23.2 |
| RSTNet[44] | 39.3 | 29.4 | 58.8 | 133.3 | 23.0 |
| ConCap[36] | 40.5 | 30.9 | | 133.7 | 23.8 |
| EAFCC(本文) | 40.6 | 29.8 | 59.8 | 135.2 | 23.2 |
), ArticleFig(id=1329435709796102765, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, label=表2, caption=
EAFCC模型在COCO2014数据集上对比实验结果
, figureFileSmall=null, figureFileBig=null, tableContent=
| 模型 | B4↑ | M↑ | R↑ | C↑ | S↑ |
|---|
| SCST [39] | 34.3 | 26.7 | 55.7 | 114.0 | |
| Up-Down[25] | 36.3 | 27.7 | 56.9 | 120.1 | 21.4 |
| GCN-LSTM[29] | 38.2 | 28.5 | 58.5 | 128.3 | 22.0 |
| AoANet[10] | 38.9 | 29.2 | 58.8 | 129.8 | 22.4 |
| EAT[11] | 39.9 | 28.9 | 59.0 | 127.6 | 22.6 |
| $ {\text{M}}^{2} $Transformer[12] | 39.1 | 29.2 | 58.6 | 131.2 | 22.6 |
| X-Transformer[13] | 39.7 | 29.5 | 59.1 | 132.8 | 23.4 |
| DLCT[14] | 39.8 | 29.5 | 59.1 | 133.8 | 23.0 |
| CIIC[43] | 40.2 | 29.5 | 59.4 | 133.1 | 23.2 |
| RSTNet[44] | 39.3 | 29.4 | 58.8 | 133.3 | 23.0 |
| ConCap[36] | 40.5 | 30.9 | | 133.7 | 23.8 |
| EAFCC(本文) | 40.6 | 29.8 | 59.8 | 135.2 | 23.2 |
), ArticleFig(id=1329435709879988846, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, label=Table 3, caption=
Comparative Results of Cross-Entropy Loss Training Across Different Models in Stage 1
, figureFileSmall=null, figureFileBig=null, tableContent=
| 模型 | B1↑ | B4↑ | M↑ | R↑ | C↑ | S↑ |
|---|
| Up-Down[25] | 77.2 | 36.2 | 27.0 | 56.4 | 113.5 | 20.3 |
| GCN-LSTM[29] | 77.3 | 36.8 | 27.9 | 57.0 | 116.3 | 20.9 |
| SGAE [28] | 77.6 | 36.9 | 27.7 | 57.2 | 116.7 | 20.9 |
| AoANet[10] | 77.4 | 37.2 | 28.4 | 57.5 | 119.8 | 21.3 |
| X-Transformer[13] | 77.3 | 37.0 | 28.7 | 57.5 | 120.0 | 21.8 |
| SMALLCAP[34] | | 37.0 | 27.9 | | 119.7 | 21.3 |
| EAFCC(本文) | 77.6 | 37.5 | 28.8 | 57.6 | 120.2 | 21.8 |
), ArticleFig(id=1329435709947097711, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, label=表3, caption=
不同模型在阶段1的交叉熵损失训练对比结果
, figureFileSmall=null, figureFileBig=null, tableContent=
| 模型 | B1↑ | B4↑ | M↑ | R↑ | C↑ | S↑ |
|---|
| Up-Down[25] | 77.2 | 36.2 | 27.0 | 56.4 | 113.5 | 20.3 |
| GCN-LSTM[29] | 77.3 | 36.8 | 27.9 | 57.0 | 116.3 | 20.9 |
| SGAE [28] | 77.6 | 36.9 | 27.7 | 57.2 | 116.7 | 20.9 |
| AoANet[10] | 77.4 | 37.2 | 28.4 | 57.5 | 119.8 | 21.3 |
| X-Transformer[13] | 77.3 | 37.0 | 28.7 | 57.5 | 120.0 | 21.8 |
| SMALLCAP[34] | | 37.0 | 27.9 | | 119.7 | 21.3 |
| EAFCC(本文) | 77.6 | 37.5 | 28.8 | 57.6 | 120.2 | 21.8 |
), ArticleFig(id=1329435710093898352, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, label=Table 4, caption=
Comparative Results of Pre-Trained Vision-Language Models
, figureFileSmall=null, figureFileBig=null, tableContent=
| 模型 | 预训练图像数量 | 参数量 | B4↑ | C↑ |
|---|
| BLIPbase[17] | 14×106 | 223×106 | 38.6 | 129.7 |
| BLIPCapFilt-L[17] | 129×106 | | 39.7 | 133.3 |
| OFASys[19] | 20×106 | 270×106 | | 134.8 |
| OFATiny[18] | 20×106 | 33×106 | 38.1 | 128.7 |
| SimVLMbase[20] | 1.8×109 | 86×106 | 39.0 | 134.8 |
| UNIMO[21] | | | 39.6 | 127.7 |
| Unified VLP[22] | 3×106 | | 39.5 | 129.3 |
| EAFCC(本文) | | 31.9×106 | 40.6 | 135.2 |
), ArticleFig(id=1329435710165201521, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, label=表4, caption=
预训练视觉语言模型对比结果
, figureFileSmall=null, figureFileBig=null, tableContent=
| 模型 | 预训练图像数量 | 参数量 | B4↑ | C↑ |
|---|
| BLIPbase[17] | 14×106 | 223×106 | 38.6 | 129.7 |
| BLIPCapFilt-L[17] | 129×106 | | 39.7 | 133.3 |
| OFASys[19] | 20×106 | 270×106 | | 134.8 |
| OFATiny[18] | 20×106 | 33×106 | 38.1 | 128.7 |
| SimVLMbase[20] | 1.8×109 | 86×106 | 39.0 | 134.8 |
| UNIMO[21] | | | 39.6 | 127.7 |
| Unified VLP[22] | 3×106 | | 39.5 | 129.3 |
| EAFCC(本文) | | 31.9×106 | 40.6 | 135.2 |
), ArticleFig(id=1329435710232310386, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, label=Table 5, caption=
Ablation Experiment Results of EAFCC Model on COCO2014 Dataset
, figureFileSmall=null, figureFileBig=null, tableContent=
| 模型 | 增加模块 | B1↑ | B2↑ | B3↑ | B4↑ | M↑ | R↑ | C↑ | S↑ |
|---|
| $ {\text{M}}^{2} $Transformer[12] | | 80.8 | | | 39.1 | 29.2 | 58.6 | 131.2 | 22.6 |
| DLCT[14] | | 81.4 | | | 39.8 | 29.5 | 59.1 | 133.8 | 23.0 |
| DIFNet[16] | | 81.5 | | | 40.0 | 29.5 | 59.2 | 134.9 | 23.1 |
| EAFCC-base | | 81.6 | 66.4 | 52.2 | 40.3 | 29.6 | 59.2 | 134.0 | 23.0 |
| EAFCC-base | PF | 81.4 | 66.4 | 52.0 | 39.9 | 29.6 | 59.1 | 133.4 | 22.9 |
| EAFCC-base | FSC | 81.5 | 66.5 | 52.0 | 40.0 | 29.7 | 59.3 | 133.3 | 23.0 |
| EAFCC-base | PF+FSC | 81.8 | 66.7 | 52.2 | 40.5 | 29.7 | 59.5 | 134.3 | 23.1 |
| EAFCC-base | PF+FSC+ CL | 82.0 | 66.7 | 52.3 | 40.5 | 29.6 | 59.5 | 134.7 | 23.0 |
| EAFCC-base | PF+FSC+ MMF | 82.1 | 66.8 | 52.4 | 40.4 | 29.8 | 59.4 | 134.9 | 23.1 |
| EAFCC-base | 全部模块 | 82.4 | 67.3 | 52.6 | 40.6 | 29.8 | 59.8 | 135.2 | 23.2 |
), ArticleFig(id=1329435710312002163, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, label=表5, caption=
EAFCC模型在COCO2014数据集上的消融实验结果
, figureFileSmall=null, figureFileBig=null, tableContent=
| 模型 | 增加模块 | B1↑ | B2↑ | B3↑ | B4↑ | M↑ | R↑ | C↑ | S↑ |
|---|
| $ {\text{M}}^{2} $Transformer[12] | | 80.8 | | | 39.1 | 29.2 | 58.6 | 131.2 | 22.6 |
| DLCT[14] | | 81.4 | | | 39.8 | 29.5 | 59.1 | 133.8 | 23.0 |
| DIFNet[16] | | 81.5 | | | 40.0 | 29.5 | 59.2 | 134.9 | 23.1 |
| EAFCC-base | | 81.6 | 66.4 | 52.2 | 40.3 | 29.6 | 59.2 | 134.0 | 23.0 |
| EAFCC-base | PF | 81.4 | 66.4 | 52.0 | 39.9 | 29.6 | 59.1 | 133.4 | 22.9 |
| EAFCC-base | FSC | 81.5 | 66.5 | 52.0 | 40.0 | 29.7 | 59.3 | 133.3 | 23.0 |
| EAFCC-base | PF+FSC | 81.8 | 66.7 | 52.2 | 40.5 | 29.7 | 59.5 | 134.3 | 23.1 |
| EAFCC-base | PF+FSC+ CL | 82.0 | 66.7 | 52.3 | 40.5 | 29.6 | 59.5 | 134.7 | 23.0 |
| EAFCC-base | PF+FSC+ MMF | 82.1 | 66.8 | 52.4 | 40.4 | 29.8 | 59.4 | 134.9 | 23.1 |
| EAFCC-base | 全部模块 | 82.4 | 67.3 | 52.6 | 40.6 | 29.8 | 59.8 | 135.2 | 23.2 |
), ArticleFig(id=1329435710391693940, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, label=Table 6, caption=
Ablation Experimental Results of Iteration Numbers in PF Module of EAFCC-base
, figureFileSmall=null, figureFileBig=null, tableContent=
迭代次 数N1 | B1↑ | B2↑ | B3↑ | B4↑ | M↑ | R↑ | C↑ | S↑ |
|---|
| 1 | 81.5 | 66.5 | 52.0 | 40.2 | 29.6 | 59.2 | 133.6 | 22.9 |
| 2 | 81.6 | 66.7 | 52.3 | 40.3 | 29.6 | 59.5 | 134.0 | 23.0 |
| 3 | 81.7 | 66.4 | 51.9 | 40.0 | 29.5 | 59.3 | 134.0 | 23.0 |
| 4 | 81.4 | 66.3 | 51.8 | 39.8 | 29.5 | 59.1 | 133.9 | 22.9 |
), ArticleFig(id=1329435710475580021, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, label=表6, caption=
EAFCC-base的PF模块中迭代次数消融实验结果
, figureFileSmall=null, figureFileBig=null, tableContent=
迭代次 数N1 | B1↑ | B2↑ | B3↑ | B4↑ | M↑ | R↑ | C↑ | S↑ |
|---|
| 1 | 81.5 | 66.5 | 52.0 | 40.2 | 29.6 | 59.2 | 133.6 | 22.9 |
| 2 | 81.6 | 66.7 | 52.3 | 40.3 | 29.6 | 59.5 | 134.0 | 23.0 |
| 3 | 81.7 | 66.4 | 51.9 | 40.0 | 29.5 | 59.3 | 134.0 | 23.0 |
| 4 | 81.4 | 66.3 | 51.8 | 39.8 | 29.5 | 59.1 | 133.9 | 22.9 |
), ArticleFig(id=1329435710593020534, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, label=Table 7, caption=
Ablation Experimental Results of Skip Intervals in the EAFCC-base Skip-Connection Encoder
, figureFileSmall=null, figureFileBig=null, tableContent=
| 跳跃间隔 | B1↑ | B2↑ | B3↑ | B4↑ | M↑ | R↑ | C↑ | S↑ |
|---|
| 1 | 81.4 | 66.4 | 51.8 | 40.1 | 29.3 | 58.7 | 133.7 | 22.8 |
| 3 | 81.6 | 66.4 | 51.5 | 40.3 | 29.1 | 58.5 | 133.5 | 22.7 |
| 6 | 81.2 | 66.2 | 51.5 | 40.0 | 29.0 | 58.1 | 133.1 | 22.7 |
| 1,6 | 81.6 | 66.7 | 52.3 | 40.4 | 29.6 | 59.5 | 134.0 | 23.0 |
| 3,6 | 81.6 | 66.5 | 51.7 | 40.4 | 29.4 | 59.1 | 133.9 | 23.0 |
| 全连接 | 81.3 | 66.2 | 51.3 | 40.0 | 29.0 | 58.0 | 133.1 | 22.6 |
), ArticleFig(id=1329435710668518007, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, label=表7, caption=
EAFCC-base跳跃连接编码器中跳跃间隔消融实验结果
, figureFileSmall=null, figureFileBig=null, tableContent=
| 跳跃间隔 | B1↑ | B2↑ | B3↑ | B4↑ | M↑ | R↑ | C↑ | S↑ |
|---|
| 1 | 81.4 | 66.4 | 51.8 | 40.1 | 29.3 | 58.7 | 133.7 | 22.8 |
| 3 | 81.6 | 66.4 | 51.5 | 40.3 | 29.1 | 58.5 | 133.5 | 22.7 |
| 6 | 81.2 | 66.2 | 51.5 | 40.0 | 29.0 | 58.1 | 133.1 | 22.7 |
| 1,6 | 81.6 | 66.7 | 52.3 | 40.4 | 29.6 | 59.5 | 134.0 | 23.0 |
| 3,6 | 81.6 | 66.5 | 51.7 | 40.4 | 29.4 | 59.1 | 133.9 | 23.0 |
| 全连接 | 81.3 | 66.2 | 51.3 | 40.0 | 29.0 | 58.0 | 133.1 | 22.6 |
), ArticleFig(id=1329435710777569912, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, label=Table 8, caption=
Sensitivity Analysis of Negative Sample Number K
, figureFileSmall=null, figureFileBig=null, tableContent=
| 评价指标 | K=2 | K=3 | K=5 | K=16 | K=32 | K=50 | K=64 |
|---|
| BLEU-1 | 75.5 | 75.8 | 75.6 | 75.8 | 76.0 | 76.0 | 75.8 |
| BLEU-4 | 35.8 | 35.9 | 35.9 | 35.7 | 36.0 | 36.3 | 36.1 |
| METEOR | 27.9 | 28.0 | 27.8 | 27.9 | 27.9 | 27.9 | 27.9 |
| ROUGE | 56.3 | 56.5 | 56.2 | 56.4 | 56.4 | 56.6 | 56.3 |
| CIDEr-D | 114.9 | 115.5 | 115.3 | 115.1 | 115.5 | 116.3 | 115.4 |
), ArticleFig(id=1329435710848873081, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, label=表8, caption=
负样本数K的敏感性分析
, figureFileSmall=null, figureFileBig=null, tableContent=
| 评价指标 | K=2 | K=3 | K=5 | K=16 | K=32 | K=50 | K=64 |
|---|
| BLEU-1 | 75.5 | 75.8 | 75.6 | 75.8 | 76.0 | 76.0 | 75.8 |
| BLEU-4 | 35.8 | 35.9 | 35.9 | 35.7 | 36.0 | 36.3 | 36.1 |
| METEOR | 27.9 | 28.0 | 27.8 | 27.9 | 27.9 | 27.9 | 27.9 |
| ROUGE | 56.3 | 56.5 | 56.2 | 56.4 | 56.4 | 56.6 | 56.3 |
| CIDEr-D | 114.9 | 115.5 | 115.3 | 115.1 | 115.5 | 116.3 | 115.4 |
), ArticleFig(id=1329435710932759162, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=EN, label=Table 9, caption=
Comparison of Image Captioning Generative Results on the Test Set
, figureFileSmall=null, figureFileBig=null, tableContent=
| 图像 | 描述 |
|---|
 | GT1:A giraffe is standing in the middle of the grass. GT2:A giraffe standing alone in a forest. $ {\text{M}}^{\mathbf{2}} $Transformer:A giraffe standing in the forest. DLCT:A giraffe standing alone in the green field. DIFNet:A giraffe standing in forest. OFATiny:A giraffe is brown and white. EAFCC: A giraffe is standing in the tall green bushes. |
 | GT1:A young giraffe leaning over a tall bush in a dry field. GT2:Giraffe eating leaves off of a tree in tall bushes. $ {\text{M}}^{\mathbf{2}} $Transformer:A giraffe is eating leaves in the field. DLCT:A giraffe is eating green leaves in the field. DIFNet:A giraffe eating leaves in a field. OFATiny:A giraffe is standing in the grass. EAFCC:A young giraffe is eating green leaves in the dry bushes. |
 | GT1:Well made be in clean and neat bedroom setting. GT2:A very luxurious bedroom filled with nice furniture. $ {\text{M}}^{\mathbf{2}} $Transformer:A bedroom filled with a lot of furniture. DLCT:A grand bedroom includes a bed and a sofa. DIFNet:A bedroom filled with nice furniture. OFATiny:A bedroom with a bed or beds in a hotel room. EAFCC:A luxurious and clean bedroom filled with nice furniture such as bed and sofa. |
 | GT1:A man in a vest and plaid pants rides a bicycle. GT2:A man rides a bicycle down the street. $ {\text{M}}^{\mathbf{2}} $Transformer:A man rides a bicycle on the street. DLCT:A man with plaid and hat rides a is riding a bicycle. DIFNet: A man riding a bicycle on the street. OFATiny:A man with a hat riding a bicycle. EAFCC:A man with leather hat riding a bicycle on road with greenery in the middle. |
), ArticleFig(id=1329435711033422459, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329435702628037129, language=CN, label=表9, caption=
测试集图像描述生成结果对比
, figureFileSmall=null, figureFileBig=null, tableContent=
| 图像 | 描述 |
|---|
 | GT1:A giraffe is standing in the middle of the grass. GT2:A giraffe standing alone in a forest. $ {\text{M}}^{\mathbf{2}} $Transformer:A giraffe standing in the forest. DLCT:A giraffe standing alone in the green field. DIFNet:A giraffe standing in forest. OFATiny:A giraffe is brown and white. EAFCC: A giraffe is standing in the tall green bushes. |
 | GT1:A young giraffe leaning over a tall bush in a dry field. GT2:Giraffe eating leaves off of a tree in tall bushes. $ {\text{M}}^{\mathbf{2}} $Transformer:A giraffe is eating leaves in the field. DLCT:A giraffe is eating green leaves in the field. DIFNet:A giraffe eating leaves in a field. OFATiny:A giraffe is standing in the grass. EAFCC:A young giraffe is eating green leaves in the dry bushes. |
 | GT1:Well made be in clean and neat bedroom setting. GT2:A very luxurious bedroom filled with nice furniture. $ {\text{M}}^{\mathbf{2}} $Transformer:A bedroom filled with a lot of furniture. DLCT:A grand bedroom includes a bed and a sofa. DIFNet:A bedroom filled with nice furniture. OFATiny:A bedroom with a bed or beds in a hotel room. EAFCC:A luxurious and clean bedroom filled with nice furniture such as bed and sofa. |
 | GT1:A man in a vest and plaid pants rides a bicycle. GT2:A man rides a bicycle down the street. $ {\text{M}}^{\mathbf{2}} $Transformer:A man rides a bicycle on the street. DLCT:A man with plaid and hat rides a is riding a bicycle. DIFNet: A man riding a bicycle on the street. OFATiny:A man with a hat riding a bicycle. EAFCC:A man with leather hat riding a bicycle on road with greenery in the middle. |
)], attaches=null, journal=Journal(id=1324759606947315908, delFlag=0, nameCn=计算机研究与发展, nameEn=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, issn=1000-1239, eissn=null, cn=11-1777/TP, coden=null, periodic=0, language=CN, oaType=null, ccby=null, superviseOffice=null, ownerOffice=null, pubOffice=null, editorOffice=null, officeType=null, aims=null, clcCode=null, officeProv=null, officeCity=null, officeAddr=null, officeZip=null, officeEmail=null, officePhone=null, editDirector=null, officeDirector=null, officeDirectorPhone=null, officeStaffNum=null, officeEmpNum=null, coverPicUrl=k2l8zgqwstuxxevFX2qxVA==, journalPrice=null, startedYear=null, abbrevIsoEn=Journal of Computer Research and Development, journalRemark=null, publicationField=null, createdTime=1793776903141, updatedTime=1793850698612, createdBy=18614031015, updatedBy=13701087609, firstLetterCn=J, firstLetterEn=J, subjectCode=Electronic Information, subjectName=null, subjectCodeEn=Electronic Information, subjectNameEn=null, picCn=k2l8zgqwstuxxevFX2qxVA==, picEn=r6k0Ry+RD+c/VXj5QarmDw==, jcr=null, cjcr=null, exts=[JournalExt(id=1325069127710286688, language=CN, name=计算机研究与发展, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698640, updatedTime=1793850698640, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""}), JournalExt(id=1325069127777395553, language=EN, name=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698656, updatedTime=1793850698656, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""})], databaseList=null, tenantJournalId=1324759914146529478, websiteList=[Website(id=1325070300139892771, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN, language=CN, createTime=1793850978169, createBy=18614031015, updateTime=1793851040259, updateBy=18614031015, name=计算机研究与发展-中文, tplId=1146099689490845704, title=计算机研究与发展, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070893751349360, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=articleTextType, value=kx, createTime=1793851119697, updateTime=1793851119697, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893730377837, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=banner, value=null, createTime=1793851119692, updateTime=1793851119692, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893772320883, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=grayFlag, value=0, createTime=1793851119702, updateTime=1793851119702, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893721989228, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851119690, updateTime=1793851119690, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893784903797, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=minRunFlag, value=0, createTime=1793851119705, updateTime=1793851119705, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893742960751, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic, createTime=1793851119695, updateTime=1793851119695, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893780709492, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=silenceFlag, value=0, createTime=1793851119704, updateTime=1793851119704, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893738766446, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_cn_619/, createTime=1793851119694, updateTime=1793851119694, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893755543665, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeColor, value=null, createTime=1793851119698, updateTime=1793851119698, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893763932274, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeStyle, value=null, createTime=1793851119700, updateTime=1793851119700, creator=18614031015, updator=18614031015)]), Website(id=1325070300580294698, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN, language=EN, createTime=1793850978274, createBy=18614031015, updateTime=1793851036638, updateBy=18614031015, name=计算机研究与发展-英文, tplId=1146101810881728533, title=Journal of Computer Research and Development, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070925288321148, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=articleTextType, value=kx, createTime=1793851127216, updateTime=1793851127216, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925263155321, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=banner, value=null, createTime=1793851127210, updateTime=1793851127210, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925309292671, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=grayFlag, value=0, createTime=1793851127221, updateTime=1793851127221, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925254766712, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851127208, updateTime=1793851127208, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925326069889, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=minRunFlag, value=0, createTime=1793851127225, updateTime=1793851127225, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925279932539, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic, createTime=1793851127214, updateTime=1793851127214, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925317681280, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=silenceFlag, value=0, createTime=1793851127223, updateTime=1793851127223, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925271543930, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_en_623/, createTime=1793851127212, updateTime=1793851127212, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925292515453, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeColor, value=null, createTime=1793851127217, updateTime=1793851127217, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925300904062, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeStyle, value=null, createTime=1793851127219, updateTime=1793851127219, creator=18614031015, updator=18614031015)])], journalTitle=计算机研究与发展, weixinUrl=null, journalUrl=https://crad.ict.ac.cn/, iacademicId=null, status=1, seqNo=null, journalTitleEn=Journal of Computer Research and Development, journalPhotoCn=k2l8zgqwstuxxevFX2qxVA==, journalPhotoEn=r6k0Ry+RD+c/VXj5QarmDw==, journalFirstLetter=J, journalRecommend=null, journalNew=null, journalCollection=null, jcrJf=null, cjcrJf=null, jcrJfStr=null, cjcrJfStr=null, submissionFirstDecision=null, sciSubjectClassification=null, casSubjectClassification=null, citeScore=null, totalCitationFrequency=null, icpCode=null, psCode=null, advertisingLicenseCode=null, copyrightInformation=null, country=null, option=, provinceCode=null, provinceName=null, collectFlag=false, interPubPlatform=, interPubPlatformUrl=null), detailUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202550936, detailUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202550936, pdfUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/PDF/10.7544/issn1000-1239.202550936, pdfUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/PDF/10.7544/issn1000-1239.202550936, aliStartDate=0, aliEndDate=0, collectionFlag=false, citedCount=null, citedUrl=null, previewStatus=0, delFlag=0, hasFullText=1, orderTime=1793462400000, fullTextJson=null, articleText=null, reference=null)