Article(id=1327012629421339208, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1327012311115608215, articleNumber=null, orderNo=null, doi=10.7544/issn1000-1239.202440123, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=research-article, receivedDate=1709222400000, receivedDateStr=2024-03-01, revisedDate=1730304000000, revisedDateStr=2024-10-31, acceptedDate=1731427200000, acceptedDateStr=2024-11-13, onlineDate=1794314065555, onlineDateStr=2026-11-10, pubDate=1738339200000, pubDateStr=2025-02-01, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1794314065555, onlineIssueDateStr=2026-11-10, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1794314065555, creator=13701087609, updateTime=1794314065555, updator=13701087609, issue=Issue{id=1327012311115608215, tenantId=1146029695717560320, journalId=1324759914146529478, year='2025', volume='62', issue='2', pageStart='285', pageEnd='541', issueExtLink='null', onlineDate='null', pubDate='1738339200000', pubDateStr='2025-02-01', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=1, issueType=-1, specialIssue=null, createTime=1794313989665, creator='13701087609', updateTime=1794376696965, updator='13701087609', preIssue=null, nextIssue=null, articleTotal=null, ext={EN=IssueExt(id=1327275325060395748, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1327012311115608215, language=EN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=), CN=IssueExt(id=1327275325060395749, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1327012311115608215, language=CN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=)}, issueFiles=null, downloadFileDto=null}, startPage=432, endPage=442, ext={EN=ArticleExt(id=1327012631250055753, articleId=1327012629421339208, tenantId=1146029695717560320, journalId=1324759914146529478, language=EN, title=Semi-supervised Open Vocabulary Multi-label Learning Based on Graph Prompting, columnId=1325489891789910943, journalTitle=Journal of Computer Research and Development, columnName=Artificial Intelligence, runingTitle=null, highlight=null, articleAbstract=

Semi-supervised multi-label learning employs labeled and unlabeled data to train a model, which effectively achieves good results and reduces the labeling cost of multi-label data. Therefore, semi-supervised multi-label learning has attracted many researchers dedicated to this field. However, in the semi-supervised annotation process, due to the large number of labels, it is a common situation that some labels lack of samples, and these labels are called open vocabulary. It is difficult for the model to learn the label information of the open vocabulary, which leads to the degradation of its performance. To address the above problem, we propose a semi-supervised open vocabulary multi-label learning method based on graph prompting. Specifically, this method uses a graph neural network via prompt to fine-tune the pre-trained model and explore the relationship between open vocabulary and supervised samples. By using images and text, we construct a graph neural network as the input of text for the pre-trained model. Furthermore, by leveraging the generalization ability of the pre-trained model on open vocabulary, pseudo-labels are generated for unsupervised samples. Then we use pseudo-labels to train the classification layer and to enable the model to achieve better performance in classifying open vocabulary. Experimental results on multiple benchmark datasets, including VOC, COCO, CUB, and NUS, consistently demonstrate that the proposed method outperforms existing methods and achieves state-of-the-art performance.

, authors=Zhongnian Li1, 3, Zhiyu Huangfu1, Kaijie Yang1, Peng Ying1, Tongfeng Sun1, 3, Xinzheng Xu1, 2, 3, *, authorsList=Zhongnian Li, Zhiyu Huangfu, Kaijie Yang, Peng Ying, Tongfeng Sun, Xinzheng Xu, authorCompany=null, correspAuthors=Xinzheng Xu, authorNote=

born in 1990. PhD, lecturer. Member of CCF. His main research interests include data mining, machine learning, and information system

born in 2000. Master. His main research interests include artificial intelligence and machine learning

born in 2001. Master. His main research interests include artificial intelligence and machine learning

born in 2001. PhD. Student member of CCF. His main research interests include artificial intelligence, data mining, and machine learning

born in 1977. PhD, associate professor. Member of CCF. His main research interests include artificial intelligence, machine learning, and data mining

born in 1980. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include artificial intelligence, machine learning, and medical image processing

, correspAuthorsNote=null, copyrightStatement=Copyright © 2025 Journal of Computer Research and Development. All rights reserved., copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=null, magXml=null, pdfUrl=null, pdf=null, pdfFileSize=null, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=null, mapNumber=null, fund=null), CN=ArticleExt(id=1327012633494008419, articleId=1327012629421339208, tenantId=1146029695717560320, journalId=1324759914146529478, language=CN, title=基于图提示的半监督开放词汇多标记学习, columnId=1325489892104483745, journalTitle=计算机研究与发展, columnName=人工智能, runingTitle=null, highlight=null, articleAbstract=

半监督多标记学习利用有标记数据和无标记数据进行模型的训练,降低了多标记数据的标记成本并取得了不错的结果,吸引了很多研究者不断进行研究. 然而,在半监督标注过程中,由于标记的数量较多,往往会出现某些标记缺失标注样本的情况,这些标记被称为开放词汇. 开放词汇会导致模型无法学习到该类别的标记信息,使得模型性能下降. 针对上述问题,提出了基于图提示的半监督开放词汇多标记学习方法. 具体地,该方法利用基于提示的图神经网络对预训练大模型进行微调,挖掘和探索开放词汇与监督样本之间的关系. 通过使用包含图像与文本的多模态数据构造图神经网络作为预训练大模型的文本输入进行学习. 其次利用预训练大模型在开放词汇上的泛化能力,对无监督样本生成伪标记,实现对输出分类层的微调,使模型在对开放词汇进行分类时能获得更加理想的效果. 多个基准数据集上的实验结果均显示,基于图提示的半监督开放词汇多标记学习方法优于目前的主流方法,在VOC,COCO,CUB,NUS等基准数据集上均取得了最优的效果.

, authors=李仲年1, 3, 皇甫志宇1, 杨凯杰1, 营鹏1, 孙统风1, 3, 许新征1, 2, 3, *, authorsList=李仲年, 皇甫志宇, 杨凯杰, 营鹏, 孙统风, 许新征, authorCompany=null, correspAuthors=许新征, authorNote=

1990年生. 博士,讲师. CCF会员. 主要研究方向为数据挖掘、机器学习、信息系统

2000年生. 硕士. 主要研究方向为人工智能、机器学习

2001年生. 硕士. 主要研究方向为人工智能、机器学习

2001年生. 博士. CCF学生会员. 主要研究方向为人工智能、数据挖掘、机器学习

1977年生. 博士,副教授. CCF会员. 主要研究方向为人工智能、机器学习、数据挖掘

1980年生. 博士,教授,博士生导师. CCF高级会员. 主要研究方向为人工智能、机器学习、医学图像处理

, correspAuthorsNote=
许新征()
, copyrightStatement=版权所有 © 2025 计算机研究与发展编辑部, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=cTAjtK/5nEj74jr6earRJA==, magXml=nNHJOPDqL8YWVc3d+X+PxA==, pdfUrl=null, pdf=8h65S9f0aYr6dRcfqA7Xtg==, pdfFileSize=4265513, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=7MrweChiBGJZ0UeM75Zdug==, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=ZobU0kfizofudWLSZVnZeg==, mapNumber=null, fund=null)}, authors=[Author(id=1327012635612131951, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=zhongnianli@cumt.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327012635716989555, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012635612131951, language=EN, stringName=Zhongnian Li, firstName=Zhongnian, middleName=null, lastName=Li, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 3, address=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116
3Mine Digitization Engineering Research Center (China University of Mining and Technology), Ministry of Education, Xuzhou, Jiangsu 221116, bio={"content":"

born in 1990. PhD, lecturer. Member of CCF. His main research interests include data mining, machine learning, and information system

"}, bioImg=null, bioContent=

born in 1990. PhD, lecturer. Member of CCF. His main research interests include data mining, machine learning, and information system

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327012635826041460, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012635612131951, language=CN, stringName=李仲年, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 3, address=1中国矿业大学计算机科学与技术学院 江苏徐州 221116
3矿山数字化教育部工程研究中心 (中国矿业大学) 江苏徐州 221116, bio={"img":"nB581yCVqKd+6mLXtztdEg==","content":"

1990年生. 博士,讲师. CCF会员. 主要研究方向为数据挖掘、机器学习、信息系统

"}, bioImg=nB581yCVqKd+6mLXtztdEg==, bioContent=

1990年生. 博士,讲师. CCF会员. 主要研究方向为数据挖掘、机器学习、信息系统

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327012633720500836, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=1, ext=[AuthorCompanyExt(id=1327012633728889445, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116), AuthorCompanyExt(id=1327012633737278054, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1中国矿业大学计算机科学与技术学院 江苏徐州 221116)]), AuthorCompany(id=1327012633909244522, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=3, ext=[AuthorCompanyExt(id=1327012633917633131, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633909244522, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3Mine Digitization Engineering Research Center (China University of Mining and Technology), Ministry of Education, Xuzhou, Jiangsu 221116), AuthorCompanyExt(id=1327012633926021740, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633909244522, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3矿山数字化教育部工程研究中心 (中国矿业大学) 江苏徐州 221116)])]), Author(id=1327012635914121846, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327012636098671224, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012635914121846, language=EN, stringName=Zhiyu Huangfu, firstName=Zhiyu, middleName=null, lastName=Huangfu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116, bio={"content":"

born in 2000. Master. His main research interests include artificial intelligence and machine learning

"}, bioImg=null, bioContent=

born in 2000. Master. His main research interests include artificial intelligence and machine learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327012636203528825, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012635914121846, language=CN, stringName=皇甫志宇, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1中国矿业大学计算机科学与技术学院 江苏徐州 221116, bio={"img":"JccGJzRpnRZm/3SaoY7XiQ==","content":"

2000年生. 硕士. 主要研究方向为人工智能、机器学习

"}, bioImg=JccGJzRpnRZm/3SaoY7XiQ==, bioContent=

2000年生. 硕士. 主要研究方向为人工智能、机器学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327012633720500836, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=1, ext=[AuthorCompanyExt(id=1327012633728889445, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116), AuthorCompanyExt(id=1327012633737278054, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1中国矿业大学计算机科学与技术学院 江苏徐州 221116)])]), Author(id=1327012636279026299, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327012636354523773, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012636279026299, language=EN, stringName=Kaijie Yang, firstName=Kaijie, middleName=null, lastName=Yang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116, bio={"content":"

born in 2001. Master. His main research interests include artificial intelligence and machine learning

"}, bioImg=null, bioContent=

born in 2001. Master. His main research interests include artificial intelligence and machine learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327012636450992766, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012636279026299, language=CN, stringName=杨凯杰, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1中国矿业大学计算机科学与技术学院 江苏徐州 221116, bio={"img":"cTwGhFAkXWR2wpZNSbDFgg==","content":"

2001年生. 硕士. 主要研究方向为人工智能、机器学习

"}, bioImg=cTwGhFAkXWR2wpZNSbDFgg==, bioContent=

2001年生. 硕士. 主要研究方向为人工智能、机器学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327012633720500836, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=1, ext=[AuthorCompanyExt(id=1327012633728889445, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116), AuthorCompanyExt(id=1327012633737278054, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1中国矿业大学计算机科学与技术学院 江苏徐州 221116)])]), Author(id=1327012636526490240, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327012636601987714, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012636526490240, language=EN, stringName=Peng Ying, firstName=Peng, middleName=null, lastName=Ying, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116, bio={"content":"

born in 2001. PhD. Student member of CCF. His main research interests include artificial intelligence, data mining, and machine learning

"}, bioImg=null, bioContent=

born in 2001. PhD. Student member of CCF. His main research interests include artificial intelligence, data mining, and machine learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327012636685873795, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012636526490240, language=CN, stringName=营鹏, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1中国矿业大学计算机科学与技术学院 江苏徐州 221116, bio={"img":"21IZRteQOG27AX9NteABmQ==","content":"

2001年生. 博士. CCF学生会员. 主要研究方向为人工智能、数据挖掘、机器学习

"}, bioImg=21IZRteQOG27AX9NteABmQ==, bioContent=

2001年生. 博士. CCF学生会员. 主要研究方向为人工智能、数据挖掘、机器学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327012633720500836, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=1, ext=[AuthorCompanyExt(id=1327012633728889445, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116), AuthorCompanyExt(id=1327012633737278054, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1中国矿业大学计算机科学与技术学院 江苏徐州 221116)])]), Author(id=1327012636786537093, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327012636929143432, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012636786537093, language=EN, stringName=Tongfeng Sun, firstName=Tongfeng, middleName=null, lastName=Sun, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 3, address=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116
3Mine Digitization Engineering Research Center (China University of Mining and Technology), Ministry of Education, Xuzhou, Jiangsu 221116, bio={"content":"

born in 1977. PhD, associate professor. Member of CCF. His main research interests include artificial intelligence, machine learning, and data mining

"}, bioImg=null, bioContent=

born in 1977. PhD, associate professor. Member of CCF. His main research interests include artificial intelligence, machine learning, and data mining

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327012637000446601, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012636786537093, language=CN, stringName=孙统风, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 3, address=1中国矿业大学计算机科学与技术学院 江苏徐州 221116
3矿山数字化教育部工程研究中心 (中国矿业大学) 江苏徐州 221116, bio={"img":"EoArBWxvmDLbNKS2IYh1TQ==","content":"

1977年生. 博士,副教授. CCF会员. 主要研究方向为人工智能、机器学习、数据挖掘

"}, bioImg=EoArBWxvmDLbNKS2IYh1TQ==, bioContent=

1977年生. 博士,副教授. CCF会员. 主要研究方向为人工智能、机器学习、数据挖掘

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327012633720500836, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=1, ext=[AuthorCompanyExt(id=1327012633728889445, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116), AuthorCompanyExt(id=1327012633737278054, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1中国矿业大学计算机科学与技术学院 江苏徐州 221116)]), AuthorCompany(id=1327012633909244522, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=3, ext=[AuthorCompanyExt(id=1327012633917633131, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633909244522, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3Mine Digitization Engineering Research Center (China University of Mining and Technology), Ministry of Education, Xuzhou, Jiangsu 221116), AuthorCompanyExt(id=1327012633926021740, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633909244522, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3矿山数字化教育部工程研究中心 (中国矿业大学) 江苏徐州 221116)])]), Author(id=1327012637059166859, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, orderNo=5, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=xxzheng@cumt.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1327012637176607375, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012637059166859, language=EN, stringName=Xinzheng Xu, firstName=Xinzheng, middleName=null, lastName=Xu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, 3, *, address=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116
2State Key Laboratory of CAD & CG (Zhejiang University), Hangzhou 310058
3Mine Digitization Engineering Research Center (China University of Mining and Technology), Ministry of Education, Xuzhou, Jiangsu 221116, bio={"content":"

born in 1980. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include artificial intelligence, machine learning, and medical image processing

"}, bioImg=null, bioContent=

born in 1980. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include artificial intelligence, machine learning, and medical image processing

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327012637256299152, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012637059166859, language=CN, stringName=许新征, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, 3, *, address=1中国矿业大学计算机科学与技术学院 江苏徐州 221116
2计算机辅助设计与图形系统全国重点实验室 (浙江大学) 杭州 310058
3矿山数字化教育部工程研究中心 (中国矿业大学) 江苏徐州 221116, bio={"img":"9zuDj17ns1PX4bZIYOzD1A==","content":"

1980年生. 博士,教授,博士生导师. CCF高级会员. 主要研究方向为人工智能、机器学习、医学图像处理

"}, bioImg=9zuDj17ns1PX4bZIYOzD1A==, bioContent=

1980年生. 博士,教授,博士生导师. CCF高级会员. 主要研究方向为人工智能、机器学习、医学图像处理

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327012633720500836, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=1, ext=[AuthorCompanyExt(id=1327012633728889445, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116), AuthorCompanyExt(id=1327012633737278054, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1中国矿业大学计算机科学与技术学院 江苏徐州 221116)]), AuthorCompany(id=1327012633804386919, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=2, ext=[AuthorCompanyExt(id=1327012633816969832, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633804386919, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2State Key Laboratory of CAD & CG (Zhejiang University), Hangzhou 310058), AuthorCompanyExt(id=1327012633829552745, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633804386919, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2计算机辅助设计与图形系统全国重点实验室 (浙江大学) 杭州 310058)]), AuthorCompany(id=1327012633909244522, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=3, ext=[AuthorCompanyExt(id=1327012633917633131, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633909244522, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3Mine Digitization Engineering Research Center (China University of Mining and Technology), Ministry of Education, Xuzhou, Jiangsu 221116), AuthorCompanyExt(id=1327012633926021740, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633909244522, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3矿山数字化教育部工程研究中心 (中国矿业大学) 江苏徐州 221116)])])], keywords=[Keyword(id=1327012637365351057, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=EN, orderNo=1, keyword=semi-supervised multi-label learning), Keyword(id=1327012637419877010, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=EN, orderNo=2, keyword=pre-trained model), Keyword(id=1327012637482791571, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=EN, orderNo=3, keyword=graph neural network), Keyword(id=1327012637545706132, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=EN, orderNo=4, keyword=open vocabulary), Keyword(id=1327012637604426389, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=EN, orderNo=5, keyword=prompt), Keyword(id=1327012637675729558, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=CN, orderNo=1, keyword=半监督多标记学习), Keyword(id=1327012637747032727, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=CN, orderNo=2, keyword=预训练模型), Keyword(id=1327012637822530200, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=CN, orderNo=3, keyword=图神经网络), Keyword(id=1327012637948359321, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=CN, orderNo=4, keyword=开放词汇), Keyword(id=1327012638011273882, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=CN, orderNo=5, keyword=提示)], refs=[Reference(id=1327012641836479155, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=1, rfOrder=0, authorNames=null, journalName=null, refType=null, unstructuredReference=Tian Yingjie, Yu Xiaotong, Fu Saiji. Partial label learning: Taxonomy, analysis and outlook[J]. Neural networks, 2023, 161(1): 708−734, articleTitle=null, refAbstract=null), Reference(id=1327012641911976628, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=2, rfOrder=1, authorNames=null, journalName=null, refType=null, unstructuredReference=Rastogi R, Kumar S. Discriminatory label-specific weights for multi-label learning with missing labels[J]. Neural Processing Letters, 2023, 55(2): 1397−1431, articleTitle=null, refAbstract=null), Reference(id=1327012641962308277, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=3, rfOrder=2, authorNames=null, journalName=null, refType=null, unstructuredReference=李宇峰,黄圣君,周志华. 一种基于正则化的半监督多标记学习方法[J]. 计算机研究与发展,2012,49(6):1272−1278, articleTitle=null, refAbstract=null), Reference(id=1327012642033611446, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=3, rfOrder=3, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Yufeng, Huang Shengjun, Zhou Zhihua. Regularized semi-supervised multi-label learning[J]. Journal of Computer Research and Development, 2012, 49(6): 1272−1278 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327012642121691831, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=4, rfOrder=4, authorNames=null, journalName=null, refType=null, unstructuredReference=Lai Zhengfeng, Wang Chao, Cheung S C , et al. SAR: Self-adaptive refinement on pseudo labels for multiclass-imbalanced semi-supervised learning[C]//Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2022: 4091−4100, articleTitle=null, refAbstract=null), Reference(id=1327012642180412088, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=5, rfOrder=5, authorNames=null, journalName=null, refType=null, unstructuredReference=张永,陈蓉蓉,张晶. 基于交叉熵的安全Tri-training算法[J]. 计算机研究与发展,2021,58(1):60−69, articleTitle=null, refAbstract=null), Reference(id=1327012642323018425, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=5, rfOrder=6, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Yong, Chen Rongrong, Zhang Jing. Safe tri-training algorithm based on cross entropy[J]. Journal of Computer Research and Development, 2021, 58(1): 60−69 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327012642478207674, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=6, rfOrder=7, authorNames=null, journalName=null, refType=null, unstructuredReference=Fan Yue, Kukleva A, Dai Dengxin, et al. Revisiting consistency regularization for semi-supervised learning[J]. International Journal of Computer Vision, 2023, 131(3): 626−643, articleTitle=null, refAbstract=null), Reference(id=1327012642557899451, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=7, rfOrder=8, authorNames=null, journalName=null, refType=null, unstructuredReference=Qiu Ye, Gong Xiaolong, Ma Zhiyi, et al. MixLab: An informative semi-supervised method for multi-label classification[C]//Proc of the 9th CCF Int Conf on Natural Language Processing and Chinese Computing. Berlin: Springer, 2020: 506−518, articleTitle=null, refAbstract=null), Reference(id=1327012642633396924, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=8, rfOrder=9, authorNames=null, journalName=null, refType=null, unstructuredReference=Huang Teng, Jia Binbin, Zhang Minling. Progressive label propagation for semi-supervised multi-dimensional classification[C]//Proc of the 32nd Int Joint Conf on Artificial Intelligence. San Francisco, CA: Margan Kaufmann, 2023: 3821−3829, articleTitle=null, refAbstract=null), Reference(id=1327012642704700093, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=9, rfOrder=10, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Xuan, Shi Tianyi, Zhou Guohui, et al. Emotion classification for short texts: An improved multi-label method[J]. Humanities and Social Sciences Communications, 2023, 10(1): 1−9, articleTitle=null, refAbstract=null), Reference(id=1327012642788586174, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=10, rfOrder=11, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Shuo, Zhang Jiaojiao, Tian Biao, et al. Multi-modal contrastive mutual learning and pseudo-label re-learning for semi-supervised medical image segmentation[J]. Medical Image Analysis, 2022, 83(1): 102656−102656, articleTitle=null, refAbstract=null), Reference(id=1327012642859889343, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=11, rfOrder=12, authorNames=null, journalName=null, refType=null, unstructuredReference=Song Kun, Ma Huimin, Zou Bochao, et al. FD-Align: Feature discrimination alignment for fine-tuning pre-trained models in few-shot learning[C]//Proc of the 37th Conf on Neural Information Processing Systems. New York: Curran Associates, 2023: 43579−43592, articleTitle=null, refAbstract=null), Reference(id=1327012644520833728, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=12, rfOrder=13, authorNames=null, journalName=null, refType=null, unstructuredReference=Radford A, Kim J W, Hallacy C, et al. Learning transferable visual models from natural language supervision[C]//Proc of the 38th Int Conf on Machine Learning. New York: ACM, 2021: 8748−8763, articleTitle=null, refAbstract=null), Reference(id=1327012644621497026, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=13, rfOrder=14, authorNames=null, journalName=null, refType=null, unstructuredReference=Mo S, Kim M, Lee K, et al. S-CLIP: Semi-supervised vision-language learning using few specialist captions[C]//Proc of the 37th Conf on Neural Information Processing Systems. New York: Curran Associates, 2023: 61187−61212, articleTitle=null, refAbstract=null), Reference(id=1327012644709577411, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=14, rfOrder=15, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Ao, Chen Hui, Lin Zijia, et al. Hierarchical prompt learning using CLIP for multi-label classification with single positive labels[C]//Proc of the 31st ACM Int Conf on Multimedia. New York: ACM, 2023: 5594−5604, articleTitle=null, refAbstract=null), Reference(id=1327012644801852100, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=15, rfOrder=16, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Youcai, Huang Xinyu, Ma Jinyu, et al. Recognize anything: A strong image tagging model[J]. arXiv preprint, arXiv: 2306.03514, 2023, articleTitle=null, refAbstract=null), Reference(id=1327012644915098309, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=16, rfOrder=17, authorNames=null, journalName=null, refType=null, unstructuredReference=Read J, Pfahringer B, Holmes G, et al. Classifier chains for multi-label classification[J]. Machine Learning, 2011, 85(3): 333−359, articleTitle=null, refAbstract=null), Reference(id=1327012644982207174, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=17, rfOrder=18, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Zhunga, Pan Quan, Dezert J, et al. Classifier fusion with contextual reliability evaluation[J]. IEEE Transactions on Cybernetics, 2017, 48(5): 1605−1618, articleTitle=null, refAbstract=null), Reference(id=1327012645066093255, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=18, rfOrder=19, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhao Xiangyun, Vemulapalli R, Mansfield P A, et al. Contrastive learning for label efficient semantic segmentation[C]//Proc of the 18th IEEE/CVF Int Conf on Computer Vision. Los Alamitos, CA: IEEE Computer Society, 2021: 10623−10633, articleTitle=null, refAbstract=null), Reference(id=1327012645129007816, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=19, rfOrder=20, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Lichen, Liu Yunyu, Qin Can, et al. Dual relation semi-supervised multi-label learning[C]//Proc of the 34th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2020: 6227−6234, articleTitle=null, refAbstract=null), Reference(id=1327012645200310985, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=20, rfOrder=21, authorNames=null, journalName=null, refType=null, unstructuredReference=Xie Mingkun, Huang Shengjun. Partial multi-label learning with noisy label identification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 44(7): 3676−3687, articleTitle=null, refAbstract=null), Reference(id=1327012645267419850, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=21, rfOrder=22, authorNames=null, journalName=null, refType=null, unstructuredReference=Duan Yue, Zhao Zhen, Qi Lei, et al. Towards semi-supervised learning with non-random missing labels[C]//Proc of the 19th IEEE/CVF Int Conf on Computer Vision. Los Alamitos, CA: IEEE Computer Society, 2023: 16121−16131, articleTitle=null, refAbstract=null), Reference(id=1327012645338723019, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=22, rfOrder=23, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhou Zhihua. Open-environment machine learning[J]. National Science Review, 2022, 9(8): nwac123, articleTitle=null, refAbstract=null), Reference(id=1327012645405831884, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=23, rfOrder=24, authorNames=null, journalName=null, refType=null, unstructuredReference=Eichstaedt J C, Kern M L, Yaden D B, et al. Closed-and open-vocabulary approaches to text analysis: A review, quantitative comparison, and recommendations[J]. Psychological Methods, 2021, 26(4): 398−398, articleTitle=null, refAbstract=null), Reference(id=1327012645481329357, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=24, rfOrder=25, authorNames=null, journalName=null, refType=null, unstructuredReference=Gu Xinyue, Lin T Y, Kuo Weicheng, et al. Open-vocabulary object detection via vision and language knowledge distillation[J]. arXiv preprint, arXiv: 2104.13921, 2021, articleTitle=null, refAbstract=null), Reference(id=1327012645544243918, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=25, rfOrder=26, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Hao, Li Feng, Zou Xueyan, et al. A simple framework for open-vocabulary segmentation and detection[C]//Proc of the 19th IEEE/CVF Int Conf on Computer Vision. Los Alamitos, CA: IEEE Computer Society, 2023: 1020−1031, articleTitle=null, refAbstract=null), Reference(id=1327012645615547087, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=26, rfOrder=27, authorNames=null, journalName=null, refType=null, unstructuredReference=He Sunan, Guo Taian, Dai Tao, et al. Open-vocabulary multi-label classification via multi-modal knowledge transfer[C]//Proc of the 37th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2023: 808−816, articleTitle=null, refAbstract=null), Reference(id=1327012645703627472, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=27, rfOrder=28, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Junnan, Li Dongxu, Xiong Caiming, et al. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation[C]//Proc of the 39th Int Conf on Machine Learning. New York: ACM, 2022: 12888−12900, articleTitle=null, refAbstract=null), Reference(id=1327012645795902161, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=28, rfOrder=29, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhou Kaiyang, Yang Jingkang, Loy C C, et al. Learning to prompt for vision-language models[J]. International Journal of Computer Vision, 2022, 130(9): 2337−2348, articleTitle=null, refAbstract=null), Reference(id=1327012645896565458, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=29, rfOrder=30, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhou Kaiyang, Yang Jingkang, Loy C C, et al. Conditional prompt learning for vision-language models[C]//Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2022: 16816−16825, articleTitle=null, refAbstract=null), Reference(id=1327012645963674323, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=30, rfOrder=31, authorNames=null, journalName=null, refType=null, unstructuredReference=Kingma D P, Ba J. Adam: A method for stochastic optimization[J]. arXiv preprint, arXiv: 1412.6980, 2014, articleTitle=null, refAbstract=null), Reference(id=1327012646026588884, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=31, rfOrder=32, authorNames=null, journalName=null, refType=null, unstructuredReference=Ridnik T, Ben-Baruch E, Zamir N, et al. Asymmetric loss for multi-label classification[C]//Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2021: 82−91, articleTitle=null, refAbstract=null), Reference(id=1327012646097892053, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=32, rfOrder=33, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhu Ke, Wu Jianxin. Residual attention: A simple but effective method for multi-label recognition[C]//Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2021: 184−193, articleTitle=null, refAbstract=null), Reference(id=1327012646173389526, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=33, rfOrder=34, authorNames=null, journalName=null, refType=null, unstructuredReference=Lanchantin J, Wang Tianlu, Ordonez V, et al. General multi-label image classification with transformers[C]//Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2021: 16478−16488, articleTitle=null, refAbstract=null)], funds=null, companyList=[AuthorCompany(id=1327012633720500836, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=1, ext=[AuthorCompanyExt(id=1327012633728889445, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116), AuthorCompanyExt(id=1327012633737278054, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1中国矿业大学计算机科学与技术学院 江苏徐州 221116)]), AuthorCompany(id=1327012633804386919, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=2, ext=[AuthorCompanyExt(id=1327012633816969832, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633804386919, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2State Key Laboratory of CAD & CG (Zhejiang University), Hangzhou 310058), AuthorCompanyExt(id=1327012633829552745, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633804386919, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2计算机辅助设计与图形系统全国重点实验室 (浙江大学) 杭州 310058)]), AuthorCompany(id=1327012633909244522, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=3, ext=[AuthorCompanyExt(id=1327012633917633131, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633909244522, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3Mine Digitization Engineering Research Center (China University of Mining and Technology), Ministry of Education, Xuzhou, Jiangsu 221116), AuthorCompanyExt(id=1327012633926021740, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633909244522, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3矿山数字化教育部工程研究中心 (中国矿业大学) 江苏徐州 221116)])], figs=[ArticleFig(id=1327012638166463131, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=EN, label=Figure 1, caption=Open-vocabulary multi-label learning, figureFileSmall=T2cUmuQau2gAFpITceq7fw==, figureFileBig=7MrweChiBGJZ0UeM75Zdug==, tableContent=null), ArticleFig(id=1327012638241960604, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=CN, label=图1, caption=开放词汇多标记学习, figureFileSmall=T2cUmuQau2gAFpITceq7fw==, figureFileBig=7MrweChiBGJZ0UeM75Zdug==, tableContent=null), ArticleFig(id=1327012640091648670, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=EN, label=Figure 2, caption=Difference between open vocabulary semi-supervised and traditional semi-supervised, figureFileSmall=BnpbIHmB6SYKyJWYNRONmA==, figureFileBig=gWuAj/1ivBhDG9/XsqmcMg==, tableContent=null), ArticleFig(id=1327012640158757535, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=CN, label=图2, caption=开放词汇半监督与传统半监督的区别, figureFileSmall=BnpbIHmB6SYKyJWYNRONmA==, figureFileBig=gWuAj/1ivBhDG9/XsqmcMg==, tableContent=null), ArticleFig(id=1327012640225866400, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=EN, label=Figure 3, caption=Overall framework of the proposed model, figureFileSmall=iPRc6o+Jnm0yyw1EHSHqqQ==, figureFileBig=XXPLCzkDErD1dohsdgi/jA==, tableContent=null), ArticleFig(id=1327012640330724001, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=CN, label=图3, caption=本文提出的模型总体框架, figureFileSmall=iPRc6o+Jnm0yyw1EHSHqqQ==, figureFileBig=XXPLCzkDErD1dohsdgi/jA==, tableContent=null), ArticleFig(id=1327012640473330338, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=EN, label=Figure 4, caption=Multi-modal prompt graph construction method, figureFileSmall=0GRvcbuezw24EbGR8oT5tA==, figureFileBig=a6J2S6QxZKkNQ2baMWQT9w==, tableContent=null), ArticleFig(id=1327012640548827811, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=CN, label=图4, caption=多模态提示图构建方法, figureFileSmall=0GRvcbuezw24EbGR8oT5tA==, figureFileBig=a6J2S6QxZKkNQ2baMWQT9w==, tableContent=null), ArticleFig(id=1327012640632713892, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=EN, label=Figure 5, caption=mAP under the proportion of open vocabulary and number of prompts, figureFileSmall=QbuFTqwqN/BNR11m1lPRWA==, figureFileBig=+NoVFWVLN7gMnXVbOhefbA==, tableContent=null), ArticleFig(id=1327012640708211365, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=CN, label=图5, caption=开放词汇比例及提示个数下的mAP, figureFileSmall=QbuFTqwqN/BNR11m1lPRWA==, figureFileBig=+NoVFWVLN7gMnXVbOhefbA==, tableContent=null), ArticleFig(id=1327012640771125926, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=EN, label=Figure 6, caption=Convergence of algorithm, figureFileSmall=dEwxjQOSI4/E/nmxW4bLOQ==, figureFileBig=k342AoC4ndruLqgkyjAtwQ==, tableContent=null), ArticleFig(id=1327012640850817703, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=CN, label=图6, caption=算法收敛性, figureFileSmall=dEwxjQOSI4/E/nmxW4bLOQ==, figureFileBig=k342AoC4ndruLqgkyjAtwQ==, tableContent=null), ArticleFig(id=1327012640913732264, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=EN, label=Table 1, caption=Benchmark Datasets, figureFileSmall=null, figureFileBig=null, tableContent=
数据集标记数总标记数训练集数目验证集数目
VOC202745057175823
COCO8011726611828740670
CUB2001178859945794
NUS815018161789107859
), ArticleFig(id=1327012640997618345, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=CN, label=表1, caption=

基准数据集

, figureFileSmall=null, figureFileBig=null, tableContent=
数据集标记数总标记数训练集数目验证集数目
VOC202745057175823
COCO8011726611828740670
CUB2001178859945794
NUS815018161789107859
), ArticleFig(id=1327012641106670250, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=EN, label=Table 2, caption=Experimental Results on Baseline Datasets %, figureFileSmall=null, figureFileBig=null, tableContent=
方法mAP
VOCCOCOCUBNUS
强监督93.8482.7933.3459.04
零次学习90.8875.7121.6741.34
线性探针90.6872.7621.2641.22
MKT[26]68.7462.3810.6132.44
ASL[31]79.5071.2710.9240.99
CSRA[32]86.2178.9720.8642.87
CTran[33]88.6077.8910.8643.23
本文方法92.0879.7122.6644.20
), ArticleFig(id=1327012641198944939, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=CN, label=表2, caption=

基准数据集实验结果

, figureFileSmall=null, figureFileBig=null, tableContent=
方法mAP
VOCCOCOCUBNUS
强监督93.8482.7933.3459.04
零次学习90.8875.7121.6741.34
线性探针90.6872.7621.2641.22
MKT[26]68.7462.3810.6132.44
ASL[31]79.5071.2710.9240.99
CSRA[32]86.2178.9720.8642.87
CTran[33]88.6077.8910.8643.23
本文方法92.0879.7122.6644.20
), ArticleFig(id=1327012641287025324, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=EN, label=Table 3, caption=Graph Neural Network Ablation Experiments on VOC Dataset %, figureFileSmall=null, figureFileBig=null, tableContent=
方法mAP
不使用提示图网络90.45
使用提示图网络92.08
), ArticleFig(id=1327012641379300013, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=CN, label=表3, caption=

VOC数据集上图神经网络消融实验

, figureFileSmall=null, figureFileBig=null, tableContent=
方法mAP
不使用提示图网络90.45
使用提示图网络92.08
), ArticleFig(id=1327012641450603182, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=EN, label=Table 4, caption=Ablation Experiments Based on the Proportion of Supervised Samples %, figureFileSmall=null, figureFileBig=null, tableContent=
标记比例mAP
VOCCOCOCUBNUS
2089.4776.1621.6642.90
5092.0879.7122.6644.20
8093.0180.0723.5148.72
10093.8482.7933.3459.04
), ArticleFig(id=1327012641547072176, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=CN, label=表4, caption=

基于监督样本比例的消融实验

, figureFileSmall=null, figureFileBig=null, tableContent=
标记比例mAP
VOCCOCOCUBNUS
2089.4776.1621.6642.90
5092.0879.7122.6644.20
8093.0180.0723.5148.72
10093.8482.7933.3459.04
), ArticleFig(id=1327012641630958257, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=EN, label=Table 5, caption=Execution Time of Algorithm s, figureFileSmall=null, figureFileBig=null, tableContent=
数据集执行方式MKT[26]ASL[31]CSRA[32]CTran[33]本文方法*本文方法
注:本文方法*表示未使用多模态提示图时所需训练与测试时间.
VOC训练545357505555
测试555357505555
COCO训练119011051259113012151168
测试515151515151
), ArticleFig(id=1327012641710650034, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, language=CN, label=表5, caption=

算法执行时间

, figureFileSmall=null, figureFileBig=null, tableContent=
数据集执行方式MKT[26]ASL[31]CSRA[32]CTran[33]本文方法*本文方法
注:本文方法*表示未使用多模态提示图时所需训练与测试时间.
VOC训练545357505555
测试555357505555
COCO训练119011051259113012151168
测试515151515151
)], attaches=null, journal=Journal(id=1324759606947315908, delFlag=0, nameCn=计算机研究与发展, nameEn=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, issn=1000-1239, eissn=null, cn=11-1777/TP, coden=null, periodic=0, language=CN, oaType=null, ccby=null, superviseOffice=null, ownerOffice=null, pubOffice=null, editorOffice=null, officeType=null, aims=null, clcCode=null, officeProv=null, officeCity=null, officeAddr=null, officeZip=null, officeEmail=null, officePhone=null, editDirector=null, officeDirector=null, officeDirectorPhone=null, officeStaffNum=null, officeEmpNum=null, coverPicUrl=k2l8zgqwstuxxevFX2qxVA==, journalPrice=null, startedYear=null, abbrevIsoEn=Journal of Computer Research and Development, journalRemark=null, publicationField=null, createdTime=1793776903141, updatedTime=1793850698612, createdBy=18614031015, updatedBy=13701087609, firstLetterCn=J, firstLetterEn=J, subjectCode=Electronic Information, subjectName=null, subjectCodeEn=Electronic Information, subjectNameEn=null, picCn=k2l8zgqwstuxxevFX2qxVA==, picEn=r6k0Ry+RD+c/VXj5QarmDw==, jcr=null, cjcr=null, exts=[JournalExt(id=1325069127710286688, language=CN, name=计算机研究与发展, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698640, updatedTime=1793850698640, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""}), JournalExt(id=1325069127777395553, language=EN, name=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698656, updatedTime=1793850698656, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""})], databaseList=null, tenantJournalId=1324759914146529478, websiteList=[Website(id=1325070300139892771, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN, language=CN, createTime=1793850978169, createBy=18614031015, updateTime=1793851040259, updateBy=18614031015, name=计算机研究与发展-中文, tplId=1146099689490845704, title=计算机研究与发展, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070893751349360, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=articleTextType, value=kx, createTime=1793851119697, updateTime=1793851119697, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893730377837, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=banner, value=null, createTime=1793851119692, updateTime=1793851119692, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893772320883, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=grayFlag, value=0, createTime=1793851119702, updateTime=1793851119702, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893721989228, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851119690, updateTime=1793851119690, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893784903797, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=minRunFlag, value=0, createTime=1793851119705, updateTime=1793851119705, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893742960751, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic, createTime=1793851119695, updateTime=1793851119695, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893780709492, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=silenceFlag, value=0, createTime=1793851119704, updateTime=1793851119704, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893738766446, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_cn_619/, createTime=1793851119694, updateTime=1793851119694, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893755543665, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeColor, value=null, createTime=1793851119698, updateTime=1793851119698, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893763932274, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeStyle, value=null, createTime=1793851119700, updateTime=1793851119700, creator=18614031015, updator=18614031015)]), Website(id=1325070300580294698, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN, language=EN, createTime=1793850978274, createBy=18614031015, updateTime=1793851036638, updateBy=18614031015, name=计算机研究与发展-英文, tplId=1146101810881728533, title=Journal of Computer Research and Development, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070925288321148, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=articleTextType, value=kx, createTime=1793851127216, updateTime=1793851127216, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925263155321, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=banner, value=null, createTime=1793851127210, updateTime=1793851127210, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925309292671, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=grayFlag, value=0, createTime=1793851127221, updateTime=1793851127221, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925254766712, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851127208, updateTime=1793851127208, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925326069889, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=minRunFlag, value=0, createTime=1793851127225, updateTime=1793851127225, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925279932539, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic, createTime=1793851127214, updateTime=1793851127214, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925317681280, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=silenceFlag, value=0, createTime=1793851127223, updateTime=1793851127223, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925271543930, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_en_623/, createTime=1793851127212, updateTime=1793851127212, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925292515453, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeColor, value=null, createTime=1793851127217, updateTime=1793851127217, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925300904062, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeStyle, value=null, createTime=1793851127219, updateTime=1793851127219, creator=18614031015, updator=18614031015)])], journalTitle=计算机研究与发展, weixinUrl=null, journalUrl=https://crad.ict.ac.cn/, iacademicId=null, status=1, seqNo=null, journalTitleEn=Journal of Computer Research and Development, journalPhotoCn=k2l8zgqwstuxxevFX2qxVA==, journalPhotoEn=r6k0Ry+RD+c/VXj5QarmDw==, journalFirstLetter=J, journalRecommend=null, journalNew=null, journalCollection=null, jcrJf=null, cjcrJf=null, jcrJfStr=null, cjcrJfStr=null, submissionFirstDecision=null, sciSubjectClassification=null, casSubjectClassification=null, citeScore=null, totalCitationFrequency=null, icpCode=null, psCode=null, advertisingLicenseCode=null, copyrightInformation=null, country=null, option=, provinceCode=null, provinceName=null, collectFlag=false, interPubPlatform=, interPubPlatformUrl=null), detailUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202440123, detailUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202440123, pdfUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/PDF/10.7544/issn1000-1239.202440123, pdfUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/PDF/10.7544/issn1000-1239.202440123, aliStartDate=0, aliEndDate=0, collectionFlag=false, citedCount=null, citedUrl=null, previewStatus=0, delFlag=0, hasFullText=1, orderTime=1738339200000, fullTextJson=null, articleText=null, reference=null)
收藏切换
基于图提示的半监督开放词汇多标记学习
李仲年 1, 3 , 皇甫志宇 1 , 杨凯杰 1 , 营鹏 1 , 孙统风 1, 3 , 许新征 1, 2, 3, *
计算机研究与发展 | 人工智能 2025,62(2): 432-442
收起
收藏切换
计算机研究与发展 |人工智能 2025 , 62 (2) : 432 -442
基于图提示的半监督开放词汇多标记学习
全屏
[Author(id=1327012635612131951, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=zhongnianli@cumt.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327012635716989555, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012635612131951, language=EN, stringName=Zhongnian Li, firstName=Zhongnian, middleName=null, lastName=Li, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 3, address=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116
3Mine Digitization Engineering Research Center (China University of Mining and Technology), Ministry of Education, Xuzhou, Jiangsu 221116, bio={"content":"

born in 1990. PhD, lecturer. Member of CCF. His main research interests include data mining, machine learning, and information system

"}, bioImg=null, bioContent=

born in 1990. PhD, lecturer. Member of CCF. His main research interests include data mining, machine learning, and information system

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327012635826041460, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012635612131951, language=CN, stringName=李仲年, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 3, address=1中国矿业大学计算机科学与技术学院 江苏徐州 221116
3矿山数字化教育部工程研究中心 (中国矿业大学) 江苏徐州 221116, bio={"img":"nB581yCVqKd+6mLXtztdEg==","content":"

1990年生. 博士,讲师. CCF会员. 主要研究方向为数据挖掘、机器学习、信息系统

"}, bioImg=nB581yCVqKd+6mLXtztdEg==, bioContent=

1990年生. 博士,讲师. CCF会员. 主要研究方向为数据挖掘、机器学习、信息系统

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327012633720500836, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=1, ext=[AuthorCompanyExt(id=1327012633728889445, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116), AuthorCompanyExt(id=1327012633737278054, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1中国矿业大学计算机科学与技术学院 江苏徐州 221116)]), AuthorCompany(id=1327012633909244522, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=3, ext=[AuthorCompanyExt(id=1327012633917633131, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633909244522, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3Mine Digitization Engineering Research Center (China University of Mining and Technology), Ministry of Education, Xuzhou, Jiangsu 221116), AuthorCompanyExt(id=1327012633926021740, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633909244522, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3矿山数字化教育部工程研究中心 (中国矿业大学) 江苏徐州 221116)])]), Author(id=1327012635914121846, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327012636098671224, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012635914121846, language=EN, stringName=Zhiyu Huangfu, firstName=Zhiyu, middleName=null, lastName=Huangfu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116, bio={"content":"

born in 2000. Master. His main research interests include artificial intelligence and machine learning

"}, bioImg=null, bioContent=

born in 2000. Master. His main research interests include artificial intelligence and machine learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327012636203528825, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012635914121846, language=CN, stringName=皇甫志宇, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1中国矿业大学计算机科学与技术学院 江苏徐州 221116, bio={"img":"JccGJzRpnRZm/3SaoY7XiQ==","content":"

2000年生. 硕士. 主要研究方向为人工智能、机器学习

"}, bioImg=JccGJzRpnRZm/3SaoY7XiQ==, bioContent=

2000年生. 硕士. 主要研究方向为人工智能、机器学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327012633720500836, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=1, ext=[AuthorCompanyExt(id=1327012633728889445, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116), AuthorCompanyExt(id=1327012633737278054, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1中国矿业大学计算机科学与技术学院 江苏徐州 221116)])]), Author(id=1327012636279026299, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327012636354523773, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012636279026299, language=EN, stringName=Kaijie Yang, firstName=Kaijie, middleName=null, lastName=Yang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116, bio={"content":"

born in 2001. Master. His main research interests include artificial intelligence and machine learning

"}, bioImg=null, bioContent=

born in 2001. Master. His main research interests include artificial intelligence and machine learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327012636450992766, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012636279026299, language=CN, stringName=杨凯杰, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1中国矿业大学计算机科学与技术学院 江苏徐州 221116, bio={"img":"cTwGhFAkXWR2wpZNSbDFgg==","content":"

2001年生. 硕士. 主要研究方向为人工智能、机器学习

"}, bioImg=cTwGhFAkXWR2wpZNSbDFgg==, bioContent=

2001年生. 硕士. 主要研究方向为人工智能、机器学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327012633720500836, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=1, ext=[AuthorCompanyExt(id=1327012633728889445, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116), AuthorCompanyExt(id=1327012633737278054, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1中国矿业大学计算机科学与技术学院 江苏徐州 221116)])]), Author(id=1327012636526490240, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327012636601987714, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012636526490240, language=EN, stringName=Peng Ying, firstName=Peng, middleName=null, lastName=Ying, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116, bio={"content":"

born in 2001. PhD. Student member of CCF. His main research interests include artificial intelligence, data mining, and machine learning

"}, bioImg=null, bioContent=

born in 2001. PhD. Student member of CCF. His main research interests include artificial intelligence, data mining, and machine learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327012636685873795, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012636526490240, language=CN, stringName=营鹏, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1中国矿业大学计算机科学与技术学院 江苏徐州 221116, bio={"img":"21IZRteQOG27AX9NteABmQ==","content":"

2001年生. 博士. CCF学生会员. 主要研究方向为人工智能、数据挖掘、机器学习

"}, bioImg=21IZRteQOG27AX9NteABmQ==, bioContent=

2001年生. 博士. CCF学生会员. 主要研究方向为人工智能、数据挖掘、机器学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327012633720500836, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=1, ext=[AuthorCompanyExt(id=1327012633728889445, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116), AuthorCompanyExt(id=1327012633737278054, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1中国矿业大学计算机科学与技术学院 江苏徐州 221116)])]), Author(id=1327012636786537093, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327012636929143432, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012636786537093, language=EN, stringName=Tongfeng Sun, firstName=Tongfeng, middleName=null, lastName=Sun, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 3, address=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116
3Mine Digitization Engineering Research Center (China University of Mining and Technology), Ministry of Education, Xuzhou, Jiangsu 221116, bio={"content":"

born in 1977. PhD, associate professor. Member of CCF. His main research interests include artificial intelligence, machine learning, and data mining

"}, bioImg=null, bioContent=

born in 1977. PhD, associate professor. Member of CCF. His main research interests include artificial intelligence, machine learning, and data mining

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327012637000446601, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012636786537093, language=CN, stringName=孙统风, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 3, address=1中国矿业大学计算机科学与技术学院 江苏徐州 221116
3矿山数字化教育部工程研究中心 (中国矿业大学) 江苏徐州 221116, bio={"img":"EoArBWxvmDLbNKS2IYh1TQ==","content":"

1977年生. 博士,副教授. CCF会员. 主要研究方向为人工智能、机器学习、数据挖掘

"}, bioImg=EoArBWxvmDLbNKS2IYh1TQ==, bioContent=

1977年生. 博士,副教授. CCF会员. 主要研究方向为人工智能、机器学习、数据挖掘

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327012633720500836, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=1, ext=[AuthorCompanyExt(id=1327012633728889445, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116), AuthorCompanyExt(id=1327012633737278054, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1中国矿业大学计算机科学与技术学院 江苏徐州 221116)]), AuthorCompany(id=1327012633909244522, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=3, ext=[AuthorCompanyExt(id=1327012633917633131, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633909244522, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3Mine Digitization Engineering Research Center (China University of Mining and Technology), Ministry of Education, Xuzhou, Jiangsu 221116), AuthorCompanyExt(id=1327012633926021740, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633909244522, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3矿山数字化教育部工程研究中心 (中国矿业大学) 江苏徐州 221116)])]), Author(id=1327012637059166859, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, orderNo=5, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=xxzheng@cumt.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1327012637176607375, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012637059166859, language=EN, stringName=Xinzheng Xu, firstName=Xinzheng, middleName=null, lastName=Xu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, 3, *, address=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116
2State Key Laboratory of CAD & CG (Zhejiang University), Hangzhou 310058
3Mine Digitization Engineering Research Center (China University of Mining and Technology), Ministry of Education, Xuzhou, Jiangsu 221116, bio={"content":"

born in 1980. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include artificial intelligence, machine learning, and medical image processing

"}, bioImg=null, bioContent=

born in 1980. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include artificial intelligence, machine learning, and medical image processing

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327012637256299152, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, authorId=1327012637059166859, language=CN, stringName=许新征, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, 2, 3, *, address=1中国矿业大学计算机科学与技术学院 江苏徐州 221116
2计算机辅助设计与图形系统全国重点实验室 (浙江大学) 杭州 310058
3矿山数字化教育部工程研究中心 (中国矿业大学) 江苏徐州 221116, bio={"img":"9zuDj17ns1PX4bZIYOzD1A==","content":"

1980年生. 博士,教授,博士生导师. CCF高级会员. 主要研究方向为人工智能、机器学习、医学图像处理

"}, bioImg=9zuDj17ns1PX4bZIYOzD1A==, bioContent=

1980年生. 博士,教授,博士生导师. CCF高级会员. 主要研究方向为人工智能、机器学习、医学图像处理

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1327012633720500836, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=1, ext=[AuthorCompanyExt(id=1327012633728889445, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116), AuthorCompanyExt(id=1327012633737278054, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633720500836, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1中国矿业大学计算机科学与技术学院 江苏徐州 221116)]), AuthorCompany(id=1327012633804386919, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=2, ext=[AuthorCompanyExt(id=1327012633816969832, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633804386919, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2State Key Laboratory of CAD & CG (Zhejiang University), Hangzhou 310058), AuthorCompanyExt(id=1327012633829552745, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633804386919, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2计算机辅助设计与图形系统全国重点实验室 (浙江大学) 杭州 310058)]), AuthorCompany(id=1327012633909244522, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, xref=3, ext=[AuthorCompanyExt(id=1327012633917633131, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633909244522, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3Mine Digitization Engineering Research Center (China University of Mining and Technology), Ministry of Education, Xuzhou, Jiangsu 221116), AuthorCompanyExt(id=1327012633926021740, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327012629421339208, companyId=1327012633909244522, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=3矿山数字化教育部工程研究中心 (中国矿业大学) 江苏徐州 221116)])])]
李仲年1, 3 , 皇甫志宇1, 杨凯杰1, 营鹏1, 孙统风1, 3, 许新征1, 2, 3, *
作者信息
  • 1中国矿业大学计算机科学与技术学院 江苏徐州 221116
  • 2计算机辅助设计与图形系统全国重点实验室 (浙江大学) 杭州 310058
  • 3矿山数字化教育部工程研究中心 (中国矿业大学) 江苏徐州 221116
通讯作者:
许新征()
作者简介:

1990年生. 博士,讲师. CCF会员. 主要研究方向为数据挖掘、机器学习、信息系统

2000年生. 硕士. 主要研究方向为人工智能、机器学习

2001年生. 硕士. 主要研究方向为人工智能、机器学习

2001年生. 博士. CCF学生会员. 主要研究方向为人工智能、数据挖掘、机器学习

1977年生. 博士,副教授. CCF会员. 主要研究方向为人工智能、机器学习、数据挖掘

1980年生. 博士,教授,博士生导师. CCF高级会员. 主要研究方向为人工智能、机器学习、医学图像处理

Semi-supervised Open Vocabulary Multi-label Learning Based on Graph Prompting
Zhongnian Li1, 3 , Zhiyu Huangfu1, Kaijie Yang1, Peng Ying1, Tongfeng Sun1, 3, Xinzheng Xu1, 2, 3, *
Affiliations
  • 1School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116
  • 2State Key Laboratory of CAD & CG (Zhejiang University), Hangzhou 310058
  • 3Mine Digitization Engineering Research Center (China University of Mining and Technology), Ministry of Education, Xuzhou, Jiangsu 221116
  • born in 1990. PhD, lecturer. Member of CCF. His main research interests include data mining, machine learning, and information system

    born in 2000. Master. His main research interests include artificial intelligence and machine learning

    born in 2001. Master. His main research interests include artificial intelligence and machine learning

    born in 2001. PhD. Student member of CCF. His main research interests include artificial intelligence, data mining, and machine learning

    born in 1977. PhD, associate professor. Member of CCF. His main research interests include artificial intelligence, machine learning, and data mining

    born in 1980. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include artificial intelligence, machine learning, and medical image processing

出版时间: 2025-02-01 doi: 10.7544/issn1000-1239.202440123
文章导航
收藏切换

半监督多标记学习利用有标记数据和无标记数据进行模型的训练,降低了多标记数据的标记成本并取得了不错的结果,吸引了很多研究者不断进行研究. 然而,在半监督标注过程中,由于标记的数量较多,往往会出现某些标记缺失标注样本的情况,这些标记被称为开放词汇. 开放词汇会导致模型无法学习到该类别的标记信息,使得模型性能下降. 针对上述问题,提出了基于图提示的半监督开放词汇多标记学习方法. 具体地,该方法利用基于提示的图神经网络对预训练大模型进行微调,挖掘和探索开放词汇与监督样本之间的关系. 通过使用包含图像与文本的多模态数据构造图神经网络作为预训练大模型的文本输入进行学习. 其次利用预训练大模型在开放词汇上的泛化能力,对无监督样本生成伪标记,实现对输出分类层的微调,使模型在对开放词汇进行分类时能获得更加理想的效果. 多个基准数据集上的实验结果均显示,基于图提示的半监督开放词汇多标记学习方法优于目前的主流方法,在VOC,COCO,CUB,NUS等基准数据集上均取得了最优的效果.

半监督多标记学习  /  预训练模型  /  图神经网络  /  开放词汇  /  提示

Semi-supervised multi-label learning employs labeled and unlabeled data to train a model, which effectively achieves good results and reduces the labeling cost of multi-label data. Therefore, semi-supervised multi-label learning has attracted many researchers dedicated to this field. However, in the semi-supervised annotation process, due to the large number of labels, it is a common situation that some labels lack of samples, and these labels are called open vocabulary. It is difficult for the model to learn the label information of the open vocabulary, which leads to the degradation of its performance. To address the above problem, we propose a semi-supervised open vocabulary multi-label learning method based on graph prompting. Specifically, this method uses a graph neural network via prompt to fine-tune the pre-trained model and explore the relationship between open vocabulary and supervised samples. By using images and text, we construct a graph neural network as the input of text for the pre-trained model. Furthermore, by leveraging the generalization ability of the pre-trained model on open vocabulary, pseudo-labels are generated for unsupervised samples. Then we use pseudo-labels to train the classification layer and to enable the model to achieve better performance in classifying open vocabulary. Experimental results on multiple benchmark datasets, including VOC, COCO, CUB, and NUS, consistently demonstrate that the proposed method outperforms existing methods and achieves state-of-the-art performance.

semi-supervised multi-label learning  /  pre-trained model  /  graph neural network  /  open vocabulary  /  prompt
李仲年, 皇甫志宇, 杨凯杰, 营鹏, 孙统风, 许新征. 基于图提示的半监督开放词汇多标记学习. 计算机研究与发展, 2025 , 62 (2) : 432 -442 . DOI: 10.7544/issn1000-1239.202440123
Zhongnian Li, Zhiyu Huangfu, Kaijie Yang, Peng Ying, Tongfeng Sun, Xinzheng Xu. Semi-supervised Open Vocabulary Multi-label Learning Based on Graph Prompting[J]. Journal of Computer Research and Development, 2025 , 62 (2) : 432 -442 . DOI: 10.7544/issn1000-1239.202440123
在传统的机器学习任务中,一个样本通常对应一个明确的固定标记. 但随着机器学习技术的进步,许多机器学习任务中的样本可以具有多个标记,这被称为多标记学习. 多标记学习中的样本与多个标记相关联,其标记数量众多且具有复杂的语义特征. 这使得在实际应用中获取数据的真实标记常常代价过高,需要投入大量的时间和人力物力成本. 因此,许多研究者选择研究一种新的机器学习范式,即弱监督多标记学习.
常见的弱监督多标记学习包括偏多标记学习[1]、标记缺失学习[2]、半监督多标记学习[3]. 在这些多标记问题中,半监督多标记学习吸引了大量研究者的关注. 半监督学习是指在训练模型时,同时利用有标记数据和无标记数据进行学习,其目标是通过利用有标记数据和无标记数据的信息来提高模型的性能. 通常情况下,有标记数据的数量相对较少,而无标记数据的数量相对较多. 目前,半监督多标记学习在计算机视觉[4-6]、自然语言处理[7-8]、社交媒体分析[9]、生物医学[10]等方面均具有广泛应用.
针对半监督学习中缺乏标记的问题,预训练大模型和图神经网络被用于解决这种问题. 在大模型使用方面,Song等人[11]提出了FD-Align(feature discri-mination alignment)方法,通过在训练过程中保持虚假特征的一致性以增强模型的可推广性来提高CLIP(contrastive language-image pre-training)[12]大模型在半监督任务上的表现效果. Mo等人[13]则提出了S-CLIP(semi-supervised vision-language pre-training)方法,使用专门设计的伪标记策略,通过配对的图像文本提供的关键字级伪标记,以及微调CLIP将CLIP的零样本分类性能提高了10%. 同时,在图网络方面,Wang等人[14]提出的HSPNet(hierarchical semantic prompt network)则通过对标记进行建图来解决单一正标记问题.
然而在传统的半监督多标记学习中,每个标记都有被标注的样本,这与真实的情况不符. 在真实情况中,由于多标记任务中标记的数量较多、被标记的样本比例很小,导致一些标记在所有样本中都未被标记,如图1所示. 在图1中,由于盆栽和桌子标记不常见,这导致了它们在有监督样本中未被标记,但出现在无监督样本中,本文称这些标记为开放词汇(open vocabulary). 然而,在传统的半监督学习中,如图2所示,对所有标记来说都有被标注到的样本,其有监督标记集与无监督标记集一致,即不存在开放词汇.
在这个设定下,现有的模型在面对开放词汇的标记时,没有挖掘和探索开放词汇的信息,因此会带来模型性能的下降. 这激发了本文对于该问题的探索. 为解决半监督多标记任务中的开放词汇问题,提出了一种基于图提示的微调大模型方法. 通过使用包含图像与文本的多模态数据构造图神经网络,将图神经网络作为预训练大模型RAM(recognize anything model)[15]的文本输入进行学习. 图神经网络由节点和边组成,它使用标记与图像作为节点,使用边存储节点间的信息,在图神经网络中存储标记间的联系,以提高模型在开放词汇半监督学习中的表现效果.
本文的主要贡献包括3个方面:
1)提出了一种基于图卷积网络的微调大模型算法. 该算法在训练过程中利用大模型的输出作为伪标记来参与损失计算,以提高大模型在开放词汇半监督多标记学习中的分类性能;
2)构建了基于提示的图神经网络,通过对数据集的文本特征和图像特征进行建图,使用图神经网络来学习图像与文本之间的联系,以达到提高模型分类性能的效果;
3)多个基准数据集上的实验结果表明,基于图提示的大模型微调方法在开放词汇半监督多标记学习中具有最佳的效果.
多标记学习是机器学习领域的一个重要研究方向,旨在解决实际问题中1张图像或样本存在多个相关标记的情况. 在过去的几十年里,研究者提出了许多方法和算法以处理多标记学习问题. 其中,一些方法基于传统的机器学习技术,如二分类器链[16]、分类器融合[17]和标记空间分割[18]等. 这些方法通过将多标记问题转化为一系列单标记问题的组合或分割,来实现多标记的预测.
随着深度学习的兴起,越来越多的研究工作将深度神经网络应用于多标记学习. 不同设置下的多标记分类问题在过去几年中取得了很大进展,对损失函数和骨干网络等的改进在完全标记和部分标记的设置下都取得了很大成功. 其中,Wang等人[19]提出了一种双重关系半监督多标记学习方法,提出了一种双分类器域自适应策略,在生成伪标记的同时对齐特征以提高学习性能. Xie等人[20]提出了一种局部多标记学习方法来同时恢复真值信息和识别噪声标记,结合标记相关性挖掘和特征诱导噪声模型,对多标记分类器和带噪声标记识别器进行联合优化. Duan等人[21]提出了一种基于类转移跟踪的伪整流制导方法,统一了伪校正过程引起的类分布和类转移的历史信息,提高了标记非随机丢失问题中的伪标记质量.
开放词汇[22]原指在自然语言处理任务中,不事先限定词汇表的大小或内容,而允许使用任意的、未知的词汇. 相比于封闭词汇(closed vocabulary)[23]的模型,开放词汇模型具有更大的词汇表覆盖范围,可以处理未见过的词汇和新兴词汇. 在计算机视觉任务中,开放词汇模型可以对未见过的物体进行标记. 传统的封闭词汇模型通常仅能识别预定义的有限标记,而开放词汇模型可以对新的、未知的物体进行分类和识别.
在目标检测方面,Gu等人[24]使用大模型强大的泛化性能,利用知识蒸馏技术对CLIP进行蒸馏,以解决开放词汇目标检测问题. 在目标分割方面,Zhang等人[25]通过引入一个预训练的文本编码器,在分割任务中对所有视觉概念进行编码,取得了优异的结果. 在多标记学习方面,He等人[26]提出了一种采用知识蒸馏来保证图像和标记嵌入的一致性的方法,使用提示调整以进一步更新标记嵌入. 为了进一步实现对多个物体的识别,开发了一个简单但有效的双流模块来捕获局部和全局特征.
大模型的泛化能力使得其在零样本学习中能够取得较好的结果. 零样本学习属于迁移学习,其目的是使训练的模型能够正确识别,仅存在于测试集中,不存在于训练集中的样本中. 这与开放词汇的概念不谋而合.
CLIP通过对比学习的训练方式,同时处理图像和文本2种模态的数据,有效地提高了模型的跨模态理解能力,更好地挖掘出语言和图像之间的内在联系,提高模型的泛化能力,使得其在开放词汇任务中具有较好的性能.
BLIP(bootstrapping language-image pre-training)[27]不仅使用文本生成器合成文本,而且使用过滤器去除噪声文本,在预训练过程中实现了编码器-编码器的多模式混合,具有强大的泛化能力,同样在开放词汇任务中取得了较为理想的结果.
RAM通过文本语义自动解析,使用大模型获取无标注的图像标记;随后通过统一标注,使用一个初步的自动标注模型产生自动标注,并使用数据引擎生成额外的注释;最后使用处理后的数据进行重新训练,以达到处理开放词汇多标记任务的效果.
CoOp(context optimization)[28],CoCoOp(condi-tional context optimization)[29]使用提示学习的方法,尝试使用大模型更好地适应下游视觉任务,为微调大模型提出了崭新范式.
在本节中,提出了基于图提示的微调大模型方法. 主要介绍在开放词汇半监督多标记问题中如何使用大模型输出作为伪标记以计算损失以及如何通过图卷积神经网络提高模型的开放词汇多标记学习能力. 该方法旨在使用图卷积神经网络对图像和文本构造图神经网络,然后在图神经网络中学习开放词汇与封闭词汇之间的关系,从而实现对开放词汇样本的分类.
具体的,首先使用CLIP的编码器提取一个样本中的文本和图像特征. 然后将每个标记设置为1个节点,从而构建一张包含所有标记的图网络(见2.2节). 之后使用该图网络替换RAM中的文本输入(见2.1节). 在计算损失环节,对于有监督数据使用其原本标记的计算损失. 对于无监督数据使用大模型的输出作为伪标记计算其损失(见2.3节).
在本文中,$ M $表示标记的个数. $ k $表示构建多模态特征图使用的图像数量. $ D $表示实例集. $ W $表示标记集. $ T $表示经过文本编码器获得的文本特征集. $ L $表示实例集$ D $经过图像编码器获得的图像特征集. $ O $表示最终的多模态提示图. $ G=\{{g}_{1},{g}_{2},…,{g}_{k}\} $表示标记对应的多模态特征图. $ {\boldsymbol{A}}_{ii}^{{g}_{i}} $表示图$ {g}_{i} $的邻接矩阵,$ {\boldsymbol{B}}_{ij} $表示图$ O $对应的邻接矩阵. $ {X}^{\mathrm{s}}=\{{x}_{1}^{\mathrm{s}}{,x}_{2}^{\mathrm{s}},…,{x}_{{n}_{\mathrm{s}}}^{\mathrm{s}}\} $表示有监督样本,$ {X}^{\mathrm{u}}=\{{x}_{1}^{\mathrm{u}},{x}_{2}^{\mathrm{u}},…,{x}_{{n}_{\mathrm{u}}}^{\mathrm{u}}\} $表示无监督样本,其中$ {n}_{\mathrm{s}} $表示有监督样本数量,$ {n}_{\mathrm{u}} $表示无监督样本数量. $ {Y}^{\mathrm{s}}=\{{y}_{1}^{\mathrm{s}}, {y}_{2,}^{\mathrm{s}},…,{y}_{{n}_{\mathrm{s}}}^{\mathrm{s}}\} $表示标记,$ {Y}^{\mathrm{u}}=\{{y}_{1}^{\mathrm{u}},{y}_{2}^{\mathrm{u}},…,{y}_{{n}_{\mathrm{u}}}^{\mathrm{u}}\} $表示伪标记.
如图3所示,本文提出的框架包含被冻结RAM预训练大模型、提示图网络、无监督损失与有监督损失. 其中,冻结RAM预训练大模型获取所有样本特征,包括有监督样本及无监督样本. 使用“A photo of a [Class]”的格式构建提示,与图像一起构建提示图网络,使用构建出的多模态提示图替换RAM大模型中的文本输入. 对于有监督样本,直接使用有监督标记参与损失计算,对于无监督样本,使用大模型的输出作为伪标记参与损失计算. 最后将得到的有监督损失与无监督损失相加微调大模型.
关于提示的选取,在图的构建过程中共有M个标记,再针对每个标记获取数据集外的k张图像,将这k张图像组成实例集$ {D}_{m}=\{{d}_{{m}_{1}},{d}_{{m}_{2}},…,{d}_{{m}_{k}}\} $. 同时获取其标记集$ W=\left\{{w}_{1},{w}_{2},…,{w}_{m}\right\} $. 针对标记集中的每个标记,都有与其唯一对应的图像集$ D $. 然后将第i个标记通过模板“A photo of a [Class]”构造一个提示,并将提示输入到CLIP的文本编码器,提取512维的文本特征$ {T}_{i} $,将第i个标记的图像经过CLIP的图像编码器来提取512维的图像特征$ {L}_{i} $. 其次,将第$ i $个标记的图像特征与文本特征构建为多模态特征图,记为$ {g}_{i} $. 其中,边的信息使用余弦相似度$ S im(\cdot) $计算得到,使用$ {\boldsymbol{A}}_{ii}^{{g}_{i}} $表示其邻接矩阵. 具体计算方法如式(1)所示.
$ \begin{array}{c}\boldsymbol{A}_{ii}^{g_i}=S im\left(T_i\times L_i\right),\end{array} $
$ \begin{array}{c}\boldsymbol{B}_{ij}=S im\left(\boldsymbol{n}_i\cdot\boldsymbol{n}_j\right).\end{array} $
在得到所有标记的多模态特征图集合$ G=\left\{{g}_{1},{g}_{2},…, {g}_{m}\right\} $后,将其经过2层图卷积层和1层池化层得到节点集$ N=\left\{{\boldsymbol{n}}_{1},{\boldsymbol{n}}_{2},…,{\boldsymbol{n}}_{m}\right\} $,并对得到的节点集再次经过余弦相似度$ S im(\cdot) $计算,得到最终的多模态提示图$ O $,使用$ {\boldsymbol{B}}_{ij} $表示其邻接矩阵(见式(2)). 最后,使用多模态提示图$ O $替换RAM大模型中的文本输入. 具体构建过程如图4所示.
通过构建出的图神经网络替换RAM文本嵌入,使得模型在面对开放词汇时能够通过图神经网络中的标记关系得到更多开放词汇的信息. 在计算损失时,面对已知类信息可以直接使用样本真实标记和直接参与计算,而未知标记信息则需使用伪标记作为真实标记进行损失计算.
得益于大模型优秀的泛化性能和零样本推理能力,使用RAM大模型的无标记损失作为伪标记参与损失计算,提出了新的经验风险函数:
$ \begin{split}\hat{R}\left({\boldsymbol f}_{\boldsymbol \theta }\right)=&\dfrac{1}{{n}_{\mathrm{s}}}\displaystyle\sum\limits _{i=1}^{{n}_{\mathrm{s}}}L\left[{\boldsymbol f}_{\boldsymbol \theta }\left[{f}_{\mathrm{R}\mathrm{A}\mathrm{M}}\left({x}_{i}^{\mathrm{s}},P\right)\right],{y}_{i}^{\mathrm{s}}\right]+\\&\dfrac{1}{{n}_{\mathrm{u}}}\displaystyle\sum\limits _{i=1}^{{n}_{\mathrm{u}}}L\left[{\boldsymbol f}_{\boldsymbol \theta }\left[{f}_{\mathrm{R}\mathrm{A}\mathrm{M}}\left({x}_{i}^{\mathrm{u}},P\right)\right],{y}_{i}^{\mathrm{u}}\right],\end{split} $
其中$ {f}_{\mathrm{R}\mathrm{A}\mathrm{M}} $表示预训练大模型的输出,$ {\boldsymbol f}_{\boldsymbol \theta } $表示全连接层,$ P $表示图像特征,$ L $表示均方误差损失,计算方法如式(4)所示,其中,$ {\hat{c}}_{i} $表示模型的输出,$ {c}_{i} $表示给定的标记.
$ \begin{array}{c}L\left[\hat{c}_i,c_i\right]=\dfrac{1}{M}\displaystyle\sum\limits_{i=1}^m\left(\hat{c}_i-c_i\right)^2,\end{array} $
$ \begin{array}{c}{y}_{i}^{u}={\mathrm{sgn}}\left[{f}_{\mathrm{R}\mathrm{A}\mathrm{M}}\left({x}_{i}^{\mathrm{u}},P\right)-\sigma \right]. \end{array} $
使用式(5)将预训练大模型的输出作为伪标记$ {y}_{i}^{\mathrm{u}} $,其中$ \mathrm{s}\mathrm{g}\mathrm{n} $表示符号函数,$ \sigma $表示选择伪标记时的置信度阈值.
该经验风险函数在计算已知类信息时使用样本与其真实标记直接参与计算. 面对未知类时,使用大模型对其进行推理,将RAM大模型的预测输出当作该样本的伪标记,以计算标记缺失样本的损失.
本文方法通过构造多模态提示图存储类间信息以实现开放词汇多标记任务,包括训练和测试2个过程.
计算空间复杂度时,由于本文方法针对每个数据集只需要构建1次多模态提示图,其空间占用不变,不会随训练样本数量的增加而增加,因此空间复杂度$ S\left(n\right)=O\left(1\right) $.
计算时间复杂度时,训练过程主要分为2个阶段,即多模态提示图构建阶段和反向传播优化阶段. 多模态提示图构建阶段与模型训练无关,因此本文针对每个数据集单独构建一次多模态提示图,即时间复杂度为图卷积计算复杂度$ O(n^3) $,n表示样本数量. 反向传播优化过程中,根据样本数量和迭代次数,时间复杂度为$ O(T\times n\times S) $,S表示单样本梯度更新时间,T表示迭代次数. 因此,本文方法训练阶段的时间复杂度为$ O(n^3+T\times n\times S) $. 在测试过程中,不需要额外构建多模态提示图,所需的时间仅为前向传播的时间,时间复杂度为$ O(n\times U) $,U表示单样本前向传播时间.
理论1[30]. 假设第$ t $步目标函数$ {f}_{t} $梯度有界,其中$ t\in \{\mathrm{1,2},…,T\} $. 对于所有$ \boldsymbol \theta \in {\mathbb{R}}^{d} $都满足$ {||\nabla {f}_{t}\left(\boldsymbol \theta \right)||}_{2}\le G $,$ {||\nabla {f}_{t}\left(\boldsymbol \theta \right)||}_{\infty }\le {G}_{\infty } $. 假设使用Adam生成的任意$ {\theta }_{t} $之间的距离是有界的,对于任意$ m,n\in \{\mathrm{1,2},…,T\} $,$ {||{\theta }_{n}-{\theta }_{m}||}_{2}\le D $,$ {||{\theta }_{n}-{\theta }_{m}||}_{\infty }\le {D}_{\infty } $,其中$ d $表示参数量,$ G $,$ {G}_{\infty } $,$ D $,$ {D}_{\infty } $表示常数. 对于任意$ T\ge 1 $,Adam实现如下保证:
$ \begin{array}{c}\dfrac{R\left(T\right)}{T}=O\left(\dfrac{1}{\sqrt{T}}\right),\end{array} $
其中,$ R\left(T\right)=\underset{t=1}{\displaystyle\sum\limits ^{T}}[{f}_{t}\left({\theta }_{t}\right)-{f}_{t}\left({\theta }^{*}\right)] $,$ {\theta }^{*} $=$ \underset{\theta \in x}{\mathrm{arg}\,\mathrm{m}\mathrm{i}\mathrm{n}}\underset{t=1}{\displaystyle\sum\limits ^{T}}{f}_{t}\left(\theta \right) $.
推论1. 假设第$ t $步经验风险$ {\hat{R}}^{t}\left({\boldsymbol f}_{\boldsymbol \theta }\right) $梯度有界,其中$ t\in \{\mathrm{1,2},…,T\} $. 对于所有$ {\boldsymbol f}_{\boldsymbol \theta }\in {\mathbb{R}}^{d} $都满足$ {||\nabla {\hat{R}}^{t}\left({\boldsymbol f}_{\boldsymbol \theta }\right)||}_{2}\le G $,$ {||\nabla {\hat{R}}^{t}\left({\boldsymbol f}_{\boldsymbol \theta }\right)||}_{\infty }\le {G}_{\infty } $. 假设使用Adam生成的任意$ {\boldsymbol f}_{\boldsymbol \theta }^{t} $之间的距离是有界的,对于任意$ m,n\in \{\mathrm{1,2},…,T\} $,$ {||{\boldsymbol f}_{\boldsymbol \theta }^{n}-{\boldsymbol f}_{\boldsymbol \theta }^{m}||}_{2}\le D $,$ {||{\boldsymbol f}_{\boldsymbol \theta }^{n}-{\boldsymbol f}_{\boldsymbol \theta }^{m}||}_{\infty }\le {D}_{\infty } $. 在使用Adam作为优化器的情况下,本文的经验风险$ \hat{R}\left({\boldsymbol f}_{\boldsymbol \theta }\right) $收敛.
证明. 根据理论1与推论1中假设可得式(7):
$ \begin{array}{c}\dfrac{1}{T}\displaystyle\sum\limits_{t=1}^T\left[\hat{R}^t\left(\boldsymbol{f}_{\boldsymbol{\theta}}\right)-\hat{R}^t\left(\boldsymbol{f}_{\boldsymbol{\theta}}^*\right)\right]=O\left(\dfrac{1}{\sqrt{T}}\right),\end{array} $
其中$ {\boldsymbol f}_{\boldsymbol \theta }^{*}=\underset{\theta \in x}{\mathrm{arg}\,\mathrm{m}\mathrm{i}\mathrm{n}}\underset{t=1}{\displaystyle\sum\limits ^{T}}{\hat{R}}^{t}\left({\boldsymbol f}_{\boldsymbol \theta }\right) $,令$ T\to \infty $可得$ \underset{T\to \infty }{\mathrm{lim}}\Bigg(\dfrac{1}{T}\;\underset{t=1}{\displaystyle\sum\limits ^{T}}\;\cdot \Big[{\hat{R}}^{t}\left({\boldsymbol f}_{\boldsymbol \theta }\right)-{\hat{R}}^{t}\left({\boldsymbol f}_{\boldsymbol \theta }^{*}\right)\Big]\Bigg)=0 $,因此,使用Adam作为优化器的情况下,本文的经验风险$ \hat{R}\left({\boldsymbol f}_{\boldsymbol \theta }\right) $收敛.证毕.
算法过程主要分为2个阶段,即多模态图提示图构建阶段和模型训练阶段,本节分别给出了2个阶段的算法伪代码,其中多模态提示图的构建算法伪代码如算法1所示.
算法1. 多模态提示图的构建.
输入:实例集$ D $,标记集$ W $,标记数$ M $;
输出:$ O $.
① for m = 1 to $ M $ do
② 选取实例集中图像$ {D}_{m} $经过图像编码器获得  图像特征$ {L}_{i} $;
③ 选取标记集中标记$ {W}_{m} $经过文本编码器获得  文本特征$ {T}_{i} $;
④ 根据式(1)计算多模态特征图$ {G}_{m} $;
⑤ 将多模态特征图$ {G}_{m} $经过图卷积和池化后得  到节点集N;
⑥ 根据式(2)计算多模态提示图$ O $;
⑦ end for
使用算法1单独为每一个数据集构建多模态提示图,该构建过程独立于训练过程之外. 在完成多模态提示图构建后,使用算法2进行模型微调,具体模型微调过程伪代码如算法2所示.
算法2. 模型微调过程.
输入:多模态提示图$ O $,训练集$ Z $,迭代次数 $ t $,$ \varepsilon =10\times {\mathrm{e}}^{-5} $;
输出:模型$ {\boldsymbol f}_{\boldsymbol \theta }^{t} $.
① while $ \left({\hat{R}}^{t}\left({\boldsymbol f}_{\boldsymbol \theta }^{t}\right)-{\hat{R}}^{t-1}\left({\boldsymbol f}_{\boldsymbol \theta }^{t-1}\right)\right)^2\ge \varepsilon $ do
② 选取训练集图像$ Z $经过图像编码器得到图  像特征$ {P}_{i} $;
③ 根据式(3)计算经验风险$ {\hat{R}}^{t}\left({\boldsymbol f}_{\boldsymbol \theta }^{t}\right) $;
  使用Adam更新参数$ {\boldsymbol f}_{\boldsymbol \theta }^{t} $;
④ end while
为验证本文方法,在基准数据集PASCAL VOC 2012(VOC),COCO-80,CUB-200-2011(CUB),NUSWIDE(NUS)数据集上进行了实验. VOC数据集共有17125张图像,20个标记. COCO数据集共有158957张图像,80个标记. CUB数据集共有11788张图像,200个鸟类标记. NUS数据集共有269648张图像,81个标记. 数据集具体信息如表1所示.
针对每一个数据集,在训练集中随机选出25%的标记定义为开放词汇,将含有这些标记的样本全部置为不可见样本来模拟在实际工作中遇到的开放词汇情况. 针对剩余的75%的标记,随机选出50%置为未知标记来实现半监督训练.
本文选取RAM预训练大模型作为基模型,输入的图像大小为384×384,Batch大小设置为128,使用Adam优化器,学习率设置为0.01,学习率衰减设置为0.9,每个数据集训练30轮,使用平均精度均值(mean average precision,mAP)作为评估标准,置信度阈值$ \sigma $设置为0.9.
将本文方法与现有的最先进的半监督多标记学习进行了比较,包括MKT[26],ASL[31],CSRA[32],CTran[33]. 其中,MKT提出联合捕获局部和全局特征的双流特征提取模块,应用知识迁移引入开放词汇的前置知识,同时对文本特征使用提示学习,以适应不同的下游任务. ASL基于局部损失(focal loss)解决正负样本不平衡问题,提出了局部损失的改进版,一种非对称的损失,即非对称损失(asymmetric loss). CSRA通过提出一个简单的空间注意力得分来生成每个标记的特定标记特征,然后将其与标记无关的平均池化特征相结合以解决多标记分类问题. CTran提出了一个多标记图像分类的通用框架,使用Transformer来利用视觉特征和标记之间的复杂依赖关系,设定一个标记掩码训练目标来解决训练过程中标记状态为正、负或未知的问题.
为了确保公平性,本文使用了以上方法发布的最新代码在表1中的数据集上进行实验.
在4个基准数据集的对比实验如表2所示. 从表2中可以看出,在所有基准数据集上,本文方法均优于现有的最先进的方法,其中方法中的第1行表示强监督下进行训练的结果,第2行表示使用RAM大模型进行零次推理得到的结果,第3行表示对RAM添加线性探针得到的结果. 具体而言,使用本文方法得到的mAP在VOC数据集上比直接使用RAM大模型得到的mAP高出约1.2个百分点,比MKT方法高出约20个百分点,比ASL方法高出约10个百分点,比CSRA方法高出约6个百分点,比CTran方法高出约4个百分点. 同时在COCO,CUB,NUS数据集上,通过使用基于图提示的微调大模型方法,均具有更好的效果.
本文认为,性能的提升是由于使用了图神经网络挖掘了开放词汇与监督标记之间的关系,充分探索了不同标记间的相关信息,发挥了图神经网络学习标记间信息的能力,使得模型在面对开放词汇半监督多标记问题时表现出了更优的效果.
为了分析图卷积神经网络的有效性,本节进行了消融实验,消融实验通过去除图卷积神经网络,使用RAM直接进行开放词汇半监督多标记学习训练,得到的结果如表3所示.
从表3可以看出,在不使用提示图网络,利用RAM预训练大模型对VOC数据集进行直接推理时得到的mAP为90.45%. 而在使用提示图网络替换RAM大模型文本输入,并对线性层进行训练后,得到的mAP提升到了92.08%.
这个结果说明了多模态提示图在解决开放词汇半监督学习问题时,能够使用图充分挖掘出标记间的联系并保存. 使用保存了标记间关系的多模态提示图输入预训练模型进行训练,能够有效融合图像与文本的多模态信息,提高模型的开放词汇半监督多标记学习能力.
同样地,根据已有标记数量的不同,在20%已有标记、50%已有标记、80%已有标记和完全标记4种情况下均设计了消融实验,得到的实验结果如表4所示.
如表4所示,随着有监督样本比例的提高,本文模型在每个基准数据集上的mAP同样得到了提高. 同时在仅有20%有监督标记的情况下,模型仍能保持较高精度,这证明了本文方法具有鲁棒性.
但在CUB数据集及NUS数据集上半监督训练的结果要远低于全监督训练的结果. 这可能是由于CUB数据集及NUS数据集的特殊性,预训练大模型本身在这2个数据集上的基础性能不佳,在开放词汇比例为25%的情况下,预训练大模型很难做出准确的预测,这样就导致伪标记的mAP降低,最终模型的效果变差.
为验证提示个数及开放词汇比例对结果的影响,在提示个数为0,1,10,20和开放词汇比例为0,0.2,0.4,0.6,0.8的情况下,在VOC数据集上进行了消融实验. 得到的实验结果如图5所示.
在图5中,横坐标表示开放词汇比例,即在整个数据集中完全未标记到的标记占整个数据集的比例. 纵坐标表示mAP. 在不同的开放词汇比例下,当提示个数从0增加至1时,模型的表现有较大提升,但当提示个数从1增加至10及从10增加至20时,模型的表现变化并不明显. 比如开放词汇比例为0.6时,当图像个数从0增加至1时,mAP由90.42%提升至91.56%. 但当图像个数从1增加至20时,mAP提升了0.09个百分点. 这个结果说明了本文方法对于提示的图像个数是鲁棒的.
为了验证算法的复杂度,本节在VOC数据集和COCO数据集上对每个对比方法进行了算法复杂度分析. 得到的实验结果如表5所示.
在COCO和VOC数据集上,本文方法在提升了mAP的同时并没有占用过多的时间进行训练. 与CSRA方法相比,本文方法在提升了mAP的同时占用更少的训练时间. 同时本节设计了不使用多模态提示图时进行训练与测试的消融实验,实验结果表明多模态提示图并未占据更多时间.
通过在VOC数据集和COCO数据集上进行实验,得到的实验结果如图6所示. 从图6可以看出,在VOC数据集上,本文方法在25个迭代次数内即完成收敛;在COCO数据集上,本文方法在30个迭代次数内即完成收敛. 同时在2个数据集上,本文方法都在30个迭代次数内完成收敛,这也表明了本文提出的算法具有收敛性.
本文提出了一种基于图提示的微调大模型方法,基于提示图神经网络对RAM预训练大模型进行微调,使用图神经网络学习开放词汇与有监督样本之间的关系. 同时使用预训练大模型的泛化能力以及使用大模型的输出作为伪标记参与计算无监督样本的损失,使模型在对开放词汇进行分类时能取得更加理想的效果.
在基准数据集上的实验结果表明了本文方法在解决开放词汇多标记问题时具有较好的效果. 此外,本文还设计了不同的消融实验验证了本文方法具有可信服性. 在未来的工作中,将考虑对图构造算法进一步优化,同时研究自适应的伪标记阈值选择方法,以进一步增强模型的开放词汇分类能力.
作者贡献声明:李仲年提出研究课题、确定研究方案;皇甫志宇撰写论文;杨凯杰负责实验的实施;营鹏负责调研和整理文献;孙统风负责实验的指导;许新征负责论文的修改.

参考文献 引证文献
排序方式:
1
Tian Yingjie, Yu Xiaotong, Fu Saiji. Partial label learning: Taxonomy, analysis and outlook[J]. Neural networks, 2023, 161(1): 708−734
2
Rastogi R, Kumar S. Discriminatory label-specific weights for multi-label learning with missing labels[J]. Neural Processing Letters, 2023, 55(2): 1397−1431
3
李宇峰,黄圣君,周志华. 一种基于正则化的半监督多标记学习方法[J]. 计算机研究与发展,2012,49(6):1272−1278
Li Yufeng, Huang Shengjun, Zhou Zhihua. Regularized semi-supervised multi-label learning[J]. Journal of Computer Research and Development, 2012, 49(6): 1272−1278 (in Chinese)
4
Lai Zhengfeng, Wang Chao, Cheung S C , et al. SAR: Self-adaptive refinement on pseudo labels for multiclass-imbalanced semi-supervised learning[C]//Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2022: 4091−4100
5
张永,陈蓉蓉,张晶. 基于交叉熵的安全Tri-training算法[J]. 计算机研究与发展,2021,58(1):60−69
Zhang Yong, Chen Rongrong, Zhang Jing. Safe tri-training algorithm based on cross entropy[J]. Journal of Computer Research and Development, 2021, 58(1): 60−69 (in Chinese)
6
Fan Yue, Kukleva A, Dai Dengxin, et al. Revisiting consistency regularization for semi-supervised learning[J]. International Journal of Computer Vision, 2023, 131(3): 626−643
7
Qiu Ye, Gong Xiaolong, Ma Zhiyi, et al. MixLab: An informative semi-supervised method for multi-label classification[C]//Proc of the 9th CCF Int Conf on Natural Language Processing and Chinese Computing. Berlin: Springer, 2020: 506−518
8
Huang Teng, Jia Binbin, Zhang Minling. Progressive label propagation for semi-supervised multi-dimensional classification[C]//Proc of the 32nd Int Joint Conf on Artificial Intelligence. San Francisco, CA: Margan Kaufmann, 2023: 3821−3829
9
Liu Xuan, Shi Tianyi, Zhou Guohui, et al. Emotion classification for short texts: An improved multi-label method[J]. Humanities and Social Sciences Communications, 2023, 10(1): 1−9
10
Zhang Shuo, Zhang Jiaojiao, Tian Biao, et al. Multi-modal contrastive mutual learning and pseudo-label re-learning for semi-supervised medical image segmentation[J]. Medical Image Analysis, 2022, 83(1): 102656−102656
11
Song Kun, Ma Huimin, Zou Bochao, et al. FD-Align: Feature discrimination alignment for fine-tuning pre-trained models in few-shot learning[C]//Proc of the 37th Conf on Neural Information Processing Systems. New York: Curran Associates, 2023: 43579−43592
12
Radford A, Kim J W, Hallacy C, et al. Learning transferable visual models from natural language supervision[C]//Proc of the 38th Int Conf on Machine Learning. New York: ACM, 2021: 8748−8763
13
Mo S, Kim M, Lee K, et al. S-CLIP: Semi-supervised vision-language learning using few specialist captions[C]//Proc of the 37th Conf on Neural Information Processing Systems. New York: Curran Associates, 2023: 61187−61212
14
Wang Ao, Chen Hui, Lin Zijia, et al. Hierarchical prompt learning using CLIP for multi-label classification with single positive labels[C]//Proc of the 31st ACM Int Conf on Multimedia. New York: ACM, 2023: 5594−5604
15
Zhang Youcai, Huang Xinyu, Ma Jinyu, et al. Recognize anything: A strong image tagging model[J]. arXiv preprint, arXiv: 2306.03514, 2023
16
Read J, Pfahringer B, Holmes G, et al. Classifier chains for multi-label classification[J]. Machine Learning, 2011, 85(3): 333−359
17
Liu Zhunga, Pan Quan, Dezert J, et al. Classifier fusion with contextual reliability evaluation[J]. IEEE Transactions on Cybernetics, 2017, 48(5): 1605−1618
18
Zhao Xiangyun, Vemulapalli R, Mansfield P A, et al. Contrastive learning for label efficient semantic segmentation[C]//Proc of the 18th IEEE/CVF Int Conf on Computer Vision. Los Alamitos, CA: IEEE Computer Society, 2021: 10623−10633
19
Wang Lichen, Liu Yunyu, Qin Can, et al. Dual relation semi-supervised multi-label learning[C]//Proc of the 34th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2020: 6227−6234
20
Xie Mingkun, Huang Shengjun. Partial multi-label learning with noisy label identification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 44(7): 3676−3687
21
Duan Yue, Zhao Zhen, Qi Lei, et al. Towards semi-supervised learning with non-random missing labels[C]//Proc of the 19th IEEE/CVF Int Conf on Computer Vision. Los Alamitos, CA: IEEE Computer Society, 2023: 16121−16131
22
Zhou Zhihua. Open-environment machine learning[J]. National Science Review, 2022, 9(8): nwac123
23
Eichstaedt J C, Kern M L, Yaden D B, et al. Closed-and open-vocabulary approaches to text analysis: A review, quantitative comparison, and recommendations[J]. Psychological Methods, 2021, 26(4): 398−398
24
Gu Xinyue, Lin T Y, Kuo Weicheng, et al. Open-vocabulary object detection via vision and language knowledge distillation[J]. arXiv preprint, arXiv: 2104.13921, 2021
25
Zhang Hao, Li Feng, Zou Xueyan, et al. A simple framework for open-vocabulary segmentation and detection[C]//Proc of the 19th IEEE/CVF Int Conf on Computer Vision. Los Alamitos, CA: IEEE Computer Society, 2023: 1020−1031
26
He Sunan, Guo Taian, Dai Tao, et al. Open-vocabulary multi-label classification via multi-modal knowledge transfer[C]//Proc of the 37th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2023: 808−816
27
Li Junnan, Li Dongxu, Xiong Caiming, et al. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation[C]//Proc of the 39th Int Conf on Machine Learning. New York: ACM, 2022: 12888−12900
28
Zhou Kaiyang, Yang Jingkang, Loy C C, et al. Learning to prompt for vision-language models[J]. International Journal of Computer Vision, 2022, 130(9): 2337−2348
29
Zhou Kaiyang, Yang Jingkang, Loy C C, et al. Conditional prompt learning for vision-language models[C]//Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2022: 16816−16825
30
Kingma D P, Ba J. Adam: A method for stochastic optimization[J]. arXiv preprint, arXiv: 1412.6980, 2014
31
Ridnik T, Ben-Baruch E, Zamir N, et al. Asymmetric loss for multi-label classification[C]//Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2021: 82−91
32
Zhu Ke, Wu Jianxin. Residual attention: A simple but effective method for multi-label recognition[C]//Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2021: 184−193
33
Lanchantin J, Wang Tianlu, Ordonez V, et al. General multi-label image classification with transformers[C]//Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2021: 16478−16488
2025年第62卷第2期
117
44
0
引用本文
BibTeX
文章信息
doi: 10.7544/issn1000-1239.202440123
  • 接收时间:2024-03-01
  • 首发时间:2026-11-10
  • 出版时间:2025-02-01
补充材料
相关文章
文章信息
作者
出版历史
  • 收稿日期:2024-03-01
  • 修回日期:2024-10-31
  • 录用日期:2024-11-13
基金
作者信息
    1中国矿业大学计算机科学与技术学院 江苏徐州 221116
    2计算机辅助设计与图形系统全国重点实验室 (浙江大学) 杭州 310058
    3矿山数字化教育部工程研究中心 (中国矿业大学) 江苏徐州 221116

通讯作者:

许新征()
参考文献
分享链接
https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202440123
分享至
全文二维码

扫描看全文

引用本文
BibTeX
本文的引用情况
2种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏