Article(id=1329460769680372512, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1329460564176253901, articleNumber=null, orderNo=null, doi=10.7544/issn1000-1239.202330433, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=null, receivedDate=1685894400000, receivedDateStr=2023-06-05, revisedDate=1702915200000, revisedDateStr=2023-12-19, acceptedDate=null, acceptedDateStr=null, onlineDate=1794897747678, onlineDateStr=2026-11-17, pubDate=1730390400000, pubDateStr=2024-11-01, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1794897747678, onlineIssueDateStr=2026-11-17, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1794897747678, creator=13701087609, updateTime=1794897747678, updator=13701087609, issue=Issue{id=1329460564176253901, tenantId=1146029695717560320, journalId=1324759914146529478, year='2024', volume='61', issue='11', pageStart='2663', pageEnd='2936', issueExtLink='null', onlineDate='null', pubDate='1730390400000', pubDateStr='2024-11-01', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=1, issueType=-1, specialIssue=null, createTime=1794897698682, creator='13701087609', updateTime=1794897698682, updator='13701087609', preIssue=null, nextIssue=null, articleTotal=null, ext=null, issueFiles=null, downloadFileDto=null}, startPage=2923, endPage=2936, ext={EN=ArticleExt(id=1329460769927836449, articleId=1329460769680372512, tenantId=1146029695717560320, journalId=1324759914146529478, language=EN, title=Online Cross-Modal Hashing with Double Structure Preserving, columnId=1325489891789910943, journalTitle=Journal of Computer Research and Development, columnName=Artificial Intelligence, runingTitle=null, highlight=null, articleAbstract=

Online cross-modal hashing has received increasing attention due to its efficiency and effectiveness in handling cross-modal streaming data retrieval. Despite promising progress, most existing methods rely on accurate and clear supervised information. There are only a few online cross-modal hashing studies concentrating on unsupervised learning mode, and numerous challenges still need to be tackled. For example, streaming data usually suffer from the unbalanced distribution problem due to the limited volume of data in each chunk. Most existing methods neglect this problem, resulting in heightened sensitivity to outlier samples and compromised robustness. Moreover, existing models typically exploit global data distribution, while ignoring local neighborhood information that can promote hash learning. To solve these problems, we propose an unsupervised online cross-modal hashing method with double structure-preserving, called SPOCH(structure preserving online cross-modal hashing). It simultaneously explores the global structure and local structure to generate the corresponding common representation; thereafter, the learned common representation can be used to guide the hash learning process. In terms of global structure-preserving, we design the loss function based on $ {L}_{2,1} $ norm, which can alleviate the sensitivity of outlier samples. In terms of local structure-preserving, we reconstruct sample representation based on neighbor relations that integrates the multi-modality information. In addition, to alleviate the forgetting problem, we propose joint optimization on streaming data, and design the corresponding update strategy to improve the training efficiency. We conduct experiments on two widely-used cross-modal retrieval datasets. Compared with the existing state-of-the-art unsupervised online cross-modal hashing methods, SPOCH achieves superior retrieval accuracy within a comparable or even shorter training time, validating the effectiveness of the proposed approach.

, authors=Xiao Kang1, Xingbo Liu2, *, Pengyu Lu2, Zhijie Zhao2, Xiushan Nie2, Shaohua Wang2, Yilong Yin1, authorsList=Xiao Kang, Xingbo Liu, Pengyu Lu, Zhijie Zhao, Xiushan Nie, Shaohua Wang, Yilong Yin, authorCompany=null, correspAuthors=Xingbo Liu, authorNote=

born in 1997. PhD. Her main research interests include multimedia retrieval and computer vision

born in 1995. PhD, professor. His main research interests include multimedia retrieval and computer vision

born in 1998. PhD candidate. Her main research interests include machine learning and artificial neural network. ()

born in 1999. Master candidate. Her main research interests include multimedia retrieval and computer vision. ()

born in 1981. PhD, professor. His main research interests include multimedia retrieval, multimedia security, and computer vision. ()

born in 1979. PhD, senior engineer. Member of CCF. Her main research interests include multimedia retrieval and computer vision. ()

born in 1972. PhD, professor, PhD supervisor. His main research interests include machine learning and data mining. ()

, correspAuthorsNote=null, copyrightStatement=Copyright © 2024 Journal of Computer Research and Development. All rights reserved., copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=null, magXml=null, pdfUrl=null, pdf=null, pdfFileSize=null, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=null, mapNumber=null, fund=null), CN=ArticleExt(id=1329460772687688511, articleId=1329460769680372512, tenantId=1146029695717560320, journalId=1324759914146529478, language=CN, title=双重结构保持的在线跨模态哈希, columnId=1325489892104483745, journalTitle=计算机研究与发展, columnName=人工智能, runingTitle=null, highlight=null, articleAbstract=

近年来,在线跨模态哈希因其能处理更为贴近现实的流数据场景而受到广泛关注. 虽然取得了不错的进展,但现有方法大都依赖准确清晰的数据标记. 目前,针对无监督学习模式下在线跨模态哈希的研究相对较少,还有很多问题有待解决. 例如,新到达的数据流通常规模较小,因此常常存在分布不平衡的现象. 而现有模型极少关注这个问题,导致模型对离群样本敏感,鲁棒性较差. 并且,现有方法大都关注样本的整体结构而忽视了邻域信息对于生成公共哈希码的帮助. 为了解决上述问题,提出了基于双重结构保持的无监督在线跨模态哈希方法,称为SPOCH(structure preserving online cross-modal hashing). 该方法的基本思想是同时挖掘样本空间的全局结构信息和邻域结构信息来生成相应的公共表示,用以指导哈希码和哈希函数的学习. 针对全局结构的学习,引入$ {L_{2,1}} $范数取代$ {L_2} $范数来约束损失函数,利用$ {L_{2,1}} $范数结构化稀疏的性质缓解模型对离群样本的敏感性;针对邻域结构的学习,利用多模态融合的邻域样本进行样本重构,使得所学公共表示更好地表征多模态信息. 此外,为了缓解遗忘问题,提出在新旧数据上联合优化,并设计相应的更新策略提高算法的训练效率,实现在线检索. 在2个广泛使用的跨模态检索数据集上进行的实验结果表明,较现有最先进的无监督在线跨模态哈希方法,SPOCH在可比较甚至更短的训练时间内取得了更优的检索精度,验证了所提方法的有效性.

, authors=康潇1, 刘兴波2, *, 卢鹏宇2, 赵志杰2, 聂秀山2, 王少华2, 尹义龙1, authorsList=康潇, 刘兴波, 卢鹏宇, 赵志杰, 聂秀山, 王少华, 尹义龙, authorCompany=null, correspAuthors=刘兴波, authorNote=

1997年生. 博士. 主要研究方向为多媒体检索、计算机视觉

1995年生. 博士,教授. 主要研究方向为多媒体检索、计算机视觉

1998年生. 博士研究生. 主要研究方向为机器学习、人工神经网络

1999年生. 硕士研究生. 主要研究方向为多媒体检索、计算机视觉

1981年生. 博士,教授. 主要研究方向为多媒体检索、多媒体安全、计算机视觉

1979年生. 博士,高级工程师. CCF会员. 主要研究方向为多媒体检索、计算机视觉

1972年生. 博士,教授,博士生导师. 主要研究方向为机器学习、数据挖掘

, correspAuthorsNote=
刘兴波()
, copyrightStatement=版权所有 © 2024 计算机研究与发展编辑部, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=66ouuHgjmfBS6kATugpoIg==, magXml=9kzjt5pkTxsWNYqeg+b+4g==, pdfUrl=null, pdf=Ety94LtXXyUTX/g3QNMZCQ==, pdfFileSize=1678038, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=P6rFt6fk/wNgc7aM3ZwU2g==, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=dhL06TMswIogVD7fXLkvgw==, mapNumber=null, fund=null)}, authors=[Author(id=1329460773065175879, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=sckx@mail.sdu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329460774671594313, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460773065175879, language=EN, stringName=Xiao Kang, firstName=Xiao, middleName=null, lastName=Kang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1School of Software, Shandong University, Jinan 250101, bio={"content":"

born in 1997. PhD. Her main research interests include multimedia retrieval and computer vision

"}, bioImg=null, bioContent=

born in 1997. PhD. Her main research interests include multimedia retrieval and computer vision

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329460774747091786, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460773065175879, language=CN, stringName=康潇, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1山东大学软件学院 济南 250101, bio={"img":"SquQSx655zSr1S7XsUfw2g==","content":"

1997年生. 博士. 主要研究方向为多媒体检索、计算机视觉

"}, bioImg=SquQSx655zSr1S7XsUfw2g==, bioContent=

1997年生. 博士. 主要研究方向为多媒体检索、计算机视觉

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329460772884820800, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, xref=1, ext=[AuthorCompanyExt(id=1329460772893209409, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772884820800, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Software, Shandong University, Jinan 250101), AuthorCompanyExt(id=1329460772901598018, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772884820800, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1山东大学软件学院 济南 250101)])]), Author(id=1329460774810006348, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=sclxb@mail.sdu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1329460774889698126, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460774810006348, language=EN, stringName=Xingbo Liu, firstName=Xingbo, middleName=null, lastName=Liu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, *, address=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101, bio={"content":"

born in 1995. PhD, professor. His main research interests include multimedia retrieval and computer vision

"}, bioImg=null, bioContent=

born in 1995. PhD, professor. His main research interests include multimedia retrieval and computer vision

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329460774948418383, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460774810006348, language=CN, stringName=刘兴波, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, *, address=2山东建筑大学计算机科学与技术学院 济南 250101, bio={"img":"15tEwcC9XgZf0hSW6CZDrw==","content":"

1995年生. 博士,教授. 主要研究方向为多媒体检索、计算机视觉

"}, bioImg=15tEwcC9XgZf0hSW6CZDrw==, bioContent=

1995年生. 博士,教授. 主要研究方向为多媒体检索、计算机视觉

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329460772981289795, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, xref=2, ext=[AuthorCompanyExt(id=1329460772985484100, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101), AuthorCompanyExt(id=1329460772993872709, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2山东建筑大学计算机科学与技术学院 济南 250101)])]), Author(id=1329460775049081681, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=periispengyu@163.com, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329460775132967763, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460775049081681, language=EN, stringName=Pengyu Lu, firstName=Pengyu, middleName=null, lastName=Lu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101, bio={"content":"

born in 1998. PhD candidate. Her main research interests include machine learning and artificial neural network. ()

"}, bioImg=null, bioContent=

born in 1998. PhD candidate. Her main research interests include machine learning and artificial neural network. ()

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329460775200076628, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460775049081681, language=CN, stringName=卢鹏宇, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2山东建筑大学计算机科学与技术学院 济南 250101, bio={"img":"irtH+HFs0u7qIfB2gv7k5Q==","content":"

1998年生. 博士研究生. 主要研究方向为机器学习、人工神经网络

"}, bioImg=irtH+HFs0u7qIfB2gv7k5Q==, bioContent=

1998年生. 博士研究生. 主要研究方向为机器学习、人工神经网络

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329460772981289795, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, xref=2, ext=[AuthorCompanyExt(id=1329460772985484100, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101), AuthorCompanyExt(id=1329460772993872709, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2山东建筑大学计算机科学与技术学院 济南 250101)])]), Author(id=1329460775267185494, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=zhaozhijie_01@163.com, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329460775372043096, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460775267185494, language=EN, stringName=Zhijie Zhao, firstName=Zhijie, middleName=null, lastName=Zhao, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101, bio={"content":"

born in 1999. Master candidate. Her main research interests include multimedia retrieval and computer vision. ()

"}, bioImg=null, bioContent=

born in 1999. Master candidate. Her main research interests include multimedia retrieval and computer vision. ()

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329460775426569049, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460775267185494, language=CN, stringName=赵志杰, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2山东建筑大学计算机科学与技术学院 济南 250101, bio={"img":"S2tA8yYlKbb4AbG8hr9Qeg==","content":"

1999年生. 硕士研究生. 主要研究方向为多媒体检索、计算机视觉

"}, bioImg=S2tA8yYlKbb4AbG8hr9Qeg==, bioContent=

1999年生. 硕士研究生. 主要研究方向为多媒体检索、计算机视觉

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329460772981289795, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, xref=2, ext=[AuthorCompanyExt(id=1329460772985484100, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101), AuthorCompanyExt(id=1329460772993872709, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2山东建筑大学计算机科学与技术学院 济南 250101)])]), Author(id=1329460775485289307, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=niexiushan@163.com, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329460775564981085, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460775485289307, language=EN, stringName=Xiushan Nie, firstName=Xiushan, middleName=null, lastName=Nie, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101, bio={"content":"

born in 1981. PhD, professor. His main research interests include multimedia retrieval, multimedia security, and computer vision. ()

"}, bioImg=null, bioContent=

born in 1981. PhD, professor. His main research interests include multimedia retrieval, multimedia security, and computer vision. ()

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329460775665644382, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460775485289307, language=CN, stringName=聂秀山, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2山东建筑大学计算机科学与技术学院 济南 250101, bio={"img":"Mfz7A8PQK4as/7fYYJl2ug==","content":"

1981年生. 博士,教授. 主要研究方向为多媒体检索、多媒体安全、计算机视觉

"}, bioImg=Mfz7A8PQK4as/7fYYJl2ug==, bioContent=

1981年生. 博士,教授. 主要研究方向为多媒体检索、多媒体安全、计算机视觉

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329460772981289795, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, xref=2, ext=[AuthorCompanyExt(id=1329460772985484100, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101), AuthorCompanyExt(id=1329460772993872709, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2山东建筑大学计算机科学与技术学院 济南 250101)])]), Author(id=1329460775745336160, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, orderNo=5, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=vision.wsh@sdjzu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329460775825027938, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460775745336160, language=EN, stringName=Shaohua Wang, firstName=Shaohua, middleName=null, lastName=Wang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101, bio={"content":"

born in 1979. PhD, senior engineer. Member of CCF. Her main research interests include multimedia retrieval and computer vision. ()

"}, bioImg=null, bioContent=

born in 1979. PhD, senior engineer. Member of CCF. Her main research interests include multimedia retrieval and computer vision. ()

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329460775896331107, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460775745336160, language=CN, stringName=王少华, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2山东建筑大学计算机科学与技术学院 济南 250101, bio={"img":"sChZGZDyPkJfBX17lVzlUw==","content":"

1979年生. 博士,高级工程师. CCF会员. 主要研究方向为多媒体检索、计算机视觉

"}, bioImg=sChZGZDyPkJfBX17lVzlUw==, bioContent=

1979年生. 博士,高级工程师. CCF会员. 主要研究方向为多媒体检索、计算机视觉

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329460772981289795, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, xref=2, ext=[AuthorCompanyExt(id=1329460772985484100, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101), AuthorCompanyExt(id=1329460772993872709, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2山东建筑大学计算机科学与技术学院 济南 250101)])]), Author(id=1329460775963439973, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, orderNo=6, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=mining.ylyin@sdu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329460776030548839, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460775963439973, language=EN, stringName=Yilong Yin, firstName=Yilong, middleName=null, lastName=Yin, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1School of Software, Shandong University, Jinan 250101, bio={"content":"

born in 1972. PhD, professor, PhD supervisor. His main research interests include machine learning and data mining. ()

"}, bioImg=null, bioContent=

born in 1972. PhD, professor, PhD supervisor. His main research interests include machine learning and data mining. ()

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329460776106046312, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460775963439973, language=CN, stringName=尹义龙, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1山东大学软件学院 济南 250101, bio={"img":"aQ/aL9vXhSwuErMTVIRccQ==","content":"

1972年生. 博士,教授,博士生导师. 主要研究方向为机器学习、数据挖掘

"}, bioImg=aQ/aL9vXhSwuErMTVIRccQ==, bioContent=

1972年生. 博士,教授,博士生导师. 主要研究方向为机器学习、数据挖掘

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329460772884820800, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, xref=1, ext=[AuthorCompanyExt(id=1329460772893209409, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772884820800, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Software, Shandong University, Jinan 250101), AuthorCompanyExt(id=1329460772901598018, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772884820800, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1山东大学软件学院 济南 250101)])])], keywords=[Keyword(id=1329460776206709609, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=EN, orderNo=1, keyword=cross-modal retrieval), Keyword(id=1329460776261235562, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=EN, orderNo=2, keyword=unsupervised learning), Keyword(id=1329460776319955819, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=EN, orderNo=3, keyword=online hashing), Keyword(id=1329460776378676076, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=EN, orderNo=4, keyword=double structure preserving), Keyword(id=1329460776470950765, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=EN, orderNo=5, keyword=robustness), Keyword(id=1329460776529671022, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=CN, orderNo=1, keyword=跨模态检索), Keyword(id=1329460776596779887, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=CN, orderNo=2, keyword=无监督学习), Keyword(id=1329460776655500144, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=CN, orderNo=3, keyword=在线哈希), Keyword(id=1329460776710026097, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=CN, orderNo=4, keyword=双重结构保持), Keyword(id=1329460776768746354, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=CN, orderNo=5, keyword=鲁棒性)], refs=[Reference(id=1329460780119995273, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=1, rfOrder=0, authorNames=null, journalName=null, refType=null, unstructuredReference=Xu Xing, Shen Fumin, Yang Yang, et al. Learning discriminative binary codes for large-scale cross-modal retrieval[J]. IEEE Transactions on Image Processing, 2017, 26(5): 2494−2507, articleTitle=null, refAbstract=null), Reference(id=1329460780199687051, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=2, rfOrder=1, authorNames=null, journalName=null, refType=null, unstructuredReference=Raginsky M, Lazebnik S. Locality-sensitive binary codes from shift-invariant kernels[C]//Proc of the 22nd Int Conf on Neural Information Processing Systems. New York: ACM, 2009, 1509–1517, articleTitle=null, refAbstract=null), Reference(id=1329460780266795915, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=3, rfOrder=2, authorNames=null, journalName=null, refType=null, unstructuredReference=刘兴波,聂秀山,尹义龙. 基于双向线性回归的监督离散跨模态散列方法[J]. 计算机研究与发展,2020,57(8):1707−1714, articleTitle=null, refAbstract=null), Reference(id=1329460780317127564, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=3, rfOrder=3, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Xingbo, Nie Xiushan, Yin Yilong. Supervised discrete cross-modal hashing method based on two-way linear regression[J]. Journal of Computer Research and Development, 2020, 57(8): 1707−1714 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1329460780384236429, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=4, rfOrder=4, authorNames=null, journalName=null, refType=null, unstructuredReference=Zheng Chaoqun, Zhu Lei, Zhang Zheng, et al. Efficient semi-supervised multimodal hashing with importance differentiation regression[J]. IEEE Transactions on Image Processing, 2022, 31: 5881−5892, articleTitle=null, refAbstract=null), Reference(id=1329460780447150990, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=5, rfOrder=5, authorNames=null, journalName=null, refType=null, unstructuredReference=卓君宝,苏驰,王树徽,等. 最小熵迁移对抗散列方法[J]. 计算机研究与发展,2020,57(4):888−896, articleTitle=null, refAbstract=null), Reference(id=1329460780518454159, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=5, rfOrder=6, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhuo Junbao, Su Chi, Wang Shuhui, et al. Least entropy transfer against hashing method[J]. Journal of Computer Research and Development, 2020, 57(4): 888−896 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1329460780572980112, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=6, rfOrder=7, authorNames=null, journalName=null, refType=null, unstructuredReference=Lin Zijia, Ding Guiguang, Hu Mingqing, et al. Semantics-preserving hashing for cross-view retrieval[C]// Proc of the IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2015: 3864−3872, articleTitle=null, refAbstract=null), Reference(id=1329460780644283281, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=7, rfOrder=8, authorNames=null, journalName=null, refType=null, unstructuredReference=Xie Liang, Shen Jialie, Zhu Lei, et al. Online cross-modal hashing for web image retrieval[C]// Proc of the 30th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2016: 294−300, articleTitle=null, refAbstract=null), Reference(id=1329460780715586450, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=8, rfOrder=9, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Di, Wang Quan, An Yaqiang, et al. Online collective matrix factorization hashing for large-scale cross-media retrieval[C]// Proc of the 43rd Int ACM SIGIR Conf on Research and Development in Information Retrieval. New York: ACM, 2020: 1409−1418, articleTitle=null, refAbstract=null), Reference(id=1329460780782695315, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=9, rfOrder=10, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Xuan, Wu Wei, Yuan Yunhao, et al. Online unsupervised cross-view discrete hashing for large-scale retrieval[J]. Applied Intelligence, 2022, 52(13): 14905−14917, articleTitle=null, refAbstract=null), Reference(id=1329460780837221268, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=10, rfOrder=11, authorNames=null, journalName=null, refType=null, unstructuredReference=Yao Tao, Wang Gang, Yan Lianshan, et al. Online latent semantic hashing for cross-media retrieval[J]. Pattern Recognition, 2019, 89: 1−11, articleTitle=null, refAbstract=null), Reference(id=1329460780891747221, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=11, rfOrder=12, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Yongxin, Luo Xin, Xu Xinshun, et al. Label embedding online hashing for cross-modal retrieval[C]// Proc of the 28th ACM Int Conf on Multimedia. New York: ACM, 2020: 871−879, articleTitle=null, refAbstract=null), Reference(id=1329460780954661782, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=12, rfOrder=13, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhan Yuwei, Wang Yongxin, Sun Yu, et al. Discrete online cross-modal hashing[J]. Pattern Recognition, 2022, 122: 108262, articleTitle=null, refAbstract=null), Reference(id=1329460781009187735, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=13, rfOrder=14, authorNames=null, journalName=null, refType=null, unstructuredReference=Lin Mingbao, Ji Rongrong, Liu Hong, et al. Towards optimal discrete online hashing with balanced similarity[C]//Proc of the 33rd AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2019: 8722−8729, articleTitle=null, refAbstract=null), Reference(id=1329460781072102296, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=14, rfOrder=15, authorNames=null, journalName=null, refType=null, unstructuredReference=Cakir F, Sclaroff S. Online supervised hashing[C]// Proc of the IEEE Int Conf on Image Processing. Piscataway, NJ: IEEE, 2015: 2606−2610, articleTitle=null, refAbstract=null), Reference(id=1329460781147599769, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=15, rfOrder=16, authorNames=null, journalName=null, refType=null, unstructuredReference=Lin Mingbao, Ji Rongrong, Liu Hong, et al. Hadamard matrix guided online hashing[J]. International Journal of Computer Vision, 2020, 128(8): 2279−2306, articleTitle=null, refAbstract=null), Reference(id=1329460781210514330, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=16, rfOrder=17, authorNames=null, journalName=null, refType=null, unstructuredReference=Lin Mingbao, Ji Rongrong, Chen Shen, et al. Similarity-preserving linkage hashing for online image retrieval[J]. IEEE Transactions on Image Processing, 2020, 29: 5289−5300, articleTitle=null, refAbstract=null), Reference(id=1329460781273428891, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=17, rfOrder=18, authorNames=null, journalName=null, refType=null, unstructuredReference=Lin Mingbao, Ji Rongrong, Sun Xiaoshuai, et al. Fast class-wise updating for online hashing[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(5): 2453−2467, articleTitle=null, refAbstract=null), Reference(id=1329460781332149148, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=18, rfOrder=19, authorNames=null, journalName=null, refType=null, unstructuredReference=Fang Yuzhi, Liu Li. Scalable supervised online hashing for image retrieval[J]. Journal of Computational Design and Engineering, 2021, 8(5): 1391−1406, articleTitle=null, refAbstract=null), Reference(id=1329460781386675101, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=19, rfOrder=20, authorNames=null, journalName=null, refType=null, unstructuredReference=Hao Yanbin, Mu Tingting, Goulermas J, et al. Unsupervised t-distributed video hashing and its deep hashing extension[J]. IEEE Transactions on Image Processing, 2017, 26(11): 5531−5544, articleTitle=null, refAbstract=null), Reference(id=1329460781470561182, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=20, rfOrder=21, authorNames=null, journalName=null, refType=null, unstructuredReference=Huang Longkai, Yang Qiang, Zheng Weishi. Online hashing[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(6): 2309−2322, articleTitle=null, refAbstract=null), Reference(id=1329460781533475743, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=21, rfOrder=22, authorNames=null, journalName=null, refType=null, unstructuredReference=Leng Cong, Wu Jiaxiang, Cheng Jian, et al. Online sketching hashing[C]// Proc of the IEEE Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2015: 2503−2511, articleTitle=null, refAbstract=null), Reference(id=1329460781588001696, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=22, rfOrder=23, authorNames=null, journalName=null, refType=null, unstructuredReference=Chen Xixian, King I, Lyu M. FROSH: Faster online sketching hashing[C/OL]// Proc of the 30th Conf on Uncertainty in Artificial Intelligence. Corvallis: AUAI, 2017[2023-11-21]. http://auai.org/uai2017/proceedings/papers/12.pdf, articleTitle=null, refAbstract=null), Reference(id=1329460781655110561, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=23, rfOrder=24, authorNames=null, journalName=null, refType=null, unstructuredReference=Chen Xixian, Yang Haiqin, Zhao Shenglin, et al. Making online sketching hashing even faster[J]. IEEE Transactions on Knowledge and Data Engineering, 2021, 33(3): 1089−1101, articleTitle=null, refAbstract=null), Reference(id=1329460781726413730, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=24, rfOrder=25, authorNames=null, journalName=null, refType=null, unstructuredReference=Ding Guiguang, Guo Yuchen, Zhou Jile. Collective matrix factorization hashing for multimodal data[C]//Proc of the IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2014: 2083−2090, articleTitle=null, refAbstract=null), Reference(id=1329460781789328291, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=25, rfOrder=26, authorNames=null, journalName=null, refType=null, unstructuredReference=Sun Yuan, Wang Xu , Peng Dezhong, et al. Hierarchical hashing learning for image set classification[J]. IEEE Transactions on Image Processing, 2023, 32: 1732−1744, articleTitle=null, refAbstract=null), Reference(id=1329460781873214372, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=26, rfOrder=27, authorNames=null, journalName=null, refType=null, unstructuredReference=Hoerl A, Kennard R. Ridge regression: Applications to nonorthogonal problems[J]. Technometrics, 1970, 12(1): 69−82, articleTitle=null, refAbstract=null), Reference(id=1329460781931934629, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=27, rfOrder=28, authorNames=null, journalName=null, refType=null, unstructuredReference=Roweis T, Saul K. Nonlinear dimensionality reduction by locally linear embedding[J]. Science, 2000, 290(5500): 2323−2326, articleTitle=null, refAbstract=null), Reference(id=1329460781990654886, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=28, rfOrder=29, authorNames=null, journalName=null, refType=null, unstructuredReference=He Xiaofei, Cai Deng, Yan Shuicheng, et al. Neighborhood preserving embedding[C]// Proc of the 10th IEEE Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2005: 1208−1213, articleTitle=null, refAbstract=null), Reference(id=1329460783597073319, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=29, rfOrder=30, authorNames=null, journalName=null, refType=null, unstructuredReference=Escalante H, Hernandez C, Gonzalez J, et al. The segmented and annotated IAPRTC-12 benchmark[J]. Computer Vision and Image Understanding, 2010, 144(4): 419−428, articleTitle=null, refAbstract=null), Reference(id=1329460783680959400, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=30, rfOrder=31, authorNames=null, journalName=null, refType=null, unstructuredReference=Huiskes M , Lew M. The MIR flickr retrieval evaluation[C]// Proc of the 1st ACM SIGMM Int Conf on Multimedia Information Retrieval. New York: ACM, 2008: 9−43, articleTitle=null, refAbstract=null), Reference(id=1329460783748068265, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=31, rfOrder=32, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Di, Wang Quan, Gao Xinbo. Robust and flexible discrete hashing for cross-modal similarity search[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2018, 28(10): 2703−2715, articleTitle=null, refAbstract=null), Reference(id=1329460783911646122, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=32, rfOrder=33, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Di, Wang Quan, He Lihuo, et al. Joint and individual matrix factorization hashing for large-scale cross-modal retrieval[J]. Pattern Recognition, 2020, 107: 107479, articleTitle=null, refAbstract=null), Reference(id=1329460783982949291, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=33, rfOrder=34, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang, Di, Caiping Zhang, Quan Wang, et al. Hierarchical semantic structure preserving hashing for cross-modal retrieval[J]. IEEE Transactions on Multimedia, 2023, 25: 1217−1229, articleTitle=null, refAbstract=null), Reference(id=1329460784079418284, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=34, rfOrder=35, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Xin, Yi Jinhan, Cheung Yiuming, et al. OMGHUN: Online manifold-guided hashing for flexible cross-modal retrieval[J]. IEEE Transactions on Multimedia, 2023, 25: 3811−3824, articleTitle=null, refAbstract=null)], funds=null, companyList=[AuthorCompany(id=1329460772884820800, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, xref=1, ext=[AuthorCompanyExt(id=1329460772893209409, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772884820800, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Software, Shandong University, Jinan 250101), AuthorCompanyExt(id=1329460772901598018, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772884820800, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1山东大学软件学院 济南 250101)]), AuthorCompany(id=1329460772981289795, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, xref=2, ext=[AuthorCompanyExt(id=1329460772985484100, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101), AuthorCompanyExt(id=1329460772993872709, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2山东建筑大学计算机科学与技术学院 济南 250101)])], figs=[ArticleFig(id=1329460776877798259, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=EN, label=Figure 1, caption=Framework of SPOCH, figureFileSmall=SSN1+NF2Isa9lfCS1z0Y4A==, figureFileBig=P6rFt6fk/wNgc7aM3ZwU2g==, tableContent=null), ArticleFig(id=1329460776940712820, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=CN, label=图1, caption=SPOCH框架, figureFileSmall=SSN1+NF2Isa9lfCS1z0Y4A==, figureFileBig=P6rFt6fk/wNgc7aM3ZwU2g==, tableContent=null), ArticleFig(id=1329460777112679285, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=EN, label=Figure 2, caption=mAP@all varies with chunks, figureFileSmall=MHtwiJJxAZ6dPfLVVdFjLQ==, figureFileBig=7dpmSxl3+NwEDvTY6FIZxw==, tableContent=null), ArticleFig(id=1329460777183982454, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=CN, label=图2, caption=mAP@all 随数据块的变化, figureFileSmall=MHtwiJJxAZ6dPfLVVdFjLQ==, figureFileBig=7dpmSxl3+NwEDvTY6FIZxw==, tableContent=null), ArticleFig(id=1329460777246897015, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=EN, label=Figure 3, caption=Precision@100 varies with chunks, figureFileSmall=6CvFjkecJSbCYiWmW4l/RA==, figureFileBig=l0/ypIV+zHEvPuZq/GxGAw==, tableContent=null), ArticleFig(id=1329460777309811576, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=CN, label=图3, caption=Precision@100 随数据块的变化, figureFileSmall=6CvFjkecJSbCYiWmW4l/RA==, figureFileBig=l0/ypIV+zHEvPuZq/GxGAw==, tableContent=null), ArticleFig(id=1329460777389503353, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=EN, label=Figure 4, caption=Parameter sensitivity analysis result of $ {\alpha _m} $ on MIRFLICKR dataset, figureFileSmall=jvf09Bm8pl966+6cAxoBWg==, figureFileBig=EzDcqO2fOyxGtcyUXSeKWw==, tableContent=null), ArticleFig(id=1329460777460806522, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=CN, label=图4, caption=MIRFLICKR数据集上$ {\alpha _m} $参数的敏感性分析结果, figureFileSmall=jvf09Bm8pl966+6cAxoBWg==, figureFileBig=EzDcqO2fOyxGtcyUXSeKWw==, tableContent=null), ArticleFig(id=1329460777532109691, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=EN, label=Figure 5, caption=Parameter sensitivity analysis result of ${\beta _m}$ on MIRFLICKR dataset, figureFileSmall=a5LtGW4ZMerChKamUBG1Vg==, figureFileBig=M4O0m6PqhwzwzdFp7zMmkg==, tableContent=null), ArticleFig(id=1329460779121750908, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=CN, label=图5, caption=MIRFLICKR数据集上${\beta _m}$参数的敏感性分析结果, figureFileSmall=a5LtGW4ZMerChKamUBG1Vg==, figureFileBig=M4O0m6PqhwzwzdFp7zMmkg==, tableContent=null), ArticleFig(id=1329460779193054077, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=EN, label=Figure 6, caption=Parameter sensitivity analysis result of k and c on MIRFLICKR dataset, figureFileSmall=sEtx127GIk2oFFGv1Yxyaw==, figureFileBig=plLOJ1OiwBQ6u8SrXGkQlw==, tableContent=null), ArticleFig(id=1329460779247580030, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=CN, label=图6, caption=MIRFLICKR数据集上k和c参数的敏感性分析结果, figureFileSmall=sEtx127GIk2oFFGv1Yxyaw==, figureFileBig=plLOJ1OiwBQ6u8SrXGkQlw==, tableContent=null), ArticleFig(id=1329460779314688895, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=EN, label=Figure 7, caption=Training time varies with chunks on two datasets, figureFileSmall=mzZZQNc8JSJYcsSh9GVr6w==, figureFileBig=9tfE1/7/ADiR6yCxeUgw5A==, tableContent=null), ArticleFig(id=1329460779377603456, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=CN, label=图7, caption=在2个数据集上训练时间随数据块的变化, figureFileSmall=mzZZQNc8JSJYcsSh9GVr6w==, figureFileBig=9tfE1/7/ADiR6yCxeUgw5A==, tableContent=null), ArticleFig(id=1329460779448906625, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=EN, label=Table 1, caption=

Experimental Results Comparison of mAP@all and Standard Deviation on IAPRTC-12 Dataset

, figureFileSmall=null, figureFileBig=null, tableContent=
检索任务对比方法哈希码长
8b16b32b64b128b
图像检索文本CMFH0.3148±0.00280.3158±0.00360.3077±0.00220.3115±0.00200.3151±0.0028
RFDH0.3474±0.00300.3516±0.00310.3535±0.00590.3603±0.00720.3690±0.0037
JIMFH0.3224±0.00870.3317±0.00550.3367±0.00490.3444±0.00280.3482±0.0043
HSSPH0.3514±0.00780.3521±0.00670.3542±0.01010.3645±0.00720.3688±0.0088
OCMH0.3039±0.00500.3025±0.00270.3012±0.00180.3019±0.00170.3017±0.0018
OCMFH0.3007±0.00390.3006±0.00320.2989±0.00260.2983±0.00330.2984±0.0040
OUCDH0.3030±0.00400.3010±0.00320.3005±0.00310.3021±0.00500.3066±0.0034
OMGHUN0.3023±0.00660.3022±0.00660.3025±0.00640.3023±0.00650.3024±0.0063
SPOCH(本文)0.3526±0.00970.3568±0.01400.3619±0.01060.3785±0.00600.3788±0.0060
文本检索图像CMFH0.3760±0.00450.3882±0.00430.4025±0.00530.4118±0.00410.4176±0.0043
RFDH0.3477±0.00210.3560±0.00240.3646±0.00360.3731±0.00420.3838±0.0034
JIMFH0.3602±0.00570.3691±0.00170.3779±0.00250.3939±0.00320.4062±0.0036
HSSPH0.3574±0.00680.3568±0.00580.3615±0.00990.3603±0.00720.3690±0.0037
OCMH0.3010±0.01080.3031±0.00610.3029±0.00330.3007±0.00180.3037±0.0038
OCMFH0.3377±0.00420.3369±0.00360.3331±0.00320.3303±0.00200.3270±0.0029
OUCDH0.3027±0.00400.3029±0.00340.3005±0.00320.3021±0.00470.3027±0.0039
OMGHUN0.3023±0.00670.3022±0.00600.3024±0.00620.3023±0.00650.3026±0.0060
SPOCH(本文)0.3609±0.0099 0.3650±0.01510.3762±0.01300.3978±0.00730.4027±0.0074
), ArticleFig(id=1329460779528598402, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=CN, label=表1, caption=

IAPRTC-12数据集上mAP@all和标准差的实验结果对比

, figureFileSmall=null, figureFileBig=null, tableContent=
检索任务对比方法哈希码长
8b16b32b64b128b
图像检索文本CMFH0.3148±0.00280.3158±0.00360.3077±0.00220.3115±0.00200.3151±0.0028
RFDH0.3474±0.00300.3516±0.00310.3535±0.00590.3603±0.00720.3690±0.0037
JIMFH0.3224±0.00870.3317±0.00550.3367±0.00490.3444±0.00280.3482±0.0043
HSSPH0.3514±0.00780.3521±0.00670.3542±0.01010.3645±0.00720.3688±0.0088
OCMH0.3039±0.00500.3025±0.00270.3012±0.00180.3019±0.00170.3017±0.0018
OCMFH0.3007±0.00390.3006±0.00320.2989±0.00260.2983±0.00330.2984±0.0040
OUCDH0.3030±0.00400.3010±0.00320.3005±0.00310.3021±0.00500.3066±0.0034
OMGHUN0.3023±0.00660.3022±0.00660.3025±0.00640.3023±0.00650.3024±0.0063
SPOCH(本文)0.3526±0.00970.3568±0.01400.3619±0.01060.3785±0.00600.3788±0.0060
文本检索图像CMFH0.3760±0.00450.3882±0.00430.4025±0.00530.4118±0.00410.4176±0.0043
RFDH0.3477±0.00210.3560±0.00240.3646±0.00360.3731±0.00420.3838±0.0034
JIMFH0.3602±0.00570.3691±0.00170.3779±0.00250.3939±0.00320.4062±0.0036
HSSPH0.3574±0.00680.3568±0.00580.3615±0.00990.3603±0.00720.3690±0.0037
OCMH0.3010±0.01080.3031±0.00610.3029±0.00330.3007±0.00180.3037±0.0038
OCMFH0.3377±0.00420.3369±0.00360.3331±0.00320.3303±0.00200.3270±0.0029
OUCDH0.3027±0.00400.3029±0.00340.3005±0.00320.3021±0.00470.3027±0.0039
OMGHUN0.3023±0.00670.3022±0.00600.3024±0.00620.3023±0.00650.3026±0.0060
SPOCH(本文)0.3609±0.0099 0.3650±0.01510.3762±0.01300.3978±0.00730.4027±0.0074
), ArticleFig(id=1329460779612484483, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=EN, label=Table 2, caption=

Experimental Results Comparison of mAP@all and Standard Deviation on MIRFLICKR Dataset

, figureFileSmall=null, figureFileBig=null, tableContent=
检索任务对比方法哈希码长
8b16b32b64b128b
图像检索文本CMFH0.5788±0.00350.5751±0.00650.5742±0.00620.5777±0.00350.5773±0.0050
RFDH0.5592±0.00200.5800±0.00330.5821±0.00250.5840±0.00160.5843±0.0019
JIMFH0.5820±0.00740.5871±0.00710.5882±0.00700.5872±0.00600.5937±0.0069
HSSPH0.5912±0.00670.5961±0.00830.5931±0.00680.5938±0.00990.5969±0.0031
OCMH0.5553±0.00170.5565±0.00150.5551±0.00240.5557±0.00160.5556±0.0017
OCMFH0.5641±0.00450.5618±0.00390.5605±0.00370.5590±0.00330.5586±0.0034
OUCDH0.5522±0.00210.5530±0.00170.5515±0.00220.5516±0.00260.5520±0.0025
OMGHUN0.5536±0.00260.5537±0.00150.5528±0.00110.5530±0.00220.5536±0.0023
SPOCH(本文)0.5638±0.00460.5719±0.00840.5779±0.00970.5877±0.00940.5931±0.0021
文本检索图像CMFH0.5931±0.00200.5933±0.00350.5906±0.00360.5884±0.00430.5847±0.0038
RFDH0.5591±0.00240.5794±0.00350.5847±0.00460.5861±0.00330.5856±0.0024
JIMFH0.6002±0.00660.6028±0.00740.6020±0.00550.6012±0.00340.6067±0.0041
HSSPH0.5913±0.00670.6028±0.00600.6014±0.00620.6034±0.00650.6015±0.0060
OCMH0.5523±0.00300.5552±0.00240.5561±0.00390.5555±0.00170.5555±0.0018
OCMFH0.5604±0.00340.5598±0.00300.5594±0.00350.5590±0.00350.5590±0.0032
OUCDH0.5520±0.00200.5528±0.00160.5517±0.00210.5517±0.00240.5520±0.0024
OMGHUN0.5544±0.00190.5537±0.00180.5534±0.00140.5534±0.00200.5536±0.0023
SPOCH(本文)0.5637±0.00310.5723±0.00860.5804±0.00990.5900±0.00950.5976±0.0029
), ArticleFig(id=1329460779708953476, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=CN, label=表2, caption=

MIRFLICKR数据集上mAP@all和标准差的实验结果对比

, figureFileSmall=null, figureFileBig=null, tableContent=
检索任务对比方法哈希码长
8b16b32b64b128b
图像检索文本CMFH0.5788±0.00350.5751±0.00650.5742±0.00620.5777±0.00350.5773±0.0050
RFDH0.5592±0.00200.5800±0.00330.5821±0.00250.5840±0.00160.5843±0.0019
JIMFH0.5820±0.00740.5871±0.00710.5882±0.00700.5872±0.00600.5937±0.0069
HSSPH0.5912±0.00670.5961±0.00830.5931±0.00680.5938±0.00990.5969±0.0031
OCMH0.5553±0.00170.5565±0.00150.5551±0.00240.5557±0.00160.5556±0.0017
OCMFH0.5641±0.00450.5618±0.00390.5605±0.00370.5590±0.00330.5586±0.0034
OUCDH0.5522±0.00210.5530±0.00170.5515±0.00220.5516±0.00260.5520±0.0025
OMGHUN0.5536±0.00260.5537±0.00150.5528±0.00110.5530±0.00220.5536±0.0023
SPOCH(本文)0.5638±0.00460.5719±0.00840.5779±0.00970.5877±0.00940.5931±0.0021
文本检索图像CMFH0.5931±0.00200.5933±0.00350.5906±0.00360.5884±0.00430.5847±0.0038
RFDH0.5591±0.00240.5794±0.00350.5847±0.00460.5861±0.00330.5856±0.0024
JIMFH0.6002±0.00660.6028±0.00740.6020±0.00550.6012±0.00340.6067±0.0041
HSSPH0.5913±0.00670.6028±0.00600.6014±0.00620.6034±0.00650.6015±0.0060
OCMH0.5523±0.00300.5552±0.00240.5561±0.00390.5555±0.00170.5555±0.0018
OCMFH0.5604±0.00340.5598±0.00300.5594±0.00350.5590±0.00350.5590±0.0032
OUCDH0.5520±0.00200.5528±0.00160.5517±0.00210.5517±0.00240.5520±0.0024
OMGHUN0.5544±0.00190.5537±0.00180.5534±0.00140.5534±0.00200.5536±0.0023
SPOCH(本文)0.5637±0.00310.5723±0.00860.5804±0.00990.5900±0.00950.5976±0.0029
), ArticleFig(id=1329460779788645253, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=EN, label=Table 3, caption=

Ablation Study Results (mAP@100 and Standard Deviation) on IAPRTC-12 Dataset

, figureFileSmall=null, figureFileBig=null, tableContent=
检索任务对比方法哈希码长
8b16b32b64b128b
图像检索文本SPOCH-f0.3463±0.00900.3505±0.00840.3451±0.01590.3675±0.01030.3742±0.0070
SPOCH-v0.3145±0.02100.3494±0.01470.3772±0.00310.3702±0.00390.3805±0.0042
SPOCH-y0.3286±0.01590.3201±0.02120.3710±0.00650.3776±0.00380.3714±0.0059
SPOCH-w0.3079±0.00390.3086±0.00070.3059±0.00390.3112±0.00340.3101±0.0049
SPOCH(本文)0.3526±0.00970.3568±0.01400.3619±0.01060.3785±0.00600.3788±0.0060
文本检索图像SPOCH-f0.3503±0.01170.3602±0.01010.3627±0.01550.3870±0.01070.3984±0.0083
SPOCH-v0.3268±0.01810.3595±0.01370.3922±0.00360.3883±0.00610.4082±0.0044
SPOCH-y0.3374±0.01730.3266±0.02080.3845±0.00890.3955±0.00600.3949±0.0062
SPOCH-w0.3058±0.00160.3075±0.00160.3077±0.00310.3149±0.00360.3157±0.0052
SPOCH(本文)0.3609±0.00990.3650±0.01510.3762±0.01300.3978±0.00730.4027±0.0074
), ArticleFig(id=1329460779872531334, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=CN, label=表3, caption=

IAPRTC-12数据集上模型消融实验(mAP@100和标准差)结果

, figureFileSmall=null, figureFileBig=null, tableContent=
检索任务对比方法哈希码长
8b16b32b64b128b
图像检索文本SPOCH-f0.3463±0.00900.3505±0.00840.3451±0.01590.3675±0.01030.3742±0.0070
SPOCH-v0.3145±0.02100.3494±0.01470.3772±0.00310.3702±0.00390.3805±0.0042
SPOCH-y0.3286±0.01590.3201±0.02120.3710±0.00650.3776±0.00380.3714±0.0059
SPOCH-w0.3079±0.00390.3086±0.00070.3059±0.00390.3112±0.00340.3101±0.0049
SPOCH(本文)0.3526±0.00970.3568±0.01400.3619±0.01060.3785±0.00600.3788±0.0060
文本检索图像SPOCH-f0.3503±0.01170.3602±0.01010.3627±0.01550.3870±0.01070.3984±0.0083
SPOCH-v0.3268±0.01810.3595±0.01370.3922±0.00360.3883±0.00610.4082±0.0044
SPOCH-y0.3374±0.01730.3266±0.02080.3845±0.00890.3955±0.00600.3949±0.0062
SPOCH-w0.3058±0.00160.3075±0.00160.3077±0.00310.3149±0.00360.3157±0.0052
SPOCH(本文)0.3609±0.00990.3650±0.01510.3762±0.01300.3978±0.00730.4027±0.0074
), ArticleFig(id=1329460779939640199, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=EN, label=Table 4, caption=

Ablation Study Results (mAP@100 and Standard Deviation) on MIRFLICKR Dataset

, figureFileSmall=null, figureFileBig=null, tableContent=
检索任务对比方法哈希码长
8b16b32b64b128b
图像检索文本SPOCH-f0.5763±0.00860.5575±0.01830.5763±0.00750.5776±0.00330.5861±0.0017
SPOCH-v0.5584±0.01290.5676±0.01880.5787±0.00560.5889±0.00300.5872±0.0084
SPOCH-y0.5628±0.01800.5895±0.01480.5765±0.00640.5783±0.00350.5861±0.0044
SPOCH-w0.5516±0.00320.5521±0.00380.5581±0.00260.5547±0.00290.5560±0.0028
SPOCH(本文)0.5638±0.00460.5719±0.00840.5779±0.00970.5877±0.00940.5931±0.0021
文本检索图像SPOCH-f0.5777±0.00800.5579±0.01680.5772±0.00790.5820±0.00280.5929±0.0004
SPOCH-v0.5596±0.01170.5687±0.01790.5807±0.00570.5909±0.00430.5930±0.0088
SPOCH-y0.5644±0.01670.5897±0.01330.5799±0.00690.5815±0.00340.5945±0.0049
SPOCH-w0.5530±0.00230.5529±0.00360.5569±0.00180.5560±0.00250.5561±0.0026
SPOCH(本文)0.5637±0.00310.5723±0.00860.5804±0.00990.5900±0.00950.5976±0.0029
), ArticleFig(id=1329460780023526280, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, language=CN, label=表4, caption=

MIRFLICKR数据集上模型消融实验(mAP@100和标准差)结果

, figureFileSmall=null, figureFileBig=null, tableContent=
检索任务对比方法哈希码长
8b16b32b64b128b
图像检索文本SPOCH-f0.5763±0.00860.5575±0.01830.5763±0.00750.5776±0.00330.5861±0.0017
SPOCH-v0.5584±0.01290.5676±0.01880.5787±0.00560.5889±0.00300.5872±0.0084
SPOCH-y0.5628±0.01800.5895±0.01480.5765±0.00640.5783±0.00350.5861±0.0044
SPOCH-w0.5516±0.00320.5521±0.00380.5581±0.00260.5547±0.00290.5560±0.0028
SPOCH(本文)0.5638±0.00460.5719±0.00840.5779±0.00970.5877±0.00940.5931±0.0021
文本检索图像SPOCH-f0.5777±0.00800.5579±0.01680.5772±0.00790.5820±0.00280.5929±0.0004
SPOCH-v0.5596±0.01170.5687±0.01790.5807±0.00570.5909±0.00430.5930±0.0088
SPOCH-y0.5644±0.01670.5897±0.01330.5799±0.00690.5815±0.00340.5945±0.0049
SPOCH-w0.5530±0.00230.5529±0.00360.5569±0.00180.5560±0.00250.5561±0.0026
SPOCH(本文)0.5637±0.00310.5723±0.00860.5804±0.00990.5900±0.00950.5976±0.0029
)], attaches=null, journal=Journal(id=1324759606947315908, delFlag=0, nameCn=计算机研究与发展, nameEn=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, issn=1000-1239, eissn=null, cn=11-1777/TP, coden=null, periodic=0, language=CN, oaType=null, ccby=null, superviseOffice=null, ownerOffice=null, pubOffice=null, editorOffice=null, officeType=null, aims=null, clcCode=null, officeProv=null, officeCity=null, officeAddr=null, officeZip=null, officeEmail=null, officePhone=null, editDirector=null, officeDirector=null, officeDirectorPhone=null, officeStaffNum=null, officeEmpNum=null, coverPicUrl=k2l8zgqwstuxxevFX2qxVA==, journalPrice=null, startedYear=null, abbrevIsoEn=Journal of Computer Research and Development, journalRemark=null, publicationField=null, createdTime=1793776903141, updatedTime=1793850698612, createdBy=18614031015, updatedBy=13701087609, firstLetterCn=J, firstLetterEn=J, subjectCode=Electronic Information, subjectName=null, subjectCodeEn=Electronic Information, subjectNameEn=null, picCn=k2l8zgqwstuxxevFX2qxVA==, picEn=r6k0Ry+RD+c/VXj5QarmDw==, jcr=null, cjcr=null, exts=[JournalExt(id=1325069127710286688, language=CN, name=计算机研究与发展, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698640, updatedTime=1793850698640, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""}), JournalExt(id=1325069127777395553, language=EN, name=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698656, updatedTime=1793850698656, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""})], databaseList=null, tenantJournalId=1324759914146529478, websiteList=[Website(id=1325070300139892771, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN, language=CN, createTime=1793850978169, createBy=18614031015, updateTime=1793851040259, updateBy=18614031015, name=计算机研究与发展-中文, tplId=1146099689490845704, title=计算机研究与发展, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070893751349360, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=articleTextType, value=kx, createTime=1793851119697, updateTime=1793851119697, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893730377837, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=banner, value=null, createTime=1793851119692, updateTime=1793851119692, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893772320883, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=grayFlag, value=0, createTime=1793851119702, updateTime=1793851119702, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893721989228, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851119690, updateTime=1793851119690, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893784903797, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=minRunFlag, value=0, createTime=1793851119705, updateTime=1793851119705, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893742960751, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic, createTime=1793851119695, updateTime=1793851119695, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893780709492, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=silenceFlag, value=0, createTime=1793851119704, updateTime=1793851119704, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893738766446, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_cn_619/, createTime=1793851119694, updateTime=1793851119694, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893755543665, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeColor, value=null, createTime=1793851119698, updateTime=1793851119698, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893763932274, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeStyle, value=null, createTime=1793851119700, updateTime=1793851119700, creator=18614031015, updator=18614031015)]), Website(id=1325070300580294698, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN, language=EN, createTime=1793850978274, createBy=18614031015, updateTime=1793851036638, updateBy=18614031015, name=计算机研究与发展-英文, tplId=1146101810881728533, title=Journal of Computer Research and Development, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070925288321148, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=articleTextType, value=kx, createTime=1793851127216, updateTime=1793851127216, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925263155321, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=banner, value=null, createTime=1793851127210, updateTime=1793851127210, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925309292671, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=grayFlag, value=0, createTime=1793851127221, updateTime=1793851127221, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925254766712, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851127208, updateTime=1793851127208, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925326069889, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=minRunFlag, value=0, createTime=1793851127225, updateTime=1793851127225, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925279932539, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic, createTime=1793851127214, updateTime=1793851127214, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925317681280, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=silenceFlag, value=0, createTime=1793851127223, updateTime=1793851127223, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925271543930, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_en_623/, createTime=1793851127212, updateTime=1793851127212, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925292515453, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeColor, value=null, createTime=1793851127217, updateTime=1793851127217, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925300904062, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeStyle, value=null, createTime=1793851127219, updateTime=1793851127219, creator=18614031015, updator=18614031015)])], journalTitle=计算机研究与发展, weixinUrl=null, journalUrl=https://crad.ict.ac.cn/, iacademicId=null, status=1, seqNo=null, journalTitleEn=Journal of Computer Research and Development, journalPhotoCn=k2l8zgqwstuxxevFX2qxVA==, journalPhotoEn=r6k0Ry+RD+c/VXj5QarmDw==, journalFirstLetter=J, journalRecommend=null, journalNew=null, journalCollection=null, jcrJf=null, cjcrJf=null, jcrJfStr=null, cjcrJfStr=null, submissionFirstDecision=null, sciSubjectClassification=null, casSubjectClassification=null, citeScore=null, totalCitationFrequency=null, icpCode=null, psCode=null, advertisingLicenseCode=null, copyrightInformation=null, country=null, option=, provinceCode=null, provinceName=null, collectFlag=false, interPubPlatform=, interPubPlatformUrl=null), detailUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202330433, detailUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202330433, pdfUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/PDF/10.7544/issn1000-1239.202330433, pdfUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/PDF/10.7544/issn1000-1239.202330433, aliStartDate=0, aliEndDate=0, collectionFlag=false, citedCount=null, citedUrl=null, previewStatus=0, delFlag=0, hasFullText=1, orderTime=1730390400000, fullTextJson=null, articleText=null, reference=null)
收藏切换
双重结构保持的在线跨模态哈希
康潇 1 , 刘兴波 2, * , 卢鹏宇 2 , 赵志杰 2 , 聂秀山 2 , 王少华 2 , 尹义龙 1
计算机研究与发展 | 人工智能 2024,61(11): 2923-2936
收起
收藏切换
计算机研究与发展 |人工智能 2024 , 61 (11) : 2923 -2936
双重结构保持的在线跨模态哈希
全屏
[Author(id=1329460773065175879, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=sckx@mail.sdu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329460774671594313, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460773065175879, language=EN, stringName=Xiao Kang, firstName=Xiao, middleName=null, lastName=Kang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1School of Software, Shandong University, Jinan 250101, bio={"content":"

born in 1997. PhD. Her main research interests include multimedia retrieval and computer vision

"}, bioImg=null, bioContent=

born in 1997. PhD. Her main research interests include multimedia retrieval and computer vision

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329460774747091786, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460773065175879, language=CN, stringName=康潇, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1山东大学软件学院 济南 250101, bio={"img":"SquQSx655zSr1S7XsUfw2g==","content":"

1997年生. 博士. 主要研究方向为多媒体检索、计算机视觉

"}, bioImg=SquQSx655zSr1S7XsUfw2g==, bioContent=

1997年生. 博士. 主要研究方向为多媒体检索、计算机视觉

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329460772884820800, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, xref=1, ext=[AuthorCompanyExt(id=1329460772893209409, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772884820800, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Software, Shandong University, Jinan 250101), AuthorCompanyExt(id=1329460772901598018, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772884820800, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1山东大学软件学院 济南 250101)])]), Author(id=1329460774810006348, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=sclxb@mail.sdu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1329460774889698126, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460774810006348, language=EN, stringName=Xingbo Liu, firstName=Xingbo, middleName=null, lastName=Liu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, *, address=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101, bio={"content":"

born in 1995. PhD, professor. His main research interests include multimedia retrieval and computer vision

"}, bioImg=null, bioContent=

born in 1995. PhD, professor. His main research interests include multimedia retrieval and computer vision

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329460774948418383, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460774810006348, language=CN, stringName=刘兴波, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, *, address=2山东建筑大学计算机科学与技术学院 济南 250101, bio={"img":"15tEwcC9XgZf0hSW6CZDrw==","content":"

1995年生. 博士,教授. 主要研究方向为多媒体检索、计算机视觉

"}, bioImg=15tEwcC9XgZf0hSW6CZDrw==, bioContent=

1995年生. 博士,教授. 主要研究方向为多媒体检索、计算机视觉

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329460772981289795, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, xref=2, ext=[AuthorCompanyExt(id=1329460772985484100, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101), AuthorCompanyExt(id=1329460772993872709, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2山东建筑大学计算机科学与技术学院 济南 250101)])]), Author(id=1329460775049081681, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=periispengyu@163.com, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329460775132967763, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460775049081681, language=EN, stringName=Pengyu Lu, firstName=Pengyu, middleName=null, lastName=Lu, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101, bio={"content":"

born in 1998. PhD candidate. Her main research interests include machine learning and artificial neural network. ()

"}, bioImg=null, bioContent=

born in 1998. PhD candidate. Her main research interests include machine learning and artificial neural network. ()

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329460775200076628, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460775049081681, language=CN, stringName=卢鹏宇, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2山东建筑大学计算机科学与技术学院 济南 250101, bio={"img":"irtH+HFs0u7qIfB2gv7k5Q==","content":"

1998年生. 博士研究生. 主要研究方向为机器学习、人工神经网络

"}, bioImg=irtH+HFs0u7qIfB2gv7k5Q==, bioContent=

1998年生. 博士研究生. 主要研究方向为机器学习、人工神经网络

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329460772981289795, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, xref=2, ext=[AuthorCompanyExt(id=1329460772985484100, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101), AuthorCompanyExt(id=1329460772993872709, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2山东建筑大学计算机科学与技术学院 济南 250101)])]), Author(id=1329460775267185494, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=zhaozhijie_01@163.com, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329460775372043096, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460775267185494, language=EN, stringName=Zhijie Zhao, firstName=Zhijie, middleName=null, lastName=Zhao, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101, bio={"content":"

born in 1999. Master candidate. Her main research interests include multimedia retrieval and computer vision. ()

"}, bioImg=null, bioContent=

born in 1999. Master candidate. Her main research interests include multimedia retrieval and computer vision. ()

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329460775426569049, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460775267185494, language=CN, stringName=赵志杰, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2山东建筑大学计算机科学与技术学院 济南 250101, bio={"img":"S2tA8yYlKbb4AbG8hr9Qeg==","content":"

1999年生. 硕士研究生. 主要研究方向为多媒体检索、计算机视觉

"}, bioImg=S2tA8yYlKbb4AbG8hr9Qeg==, bioContent=

1999年生. 硕士研究生. 主要研究方向为多媒体检索、计算机视觉

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329460772981289795, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, xref=2, ext=[AuthorCompanyExt(id=1329460772985484100, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101), AuthorCompanyExt(id=1329460772993872709, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2山东建筑大学计算机科学与技术学院 济南 250101)])]), Author(id=1329460775485289307, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=niexiushan@163.com, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329460775564981085, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460775485289307, language=EN, stringName=Xiushan Nie, firstName=Xiushan, middleName=null, lastName=Nie, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101, bio={"content":"

born in 1981. PhD, professor. His main research interests include multimedia retrieval, multimedia security, and computer vision. ()

"}, bioImg=null, bioContent=

born in 1981. PhD, professor. His main research interests include multimedia retrieval, multimedia security, and computer vision. ()

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329460775665644382, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460775485289307, language=CN, stringName=聂秀山, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2山东建筑大学计算机科学与技术学院 济南 250101, bio={"img":"Mfz7A8PQK4as/7fYYJl2ug==","content":"

1981年生. 博士,教授. 主要研究方向为多媒体检索、多媒体安全、计算机视觉

"}, bioImg=Mfz7A8PQK4as/7fYYJl2ug==, bioContent=

1981年生. 博士,教授. 主要研究方向为多媒体检索、多媒体安全、计算机视觉

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329460772981289795, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, xref=2, ext=[AuthorCompanyExt(id=1329460772985484100, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101), AuthorCompanyExt(id=1329460772993872709, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2山东建筑大学计算机科学与技术学院 济南 250101)])]), Author(id=1329460775745336160, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, orderNo=5, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=vision.wsh@sdjzu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329460775825027938, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460775745336160, language=EN, stringName=Shaohua Wang, firstName=Shaohua, middleName=null, lastName=Wang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101, bio={"content":"

born in 1979. PhD, senior engineer. Member of CCF. Her main research interests include multimedia retrieval and computer vision. ()

"}, bioImg=null, bioContent=

born in 1979. PhD, senior engineer. Member of CCF. Her main research interests include multimedia retrieval and computer vision. ()

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329460775896331107, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460775745336160, language=CN, stringName=王少华, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2山东建筑大学计算机科学与技术学院 济南 250101, bio={"img":"sChZGZDyPkJfBX17lVzlUw==","content":"

1979年生. 博士,高级工程师. CCF会员. 主要研究方向为多媒体检索、计算机视觉

"}, bioImg=sChZGZDyPkJfBX17lVzlUw==, bioContent=

1979年生. 博士,高级工程师. CCF会员. 主要研究方向为多媒体检索、计算机视觉

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329460772981289795, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, xref=2, ext=[AuthorCompanyExt(id=1329460772985484100, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101), AuthorCompanyExt(id=1329460772993872709, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772981289795, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2山东建筑大学计算机科学与技术学院 济南 250101)])]), Author(id=1329460775963439973, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, orderNo=6, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=mining.ylyin@sdu.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329460776030548839, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460775963439973, language=EN, stringName=Yilong Yin, firstName=Yilong, middleName=null, lastName=Yin, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1School of Software, Shandong University, Jinan 250101, bio={"content":"

born in 1972. PhD, professor, PhD supervisor. His main research interests include machine learning and data mining. ()

"}, bioImg=null, bioContent=

born in 1972. PhD, professor, PhD supervisor. His main research interests include machine learning and data mining. ()

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329460776106046312, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, authorId=1329460775963439973, language=CN, stringName=尹义龙, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1山东大学软件学院 济南 250101, bio={"img":"aQ/aL9vXhSwuErMTVIRccQ==","content":"

1972年生. 博士,教授,博士生导师. 主要研究方向为机器学习、数据挖掘

"}, bioImg=aQ/aL9vXhSwuErMTVIRccQ==, bioContent=

1972年生. 博士,教授,博士生导师. 主要研究方向为机器学习、数据挖掘

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329460772884820800, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, xref=1, ext=[AuthorCompanyExt(id=1329460772893209409, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772884820800, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1School of Software, Shandong University, Jinan 250101), AuthorCompanyExt(id=1329460772901598018, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329460769680372512, companyId=1329460772884820800, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1山东大学软件学院 济南 250101)])])]
康潇1 , 刘兴波2, * , 卢鹏宇2 , 赵志杰2 , 聂秀山2 , 王少华2 , 尹义龙1
作者信息
  • 1山东大学软件学院 济南 250101
  • 2山东建筑大学计算机科学与技术学院 济南 250101
通讯作者:
作者简介:

1997年生. 博士. 主要研究方向为多媒体检索、计算机视觉

1995年生. 博士,教授. 主要研究方向为多媒体检索、计算机视觉

1998年生. 博士研究生. 主要研究方向为机器学习、人工神经网络

1999年生. 硕士研究生. 主要研究方向为多媒体检索、计算机视觉

1981年生. 博士,教授. 主要研究方向为多媒体检索、多媒体安全、计算机视觉

1979年生. 博士,高级工程师. CCF会员. 主要研究方向为多媒体检索、计算机视觉

1972年生. 博士,教授,博士生导师. 主要研究方向为机器学习、数据挖掘

Online Cross-Modal Hashing with Double Structure Preserving
Xiao Kang1 , Xingbo Liu2, * , Pengyu Lu2 , Zhijie Zhao2 , Xiushan Nie2 , Shaohua Wang2 , Yilong Yin1
Affiliations
  • 1School of Software, Shandong University, Jinan 250101
  • 2School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101
  • born in 1997. PhD. Her main research interests include multimedia retrieval and computer vision

    born in 1995. PhD, professor. His main research interests include multimedia retrieval and computer vision

    born in 1998. PhD candidate. Her main research interests include machine learning and artificial neural network. ()

    born in 1999. Master candidate. Her main research interests include multimedia retrieval and computer vision. ()

    born in 1981. PhD, professor. His main research interests include multimedia retrieval, multimedia security, and computer vision. ()

    born in 1979. PhD, senior engineer. Member of CCF. Her main research interests include multimedia retrieval and computer vision. ()

    born in 1972. PhD, professor, PhD supervisor. His main research interests include machine learning and data mining. ()

出版时间: 2024-11-01 doi: 10.7544/issn1000-1239.202330433
文章导航
收藏切换

近年来,在线跨模态哈希因其能处理更为贴近现实的流数据场景而受到广泛关注. 虽然取得了不错的进展,但现有方法大都依赖准确清晰的数据标记. 目前,针对无监督学习模式下在线跨模态哈希的研究相对较少,还有很多问题有待解决. 例如,新到达的数据流通常规模较小,因此常常存在分布不平衡的现象. 而现有模型极少关注这个问题,导致模型对离群样本敏感,鲁棒性较差. 并且,现有方法大都关注样本的整体结构而忽视了邻域信息对于生成公共哈希码的帮助. 为了解决上述问题,提出了基于双重结构保持的无监督在线跨模态哈希方法,称为SPOCH(structure preserving online cross-modal hashing). 该方法的基本思想是同时挖掘样本空间的全局结构信息和邻域结构信息来生成相应的公共表示,用以指导哈希码和哈希函数的学习. 针对全局结构的学习,引入$ {L_{2,1}} $范数取代$ {L_2} $范数来约束损失函数,利用$ {L_{2,1}} $范数结构化稀疏的性质缓解模型对离群样本的敏感性;针对邻域结构的学习,利用多模态融合的邻域样本进行样本重构,使得所学公共表示更好地表征多模态信息. 此外,为了缓解遗忘问题,提出在新旧数据上联合优化,并设计相应的更新策略提高算法的训练效率,实现在线检索. 在2个广泛使用的跨模态检索数据集上进行的实验结果表明,较现有最先进的无监督在线跨模态哈希方法,SPOCH在可比较甚至更短的训练时间内取得了更优的检索精度,验证了所提方法的有效性.

跨模态检索  /  无监督学习  /  在线哈希  /  双重结构保持  /  鲁棒性

Online cross-modal hashing has received increasing attention due to its efficiency and effectiveness in handling cross-modal streaming data retrieval. Despite promising progress, most existing methods rely on accurate and clear supervised information. There are only a few online cross-modal hashing studies concentrating on unsupervised learning mode, and numerous challenges still need to be tackled. For example, streaming data usually suffer from the unbalanced distribution problem due to the limited volume of data in each chunk. Most existing methods neglect this problem, resulting in heightened sensitivity to outlier samples and compromised robustness. Moreover, existing models typically exploit global data distribution, while ignoring local neighborhood information that can promote hash learning. To solve these problems, we propose an unsupervised online cross-modal hashing method with double structure-preserving, called SPOCH(structure preserving online cross-modal hashing). It simultaneously explores the global structure and local structure to generate the corresponding common representation; thereafter, the learned common representation can be used to guide the hash learning process. In terms of global structure-preserving, we design the loss function based on $ {L}_{2,1} $ norm, which can alleviate the sensitivity of outlier samples. In terms of local structure-preserving, we reconstruct sample representation based on neighbor relations that integrates the multi-modality information. In addition, to alleviate the forgetting problem, we propose joint optimization on streaming data, and design the corresponding update strategy to improve the training efficiency. We conduct experiments on two widely-used cross-modal retrieval datasets. Compared with the existing state-of-the-art unsupervised online cross-modal hashing methods, SPOCH achieves superior retrieval accuracy within a comparable or even shorter training time, validating the effectiveness of the proposed approach.

cross-modal retrieval  /  unsupervised learning  /  online hashing  /  double structure preserving  /  robustness
康潇, 刘兴波, 卢鹏宇, 赵志杰, 聂秀山, 王少华, 尹义龙. 双重结构保持的在线跨模态哈希. 计算机研究与发展, 2024 , 61 (11) : 2923 -2936 . DOI: 10.7544/issn1000-1239.202330433
Xiao Kang, Xingbo Liu, Pengyu Lu, Zhijie Zhao, Xiushan Nie, Shaohua Wang, Yilong Yin. Online Cross-Modal Hashing with Double Structure Preserving[J]. Journal of Computer Research and Development, 2024 , 61 (11) : 2923 -2936 . DOI: 10.7544/issn1000-1239.202330433
随着互联网的普及和信息技术的快速发展,媒体数据呈爆发式增长. 由于传播媒介的多样化,媒体数据通常包含多种模态,并且不同模态数据之间往往是相互关联的[1-2]. 用户在检索某个主题时期望返回尽可能丰富且与主题相关的多模态数据. 因此,如何高效准确地实现跨模态检索成为多媒体领域亟待解决的问题. 跨模态哈希学习作为最有代表性的跨模态检索方法之一受到研究者的广泛关注[3-6]. 其旨在通过对不同模态样本信息的挖掘,将多模态数据从原始特征空间映射到公共海明空间,并尽可能保持数据的原始相似关系,从而实现快速的跨模态检索. 尽管现已取得了较好的检索性能,但传统的跨模态哈希方法大都采用批量学习模式,即利用当前累积的所有训练样本学习哈希函数[5-6]. 当数据频繁更新且数据量不断增长时,这类方法无法根据数据的变化自适应更新模型,而是需要重新进行哈希学习,从而导致巨大的时间成本和计算开销. 为了应对这一挑战,在线跨模态哈希方法应运而生.
不同于传统的离线跨模态哈希,在线方法需要在不重复训练旧数据的前提下,为新到来的数据学习哈希码,同时不断更新哈希函数使之保持对全体数据的适应性[7-12]. 为实现这一目的,广泛采用的设计思路是尽可能地挖掘数据间的关联关系,利用样本关系实现对旧数据的记忆和重利用. 典型的策略包括挖掘新旧数据之间的相似关系和多模态特征之间的关联关系. 前者[10-11]期望通过对新旧数据间分布关系的保持达到对全局数据分布的保持;后者[8,12]则期望利用模态间的特异和互补信息,减小公共哈希码与多模态特征之间的误差. 尽管现有的在线跨模态哈希方法取得了不错的检索性能,但其模型实现通常依赖于准确的语义信息指导,即监督训练模式. 然而,在现实生活中,监督方法存在数据标注成本高、难以获取足够的标记数据、标记数据质量差等问题.
无监督在线跨模态哈希学习或许是解决这一问题的有效方案[7-9]. 其基本思想是在没有标记数据的情况下,仅利用数据的内在结构和相似性信息学习哈希码和哈希函数. 虽然避免了标记数据难获取的弊端, 但现有的无监督在线跨模态哈希方法仍存在很多问题需要解决. 例如:1)现有模型对离群样本敏感,模型鲁棒性较差. 现实场景中,流数据因更新频繁且数据块规模较小,往往存在类别分布不平衡问题[13]. 换言之,数据块中存在一些样本数量很少的类别,在无监督训练模式下,这些样本从数据分布上表现为离群点. 现有的在线跨模态哈希方法大都采用均方误差来优化损失函数. 这一损失通常用于衡量模型整体的拟合情况,因而会放大离群样本的偏差对于模型的影响,导致检索性能下降. 2)现有模型对原始空间中样本的分布信息挖掘不充分. 无监督学习模式下,由于缺乏多模态一致的高层语义信息的指导,充分挖掘多模态一致的表征信息显得尤为重要[7-8]. 现有模型大都通过学习线性映射将多模态特征转移到公共空间中,并期望学习到的公共表示具有模态一致性和样本可区分性. 然而,受限于多模态数据异构,很难仅凭线性变换充分挖掘出模态间的一致信息. 因而,公共空间的表征能力仍有待进一步提升.
为了缓解上述问题,我们提出一种新的无监督在线跨模态方法. 该方法设计了双重结构保持框架. 一方面,引入$ {L_{2,1}} $范数取代$ {L_2} $范数学习全局结构保持的公共表示,从而缓解模型对于离群样本敏感的问题,提升模型的鲁棒性. 另一方面,通过融合的多模态特征进行邻域样本重构,学习具有邻域结构保持的公共表示. 最后,将两者结合用于指导哈希码和哈希函数的学习. 此外,为了保证哈希函数对于新旧数据的适应性,我们采用新旧数据联合优化的模式,并设计相应的优化策略使模型的更新效率仅于新增数据的规模相关,从而实现在线检索.
本文的主要贡献包括3个方面:
1)提出了基于双重结构保持的在线跨模态哈希方法. 在不依赖语义标签的前提下,学习具有全局结构保持和邻域信息保持的公共表示用以指导哈希学习,从而提升哈希码的判别性和鲁棒性.
2)为了避免在线优化过程中的灾难性遗忘问题,我们引入新旧数据的联合优化,并设计相应的在线优化方案降低训练的时间复杂度,从而实现灵活高效的在线检索.
3)在2个广泛使用的多模态数据集上的对比实验表明本文方法在表现出更优的检索精度的同时,保持着不错的训练效率,证实了本文方法在在线检索中的有效性和实用性.
在本节中,我们将从单模态和跨模态这2个方面,简单回顾在线哈希学习相关的工作.
1)单模态在线哈希方法
早期的在线哈希研究主要关注单模态检索任务,根据是否利用监督信息,可以粗略地分为监督方法和无监督方法. 监督在线哈希方法主要依靠标签信息或成对相似关系来学习哈希码,以实现高效的图像检索、语义检索等任务. 2015年,Cakir等人[14]提出在线监督哈希(online supervised hashing, OSH). 该方法采用2步哈希学习框架,先根据标签信息为每个类分配1个ECOC(error correct output code)向量,然后通过优化哈希码和ECOC向量之间的损失来学习哈希函数. 在OSH的基础上,基于哈达玛矩阵的在线哈希(Hadamard matrix guided online hashing, HMOH)[15]将哈达玛矩阵引入在线哈希学习框架中. 该方法将哈达玛矩阵中的列作为类标签的代理码,并采用局部灵敏度哈希(local sensitivity hashing, LSH)映射[2] 对齐代理码和待学习哈希码,从而加快在线训练速度. 相似保持链接哈希(similarity preserving linking hashing, SPLH)[16]将语义标签和哈希码映射到链接空间,通过这个链接空间可以更准确地衡量语义相似度. 平衡相似度在线离散哈希(balance similarity online discrete hashing, BSODH)[13]尝试构造新旧数据间的非对称成对相似图,并引入平衡因子来调节相似样本对和不相似样本对的权重,从而缓解了新旧数据规模差异大所导致的更新不平衡问题. 快速类更新在线哈希(fast class-wise online hashing, FCOH)[17]则设计逐类更新的方法,在解决更新不平衡问题的同时提高哈希学习的效率.
不同于监督哈希,无监督在线哈希方法不需要借助标签或成对相似矩阵的指导,而是通过挖掘样本特征表示的内在结构和分布关系来学习哈希码和哈希函数[18-19]. 在线核哈希(online kernel hashing, OKH)[20]是无监督在线哈希领域的第1次尝试,它将核函数引入到在线哈希学习中,利用在线凸优化算法实现哈希函数的在线优化. 由于OKH在训练过程中每轮只能处理1对数据,因此不适合大规模数据检索. 为此,研究者提出基于数据草图的方法,如OSketchH(online sketching hashing)[21],FROSH(faster online sketching hashing)[22],MOSH(making online sketching hashing even faster)[23]. 这些方法从每个数据块中提取概要信息,并根据这些概要信息更新哈希模型,由此降低了计算复杂度,训练效率也得到有效提升.
2)跨模态在线哈希方法
近年来,随着流数据的爆发式增长,在线跨模态哈希引起了越来越多的关注. 现有的在线跨模态哈希方法可以大致分为2类:监督方法和无监督方法. 与单模态方法类似,监督在线跨模态方法延续了利用语义标签和成对相似关系来学习哈希码的技术路线. 例如,在线潜在语义哈希(online latent semantic hashing, OLSH)[10]将离散的标签映射到连续的潜在语义空间,从而更准确地度量多模态流数据的语义相似度. 基于标签嵌入的在线哈希(label embedding online hashing, LEMON)[11]通过建立非对称相似矩阵来探索新旧数据之间、不同模态之间的关联关系,从而缓解模型对旧数据的遗忘并弥合不同模态间的语义鸿沟. 离散在线跨模态哈希(discrete online cross-modal hashing, DOCH)方法[12]则通过极大似然函数来约束数据块之间的非对称相似性,从而更好地保持新旧数据关系.
相对地,无监督方法主要通过分解和重构特征表示来探索多模态流数据之间的底层几何结构,进而学习哈希码和哈希函数. 由于其新颖性和挑战性,仅有少量的无监督在线跨模态哈希方法被提出. Xie等人[7]提出在线跨模态哈希(online cross-modal hashing, OCMH),通过优化共享潜在码和动态转移矩阵来学习哈希码. 在线跨模态矩阵分解哈希(online cross-modal matrix factorization hashing, OCMFH)[8]通过定制的在线优化策略使模型的训练时间线性于新数据规模. 并且,OCMFH提出在不访问旧数据特征表示的前提下更新旧数据哈希码的方案,使得哈希函数和哈希码的学习更加协调. 在线无监督跨模态离散哈希(online unsupervised cross-modal discrete hashing, OUCDH)[9]引入了非对称锚点图策略,在每轮的数据块上学习代表性数据作为锚点,通过锚点的分布来表征累计数据分布,从而缓解遗忘问题.
本节将从4个方面详细介绍所提方法,包括:符号定义、模型构建、算法优化过程及时间复杂度分析.
假设训练样本以流数据的形式到达,每个样本具有M个模态表示. 在第$ t $轮训练时会加入新的数据块$ {{\boldsymbol{X}}^{(t)}} = $$ \left({\boldsymbol{X}}_1^{(t)},{\boldsymbol{X}}_2^{(t)},…,{\boldsymbol{X}}_M^{(t)}\right)$,其中$ {\boldsymbol{X}}_m^{(t)} \in {\mathbb{R}^{{d_m} \times {n_t}}} $,${d_m}$表示第$ m $个模态的样本维度,$ {n_t} $表示新到的样本数. 前$t - 1$轮的累计数据用$ {{\boldsymbol{X}}^{(t - 1)}} = \left({\boldsymbol{X}}_1^{(t - 1)},{\boldsymbol{X}}_2^{(t - 1)},…,{\boldsymbol{X}}_M^{(t - 1)}\right) $表示,其中$ {\boldsymbol{X}}_m^{(t - 1)} \in {\mathbb{R}^{{d_m} \times {N_t}}} $,$ {N_t} $表示累积数据的样本数. ${\boldsymbol{B}} = \left({{\boldsymbol{B}}^{(t)}},\right. \left. {{\boldsymbol{B}}^{(t - 1)}}\right)$表示所学哈希码,其中$ {{\boldsymbol{B}}^{(t)}} \in {\mathbb{R}^{r \times {n_t}}} $,$ {\boldsymbol{B}}^{(t-1)} \in {{\mathbb{R}}}^{r\times {N}_{t}} $,$ r $表示哈希码长. $ {\left\| { \cdot } \right\|_2} $和$ {\left\| { \cdot } \right\|_{2,1}} $分别表示$ {L_2} $范数和$ {L_{2,1}} $范数,$ {{\mathrm{sgn}}} ( \cdot ) $为符号函数,如果$ x \gt 0 $,则$ {{\mathrm{sgn}}} (x) = 1 $,否则$ {{\mathrm{sgn}}} (x) = - 1 $.
如图1所示,本文所提的SPOCH(structure preserving online cross-modal hashing)主要包括3个模块:鲁棒的全局结构保持、融合的邻域结构保持和基于双重结构保持的在线哈希学习. 在本节中,我们将从这3个部分入手,详细介绍模型构建过程.
由于缺乏可信的监督信息指导,无监督跨模态哈希学习通常需要充分挖掘多模态数据的潜在几何结构[7-9]. 广泛采纳的学习框架是联合矩阵分解[24],它通过为每个模态学习对应的特征转移矩阵$ {{\boldsymbol{U}}_m} $,将不同模态的特征表示$ {{\boldsymbol{X}}_m} $映射为连续的公共表示$ {\boldsymbol{V}} $. 这个过程可以建模为
$ \mathop {\min }\limits_{{{\boldsymbol{U}}_m},{{\boldsymbol{V}}^{}}} \sum\limits_{m = 1}^M {{\alpha _m}} {\left\| {{{\boldsymbol{X}}_m} - {{\boldsymbol{U}}_m}{\boldsymbol{V}}} \right\|_2}{\text{ }}, $
其中$ {\alpha _m} $为超参数.
尽管在传统的离线跨模态哈希方法中,式(1)所示的联合矩阵分解框架可以取得不错的检索性能,但在更为贴近现实生活的在线检索场景下,流数据样本频繁更新,每个批次的样本数据量较少,常常会出现数据类别分布不平衡的现象,从而导致每个批次可用训练样本中存在少量的离群样本[13]. 而式(1)采用均方误差来衡量优化损失,其对离群样本较为敏感,使得模型的鲁棒性较差,影响模型的检索性能.
为此,我们提出采用$ {L}_{2,1} $范数取代$ {L_2} $范数来约束损失函数. 矩阵的$ {L}_{2,1} $范数为矩阵中所有行的$ {L_2} $范数之和,即$ {\left\| {\mathcal{A}} \right\|_{2,1}} = \displaystyle\sum\limits_i^{} {\left\| {{{\mathcal{A}}_i}} \right\|} $. 不同于$ {L_2} $范数使矩阵中所有元素的均方误差最小化,$ {L_{2,1}} $范数旨在实现损失矩阵的结构化稀疏[25]. 换言之,$ {L_{2,1}} $范数期望出现更多优化损失小的样本(即更多的0或接近0的值),而非追求全部样本的平均优化损失最小化,从而弱化模型对离群样本的关注,提升鲁棒性. 进一步地,为了适应在线学习的需求,我们在新旧数据上联合优化转移矩阵$ {{\boldsymbol{U}}_m} $,并为映射矩阵$ {{\boldsymbol{U}}_m} $添加$ {L_2} $范数正则化以防止过拟合,提高模型的泛化能力[26]. 这个过程可以建模为
$\begin{split}&\mathop {\min }\limits_{{{\boldsymbol{U}}_m},{{\boldsymbol{V}}^{(t)}}} \sum\limits_{m = 1}^M {{\alpha _m}\left( {{{\left\| {{\boldsymbol{X}}_m^{(t)} - {{\boldsymbol{U}}_m}{{\boldsymbol{V}}^{(t)}}} \right\|}_{2,1}} + {{\left\| {{\boldsymbol{X}}_m^{(t - 1)} - {{\boldsymbol{U}}_m}{{\boldsymbol{V}}^{(t - 1)}}} \right\|}_{2,1}}} \right)} +\\&\sum\limits_{m = 1}^M {\gamma {{\left\| {{{\boldsymbol{U}}_m}} \right\|}_2}} .\\[-2pt]\end{split} $
多模态数据的异构性使得不同模态的特征表示之间存在巨大的语义鸿沟,这导致不同模态的数据之间很难相互检索. 在本节中,我们提出基于多模态特征融合的邻域结构保持框架,通过挖掘多模态融合后的特征$ {{\boldsymbol{X}}^{(t)}} = \left({\boldsymbol{X}}_1^{(t)},{\boldsymbol{X}}_2^{(t)},…,{\boldsymbol{X}}_m^{(t)}\right)$的邻域结构信息,为样本学习邻域结构保持的多模态公共表示. 具体地,我们首先学习基于邻域的线性组合${{\boldsymbol{P}}^{(t)}}$来对训练样本重表示,这个过程可以建模为
$ \begin{aligned}&\mathop {{\text{min}}}\limits_{{{\boldsymbol{P}}^{(t)}}} \sum\limits_{i = {\text{1}}}^{{n_t}} {{{\left\| {{\boldsymbol{x}}_i^{(t)} - \sum\limits_{j \in {Q_i}} {P_{ij}^{(t)}} {\boldsymbol{x}}_j^{(t)}} \right\|}_{\text{2}}}} , \\&{\text{s}}{\text{.t}}{\text{. }}\sum\limits_{j \in {Q_i}} {P_{ij}^{(t)}} = 1{\text{ }}, \end{aligned}$
其中$ {Q_i} $表示第$ i $个样本$ {\boldsymbol{x}}_i^{(t)} $的邻域样本集合. 训练样本的邻域结构信息将被保存在亲和矩阵${{\boldsymbol{P}}^{(t)}}$中. 利用${{\boldsymbol{P}}^{(t)}}$学习邻域结构保持的${{\boldsymbol{Y}}^{(t)}} = \left({\boldsymbol{y}}_1^{{(}t{)}},{\boldsymbol{y}}_2^{{(}t{)}},…,{\boldsymbol{y}}_{{n_t}}^{{(}t{)}}\right)$,并约束${{\boldsymbol{Y}}^{(t)}}$中的样本两两正交,从而提升其表征能力,减少冗余信息. 该过程表示为
$ \begin{aligned}&\mathop {{\text{min}}}\limits_{{{\boldsymbol{Y}}^{(t)}}} \sum\limits_{i = {\text{1}}}^{{n_t}} {{{\left\| {{\boldsymbol{y}}_i^{(t)} - \sum\limits_{j \in {Q_i}} {P_{ij}^{(t)}} {\boldsymbol{y}}_j^{(t)}} \right\|}_{\text{2}}}} , \\&{\text{s}}{\text{.t}}{\text{. }}{\boldsymbol{y}}_i^{(t{\text{)T}}}{\boldsymbol{y}}_i^{(t)} = {\text{1}}. \end{aligned}$
为了充分保持数据在原始空间的相似关系,我们将2.2.1节和2.2.2节所学的鲁棒全局公共表示${{\boldsymbol{V}}^{(t)}}$和邻域结构保持的公共表示${{\boldsymbol{Y}}^{(t)}}$进行拼接,并通过正交映射$ {\boldsymbol{R}} $将其映射到海明空间中,用以指导哈希码${{\boldsymbol{B}}^{(t)}}$的学习. 同时,为了实现样本外扩展,我们学习映射$ {{\boldsymbol{W}}_m} $,将原始特征$ {\boldsymbol{X}}_m^{(t)} $映射到海明空间中. 为了保证哈希函数对全局样本的适应性,我们在新旧数据上联合优化哈希函数,并为映射矩阵$ {{\boldsymbol{W}}_m} $添加$ {L_2} $范数正则化以防止过拟合,提高模型的泛化能力[26]. 这个过程可以表示为
$ \begin{aligned}&\mathop {\min }\limits_{{{\boldsymbol{B}}^{(t)}},{\boldsymbol{R}},{{\boldsymbol{W}}_m}} {\left\| {{{\boldsymbol{B}}^{(t)}} - {\boldsymbol{R}}\left[ {{{\boldsymbol{V}}^{(t)}};{{\boldsymbol{Y}}^{(t)}}} \right]} \right\|_2} + \sum\limits_{m = 1}^M {\gamma {{\left\| {{{\boldsymbol{W}}_m}} \right\|}_2}} + \\& \sum\limits_{m = 1}^M {{\beta _m}} {\left\| {\left[ {{{\boldsymbol{B}}^{(t - 1)}},{{\boldsymbol{B}}^{(t)}}} \right] - {{\boldsymbol{W}}_m}\left[ {{\boldsymbol{X}}_m^{(t - 1)},{\boldsymbol{X}}_m^{(t)}} \right]} \right\|_2}{\text{, }} \\ &{\text{s}}{\text{.t}}{\text{. }}{{\boldsymbol{B}}^{(t)}} \in {{{\{ - 1, + 1\} }}^{{n_t} \times r}},{{\boldsymbol{R}}^{\text{T}}}{\boldsymbol{R}} = {\boldsymbol{I}}, \end{aligned} $
其中$ {\beta _m} $和$ \gamma $为超参数.
在本节中,我们将介绍所提模型的优化过程. 该模型的优化分为3个部分:
1)鲁棒的全局结构保持模块中通过优化式(2)来学习转移矩阵$ {{\boldsymbol{U}}_m} $和鲁棒公共表示${{\boldsymbol{V}}^{(t)}}$.
2)融合的邻域结构保持模块中通过优化式(3)学习亲和矩阵${{\boldsymbol{P}}^{(t)}}$,通过优化式(4)学习邻域保持公共表示${{\boldsymbol{Y}}^{(t)}}$.
3)在线哈希学习模块中通过优化式(5)学习哈希码${{\boldsymbol{B}}^{(t)}}$,映射矩阵$ {\boldsymbol{R}} $和$ {{\boldsymbol{W}}_m} $.
由于存在$ {L}_{2,1} $范数约束,式(2)难以直接优化,为此,我们通过拆解变量的方式,将式(2)改写为
$\begin{split} &\mathop {{\text{min}}}\limits_{{{\boldsymbol{U}}_m},{{\boldsymbol{V}}^{(t)}}} \sum\limits_{m = {\text{1}}}^M \left\{ {\alpha _m}{\text{tr}}\left( \left( {{\boldsymbol{X}}_m^{(t)} - {{\boldsymbol{U}}_m}{{\boldsymbol{V}}^{(t)}}} \right)\right.\right.\cdot\\&\left.\left.{\boldsymbol{D}}_m^{(t)}{{\left( {{\boldsymbol{X}}_m^{(t)} - {{\boldsymbol{U}}_m}{{\boldsymbol{V}}^{(t)}}} \right)}^{\text{T}}} \right) + \gamma {{\left\| {{{\boldsymbol{U}}_m}} \right\|}_{\text{2}}} \right\} +\\& \sum\limits_{m = {\text{1}}}^M {\alpha _m}{\text{tr}}\left( \left( {{\boldsymbol{X}}_m^{(t - {\text{1)}}} - {{\boldsymbol{U}}_m}{{\boldsymbol{V}}^{(t - {\text{1)}}}}} \right)\right.\cdot\\&\left.{\boldsymbol{D}}_m^{(t - {\text{1)}}}{{\left( {{\boldsymbol{X}}_m^{(t - {\text{1)}}} - {{\boldsymbol{U}}_m}{{\boldsymbol{V}}^{(t - {\text{1)}}}}} \right)}^{\text{T}}} \right) ,\end{split} $
其中${{\boldsymbol{D}}_m}$是对角矩阵,其第$i$行的对角元素表示为
$\begin{aligned} &{\left( {{\boldsymbol{D}}_m^{(t)}} \right)_{i,i}} = {\text{1}}\Big/{\left\| {{{\left( {{\boldsymbol{x}}_m^{(t)}} \right)}_i} - {{\boldsymbol{U}}_m}{{\left( {{{\boldsymbol{v}}^{(t)}}} \right)}_i}} \right\|_{\text{2}}}, \\&{\left( {{\boldsymbol{D}}_m^{(t - {\text{1)}}}} \right)_{i,i}} = {\text{1}}\Big/{\left\| {{{\left( {{\boldsymbol{x}}_m^{(t - {\text{1)}}}} \right)}_i} - {{\boldsymbol{U}}_m}{{\left( {{{\boldsymbol{v}}^{(t-1)}}} \right)}_i}} \right\|_{\text{2}}}, \end{aligned}$
其中$ {\left( {{\boldsymbol{x}}_m^{(t)}} \right)_i} $表示第$ m $个模态中的第$ i $个样本的特征. 此时,式(6)可以通过迭代优化算法求解,包括U-Step和V-Step,具体步骤如下.
U-Step:固定变量${{\boldsymbol{V}}^{(t)}}$,优化模态转移矩阵$ {{\boldsymbol{U}}_m} $,那么式(6)可以被改写为
$ \begin{split}&\mathop {{\text{min}}}\limits_{{{\boldsymbol{U}}_m}} {\alpha _m}{\text{tr}}\left( {\left( {{\boldsymbol{X}}_m^{(t - {\text{1)}}} - {{\boldsymbol{U}}_m}{{\boldsymbol{V}}^{(t - {\text{1)}}}}} \right){\boldsymbol{D}}_m^{(t - {\text{1)}}}{{\left( {{\boldsymbol{X}}_m^{(t - {\text{1)}}} - {{\boldsymbol{U}}_m}{{\boldsymbol{V}}^{(t - {\text{1)}}}}} \right)}^{\text{T}}}} \right) + \\&{\alpha _m}{\text{tr}}\left( {\left( {{\boldsymbol{X}}_m^{(t)} - {{\boldsymbol{U}}_m}{{\boldsymbol{V}}^{(t)}}} \right){\boldsymbol{D}}_m^{(t)}{{\left( {{\boldsymbol{X}}_m^{(t)} - {{\boldsymbol{U}}_m}{{\boldsymbol{V}}^{(t)}}} \right)}^{\text{T}}}} \right) + \gamma {\left\| {{{\boldsymbol{U}}_m}} \right\|_{\text{2}}}{\text{. }} \\[-1pt]\end{split} $
通过将$ {{\boldsymbol{U}}_m} $的导数设为0,$ {{\boldsymbol{U}}_m} $的闭式解可表示为
$ {{\boldsymbol{U}}_m}{\text{ = }}{\boldsymbol{C}}_1^{(t)}{\left( {{\boldsymbol{C}}_2^{(t)}{\text{ + }}\frac{\gamma }{{{\alpha _m}}}{\boldsymbol{I}}} \right)^{ - 1}}, $
$ \begin{aligned}{\boldsymbol{C}}_{\text{1}}^{(t)} =& {\boldsymbol{X}}_m^{(t)}{\boldsymbol{D}}_m^{(t)}{{\boldsymbol{V}}^{(t{\text{)T}}}} + {\boldsymbol{X}}_m^{(t - {\text{1)}}}{\boldsymbol{D}}_m^{(t - {\text{1)}}}{{\boldsymbol{V}}^{(t - {\text{1)T}}}} = \\&{\boldsymbol{X}}_m^{(t)}{\boldsymbol{D}}_m^{(t)}{{\boldsymbol{V}}^{(t{\text{)T}}}} + {\boldsymbol{C}}_{\text{1}}^{(t - {\text{1)}}}, \\{\boldsymbol{C}}_{\text{2}}^{(t)} =& {{\boldsymbol{V}}^{(t)}}{\boldsymbol{D}}_m^{(t)}{{\boldsymbol{V}}^{(t{\text{)T}}}} + {{\boldsymbol{V}}^{(t - {\text{1)}}}}{\boldsymbol{D}}_m^{(t - {\text{1)}}}{{\boldsymbol{V}}^{(t - {\text{1)T}}}} = \\&{{\boldsymbol{V}}^{(t)}}{\boldsymbol{D}}_m^{(t)}{{\boldsymbol{V}}^{(t{\text{)T}}}} + {\boldsymbol{C}}_{\text{2}}^{(t - {\text{1)}}}. \end{aligned} $
V-Step:固定变量$ {{\boldsymbol{U}}_m} $,优化公共表示${{\boldsymbol{V}}^{(t)}}$,此时,该优化问题可表示为
$ \mathop {{\text{min}}}\limits_{{{\boldsymbol{V}}^{(t)}}} \sum\limits_{m = {\text{1}}}^M {{\alpha _m}} {\text{tr}}\left( {\left( {{\boldsymbol{X}}_m^{(t)} - {{\boldsymbol{U}}_m}{{\boldsymbol{V}}^{(t)}}} \right){\boldsymbol{D}}_m^{(t)}{{\left( {{\boldsymbol{X}}_m^{(t)} - {{\boldsymbol{U}}_m}{{\boldsymbol{V}}^{(t)}}} \right)}^{\text{T}}}} \right). $
对式(11)求导,并令导数为0,可得${{\boldsymbol{V}}^{(t)}}$的闭式解为
$ {{\boldsymbol{V}}^{(t)}} = \sum\limits_{m = {\text{1}}}^M {{\alpha _m}} {\left( {{\boldsymbol{U}}_m^{\text{T}}{{\boldsymbol{U}}_m}} \right)^{ - {\text{1}}}}{\boldsymbol{U}}_m^{\text{T}}{\boldsymbol{X}}_m^{(t)}{\text{. }} $
该模块中变量${{\boldsymbol{P}}^{(t)}}$和变量${{\boldsymbol{Y}}^{(t)}}$可单独优化.
P-Step:亲和矩阵${{\boldsymbol{P}}^{(t)}}$可以利用拉格朗日乘子法求解. 此时式(3)可被重写为
$\begin{aligned}&\mathop {{\text{min}}}\limits_{{{{\boldsymbol{P}}}^{(t)}}} \sum\limits_{i = {\text{1}}}^{{n_i}} {\boldsymbol{P}_i^{(t{\text{)T}}}} {{\boldsymbol{Z}}_i}{\boldsymbol{P}}_i^{(t)}{\text{ }},\\&{\text{s}}{\text{.t}}{\text{. }}\sum\limits_{j \in {Q_i}} {{\boldsymbol{P}}_i^{(t{\text{)T}}}} {\boldsymbol{1}} = {\text{1 }}, \end{aligned}$
其中$ {{\boldsymbol{Z}}_i} = \displaystyle\sum\limits_{j \in {Q_i}} {\left( {{\boldsymbol{x}}_i^{(t)} - {\boldsymbol{x}}_j^{(t)}} \right)} {\left( {{\boldsymbol{x}}_i^{(t)} - {\boldsymbol{x}}_j^{(t)}} \right)^{\text{T}}} $. 具体的优化过程可参考文献[27].
Y-Step:为了获得局域结构保持的表示${{\boldsymbol{Y}}^{(t)}}$,我们设置${{\boldsymbol{Y}}^{(t)}} = {\boldsymbol{A}}{{\boldsymbol{X}}^{(t)}}$,并将式(4)改写为
$ \begin{aligned}&\mathop {{\text{min}}}\limits_{\boldsymbol{A}} {\boldsymbol{A}}{{\boldsymbol{X}}^{(t)}}{\left( {{\boldsymbol{I}} - {{\boldsymbol{P}}^{(t)}}} \right)^{\text{T}}}\left( {{\boldsymbol{I}} - {{\boldsymbol{P}}^{(t)}}} \right){{\boldsymbol{X}}^{(t{\text{)T}}}}{{\boldsymbol{A}}^{\text{T}}}, \\&{\text{s}}{\text{.t}}{\text{. }}{\boldsymbol{A}}{{\boldsymbol{X}}^{(t)}}{{\boldsymbol{X}}^{(t){\text{T}}}}{{\boldsymbol{A}}^{\text{T}}} = {\boldsymbol{I}}.\end{aligned}$
转移矩阵${\boldsymbol{A}}$可通过以下广义特征向量问题[28]求解:
$ {{\boldsymbol{X}}^{(t)}}{\left( {{\boldsymbol{I}} - {{\boldsymbol{P}}^{(t)}}} \right)^{\text{T}}}\left( {{\boldsymbol{I}} - {{\boldsymbol{P}}^{(t)}}} \right){{\boldsymbol{X}}^{(t{\text{)T}}}}{{\boldsymbol{A}}^{\text{T}}} = {\lambda ^\prime }{{\boldsymbol{X}}^{(t)}}{{\boldsymbol{X}}^{(t{\text{)T}}}}{{\boldsymbol{A}}^{\text{T}}}{\text{. }} $
式(5)的优化为非凸非连续问题,很难直接求解,因此我们采用迭代优化. 具体步骤如下.
W-Step:固定变量${{\boldsymbol{B}}^{(t)}}$和$ {\boldsymbol{R}} $,优化映射矩阵${{\boldsymbol{W}}_m}$,式(5)可以改写为
$ \mathop {\min }\limits_{{{\boldsymbol{W}}_m}} {\beta _m}\left( {{{\left\| {{{\boldsymbol{B}}^{(t - 1)}} - {{\boldsymbol{W}}_m}{\boldsymbol{X}}_m^{(t - 1)}} \right\|}_2}} \right. + \left. {{{\left\| {{{\boldsymbol{B}}^{(t)}} - {{\boldsymbol{W}}_m}{\boldsymbol{X}}_m^{(t)}} \right\|}_2}} \right) + \gamma {\left\| {{{\boldsymbol{W}}_m}} \right\|_2}{\text{. }} $
令其导数为0,${{\boldsymbol{W}}_m}$的闭式解可表示为
$ {{\boldsymbol{W}}_m}{\text{ = }}{\boldsymbol{H}}_{{m}}^{(t)}{\left( {{\boldsymbol{F}}_m^{(t)}{\text{ + }}\frac{\gamma }{{{\beta _m}}}{\boldsymbol{I}}} \right)^{ - 1}}, $
$ \begin{aligned}&{\boldsymbol{H}}_m^{(t)} = {{\boldsymbol{B}}^{(t)}}{\boldsymbol{X}}_m^{(t{\text{)T}}} + {{\boldsymbol{B}}^{(t - {\text{1)}}}}{\boldsymbol{X}}_m^{(t - {\text{1)T}}} = {{\boldsymbol{B}}^{(t)}}{\boldsymbol{X}}_m^{(t{\text{)T}}} + {\boldsymbol{H}}_m^{(t - {\text{1)}}}, \\&{\boldsymbol{F}}_m^{(t)} = {\boldsymbol{X}}_m^{(t)}{\boldsymbol{X}}_m^{(t{\text{)T}}} + {\boldsymbol{X}}_m^{(t - {\text{1)}}}{\boldsymbol{X}}_m^{(t - {\text{1)T}}} = {\boldsymbol{X}}_m^{(t)}{\boldsymbol{X}}_m^{(t{\text{)T}}} + {\boldsymbol{F}}_m^{(t - {\text{1)}}}{\text{. }} \end{aligned} $
R-Step: 固定变量${{\boldsymbol{B}}^{(t)}}$和$ {{\boldsymbol{W}}_m} $,优化映射矩阵${\boldsymbol{R}}$. 经过简单的变换,式(5)可以改写为
$ \begin{aligned}&\mathop {{\text{max}}}\limits_{\boldsymbol{R}} {\text{tr}}\left( {\left[ {{{\boldsymbol{V}}^{(t)}};{{\boldsymbol{Y}}^{(t)}}} \right]{{\boldsymbol{B}}^{(t{\text{)T}}}}{\boldsymbol{R}}} \right), \\&{\text{s}}{\text{.t}}{\text{. }}{{\boldsymbol{R}}^{\text{T}}}{\boldsymbol{R}} = {\boldsymbol{I}}. {\text{ }}\end{aligned}$
为了优化式(19),我们首先执行奇异值分解,也就是$ {\text{[}}{{\boldsymbol{V}}^{(t)}};{{\boldsymbol{Y}}^{(t)}}{\text{]}}{{\boldsymbol{B}}^{(t{\text{)T}}}} = {\boldsymbol{K}}\boldsymbol{\tau} {{\boldsymbol{O}}^{\text{T}}} $. ${\boldsymbol{R}}$的解可以表示为${\boldsymbol{R}} = {\boldsymbol{O}}{{\boldsymbol{\hat K}}^{\mathrm{T}}}$,其中${\boldsymbol{\hat K}}$表示矩阵${\boldsymbol{K}}$的前$r$个特征向量.
B-Step: 固定$ {{\boldsymbol{W}}_m} $和$ {\boldsymbol{R}} $,优化哈希码${{\boldsymbol{B}}^{(t)}}$,此时
$\begin{aligned}&\mathop {\min }\limits_{{{\boldsymbol{B}}^{(t)}}} {\left\| {{{\boldsymbol{B}}^{(t)}} - {\boldsymbol{R}}\left[ {{{\boldsymbol{V}}^{(t)}};{{\boldsymbol{Y}}^{(t)}}} \right]} \right\|_2} + \sum\limits_{m = 1}^M {{\beta _m}} {\left\| {{{\boldsymbol{B}}^{(t)}} - {{\boldsymbol{W}}_m}{\boldsymbol{X}}_m^{(t)}} \right\|_2},\\&\quad {\text{s}}{\text{.t}}{\text{. }}{{\boldsymbol{B}}^{(t)}} \in {\{ - 1, + 1\} ^{{n_t} \times r}}. \end{aligned} $
通过公式变换可得
$ {{\boldsymbol{B}}^{(t)}} = {{\mathrm{sgn}}} \left( {{\boldsymbol{R}}\left[ {{{\boldsymbol{V}}^{(t)}};{{\boldsymbol{Y}}^{(t)}}} \right] + \sum\limits_{m = 1}^M {{\beta _m}} {{\boldsymbol{W}}_m}{\boldsymbol{X}}_m^{(t)}} \right). $
具体的优化过程见算法1、算法2.
算法1. 双重结构保持在线跨模态哈希算法.
输入:流数据特征表示$ {{\boldsymbol{X}}_m} $,参数$ {\alpha _m} $,${\beta _m}$,$ \gamma $,
邻居节点数$k$,哈希码长$r$;
输出:哈希码$ {\boldsymbol{B}} $,映射矩阵$ {{\boldsymbol{W}}_m} $.
① if t=1 then
②  初始化${{\boldsymbol{B}}^{(0)}}$,$ {\boldsymbol{C}}_1^{(0)} $,$ {\boldsymbol{C}}_2^{(0)} $,${\boldsymbol{H}}_m^{(0)}$,${\boldsymbol{F}}_m^{(0)}$为0;
③ end if
④ for t = 1: s do
⑤  通过算法2学习${{\boldsymbol{B}}^{(t)}}$,$ {\boldsymbol{C}}_1^{(t)} $,$ {\boldsymbol{C}}_2^{(t)} $,${\boldsymbol{H}}_m^{(t)}$,${\boldsymbol{F}}_m^{(t)}$;
⑥  设$ {{\boldsymbol{B}}^{(t - 1)}} = [{{\boldsymbol{B}}^{(t)}},{{\boldsymbol{B}}^{(t - 1)}}] $;
⑦ end for
算法2. 第$ t $轮更新算法.
输入:$ {\boldsymbol{X}}_m^{(t)} $,$ {\boldsymbol{C}}_1^{(t - 1)} $,$ {\boldsymbol{C}}_2^{(t - 1)} $,${\boldsymbol{H}}_m^{(t - 1)}$,${\boldsymbol{F}}_m^{(t - 1)}$,哈希码长$r$,
参数 $ {\alpha _m} $,${\beta _m}$,$ \gamma $,k,迭代次数$ T $;
输出:${{\boldsymbol{B}}^{(t)}}$,$ {\boldsymbol{C}}_1^{(t)} $,$ {\boldsymbol{C}}_2^{(t)} $,${\boldsymbol{H}}_m^{(t)}$,${\boldsymbol{F}}_m^{(t)}$,$ {{\boldsymbol{W}}_m} $.
① 随机初始化${{\boldsymbol{V}}^{(t)}}$为零均值矩阵;
② 随机初始化${{\boldsymbol{B}}^{(t)}}$为二值矩阵;
③ 初始化${\boldsymbol{D}}_m^{(t)}$为单位阵;
④ 根据式(13)更新${{\boldsymbol{P}}^{(t)}}$;
⑤ 根据式(15)更新${\boldsymbol{A}}$,计算${{\boldsymbol{Y}}^{(t)}} = {\boldsymbol{A}}{{\boldsymbol{X}}^{(t)}}$;
⑥ for i ≤ T do
⑦  根据式(9)更新$ {{\boldsymbol{U}}_m} $;
⑧  根据式(12)更新${{\boldsymbol{V}}^{(t)}}$;
⑨  根据式(7)更新${\boldsymbol{D}}_m^{(t)}$;
⑩ end for
⑪ for i ≤ T do
⑫  根据式(17)更新变量$ {{\boldsymbol{W}}_m} $;
⑬  根据式(19)更新变量${\boldsymbol{R}}$;
⑭  根据式(21)更新变量${{\boldsymbol{B}}^{(t)}}$;
⑮ end for
映射矩阵$ {{\boldsymbol{U}}_m} $、$ {{\boldsymbol{W}}_m} $和公共表示${{\boldsymbol{V}}^{(t)}}$可以直接求导得到其解析解,其时间复杂度分为$O({d_m}r{n_t} + {r^2}{n_t})$,$O({d_m}r{n_t} + {r^2}{d_m})$,$ O(d_m^2{n_t} + d_m^2r) $. 哈希码${{\boldsymbol{B}}^{(t)}}$通过符号函数优化,其时间复杂度为$O({d_m}r{n_t} + {r^2}{n_t})$. 正交映射${\boldsymbol{R}}$需要通过奇异值分解优化,其时间复杂度为$O({d_m}r{n_t} + {r^2}{n_t})$. 优化公共表示${{\boldsymbol{Y}}^{(t)}}$的时间复杂度为$O({d_m}r{n_t} + {k^2}{n_t})$. 由于样本数${n_t}$远大于$r$和$k$,因此,本文所提出的SPOCH方法时间复杂度可简化为$ T \times O(d_m^2{n_t} + d_m^{}r{n_t} + d_m^2r) $,其中$ T $表示迭代次数.
为了验证所提方法的有效性,我们在2个广泛使用的数据集上进行实验. 本节将从数据集、实验设置和实验结果分析3个方面具体介绍实验内容.
1)IAPRTC-12[29]. 该数据集包含20 000个图像文本对,分属255个类别. 我们使用512维GIST特征来描述图像模态样本,使用2 912维BOW特征来描述文本模态样本. 在实验中,我们随机选取2 000个图像文本对作为查询集,其余的作为训练集. 为了适应在线检索的场景,训练集被分成9个数据块,每个数据块包含2 000个样本对.
2)MIRFLICKR[30]. 该数据集由20 015个图像文本对组成. 每个样本对带有1个或多个标签,标签总共24类. 我们使用512维GIST特征和1 386维BOW特征分别描述该数据集中图像模态和文本模态. 不失一般性地,我们随机选取2 000个样本对作为查询集,其余的作为训练集. 为了适应在线场景,训练集被分成9个数据块,其中8个数据块包含2 000个样本对,1个数据块包含2 015个样本对.
1)对比方法. 在实验中,我们与8种当前最好的无监督跨模态方法进行对比. 其中:CMFH[24],RFDH[31],JIMFH[32],HSSPH[33]为离线方法;OCMH[7],OCMFH[8],OUCDH[9],OMGHUN[34]为在线方法. 其中OCMH,OUCDH,OMGHUN是我们根据论文复现的,其余对比方法均采用论文作者公开的源代码. 离线方法在每个数据块上重新训练累积数据,而在线方法仅使用新数据进行训练.
2)参数设置. 为了公平,对于对比方法,我们均采用原文中建议的参数设置. 对于本文所提方法,我们利用交叉验证与网格搜索结合,对参数设置为:模态平衡参数$ {\alpha _1} = {\alpha _2} = {10^4} $,$ {\beta _1} = {\beta _2} = 1 $,正则化参数$ \gamma = 10 $,邻居节点个数$ k = 50 $. 此外,根据经验,我们设置迭代次数$ T $=5. 所有的实验结果都在Intel(®) CoreTM i9-10900K CPU@ 3.70GHz 64GB RAM的机器上运行得到.
3)评价指标. 为了验证跨模态检索的性能,在本实验中,我们实施2种检索任务:文本检索图像和图像检索文本. 我们采用2种最常见的检索评估指标,即平均精确率均值(mAP@all)和前K精度(Precision@K),来评估模型检索准确性. 其中K=100. 此外,为了评估模型的训练效率,我们记录了所提的SPOCH方法及所有对比方法的运行时间.
在本节中,我们将从所提方法与对比方法的性能比较、消融实验、参数敏感性实验和训练时间4个方面展示实验结果并进行分析.
表1和表2展示了我们所提的SPOCH方法和对比方法在IAPRTC-12和MIRFLICKR 这2个广泛使用的跨态检索数据集上的mAP@all实验结果. 为了保证实验结果的可靠性,我们报道5次运行的均值并提供了标准差. 从表1和表2可以看出,SPOCH在几乎所有码长上明显优于4种无监督在线跨模态方法OCMH,OCMFH,OUCDH,OMGHUN,尤其是在IAPRTC-12数据集上取得显著提升. 可能的原因是,该数据集包含的类别数较多,在不同数据块之间的类别不平衡现象更为明显. 同时,我们还将SPOCH与4个离线跨模态哈希方法进行了对比. 一般来说,由于进行了重复训练,离线方法的性能优于在线方法,一定程度上是以时间为代价提升了检索精度. 然而,从实验结果中可以看出,SPOCH几乎取得了与离线方法相当的性能,尤其是在IAPRTC-12数据集上,SPOCH的精度在一些码长上甚至优于离线方法.
进一步地,图2和图3中分别展示了mAP@all和Precision@100这2种评估指标在32 b时随数据块的变化. 可以看出,SPOCH在不同数据块之间性能较为平稳,表明所提方法对新数据的适应性良好,适合用于在线检索的场景. 并且,在不同数据块上,所提方法的mAP@all和Precision@100性能都显著优于4种在线无监督跨模态哈希,且与离线方法相比也取得了相当甚至更优的检索表现,充分验证了所提方法在在线检索场景下的有效性.
根据模型的设计方案,我们从4个角度验证所提方法的有效性. 表3和表4展示了消融实验的结果.
1)SPOCH-f. 为了验证结构化稀疏是否可以缓解离群样本的影响,我们采用$ {L_2} $范数而非$ {L_{2,1}} $范数来学习公共表示$ {\boldsymbol{V}} $. 实验结果表明,SPOCH-f在2个数据集上都较SPOCH有所下降,验证了结构化稀疏可以提高模型鲁棒性. 特别地,SPOCH-f在IAPRTC-12数据集上下降尤为明显,可能的原因是该数据集的类别不平衡现象更为显著.
2)SPOCH-v. 为了验证全局结构信息对于哈希码学习的指导作用,我们在SPOCH-v的实验中去掉了鲁棒全局结构保持模块. 从表3和表4可以看出,该模块对于性能的提升十分重要,尤其是在短码的情况下,去掉公共表示的指导会导致性能显著下降.
3)SPOCH-y. 该模块用于验证邻域结构保持对于检索性能的影响. 为此,我们去掉了对邻域结构信息的学习,仅采用全局公共表示$ {\boldsymbol{V}} $来指导哈希码的生成. 从表3和表4可见,在短码的情况下,SPOCH-y的性能较SPOCH有显著下降. 结合SPOCH-v的结果可知,仅通过单一的指导信息无法习得高质量的短码,由此验证了本文所提的双重结构保持方案的有效性.
4)SPOCH-w. 该模块用于验证新旧数据联合优化对于检索性能的影响. 在SPOCH-w的实验中,我们去掉了转移矩阵$ {{\boldsymbol{U}}_m} $和映射矩阵$ {{\boldsymbol{W}}_m} $在新旧数据上的联合优化. SPOCH-w在2个数据集各个位上都呈现明显的性能下降,可见联合学习及相应的优化策略可以缓解流数据场景下的灾难性遗忘问题.
为了进一步验证模型的稳定性,我们对模型中的平衡参数$ {\alpha _m} $和${\beta _m}$、邻居节点个数k和邻域公共表示长度c,分别做敏感性测试. 图4和图5展示了平衡参数$ {\alpha _m} $和${\beta _m}$随着参数取值的变化时对应的mAP@all性能的变化. 从实验结果来看,平衡参数的选择对于检索性能的影响非常微弱. 由此可见,所提模型对这2个参数不敏感.
图6展示了邻居节点个数k和邻域结构保持的公共表示的长度c在参数变化时对应的mAP@all性能的变化. 从实验结果来看,邻域范围的选择及公共表示的长度对于检索性能的影响并不明显,即模型对于这2个参数具有良好的稳定性.
图7记录了2个数据集上SPOCH和对比方法的训练时间,哈希码长设置为32 b. 由于离线方法需要在累计数据集上重复训练,因此离线方法的训练时间随数据流显著增加,此时检索效率会受到很大影响. 而在线方法的训练时间与累积数据无关,因此更加适用于流数据的检索. 从实验结果可以看出,随着数据量的累积,SPOCH的效率逐渐优于离线方法. 并且,SPOCH在检索精度显著优于现有方法的前提下取得了可比较甚至更优的训练效率.
本文提出基于双重结构保持的在线跨模态哈希方法. 该模型可以在不依赖监督信息的情况下为多模态流数据学习对应的哈希函数和哈希码,用于后续的在线跨模态检索任务. 在该模型中,我们同时挖掘流数据中的全局结构信息和邻域结构信息,使得所学的公共表示可以更好地表征多模态数据,从而促进高质量哈希码的生成. 并且,我们基于$ {L_{2,1}} $范数设计损失函数,利用$ {L_{2,1}} $范数结构化稀疏的性质缓解了离群样本对模型的影响,提高了模型鲁棒性. 此外,我们设计了相应的在线优化策略,使得算法的时间复杂度线性于新数据的规模,提高了检索效率.
作者贡献声明:康潇提出算法思路,完成实验并撰写论文;刘兴波提出指导意见并修改论文;卢鹏宇、赵志杰和王少华协助完成实验和撰写论文;聂秀山和尹义龙提出指导意见和审核论文.

参考文献 引证文献
排序方式:
1
Xu Xing, Shen Fumin, Yang Yang, et al. Learning discriminative binary codes for large-scale cross-modal retrieval[J]. IEEE Transactions on Image Processing, 2017, 26(5): 2494−2507
2
Raginsky M, Lazebnik S. Locality-sensitive binary codes from shift-invariant kernels[C]//Proc of the 22nd Int Conf on Neural Information Processing Systems. New York: ACM, 2009, 1509–1517
3
刘兴波,聂秀山,尹义龙. 基于双向线性回归的监督离散跨模态散列方法[J]. 计算机研究与发展,2020,57(8):1707−1714
Liu Xingbo, Nie Xiushan, Yin Yilong. Supervised discrete cross-modal hashing method based on two-way linear regression[J]. Journal of Computer Research and Development, 2020, 57(8): 1707−1714 (in Chinese)
4
Zheng Chaoqun, Zhu Lei, Zhang Zheng, et al. Efficient semi-supervised multimodal hashing with importance differentiation regression[J]. IEEE Transactions on Image Processing, 2022, 31: 5881−5892
5
卓君宝,苏驰,王树徽,等. 最小熵迁移对抗散列方法[J]. 计算机研究与发展,2020,57(4):888−896
Zhuo Junbao, Su Chi, Wang Shuhui, et al. Least entropy transfer against hashing method[J]. Journal of Computer Research and Development, 2020, 57(4): 888−896 (in Chinese)
6
Lin Zijia, Ding Guiguang, Hu Mingqing, et al. Semantics-preserving hashing for cross-view retrieval[C]// Proc of the IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2015: 3864−3872
7
Xie Liang, Shen Jialie, Zhu Lei, et al. Online cross-modal hashing for web image retrieval[C]// Proc of the 30th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2016: 294−300
8
Wang Di, Wang Quan, An Yaqiang, et al. Online collective matrix factorization hashing for large-scale cross-media retrieval[C]// Proc of the 43rd Int ACM SIGIR Conf on Research and Development in Information Retrieval. New York: ACM, 2020: 1409−1418
9
Li Xuan, Wu Wei, Yuan Yunhao, et al. Online unsupervised cross-view discrete hashing for large-scale retrieval[J]. Applied Intelligence, 2022, 52(13): 14905−14917
10
Yao Tao, Wang Gang, Yan Lianshan, et al. Online latent semantic hashing for cross-media retrieval[J]. Pattern Recognition, 2019, 89: 1−11
11
Wang Yongxin, Luo Xin, Xu Xinshun, et al. Label embedding online hashing for cross-modal retrieval[C]// Proc of the 28th ACM Int Conf on Multimedia. New York: ACM, 2020: 871−879
12
Zhan Yuwei, Wang Yongxin, Sun Yu, et al. Discrete online cross-modal hashing[J]. Pattern Recognition, 2022, 122: 108262
13
Lin Mingbao, Ji Rongrong, Liu Hong, et al. Towards optimal discrete online hashing with balanced similarity[C]//Proc of the 33rd AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2019: 8722−8729
14
Cakir F, Sclaroff S. Online supervised hashing[C]// Proc of the IEEE Int Conf on Image Processing. Piscataway, NJ: IEEE, 2015: 2606−2610
15
Lin Mingbao, Ji Rongrong, Liu Hong, et al. Hadamard matrix guided online hashing[J]. International Journal of Computer Vision, 2020, 128(8): 2279−2306
16
Lin Mingbao, Ji Rongrong, Chen Shen, et al. Similarity-preserving linkage hashing for online image retrieval[J]. IEEE Transactions on Image Processing, 2020, 29: 5289−5300
17
Lin Mingbao, Ji Rongrong, Sun Xiaoshuai, et al. Fast class-wise updating for online hashing[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(5): 2453−2467
18
Fang Yuzhi, Liu Li. Scalable supervised online hashing for image retrieval[J]. Journal of Computational Design and Engineering, 2021, 8(5): 1391−1406
19
Hao Yanbin, Mu Tingting, Goulermas J, et al. Unsupervised t-distributed video hashing and its deep hashing extension[J]. IEEE Transactions on Image Processing, 2017, 26(11): 5531−5544
20
Huang Longkai, Yang Qiang, Zheng Weishi. Online hashing[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(6): 2309−2322
21
Leng Cong, Wu Jiaxiang, Cheng Jian, et al. Online sketching hashing[C]// Proc of the IEEE Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2015: 2503−2511
22
Chen Xixian, King I, Lyu M. FROSH: Faster online sketching hashing[C/OL]// Proc of the 30th Conf on Uncertainty in Artificial Intelligence. Corvallis: AUAI, 2017[2023-11-21]. http://auai.org/uai2017/proceedings/papers/12.pdf
23
Chen Xixian, Yang Haiqin, Zhao Shenglin, et al. Making online sketching hashing even faster[J]. IEEE Transactions on Knowledge and Data Engineering, 2021, 33(3): 1089−1101
24
Ding Guiguang, Guo Yuchen, Zhou Jile. Collective matrix factorization hashing for multimodal data[C]//Proc of the IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2014: 2083−2090
25
Sun Yuan, Wang Xu , Peng Dezhong, et al. Hierarchical hashing learning for image set classification[J]. IEEE Transactions on Image Processing, 2023, 32: 1732−1744
26
Hoerl A, Kennard R. Ridge regression: Applications to nonorthogonal problems[J]. Technometrics, 1970, 12(1): 69−82
27
Roweis T, Saul K. Nonlinear dimensionality reduction by locally linear embedding[J]. Science, 2000, 290(5500): 2323−2326
28
He Xiaofei, Cai Deng, Yan Shuicheng, et al. Neighborhood preserving embedding[C]// Proc of the 10th IEEE Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2005: 1208−1213
29
Escalante H, Hernandez C, Gonzalez J, et al. The segmented and annotated IAPRTC-12 benchmark[J]. Computer Vision and Image Understanding, 2010, 144(4): 419−428
30
Huiskes M , Lew M. The MIR flickr retrieval evaluation[C]// Proc of the 1st ACM SIGMM Int Conf on Multimedia Information Retrieval. New York: ACM, 2008: 9−43
31
Wang Di, Wang Quan, Gao Xinbo. Robust and flexible discrete hashing for cross-modal similarity search[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2018, 28(10): 2703−2715
32
Wang Di, Wang Quan, He Lihuo, et al. Joint and individual matrix factorization hashing for large-scale cross-modal retrieval[J]. Pattern Recognition, 2020, 107: 107479
33
Wang, Di, Caiping Zhang, Quan Wang, et al. Hierarchical semantic structure preserving hashing for cross-modal retrieval[J]. IEEE Transactions on Multimedia, 2023, 25: 1217−1229
34
Liu Xin, Yi Jinhan, Cheung Yiuming, et al. OMGHUN: Online manifold-guided hashing for flexible cross-modal retrieval[J]. IEEE Transactions on Multimedia, 2023, 25: 3811−3824
2024年第61卷第11期
72
34
0
引用本文
BibTeX
文章信息
doi: 10.7544/issn1000-1239.202330433
  • 接收时间:2023-06-05
  • 首发时间:2026-11-17
  • 出版时间:2024-11-01
补充材料
相关文章
文章信息
作者
出版历史
  • 收稿日期:2023-06-05
  • 修回日期:2023-12-19
基金
作者信息
    1山东大学软件学院 济南 250101
    2山东建筑大学计算机科学与技术学院 济南 250101

通讯作者:

参考文献
分享链接
https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202330433
分享至
全文二维码

扫描看全文

引用本文
BibTeX
本文的引用情况
2种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏