Article(id=1327018437068566928, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1327018256403117010, articleNumber=null, orderNo=null, doi=10.7544/issn1000-1239.202550418, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=research-article, receivedDate=1749139200000, receivedDateStr=2025-06-06, revisedDate=1752595200000, revisedDateStr=2025-07-16, acceptedDate=null, acceptedDateStr=null, onlineDate=1794315450207, onlineDateStr=2026-11-10, pubDate=1756656000000, pubDateStr=2025-09-01, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1794315450207, onlineIssueDateStr=2026-11-10, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1794315450207, creator=13701087609, updateTime=1794315450207, updator=13701087609, issue=Issue{id=1327018256403117010, tenantId=1146029695717560320, journalId=1324759914146529478, year='2025', volume='62', issue='9', pageStart='2125', pageEnd='2395', issueExtLink='null', onlineDate='null', pubDate='1756656000000', pubDateStr='2025-09-01', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=1, issueType=-1, specialIssue=null, createTime=1794315407133, creator='13701087609', updateTime=1794376916348, updator='13701087609', preIssue=null, nextIssue=null, articleTotal=null, ext={EN=IssueExt(id=1327276245596876892, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1327018256403117010, language=EN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=), CN=IssueExt(id=1327276245596876893, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1327018256403117010, language=CN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=)}, issueFiles=null, downloadFileDto=null}, startPage=2166, endPage=2182, ext={EN=ArticleExt(id=1327018437236339089, articleId=1327018437068566928, tenantId=1146029695717560320, journalId=1324759914146529478, language=EN, title=Confidence-Guided Multimodal Data Test-Time Adaptation Framework, columnId=1327018257179063251, journalTitle=Journal of Computer Research and Development, columnName=Special Issue on Advances and Applications of Data-Centric AI, runingTitle=null, highlight=null, articleAbstract=

Test-time adaptation (TTA) seeks to dynamically recalibrate a deployed model through online fine-tuning with unlabeled or sparsely labeled data, addressing performance degradation caused by distribution shift, sensor noise, illumination changes, and other real-world challenges. It has demonstrated great promise in latency- and robustness-critical applications such as autonomous driving, remote healthcare, and video surveillance. However, existing multimodal TTA methods typically overlook the quality variance across views—low-quality or faulty views data can introduce harmful gradients—and lack mechanisms to preserve internal temporal consistency, undermining stability in dynamic scenarios. To tackle these issues, we propose CVPTA (confidence-guided view pruning and temporal contrastive attention), a unified framework comprising three core modules: 1) confidence-guided dynamic attention, which computes each view’s confidence via its predictive entropy and uses a softmax over inverted entropy scores to attenuate high-entropy views data during feature fusion; 2) view pruning, which evaluates image-modal quality using the Laplacian variance (for blur) and histogram skewness (for exposure), discards views below a quality threshold, and maintains a constant view count by interpolating adjacent high-quality views data or injecting Gaussian noise—dramatically reducing noise accumulation; 3) temporal contrastive self-supervision, which treats adjacent frames of the same view as positive pairs and non-adjacent frames as negatives, and applies a contrastive loss to enforce temporal feature consistency. We evaluate EVPTA on the public Kinetics-50-C and VGGSound-C perturbation benchmarks under the same online-update settings as prior work. Results show that EVPTA boosts Top-1 accuracy by approximately 2.3% and 0.7% on each benchmark and retains over 0.2% gains even under extreme noise conditions. Ablation studies further confirm the individual and synergistic contributions of all three modules. EVPTA requires no extra annotations, integrates seamlessly with existing multimodal systems, and delivers both efficiency and robustness—offering significant theoretical insights and practical value.

, authors=Xiaolong Yin, Dechuan Zhan*, Yuan Jiang, authorsList=Xiaolong Yin, Dechuan Zhan, Yuan Jiang, authorCompany=null, correspAuthors=Dechuan Zhan, authorNote=

born in 1998. PhD candidate. His main research interests include test-time adaptation and reinforcement learning

born in 1982. PhD, professor. His main research interests include machine learning and data mining

born in 1976. PhD, professor. Her main research interests include machine learning and data mining

, correspAuthorsNote=null, copyrightStatement=Copyright © 2025 Journal of Computer Research and Development. All rights reserved., copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=null, magXml=null, pdfUrl=null, pdf=null, pdfFileSize=null, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=null, mapNumber=null, fund=null), CN=ArticleExt(id=1327018439702589868, articleId=1327018437068566928, tenantId=1146029695717560320, journalId=1324759914146529478, language=CN, title=置信引导的多模态数据测试时适应框架, columnId=1327018257363612629, journalTitle=计算机研究与发展, columnName=数据为中心的AI前沿进展及应用, runingTitle=null, highlight=null, articleAbstract=

测试时适应(test time adaptation, TTA)旨在在模型部署后的测试阶段,通过无标注或少量标注的在线微调策略来动态校正模型,以应对由于分布偏移、传感器噪声、光照变化等引发的性能退化问题,并在自动驾驶、远程医疗、视频监控等对实时性和鲁棒性要求极高的领域展现出广泛应用价值. 然而,现有多模态 TTA 方法往往忽视各视图数据的质量差异,容易因低质量或故障视图引入有害梯度,且缺乏对视图内部动态变化的刻画,难以保持时序特征的一致性和稳定性. 为解决上述挑战,提出了一种置信度引导的视图裁剪与时序对比注意力(confidence-guided view pruning and temporal contrastive attention, CVPTA)框架. 该框架包含三大模块:首先,基于模型预测分布计算视图不确定性,自适应削弱低置信度视图数据对特征融合的干扰;其次,视图裁剪策略利用 Laplacian 方差与亮度直方图偏度评估图像质量,丢弃质量分数低于阈值的视图数据,并结合相邻高质量视图重采样,显著降低噪声累积风险;最后,时序对比自监督任务将同一样本的不同视图视为正样本而非同一样本视图视为负样本,通过对比损失强化时序特征一致性. 在 Kinetics-50-C 与 VGGSound-C 两个公开多模态扰动基准上,沿用先前实验配置进行在线更新评测. 结果显示,CVPTA 在2组基准上分别将 Top-1 准确率提升约 2.3 个百分点和 0.7个百分点,在极端噪声场景中依然保持超过 0.2% 的性能增益;消融研究进一步验证了各模块的独立贡献与协同效应. 该方法无需额外标注,可无缝集成现有多模态系统,兼具高效性与鲁棒性,具有重要的理论意义与工程应用前景.

, authors=尹小龙, 詹德川*, 姜远, authorsList=尹小龙, 詹德川, 姜远, authorCompany=null, correspAuthors=詹德川, authorNote=

1998年生. 博士研究生. 主要研究方向为测试时适应、强化学习

1982年生. 博士,教授. 主要研究方向为机器学习、数据挖掘

1976年生. 博士,教授. 主要研究方向为机器学习、数据挖掘

, correspAuthorsNote=
詹德川()
, copyrightStatement=版权所有 © 2025 计算机研究与发展编辑部, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=f8JIc3JKZRbwdibH08NxaA==, magXml=T4QcoM0QvPBESxUUX+sJSA==, pdfUrl=null, pdf=ke1x5oKPS4DtcoP+mQSBJQ==, pdfFileSize=1864183, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=tVv+CyWjyGxjW1UbNt+SEA==, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=dEsdlnCPzA+lT6wP4ggDwQ==, mapNumber=null, fund=null)}, authors=[Author(id=1327018440084271540, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=yinxl@lamda.nju.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327018440159769013, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, authorId=1327018440084271540, language=EN, stringName=Xiaolong Yin, firstName=Xiaolong, middleName=null, lastName=Yin, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"content":"

born in 1998. PhD candidate. His main research interests include test-time adaptation and reinforcement learning

"}, bioImg=null, bioContent=

born in 1998. PhD candidate. His main research interests include test-time adaptation and reinforcement learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327018440243655094, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, authorId=1327018440084271540, language=CN, stringName=尹小龙, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"img":"omG/CO4PpiqwqVradpj2AQ==","content":"

1998年生. 博士研究生. 主要研究方向为测试时适应、强化学习

"}, bioImg=omG/CO4PpiqwqVradpj2AQ==, bioContent=

1998年生. 博士研究生. 主要研究方向为测试时适应、强化学习

, aboutCorrespAuthor=null)}, companyList=null), Author(id=1327018440323346872, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=zhandc@nju.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1327018440386261433, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, authorId=1327018440323346872, language=EN, stringName=Dechuan Zhan, firstName=Dechuan, middleName=null, lastName=Zhan, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=*, address=null, bio={"content":"

born in 1982. PhD, professor. His main research interests include machine learning and data mining

"}, bioImg=null, bioContent=

born in 1982. PhD, professor. His main research interests include machine learning and data mining

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327018440470147514, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, authorId=1327018440323346872, language=CN, stringName=詹德川, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=*, address=null, bio={"img":"r209S81af2k/3jWvzGkZFg==","content":"

1982年生. 博士,教授. 主要研究方向为机器学习、数据挖掘

"}, bioImg=r209S81af2k/3jWvzGkZFg==, bioContent=

1982年生. 博士,教授. 主要研究方向为机器学习、数据挖掘

, aboutCorrespAuthor=null)}, companyList=null), Author(id=1327018440545644988, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327018440608559549, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, authorId=1327018440545644988, language=EN, stringName=Yuan Jiang, firstName=Yuan, middleName=null, lastName=Jiang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"content":"

born in 1976. PhD, professor. Her main research interests include machine learning and data mining

"}, bioImg=null, bioContent=

born in 1976. PhD, professor. Her main research interests include machine learning and data mining

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327018440675668414, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, authorId=1327018440545644988, language=CN, stringName=姜远, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"img":"nufmxhUnQyFK42Rqh1QonA==","content":"

1976年生. 博士,教授. 主要研究方向为机器学习、数据挖掘

"}, bioImg=nufmxhUnQyFK42Rqh1QonA==, bioContent=

1976年生. 博士,教授. 主要研究方向为机器学习、数据挖掘

, aboutCorrespAuthor=null)}, companyList=null)], keywords=[Keyword(id=1327018440776331711, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=EN, orderNo=1, keyword=test-time adaptation), Keyword(id=1327018440856023488, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=EN, orderNo=2, keyword=multimodal learning), Keyword(id=1327018440923132353, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=EN, orderNo=3, keyword=multi-view learning), Keyword(id=1327018440981852610, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=EN, orderNo=4, keyword=view pruning), Keyword(id=1327018441044767171, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=EN, orderNo=5, keyword=temporal contrastive learning), Keyword(id=1327018441107681732, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=CN, orderNo=1, keyword=测试时适应), Keyword(id=1327018441170596293, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=CN, orderNo=2, keyword=多模态学习), Keyword(id=1327018441237705158, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=CN, orderNo=3, keyword=多视图学习), Keyword(id=1327018441300619719, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=CN, orderNo=4, keyword=视图裁剪), Keyword(id=1327018441355145672, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=CN, orderNo=5, keyword=时序对比学习)], refs=[Reference(id=1327018443590709731, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=1, rfOrder=0, authorNames=null, journalName=null, refType=null, unstructuredReference=Gama J, Žliobaitė I, Bifet A, et al. A survey on concept drift adaptation[J]. ACM Computing Surveys, 2014, 46(4): 1−37, articleTitle=null, refAbstract=null), Reference(id=1327018443695567332, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=2, rfOrder=1, authorNames=null, journalName=null, refType=null, unstructuredReference=Moreno-Torres J G, Raeder T, Alaíz-Rodríguez R, et al. A unifying view on dataset shift in classification[J]. Pattern Recognition, 2012, 45(1): 521−530, articleTitle=null, refAbstract=null), Reference(id=1327018443762676197, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=3, rfOrder=2, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Mei, Deng Weihong. Deep visual domain adaptation: A survey[J]. Neurocomputing, 2018, 312: 135−153, articleTitle=null, refAbstract=null), Reference(id=1327018443838173670, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=4, rfOrder=3, authorNames=null, journalName=null, refType=null, unstructuredReference=Sun Yu, Wang Xiaolong, Liu Zhuang, et al. Test-time training with self-supervision for generalization under distribution shifts[C]//Proc of the 37th Int Conf on Machine Learning. Online, JMLR. org, 2020: 9229−9248, articleTitle=null, refAbstract=null), Reference(id=1327018443909476839, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=5, rfOrder=4, authorNames=null, journalName=null, refType=null, unstructuredReference=Pan S J, Yang Qiang. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345−1359, articleTitle=null, refAbstract=null), Reference(id=1327018443989168616, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=6, rfOrder=5, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhou Kaiyang, Liu Ziwei, Qiao Yu, et al. Domain generalization: A survey[J/OL]. 2021[2022-08-08]. https://arxiv.org/abs/2103.02503, articleTitle=null, refAbstract=null), Reference(id=1327018444068860393, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=7, rfOrder=6, authorNames=null, journalName=null, refType=null, unstructuredReference=Liang Jian, He Ran, Tan Tieniu. A comprehensive survey on test-time adaptation under distribution shifts[J]. International Journal of Computer Vision, 2023, 133: 31−64, articleTitle=null, refAbstract=null), Reference(id=1327018444144357866, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=8, rfOrder=7, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Dequan, Shelhamer E, Liu Shaoteng, et al. Tent: Fully test-time adaptation by entropy minimization[C/OL]//Proc of the 9th In Int Conf on Learning Representations. Vienna: OpenReview. net, 2021[2021-03-18]. https://openreview.net/forum?id=uXl3bZLkr3c, articleTitle=null, refAbstract=null), Reference(id=1327018444228243947, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=9, rfOrder=8, authorNames=null, journalName=null, refType=null, unstructuredReference=Wilson G, Cook D J. A survey of unsupervised deep domain adaptation[J]. ACM Transactions on Intelligent Systems and Technology, 2020, 11(5): 1−46, articleTitle=null, refAbstract=null), Reference(id=1327018444328907244, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=10, rfOrder=9, authorNames=null, journalName=null, refType=null, unstructuredReference=Panagiotakopoulos T, Dovesi P L, Härenstam-Nielsen L, et al. Online domain adaptation for semantic segmentation in ever-changing conditions[C]// Proc of the 17th European Conf on Computer Vision (ECCV 2022). Berlin: Springer, 2022: 128−146, articleTitle=null, refAbstract=null), Reference(id=1327018444404404717, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=11, rfOrder=10, authorNames=null, journalName=null, refType=null, unstructuredReference=Yang Tao, Zhou Shenglong, Wang Yuwang, et al. Test-time batch normalization[J/OL]. 2022[2022-05-20]. https://arxiv.org/abs/2205.10210., articleTitle=null, refAbstract=null), Reference(id=1327018444475707886, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=12, rfOrder=11, authorNames=null, journalName=null, refType=null, unstructuredReference=Park H, Hwang J, Mun S, et al. MedBN: Robust test-time adaptation against malicious mest mamples[C]//Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2024: 5997−6007, articleTitle=null, refAbstract=null), Reference(id=1327018444572176879, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=13, rfOrder=12, authorNames=null, journalName=null, refType=null, unstructuredReference=Lim H, Kim B, Choo J, et al. TTN: A domain-shift aware batch normalization in test-time adaptation[C/OL]// Proc of the 9th In Int Conf on Learning Representations. Kigali, OpenReview.net, 2023[2023-02-18]. https://arxiv.org/abs/2302.05155, articleTitle=null, refAbstract=null), Reference(id=1327018444660257264, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=14, rfOrder=13, authorNames=null, journalName=null, refType=null, unstructuredReference=Hong Junyuan, Lyu Lingjuan, Zhou Jiayu, et al. MECTA: Memory-economic continual test-time model adaptation[C/OL]// Proc of the 9th In Int Conf on Learning Representations. Kigali, OpenReview. net, 2023[2023-03-04]. https://openreview.net/pdf?id=N92hjSf5NNh, articleTitle=null, refAbstract=null), Reference(id=1327018444739949041, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=15, rfOrder=14, authorNames=null, journalName=null, refType=null, unstructuredReference=Sójka D, Cygert S, Twardowski B, et al. AR-TTA: A simple method for real-world continual test-time adaptation[C/OL]//Proc of British Machine Vision Conf. Glasgow: BMVA. org, 2024[2024-10-14]. https://arxiv.org/pdf/2309.10109, articleTitle=null, refAbstract=null), Reference(id=1327018444807057906, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=16, rfOrder=15, authorNames=null, journalName=null, refType=null, unstructuredReference=Yu Yongcan, Sheng Lijun, He Ran, et al. Benchmarking test-time adaptation against distribution shifts in image classification[J]. arXiv preprint, arXiv: 2307.03133, 2023, articleTitle=null, refAbstract=null), Reference(id=1327018444869972467, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=17, rfOrder=16, authorNames=null, journalName=null, refType=null, unstructuredReference=Chen Dian, Wang Dequan, Darrell T, et al. Contrastive test-time adaptation[C]//Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 295−305, articleTitle=null, refAbstract=null), Reference(id=1327018444937081332, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=18, rfOrder=17, authorNames=null, journalName=null, refType=null, unstructuredReference=Boudiaf M, Mueller R, Ayed I B, et al. Parameter-free online test-time adaptation[C]// /Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 8334−8343, articleTitle=null, refAbstract=null), Reference(id=1327018445041938933, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=19, rfOrder=18, authorNames=null, journalName=null, refType=null, unstructuredReference=周植, 张丁楚, 李宇峰, 等. 面向开集识别的稳健测试时适应方法. 软件学报, 2024, 35(4): 1667−1681, articleTitle=null, refAbstract=null), Reference(id=1327018445192933878, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=19, rfOrder=19, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhou Zhi, Zhang Dingchu, Li Yufeng, et al. Robust test-time adaptation method for open-set recognition[J]. Journal of Software, 2024, 35(4): 1667−1681(in Chinese), articleTitle=null, refAbstract=null), Reference(id=1327018445297791479, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=20, rfOrder=20, authorNames=null, journalName=null, refType=null, unstructuredReference=Baltrušaitis T, Ahuja C, Morency L. Multimodal machine learning: A survey and taxonomy[J]. IEEE transactions on pattern analysis and machine intelligence, 2018, 41(2): 423−443, articleTitle=null, refAbstract=null), Reference(id=1327018445381677560, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=21, rfOrder=21, authorNames=null, journalName=null, refType=null, unstructuredReference=Radford A, Kim J, Hallacy C, et al. Learning transferable visual models from natural language supervision[C/OL]// Proc of Int Conf on Machine Learning. New York: ACM, 2021[2021-02-26]. https://arxiv.org/abs/2103.00020, articleTitle=null, refAbstract=null), Reference(id=1327018445461369337, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=22, rfOrder=22, authorNames=null, journalName=null, refType=null, unstructuredReference=Lei Jixiang, Pernkopf F. Two-level test-time adaptation in multimodal learning[C/OL]//Proc of ICML 2024 Workshop on Foundation Models in the Wild. New York: ACM, 2024[2025-05-10]. https://icml.cc/virtual/2024/39426., articleTitle=null, refAbstract=null), Reference(id=1327018445528478202, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=23, rfOrder=23, authorNames=null, journalName=null, refType=null, unstructuredReference=Mao Tingting, Li Chang, Zhao Yuchang, et al. Online test-time adaptation for patient-independent seizure prediction[J]. IEEE Sensors Journal 2023, 23(19): 23133−23144, articleTitle=null, refAbstract=null), Reference(id=1327018445603975675, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=24, rfOrder=24, authorNames=null, journalName=null, refType=null, unstructuredReference=Yuan Longhui, Binhui Xie, Shuang Li. Robust test-time adaptation in dynamic scenarios[C]// Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2023: 15922−15932, articleTitle=null, refAbstract=null), Reference(id=1327018445658501628, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=25, rfOrder=25, authorNames=null, journalName=null, refType=null, unstructuredReference=Varsavsky T, Orbes-Arteaga M, Sudre C H, et al. Test-time unsupervised domain adaptation[J]. arXiv preprint, arXiv: 2010.01926, 2020, articleTitle=null, refAbstract=null), Reference(id=1327018445721416189, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=26, rfOrder=26, authorNames=null, journalName=null, refType=null, unstructuredReference=Tsai Y Y , Chen F C , Chen A Y C, et al. GDA: Gneralized diffusion for robust test-time adaptation[C]//Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2024: 23242−23251, articleTitle=null, refAbstract=null), Reference(id=1327018445780136446, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=27, rfOrder=27, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang M, Levine S, Finn C, et al. Memo: Test time robustness via adaptation and augmentation[J]. Advances in Neural Information Processing Systems 2022, 35: 38629−38642, articleTitle=null, refAbstract=null), Reference(id=1327018445834662399, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=28, rfOrder=28, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Shuang, Yuan Longhui, Xie Binhui, et al. Generalized robust test-time adaptation in continuous dynamic scenarios[C/OL]// Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2023[2023-05-08]. https://arxiv.org/abs/2303.13899, articleTitle=null, refAbstract=null), Reference(id=1327018445893382656, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=29, rfOrder=29, authorNames=null, journalName=null, refType=null, unstructuredReference=Gong T, Kim Y, Lee T, et al. Sotta: Robust test-time adaptation on noisy data streams[J]. Advances in Neural Information Processing Systems 2023, 36: 14070−14093, articleTitle=null, refAbstract=null), Reference(id=1327018445956297217, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=30, rfOrder=30, authorNames=null, journalName=null, refType=null, unstructuredReference=Park H, Hwang J, Mun S, et al. Medbn: Robust test-time adaptation against malicious test samples[C] // Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2024: 5997−6007, articleTitle=null, refAbstract=null), Reference(id=1327018446027600386, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=31, rfOrder=31, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Qin, Fink O, Gool L, et al. Continual test-time domain adaptation[C/OL] // Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022[2022-06-01]. https://openaccess.thecvf.com/content/CVPR2022/papers/Wang_Continual_Test-Time_Domain_Adaptation_CVPR_2022_paper.pdf, articleTitle=null, refAbstract=null), Reference(id=1327018446086320643, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=32, rfOrder=32, authorNames=null, journalName=null, refType=null, unstructuredReference=Liu Jiaming, Xu Ran, Yang Senqiao, et al. Continual-mae: Adaptive distribution masked autoencoders for continual test-time adaptation[C]// Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2024: 28653−28663, articleTitle=null, refAbstract=null), Reference(id=1327018446145040900, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=33, rfOrder=33, authorNames=null, journalName=null, refType=null, unstructuredReference=Döbler M, Marsden R A, Yang Bin. Robust mean teacher for continual and gradual test-time adaptation[C]// Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2023: 7704−7714, articleTitle=null, refAbstract=null), Reference(id=1327018446216344069, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=34, rfOrder=34, authorNames=null, journalName=null, refType=null, unstructuredReference=Song J, Lee J, Kweon I S, et al. Ecotta: Memory-efficient continual test-time adaptation via self-distilled regularization[C]// Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2023: 11920−11929, articleTitle=null, refAbstract=null), Reference(id=1327018446312813062, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=35, rfOrder=35, authorNames=null, journalName=null, refType=null, unstructuredReference=Tang Yushun, Chen Shuoshuo, Kan Zhehan, et al. Learning visual conditioning tokens to correct domain shift for fully test-time adaptation[J]. IEEE Transactions on Multimedia 2024, 10(1145): 6260−6269, articleTitle=null, refAbstract=null), Reference(id=1327018446388310535, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=36, rfOrder=36, authorNames=null, journalName=null, refType=null, unstructuredReference=Karmanov A, Guan Dayan, Lu Shijian, et al. Efficient test-time adaptation of vision-language models[C] // Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. New York: ACM, 2024: 14162−14171, articleTitle=null, refAbstract=null), Reference(id=1327018446472196616, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=37, rfOrder=37, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhou Lihua, Ye Mao, Li Shuaifeng, et al. Bayesian test-time adaptation for vision-language models[J]. arXiv preprint, arXiv: 2503.09248, 2025, articleTitle=null, refAbstract=null), Reference(id=1327018446535111177, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=38, rfOrder=38, authorNames=null, journalName=null, refType=null, unstructuredReference=Döbler M, Marsden R A, Raichle T, et al. A lost opportunity for vision-language models: A comparative study of online test-time adaptation for vision-language models[J]. arXiv preprint, arXiv: 2405.14977, 2024, articleTitle=null, refAbstract=null), Reference(id=1327018446602220042, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=39, rfOrder=39, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Yunbei, Mehra A, Hamm J. OT-VP: Optimal transport-guided visual prompting for test-time adaptation[C] // Proc of IEEE/CVF Winter Conf on Applications of Computer Vision. Piscataway, NJ: IEEE, 2025: 1122−1132, articleTitle=null, refAbstract=null), Reference(id=1327018446690300427, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=40, rfOrder=40, authorNames=null, journalName=null, refType=null, unstructuredReference=Weijler L, Mirza M J, Sick L, et al. TTT-KD: Test-time training for 3D semantic segmentation through knowledge distillation from foundation models[J]. arXiv preprint, arXiv: 2403.11691, 2024, articleTitle=null, refAbstract=null), Reference(id=1327018446761603596, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=41, rfOrder=41, authorNames=null, journalName=null, refType=null, unstructuredReference=Bar Y, Shaer S, Romano Y. Protected test-time adaptation via online entropy matching: A betting approach[J]. Advances in Neural Information Processing Systems 2024, 37: 85467−85499, articleTitle=null, refAbstract=null), Reference(id=1327018446832906765, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=42, rfOrder=42, authorNames=null, journalName=null, refType=null, unstructuredReference=Dong Hao, Liu Moru, Zhou Kaiyang, et al. Advances in multimodal adaptation and generalization: From traditional approaches to foundation models[J]. arXiv preprint, arXiv: 2501.18592, 2025, articleTitle=null, refAbstract=null), Reference(id=1327018446929375758, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=43, rfOrder=43, authorNames=null, journalName=null, refType=null, unstructuredReference=Shu Manli, Nie Weili, Huang Dean, et al. Test-time prompt tuning for zero-shot generalization in vision-language models[J]. Advances in Neural Information Processing Systems 2022, 35: 14274−14289, articleTitle=null, refAbstract=null), Reference(id=1327018447000678927, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=44, rfOrder=44, authorNames=null, journalName=null, refType=null, unstructuredReference=Xiao Zehao, Shen Jiayi, Derakhshani M M, et al. Any-shift prompting for generalization over distributions[C] // Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2024: 13849−13860, articleTitle=null, refAbstract=null), Reference(id=1327018447067787792, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=45, rfOrder=45, authorNames=null, journalName=null, refType=null, unstructuredReference=Sun Jiachen, Ibrahim M, Hall M, et al. VPA: Fully test-time visual prompt adaptation[C/OL]//Proc of the 31st ACM Int Conf on Multimedia. New York: ACM, 2023: 5796−5806, articleTitle=null, refAbstract=null), Reference(id=1327018447168451089, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=46, rfOrder=46, authorNames=null, journalName=null, refType=null, unstructuredReference=Tsai Yun-Yun, Mao Chengzhi, Yang Junfeng. [J]. Advances in Neural Information Processing Systems, 2023, 36: 27897−27921, articleTitle=null, refAbstract=null), Reference(id=1327018447256531474, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=47, rfOrder=47, authorNames=null, journalName=null, refType=null, unstructuredReference=Hassan J, Gani H, Hussein N, et al. Align your prompts: Test-time prompting with distribution alignment for zero-shot generalization[J]. Advances in Neural Information Processing Systems, 2023, 36: 80396−80413, articleTitle=null, refAbstract=null), Reference(id=1327018447344611859, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=48, rfOrder=48, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Yanshuo, Cheraghian A, Hayder Z, et al. Backpropagation-free network for 3D test-time adaptation[C]//Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2024: 23231−23241, articleTitle=null, refAbstract=null), Reference(id=1327018447407526420, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=49, rfOrder=49, authorNames=null, journalName=null, refType=null, unstructuredReference=Osowiechi D, Noori M, Hakim G A V, et al. WATT: Weight average test-time adaptation of clip[J]. arXiv preprint, arXiv: 2406.13875, 2024, articleTitle=null, refAbstract=null), Reference(id=1327018447508189717, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=50, rfOrder=50, authorNames=null, journalName=null, refType=null, unstructuredReference=Chen Ting, Kornblith S, Norouzi M, et al. A simple framework for contrastive learning of visual representations[C/OL]//Proc of Int Conf on Machine Learning. New York: ACM, 2020[2020-07-01]. https://arxiv.org/abs/2002.05709, articleTitle=null, refAbstract=null), Reference(id=1327018447579492886, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=51, rfOrder=51, authorNames=null, journalName=null, refType=null, unstructuredReference=Xiao Zehao, Zhen Xiantong, Liao Shengcai, et al. Energy-based test sample adaptation for domain generalization[J]. arXiv preprint, arXiv: 2302.11215, 2023, articleTitle=null, refAbstract=null), Reference(id=1327018447646601751, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=52, rfOrder=52, authorNames=null, journalName=null, refType=null, unstructuredReference=Xiao Zehao, Zhen Xiantong, Shao Ling, et al. Learning to generalize across domains on single test samples[J]. arXiv preprint, arXiv: 2202.08045, 2022, articleTitle=null, refAbstract=null), Reference(id=1327018447713710616, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=53, rfOrder=53, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Yingming, Yang Ming, Zhang Zhongfei. A survey of multi-view representation learning[J]. IEEE Transactions on Knowledge and Data Engineering 2018, 31(10): 1863−1883, articleTitle=null, refAbstract=null), Reference(id=1327018447789208089, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=54, rfOrder=54, authorNames=null, journalName=null, refType=null, unstructuredReference=Trosten D J, Løkse S, Jenssen R, et al. Reconsidering representation alignment for multi-view clustering[C]// Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2021: 1255−1265, articleTitle=null, refAbstract=null), Reference(id=1327018447860511258, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=55, rfOrder=55, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Qingheng, Sun Zequn, Hu Wei, et al. Multi-view knowledge graph embedding for entity alignment[J]. arXiv preprint, arXiv: 1906.02390, 2019, articleTitle=null, refAbstract=null), Reference(id=1327018447956980251, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=56, rfOrder=56, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Ren, Sun Haoliang, Ma Yuling, et al. Metaviewer: Towards a unified multi-view representation[C]// Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2023: 11590−11599, articleTitle=null, refAbstract=null), Reference(id=1327018448024089116, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=57, rfOrder=57, authorNames=null, journalName=null, refType=null, unstructuredReference=Guo Chenfeng, Wu Dongrui. Canonical correlation analysis (CCA) based multi-view learning: An overview[J]. arXiv preprint, arXiv: 1907.01693, 2021, articleTitle=null, refAbstract=null), Reference(id=1327018448091197981, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=58, rfOrder=58, authorNames=null, journalName=null, refType=null, unstructuredReference=Blum A, Mitchell T. Combining labeled and unlabeled data with co-training[C]// Proc of the 11th Annual Conf on Computational Learning Theory. New York: ACM, 1998: 92–100, articleTitle=null, refAbstract=null), Reference(id=1327018448162501150, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=59, rfOrder=59, authorNames=null, journalName=null, refType=null, unstructuredReference=Chen Mansheng, Lin Jiaqi, Li Xianglong, et al. Representation learning in multi-view clustering: A literature review[J]. Data Science and Engineering 2022, 7(3): 225−241, articleTitle=null, refAbstract=null), Reference(id=1327018448288330271, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=60, rfOrder=60, authorNames=null, journalName=null, refType=null, unstructuredReference=Qin Zequn, Chen Jingyu, Chen Chao, et al. Unifusion: Unified multi-view fusion Transformer for spatial-temporal representation in bird's-eye-view[C]// Proc of IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2023: 8656−8665, articleTitle=null, refAbstract=null), Reference(id=1327018448368022048, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=61, rfOrder=61, authorNames=null, journalName=null, refType=null, unstructuredReference=Ke Guanzhou, Chao Guoqing, Wang Xiaoli, et al. A clustering-guided contrastive fusion for multi-view representation learning[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 34(4): 2056−2069, articleTitle=null, refAbstract=null), Reference(id=1327018448426742305, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=62, rfOrder=62, authorNames=null, journalName=null, refType=null, unstructuredReference=Cap Fuyuan, Chen Xiaohui. Multi-view attributed graph clustering with shared and specific representations[J]. Journal of Software, 2025, 36(3): 1254−1267, articleTitle=null, refAbstract=null), Reference(id=1327018448498045474, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=63, rfOrder=63, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhao Xingwang, Wang Shujun, Liu Xiaolin, et al. Joint spectral embedding for multi-view clustering via bipartite graph[J]. Journal of Software, 2024, 35(9): 4408−4424, articleTitle=null, refAbstract=null), Reference(id=1327018448569348643, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=64, rfOrder=64, authorNames=null, journalName=null, refType=null, unstructuredReference=Siddiqui N, Tirupattur P, Shah M. DVANet: Disentangling view and action features for multi-view action recognition[J]. Proceedings of the AAAI Conf on Artificial Intelligence, 2024, 38(5): 4873−4881, articleTitle=null, refAbstract=null), Reference(id=1327018448644846116, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=65, rfOrder=65, authorNames=null, journalName=null, refType=null, unstructuredReference=Bruce X B, Liu Yan, Zhang Xiang, et al. MMNet: Amodel-based multimodal network for human action recognition in RGB-D videos[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(3): 3522−3538, articleTitle=null, refAbstract=null), Reference(id=1327018448724537893, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=66, rfOrder=66, authorNames=null, journalName=null, refType=null, unstructuredReference=He Kaiming, Fan Haoqi, Wu Yuxin, et al. Momentum contrast for unsupervised visual representation learning[C] //Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2020: 9726−9735, articleTitle=null, refAbstract=null), Reference(id=1327018448787452454, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=67, rfOrder=67, authorNames=null, journalName=null, refType=null, unstructuredReference=Khosla P, Teterwak P, Wang Chen, et al. Supervised contrastive learning[J]. Advances in Neural Information Processing Systems 2020, 33: 18661−18673, articleTitle=null, refAbstract=null), Reference(id=1327018448871338535, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=68, rfOrder=68, authorNames=null, journalName=null, refType=null, unstructuredReference=Li Junnan, Zhou Pan, Xiong Caiming, et al. Prototypical contrastive learning of unsupervised representations[J]. arXiv preprint, arXiv: 2005.04966, 2021, articleTitle=null, refAbstract=null), Reference(id=1327018448934253096, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=69, rfOrder=69, authorNames=null, journalName=null, refType=null, unstructuredReference=Jia Chao, Yang Yinfei, Xia Ye, et al. Scaling up visual and vision-language representation learning with noisy text supervision[C/OL]//Proc of Int Conf on Machine Learning. New York: ACM, 2021[2021-06-11]. https://arxiv.org/abs/2102.05918, articleTitle=null, refAbstract=null), Reference(id=1327018449001361961, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=70, rfOrder=70, authorNames=null, journalName=null, refType=null, unstructuredReference=Grill J B, Strub F, Altché F, et al. Bootstrap your own latent-a new approach to self-supervised learning[C]// Proc of the 34th Int Conf on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2020: 21271−21284, articleTitle=null, refAbstract=null), Reference(id=1327018449081053738, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=71, rfOrder=71, authorNames=null, journalName=null, refType=null, unstructuredReference=Chen Xinlei, He Kaiming. Exploring simple siamese representation learning[C]// Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2021: 15745−15753, articleTitle=null, refAbstract=null), Reference(id=1327018449156551211, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=72, rfOrder=72, authorNames=null, journalName=null, refType=null, unstructuredReference=Kay W, Carreira J, Simonyan K, et al. The Kinetics human action video dataset[J]. arXiv preprint, arXiv: 1705.06950, 2017, articleTitle=null, refAbstract=null), Reference(id=1327018449227854380, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=73, rfOrder=73, authorNames=null, journalName=null, refType=null, unstructuredReference=Chen Honglie, Xie Weidi, Vedaldi A, et al. Vggsound: A large-scale audio-visual dataset[C] // Proc of IEEE Int Conf on Acoustics, Speech and Signal Processing. Piscataway, NJ: IEEE, 2020: 721−725, articleTitle=null, refAbstract=null), Reference(id=1327018449290768941, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=74, rfOrder=74, authorNames=null, journalName=null, refType=null, unstructuredReference=Hendrycks D, Dietterich T. Benchmarking neural network robustness to common corruptions and perturbations[J]. arXiv preprint, arXiv: 1903.12261, 2019, articleTitle=null, refAbstract=null), Reference(id=1327018449374655022, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=75, rfOrder=75, authorNames=null, journalName=null, refType=null, unstructuredReference=Wang Dequan, Shelhamer E, Liu Shaoteng, et al. Tent: Fully test-time adaptation by entropy minimization[C/OL]// Proc of the 9th Int Conf on Learning Representations. Vienna: OpenReview. net, 2021[2021-03-18]. https://openreview.net/forum?id=uXl3bZLkr3c, articleTitle=null, refAbstract=null), Reference(id=1327018449458541103, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=76, rfOrder=76, authorNames=null, journalName=null, refType=null, unstructuredReference=Shin I, Tsai Y, Zhuang Bingbing, et al. MM-TTA: Multi-modal test-time adaptation for 3D semantic segmentation[C] // Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 16907−16916, articleTitle=null, refAbstract=null), Reference(id=1327018449529844272, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=77, rfOrder=77, authorNames=null, journalName=null, refType=null, unstructuredReference=Niu Shuaicheng, Wu Jiaxiang, Zhang Yifan, et al. Efficient test-time model adaptation without forgetting[C/OL]//Proc of Int Conf on Machine Learning, New York: ACM, 2022[2022-05-31]. https://arxiv.org/abs/2204.02610, articleTitle=null, refAbstract=null), Reference(id=1327018449601147441, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=78, rfOrder=78, authorNames=null, journalName=null, refType=null, unstructuredReference=Niu Shuaicheng, Wu Jiaxiang, Zhang Yifan, et al. Towards stable test-time adaptation in dynamic wild world[J]. arXiv preprint, arXiv: 2302.12400, 2023, articleTitle=null, refAbstract=null), Reference(id=1327018449668256306, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=79, rfOrder=79, authorNames=null, journalName=null, refType=null, unstructuredReference=Gong Yuan, Rouditchenko A, Liu A H, et al. Contrastive audio-visual masked autoencoder[J]. arXiv preprint, arXiv: 2210.07839, 2023, articleTitle=null, refAbstract=null), Reference(id=1327018449731170867, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=80, rfOrder=80, authorNames=null, journalName=null, refType=null, unstructuredReference=Yang Mouxing, Li Yunfan, Zhang Changqing, et al. Test-time adaptation against multi-modal reliability bias[C/OL]// Proc of the 12th Int Conf on Learning Representations. Vienna: OpenReview. net, 2024[2024-01-16]. https://openreview.net/forum?id=TPZRq4FALB, articleTitle=null, refAbstract=null)], funds=null, companyList=[AuthorCompany(id=1327018439924887981, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, xref=null, ext=[AuthorCompanyExt(id=1327018439933276590, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, companyId=1327018439924887981, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=School of Artificial Intelligence, Nanjing University, Nanjing 210023), AuthorCompanyExt(id=1327018439941665199, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, companyId=1327018439924887981, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=南京大学人工智能学院 南京 210023)]), AuthorCompany(id=1327018440004579760, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, xref=null, ext=[AuthorCompanyExt(id=1327018440012968369, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, companyId=1327018440004579760, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=National Key Laboratory for Novel Software Technology (Nanjing University), Nanjing 210023), AuthorCompanyExt(id=1327018440021356978, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, companyId=1327018440004579760, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=计算机软件新技术全国重点实验室(南京大学) 南京 210023)])], figs=[ArticleFig(id=1327018441489363401, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=EN, label=Figure 1, caption=The pipeline of CVPTA, figureFileSmall=1AM676f4/lUQ3UXHrLbGqQ==, figureFileBig=tVv+CyWjyGxjW1UbNt+SEA==, tableContent=null), ArticleFig(id=1327018441552277962, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=CN, label=图1, caption=CVPTA流程图, figureFileSmall=1AM676f4/lUQ3UXHrLbGqQ==, figureFileBig=tVv+CyWjyGxjW1UbNt+SEA==, tableContent=null), ArticleFig(id=1327018441741021643, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=EN, label=Figure 2, caption=Ablation experiment for entropy-guided dynamic attention mechanisms, figureFileSmall=mtPvIlef3mkVSCzJQ5+agw==, figureFileBig=ymO9g9rQme0gZmR7G/z+Kw==, tableContent=null), ArticleFig(id=1327018441808130508, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=CN, label=图2, caption=熵引导的动态注意力机制的消融实验, figureFileSmall=mtPvIlef3mkVSCzJQ5+agw==, figureFileBig=ymO9g9rQme0gZmR7G/z+Kw==, tableContent=null), ArticleFig(id=1327018441883627981, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=EN, label=Figure 3, caption=Ablation experiment for low-quality view clipping and pseudo-resampling, figureFileSmall=EYuKbERQsH8f1OpMKjmkzw==, figureFileBig=UIGLpJzm9cSTJt5+yK41fw==, tableContent=null), ArticleFig(id=1327018441971708366, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=CN, label=图3, caption=低质视图裁剪与伪重采样的消融实验, figureFileSmall=EYuKbERQsH8f1OpMKjmkzw==, figureFileBig=UIGLpJzm9cSTJt5+yK41fw==, tableContent=null), ArticleFig(id=1327018442051400143, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=EN, label=Figure 4, caption=Ablation experiment for contrast loss, figureFileSmall=kFYw/38J50gxcQNMRMFOog==, figureFileBig=ZeXemEnGIpJuNBrUFO8VQw==, tableContent=null), ArticleFig(id=1327018442122703312, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=CN, label=图4, caption=对比损失的消融实验, figureFileSmall=kFYw/38J50gxcQNMRMFOog==, figureFileBig=ZeXemEnGIpJuNBrUFO8VQw==, tableContent=null), ArticleFig(id=1327018442198200785, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=EN, label=Figure 5, caption=Low-quality view clipping and pseudo-resampling threshold sensitivity analysis, figureFileSmall=IariQdhHKblpNJFXFA6/pQ==, figureFileBig=ZWmjuoeYuKHZ6u/5s1BhBQ==, tableContent=null), ArticleFig(id=1327018442277892562, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=CN, label=图5, caption=低质视图裁剪与伪重采样阈值敏感度分析, figureFileSmall=IariQdhHKblpNJFXFA6/pQ==, figureFileBig=ZWmjuoeYuKHZ6u/5s1BhBQ==, tableContent=null), ArticleFig(id=1327018442361778643, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=EN, label=Figure 6, caption=Time series comparative self-supervised loss weights sensitivity analysis, figureFileSmall=5fudrrJG1s8p4cBjtnttgA==, figureFileBig=QREnHuRCPm9gY2LB1ki62g==, tableContent=null), ArticleFig(id=1327018442445664724, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=CN, label=图6, caption=时序对比自监督损失权重敏感度分析, figureFileSmall=5fudrrJG1s8p4cBjtnttgA==, figureFileBig=QREnHuRCPm9gY2LB1ki62g==, tableContent=null), ArticleFig(id=1327018442504384981, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=EN, label=Figure 7, caption=Temperature parameters sensitivity analysis, figureFileSmall=sSmrwGav8TOtwY4SaiCGPQ==, figureFileBig=nJ8anyMuXr4Ggw4cthGjFQ==, tableContent=null), ArticleFig(id=1327018442575688150, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=CN, label=图7, caption=温度参数敏感度分析, figureFileSmall=sSmrwGav8TOtwY4SaiCGPQ==, figureFileBig=nJ8anyMuXr4Ggw4cthGjFQ==, tableContent=null), ArticleFig(id=1327018442642797015, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=EN, label=Figure 8, caption=Cross modal alignment loss, figureFileSmall=UKG1NWPwfBt74+tPm/9jGQ==, figureFileBig=PC6rooi2paTwgGRy//Trlg==, tableContent=null), ArticleFig(id=1327018442705711576, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=CN, label=图8, caption=跨模态对齐损失, figureFileSmall=UKG1NWPwfBt74+tPm/9jGQ==, figureFileBig=PC6rooi2paTwgGRy//Trlg==, tableContent=null), ArticleFig(id=1327018442768626137, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=EN, label=Figure 9, caption=Distribution constrains balance loss, figureFileSmall=NF+PQ76NqnAjeoEEBWBzpQ==, figureFileBig=W3e7maXAPGWBEOL53SuVUg==, tableContent=null), ArticleFig(id=1327018442844123610, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=CN, label=图9, caption=分布约束平衡损失, figureFileSmall=NF+PQ76NqnAjeoEEBWBzpQ==, figureFileBig=W3e7maXAPGWBEOL53SuVUg==, tableContent=null), ArticleFig(id=1327018442919621083, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=EN, label=Figure 10, caption=Perceived loss of reliability, figureFileSmall=PhhALFqgPnSlJ758wD7mlA==, figureFileBig=oi3NfqxlmbSa4krNig5Z7A==, tableContent=null), ArticleFig(id=1327018443020284381, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=CN, label=图10, caption=可靠性感知损失, figureFileSmall=PhhALFqgPnSlJ758wD7mlA==, figureFileBig=oi3NfqxlmbSa4krNig5Z7A==, tableContent=null), ArticleFig(id=1327018443091587549, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=EN, label=Table 1, caption=video-corruption Experimental Data on Kinetics-50 Dataset %, figureFileSmall=null, figureFileBig=null, tableContent=
方法NoiseBlurDigital
GaussianShotImpulseDefocusGlassMotionZoomContractElas.Pix.JPEG
注:黑体数值为最优值,“Elas.”是“Elastic-transform”的缩写,“Pix.”是“Pixelate”的缩写.
源((Stat.LN)& LF)31.833.431.764.054.367.561.944.060.261.756.4
MM-TTA(Dyn. LN)46.246.646.158.855.762.658.749.357.656.454.6
Tent(Dyn. LN)28.629.828.363.451.167.761.738.660.761.854.9
EATA(Dyn. LN)31.833.331.664.254.667.762.244.260.462.057.0
SAR(Dyn. LN)31.933.331.763.854.067.761.844.060.362.056.5
READ(Dyn. LN)34.034.533.865.357.768.764.948.762.964.659.2
源(Stat.(LN& AF))46.848.046.967.562.270.866.752.165.766.561.9
Tent(Dyn. LN)46.347.046.367.262.571.067.651.666.867.262.7
EATA(Dyn. LN)46.847.647.167.262.770.667.252.465.966.862.5
SAR(Dyn. LN)46.747.446.867.061.970.466.452.165.766.462.0
READ(SAF)49.449.749.068.065.171.268.253.368.270.168.3
CVPTA63.252.351.667.065.371.168.954.469.371.170.7
), ArticleFig(id=1327018443158696414, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=CN, label=表1, caption=

Kinetics-50数据集上的video-corruption实验数据

, figureFileSmall=null, figureFileBig=null, tableContent=
方法NoiseBlurDigital
GaussianShotImpulseDefocusGlassMotionZoomContractElas.Pix.JPEG
注:黑体数值为最优值,“Elas.”是“Elastic-transform”的缩写,“Pix.”是“Pixelate”的缩写.
源((Stat.LN)& LF)31.833.431.764.054.367.561.944.060.261.756.4
MM-TTA(Dyn. LN)46.246.646.158.855.762.658.749.357.656.454.6
Tent(Dyn. LN)28.629.828.363.451.167.761.738.660.761.854.9
EATA(Dyn. LN)31.833.331.664.254.667.762.244.260.462.057.0
SAR(Dyn. LN)31.933.331.763.854.067.761.844.060.362.056.5
READ(Dyn. LN)34.034.533.865.357.768.764.948.762.964.659.2
源(Stat.(LN& AF))46.848.046.967.562.270.866.752.165.766.561.9
Tent(Dyn. LN)46.347.046.367.262.571.067.651.666.867.262.7
EATA(Dyn. LN)46.847.647.167.262.770.667.252.465.966.862.5
SAR(Dyn. LN)46.747.446.867.061.970.466.452.165.766.462.0
READ(SAF)49.449.749.068.065.171.268.253.368.270.168.3
CVPTA63.252.351.667.065.371.168.954.469.371.170.7
), ArticleFig(id=1327018443238388191, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=EN, label=Table 2, caption=audio-corruption Experimental Data on Kinetics-50 Dataset %, figureFileSmall=null, figureFileBig=null, tableContent=
方法NoiseWeather平均值
GaussianTrafficCrowdRainThunderWind
注:黑体数值表示最优值.
源((Stat.LN)& LF)71.167.867.467.470.668.668.8
MM-TTA(Dyn. LN)70.869.268.569.069.869.469.4
Tent(Dyn. LN)71.168.667.867.471.268.969.2
EATA(Dyn. LN)71.267.967.567.870.968.769.0
SAR(Dyn. LN)71.167.567.467.470.668.668.8
READ(Dyn. LN)71.368.568.568.471.869.069.6
源(Stat.(LN& AF))73.765.567.970.367.970.369.3
Tent(Dyn. LN)73.967.469.270.466.570.569.6
EATA(Dyn. LN)73.766.168.570.367.970.169.4
SAR(Dyn. LN)73.765.468.269.967.270.269.1
READ(SAF)74.169.069.771.171.870.771.1
CVPTA74.969.171.071.472.471.071.6
), ArticleFig(id=1327018443322274272, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=CN, label=表2, caption=

Kinetics-50数据集上的audio-corruption实验数据

, figureFileSmall=null, figureFileBig=null, tableContent=
方法NoiseWeather平均值
GaussianTrafficCrowdRainThunderWind
注:黑体数值表示最优值.
源((Stat.LN)& LF)71.167.867.467.470.668.668.8
MM-TTA(Dyn. LN)70.869.268.569.069.869.469.4
Tent(Dyn. LN)71.168.667.867.471.268.969.2
EATA(Dyn. LN)71.267.967.567.870.968.769.0
SAR(Dyn. LN)71.167.567.467.470.668.668.8
READ(Dyn. LN)71.368.568.568.471.869.069.6
源(Stat.(LN& AF))73.765.567.970.367.970.369.3
Tent(Dyn. LN)73.967.469.270.466.570.569.6
EATA(Dyn. LN)73.766.168.570.367.970.169.4
SAR(Dyn. LN)73.765.468.269.967.270.269.1
READ(SAF)74.169.069.771.171.870.771.1
CVPTA74.969.171.071.472.471.071.6
), ArticleFig(id=1327018443393577441, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=EN, label=Table 3, caption=video-corruption Experimental Data on VGGSound Dataset, figureFileSmall=null, figureFileBig=null, tableContent=
方法NoiseBlurDigital
GaussianShotImpulseDefocusGlassMotionZoomContractElas.Pix.JPEG
注:黑体数值为最优值.“Elas.”是“Elastic-transform”的缩写,“Pix.”是“Pixelate”的缩写.
源((Stat.LN)& LF)37.736.537.852.751.355.253.745.252.551.752.3
MM-TTA(Dyn. LN)7.17.37.344.841.548.045.524.240.340.745.7
Tent(Dyn. LN)7.66.87.253.152.155.554.516.652.654.253.1
EATA(Dyn. LN)37.736.537.753.252.356.054.445.253.552.352.7
SAR(Dyn. LN)37.736.437.752.851.555.553.944.852.751.852.3
READ(Dyn. LN)42.141.542.149.350.953.552.546.252.549.150.2
源(Stat.(LN& AF))52.852.752.757.257.258.757.653.756.955.856.9
Tent(Dyn. LN)52.752.752.756.756.557.957.254.057.456.256.7
EATA(Dyn. LN)53.052.853.057.257.158.657.854.157.456.157.0
SAR(Dyn. LN)52.952.852.957.257.158.657.654.057.056.057.0
READ(SAF)52.852.952.657.157.058.858.054.658.056.657.2
CVPTA53.854.153.758.157.959.759.055.258.957.758.4
), ArticleFig(id=1327018443481657826, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, language=CN, label=表3, caption=

VGGSound数据集上的video-corruption实验数据

, figureFileSmall=null, figureFileBig=null, tableContent=
方法NoiseBlurDigital
GaussianShotImpulseDefocusGlassMotionZoomContractElas.Pix.JPEG
注:黑体数值为最优值.“Elas.”是“Elastic-transform”的缩写,“Pix.”是“Pixelate”的缩写.
源((Stat.LN)& LF)37.736.537.852.751.355.253.745.252.551.752.3
MM-TTA(Dyn. LN)7.17.37.344.841.548.045.524.240.340.745.7
Tent(Dyn. LN)7.66.87.253.152.155.554.516.652.654.253.1
EATA(Dyn. LN)37.736.537.753.252.356.054.445.253.552.352.7
SAR(Dyn. LN)37.736.437.752.851.555.553.944.852.751.852.3
READ(Dyn. LN)42.141.542.149.350.953.552.546.252.549.150.2
源(Stat.(LN& AF))52.852.752.757.257.258.757.653.756.955.856.9
Tent(Dyn. LN)52.752.752.756.756.557.957.254.057.456.256.7
EATA(Dyn. LN)53.052.853.057.257.158.657.854.157.456.157.0
SAR(Dyn. LN)52.952.852.957.257.158.657.654.057.056.057.0
READ(SAF)52.852.952.657.157.058.858.054.658.056.657.2
CVPTA53.854.153.758.157.959.759.055.258.957.758.4
)], attaches=null, journal=Journal(id=1324759606947315908, delFlag=0, nameCn=计算机研究与发展, nameEn=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, issn=1000-1239, eissn=null, cn=11-1777/TP, coden=null, periodic=0, language=CN, oaType=null, ccby=null, superviseOffice=null, ownerOffice=null, pubOffice=null, editorOffice=null, officeType=null, aims=null, clcCode=null, officeProv=null, officeCity=null, officeAddr=null, officeZip=null, officeEmail=null, officePhone=null, editDirector=null, officeDirector=null, officeDirectorPhone=null, officeStaffNum=null, officeEmpNum=null, coverPicUrl=k2l8zgqwstuxxevFX2qxVA==, journalPrice=null, startedYear=null, abbrevIsoEn=Journal of Computer Research and Development, journalRemark=null, publicationField=null, createdTime=1793776903141, updatedTime=1793850698612, createdBy=18614031015, updatedBy=13701087609, firstLetterCn=J, firstLetterEn=J, subjectCode=Electronic Information, subjectName=null, subjectCodeEn=Electronic Information, subjectNameEn=null, picCn=k2l8zgqwstuxxevFX2qxVA==, picEn=r6k0Ry+RD+c/VXj5QarmDw==, jcr=null, cjcr=null, exts=[JournalExt(id=1325069127710286688, language=CN, name=计算机研究与发展, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698640, updatedTime=1793850698640, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""}), JournalExt(id=1325069127777395553, language=EN, name=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698656, updatedTime=1793850698656, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""})], databaseList=null, tenantJournalId=1324759914146529478, websiteList=[Website(id=1325070300139892771, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN, language=CN, createTime=1793850978169, createBy=18614031015, updateTime=1793851040259, updateBy=18614031015, name=计算机研究与发展-中文, tplId=1146099689490845704, title=计算机研究与发展, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070893751349360, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=articleTextType, value=kx, createTime=1793851119697, updateTime=1793851119697, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893730377837, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=banner, value=null, createTime=1793851119692, updateTime=1793851119692, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893772320883, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=grayFlag, value=0, createTime=1793851119702, updateTime=1793851119702, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893721989228, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851119690, updateTime=1793851119690, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893784903797, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=minRunFlag, value=0, createTime=1793851119705, updateTime=1793851119705, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893742960751, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic, createTime=1793851119695, updateTime=1793851119695, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893780709492, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=silenceFlag, value=0, createTime=1793851119704, updateTime=1793851119704, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893738766446, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_cn_619/, createTime=1793851119694, updateTime=1793851119694, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893755543665, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeColor, value=null, createTime=1793851119698, updateTime=1793851119698, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893763932274, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeStyle, value=null, createTime=1793851119700, updateTime=1793851119700, creator=18614031015, updator=18614031015)]), Website(id=1325070300580294698, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN, language=EN, createTime=1793850978274, createBy=18614031015, updateTime=1793851036638, updateBy=18614031015, name=计算机研究与发展-英文, tplId=1146101810881728533, title=Journal of Computer Research and Development, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070925288321148, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=articleTextType, value=kx, createTime=1793851127216, updateTime=1793851127216, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925263155321, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=banner, value=null, createTime=1793851127210, updateTime=1793851127210, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925309292671, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=grayFlag, value=0, createTime=1793851127221, updateTime=1793851127221, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925254766712, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851127208, updateTime=1793851127208, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925326069889, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=minRunFlag, value=0, createTime=1793851127225, updateTime=1793851127225, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925279932539, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic, createTime=1793851127214, updateTime=1793851127214, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925317681280, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=silenceFlag, value=0, createTime=1793851127223, updateTime=1793851127223, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925271543930, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_en_623/, createTime=1793851127212, updateTime=1793851127212, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925292515453, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeColor, value=null, createTime=1793851127217, updateTime=1793851127217, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925300904062, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeStyle, value=null, createTime=1793851127219, updateTime=1793851127219, creator=18614031015, updator=18614031015)])], journalTitle=计算机研究与发展, weixinUrl=null, journalUrl=https://crad.ict.ac.cn/, iacademicId=null, status=1, seqNo=null, journalTitleEn=Journal of Computer Research and Development, journalPhotoCn=k2l8zgqwstuxxevFX2qxVA==, journalPhotoEn=r6k0Ry+RD+c/VXj5QarmDw==, journalFirstLetter=J, journalRecommend=null, journalNew=null, journalCollection=null, jcrJf=null, cjcrJf=null, jcrJfStr=null, cjcrJfStr=null, submissionFirstDecision=null, sciSubjectClassification=null, casSubjectClassification=null, citeScore=null, totalCitationFrequency=null, icpCode=null, psCode=null, advertisingLicenseCode=null, copyrightInformation=null, country=null, option=, provinceCode=null, provinceName=null, collectFlag=false, interPubPlatform=, interPubPlatformUrl=null), detailUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202550418, detailUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202550418, pdfUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/PDF/10.7544/issn1000-1239.202550418, pdfUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/PDF/10.7544/issn1000-1239.202550418, aliStartDate=0, aliEndDate=0, collectionFlag=false, citedCount=null, citedUrl=null, previewStatus=0, delFlag=0, hasFullText=1, orderTime=1756656000000, fullTextJson=null, articleText=null, reference=null)
收藏切换
置信引导的多模态数据测试时适应框架
尹小龙 , 詹德川 * , 姜远
计算机研究与发展 | 数据为中心的AI前沿进展及应用 2025,62(9): 2166-2182
收起
收藏切换
计算机研究与发展 |数据为中心的AI前沿进展及应用 2025 , 62 (9) : 2166 -2182
置信引导的多模态数据测试时适应框架
全屏
[Author(id=1327018440084271540, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=yinxl@lamda.nju.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327018440159769013, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, authorId=1327018440084271540, language=EN, stringName=Xiaolong Yin, firstName=Xiaolong, middleName=null, lastName=Yin, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"content":"

born in 1998. PhD candidate. His main research interests include test-time adaptation and reinforcement learning

"}, bioImg=null, bioContent=

born in 1998. PhD candidate. His main research interests include test-time adaptation and reinforcement learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327018440243655094, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, authorId=1327018440084271540, language=CN, stringName=尹小龙, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"img":"omG/CO4PpiqwqVradpj2AQ==","content":"

1998年生. 博士研究生. 主要研究方向为测试时适应、强化学习

"}, bioImg=omG/CO4PpiqwqVradpj2AQ==, bioContent=

1998年生. 博士研究生. 主要研究方向为测试时适应、强化学习

, aboutCorrespAuthor=null)}, companyList=null), Author(id=1327018440323346872, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=zhandc@nju.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=1, authorType=1, ext={EN=AuthorExt(id=1327018440386261433, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, authorId=1327018440323346872, language=EN, stringName=Dechuan Zhan, firstName=Dechuan, middleName=null, lastName=Zhan, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=*, address=null, bio={"content":"

born in 1982. PhD, professor. His main research interests include machine learning and data mining

"}, bioImg=null, bioContent=

born in 1982. PhD, professor. His main research interests include machine learning and data mining

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327018440470147514, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, authorId=1327018440323346872, language=CN, stringName=詹德川, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=*, address=null, bio={"img":"r209S81af2k/3jWvzGkZFg==","content":"

1982年生. 博士,教授. 主要研究方向为机器学习、数据挖掘

"}, bioImg=r209S81af2k/3jWvzGkZFg==, bioContent=

1982年生. 博士,教授. 主要研究方向为机器学习、数据挖掘

, aboutCorrespAuthor=null)}, companyList=null), Author(id=1327018440545644988, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1327018440608559549, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, authorId=1327018440545644988, language=EN, stringName=Yuan Jiang, firstName=Yuan, middleName=null, lastName=Jiang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"content":"

born in 1976. PhD, professor. Her main research interests include machine learning and data mining

"}, bioImg=null, bioContent=

born in 1976. PhD, professor. Her main research interests include machine learning and data mining

, aboutCorrespAuthor=null), CN=AuthorExt(id=1327018440675668414, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1327018437068566928, authorId=1327018440545644988, language=CN, stringName=姜远, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=null, address=null, bio={"img":"nufmxhUnQyFK42Rqh1QonA==","content":"

1976年生. 博士,教授. 主要研究方向为机器学习、数据挖掘

"}, bioImg=nufmxhUnQyFK42Rqh1QonA==, bioContent=

1976年生. 博士,教授. 主要研究方向为机器学习、数据挖掘

, aboutCorrespAuthor=null)}, companyList=null)]
尹小龙 , 詹德川* , 姜远
作者信息
  • 南京大学人工智能学院 南京 210023
  • 计算机软件新技术全国重点实验室(南京大学) 南京 210023
通讯作者:
詹德川()
作者简介:

1998年生. 博士研究生. 主要研究方向为测试时适应、强化学习

1982年生. 博士,教授. 主要研究方向为机器学习、数据挖掘

1976年生. 博士,教授. 主要研究方向为机器学习、数据挖掘

Confidence-Guided Multimodal Data Test-Time Adaptation Framework
Xiaolong Yin , Dechuan Zhan* , Yuan Jiang
Affiliations
  • School of Artificial Intelligence, Nanjing University, Nanjing 210023
  • National Key Laboratory for Novel Software Technology (Nanjing University), Nanjing 210023
  • born in 1998. PhD candidate. His main research interests include test-time adaptation and reinforcement learning

    born in 1982. PhD, professor. His main research interests include machine learning and data mining

    born in 1976. PhD, professor. Her main research interests include machine learning and data mining

出版时间: 2025-09-01 doi: 10.7544/issn1000-1239.202550418
文章导航
收藏切换

测试时适应(test time adaptation, TTA)旨在在模型部署后的测试阶段,通过无标注或少量标注的在线微调策略来动态校正模型,以应对由于分布偏移、传感器噪声、光照变化等引发的性能退化问题,并在自动驾驶、远程医疗、视频监控等对实时性和鲁棒性要求极高的领域展现出广泛应用价值. 然而,现有多模态 TTA 方法往往忽视各视图数据的质量差异,容易因低质量或故障视图引入有害梯度,且缺乏对视图内部动态变化的刻画,难以保持时序特征的一致性和稳定性. 为解决上述挑战,提出了一种置信度引导的视图裁剪与时序对比注意力(confidence-guided view pruning and temporal contrastive attention, CVPTA)框架. 该框架包含三大模块:首先,基于模型预测分布计算视图不确定性,自适应削弱低置信度视图数据对特征融合的干扰;其次,视图裁剪策略利用 Laplacian 方差与亮度直方图偏度评估图像质量,丢弃质量分数低于阈值的视图数据,并结合相邻高质量视图重采样,显著降低噪声累积风险;最后,时序对比自监督任务将同一样本的不同视图视为正样本而非同一样本视图视为负样本,通过对比损失强化时序特征一致性. 在 Kinetics-50-C 与 VGGSound-C 两个公开多模态扰动基准上,沿用先前实验配置进行在线更新评测. 结果显示,CVPTA 在2组基准上分别将 Top-1 准确率提升约 2.3 个百分点和 0.7个百分点,在极端噪声场景中依然保持超过 0.2% 的性能增益;消融研究进一步验证了各模块的独立贡献与协同效应. 该方法无需额外标注,可无缝集成现有多模态系统,兼具高效性与鲁棒性,具有重要的理论意义与工程应用前景.

测试时适应  /  多模态学习  /  多视图学习  /  视图裁剪  /  时序对比学习

Test-time adaptation (TTA) seeks to dynamically recalibrate a deployed model through online fine-tuning with unlabeled or sparsely labeled data, addressing performance degradation caused by distribution shift, sensor noise, illumination changes, and other real-world challenges. It has demonstrated great promise in latency- and robustness-critical applications such as autonomous driving, remote healthcare, and video surveillance. However, existing multimodal TTA methods typically overlook the quality variance across views—low-quality or faulty views data can introduce harmful gradients—and lack mechanisms to preserve internal temporal consistency, undermining stability in dynamic scenarios. To tackle these issues, we propose CVPTA (confidence-guided view pruning and temporal contrastive attention), a unified framework comprising three core modules: 1) confidence-guided dynamic attention, which computes each view’s confidence via its predictive entropy and uses a softmax over inverted entropy scores to attenuate high-entropy views data during feature fusion; 2) view pruning, which evaluates image-modal quality using the Laplacian variance (for blur) and histogram skewness (for exposure), discards views below a quality threshold, and maintains a constant view count by interpolating adjacent high-quality views data or injecting Gaussian noise—dramatically reducing noise accumulation; 3) temporal contrastive self-supervision, which treats adjacent frames of the same view as positive pairs and non-adjacent frames as negatives, and applies a contrastive loss to enforce temporal feature consistency. We evaluate EVPTA on the public Kinetics-50-C and VGGSound-C perturbation benchmarks under the same online-update settings as prior work. Results show that EVPTA boosts Top-1 accuracy by approximately 2.3% and 0.7% on each benchmark and retains over 0.2% gains even under extreme noise conditions. Ablation studies further confirm the individual and synergistic contributions of all three modules. EVPTA requires no extra annotations, integrates seamlessly with existing multimodal systems, and delivers both efficiency and robustness—offering significant theoretical insights and practical value.

test-time adaptation  /  multimodal learning  /  multi-view learning  /  view pruning  /  temporal contrastive learning
尹小龙, 詹德川, 姜远. 置信引导的多模态数据测试时适应框架. 计算机研究与发展, 2025 , 62 (9) : 2166 -2182 . DOI: 10.7544/issn1000-1239.202550418
Xiaolong Yin, Dechuan Zhan, Yuan Jiang. Confidence-Guided Multimodal Data Test-Time Adaptation Framework[J]. Journal of Computer Research and Development, 2025 , 62 (9) : 2166 -2182 . DOI: 10.7544/issn1000-1239.202550418
在传统机器学习范式中,模型在训练阶段完成参数学习与优化后即被固定为“一次训练、终身不变”的静态模型[1],依赖训练与测试分布一致性的假设,却忽视了测试部署环境中数据分布随时间和场景演进带来的数据变化问题[2]. 这一静态思路的局限性在自动驾驶、视频监控等对实时性与鲁棒性要求极高的场景下尤为明显,当模型遭遇传感器噪声、光照变化或环境干扰时,性能往往会呈现出明显的下降[3]. 此外,传统方法缺乏在线自适应更新机制,使得在测试阶段无法充分利用新采集的无标签或少量标注样本来校正模型,从而增加了训练时与测试时环境差异带来的不匹配风险[4]. 为缓解分布偏移带来的性能下降,域自适应(domain adaptation)被提出,旨在利用源域与目标域数据间的迁移或泛化能力来提高模型在新域中的表现[5]. 然而,这些方法通常需要预先获取目标域的标注或无标注数据进行联合训练,难以满足部署后的模型对实时在线适应的需求[6]. 基于此,测试时适应(test-time adaptation, TTA)作为一种域自适应改良的高级学习范式,能够通过仅使用源域训练的模型和测试阶段的无标签数据,在推理前或推理过程中进行快速自适应. TTA已在图像分类、语义分割和多模态融合等任务中展现出显著潜力[7],近期研究表明,通过在测试阶段对预训练模型进行适应,可以在多个任务中实现 5%~10% 的性能提升[8-10].
现有的TTA方法主要可以分为三大类:1)基于无监督目标的在线更新. 此类方法无需任何测试标签,直接在测试阶段利用无监督损失对模型进行实时微调. 典型策略包括通过最小化预测熵(entropy minimization)来驱动批归一化参数更新,在批归一化层中重新计算或校正统计量以适应新分布 [11],以及结合自监督辅助任务(如旋转预测、对比学习等)增强更新阶段的稳定性. 这些方法能够在无人干预的情况下,针对每个测试批次或样本进行动态校正,从而缓解分布偏移带来的性能下降. 2)增强适应性的鲁棒 TTA. 针对测试时可能出现的标签分布偏移和自然(或恶意)噪声样本,这类方法在在线更新中引入了额外的鲁棒性机制. 例如,MedBN 通过检测并剔除异常样本来抵御恶意攻击[12];测试时归一化(test-time normalization, TTN)设计了域敏感的归一化插值策略,有效平衡了源端与目标端的统计量,提升了模型的跨域鲁棒性[13]. 这些方法通过噪声过滤、动态重加权和稳健优化,使得在非理想测试环境下依然可以保持模型表现的稳定性. 3)持续流式的记忆增强 TTA. 在长期部署或流式数据场景中,模型连续在线更新会导致对早期环境的“灾难性遗忘”. 针对这一挑战,这类方法引入了记忆模块和数据缓冲机制:MECTA 使用高效的内存缓存策略保存关键中间结果,以减小在线微调的内存开销并维持旧知识的留存[14];AR-TTA 通过小规模样本缓冲区与混合增广的结合,在不断吸收新分布信息的同时保留源域知识[15]. 这些方法通过周期性回放或选择性回顾,实现了流式环境的持续自适应与知识保留.
尽管单一模态的TTA已在图像分类、语义分割等任务中取得卓越表现[16-19],但在涉及视觉、语音、文本等多源信息融合的场景下,这类方法难以同时应对各通道的分布偏移问题,不能满足跨通道的动态协同需求[20]. 多模态测试时适应(multimodal test-time adaptation, MM-TTA)需要在无标签的测试阶段,实时校正每种模态的特征分布,并在此基础上实现不同模态间的对齐与自适应融合[21],充分发挥多源数据的互补优势并保障系统整体的稳健性. 尽管已有研究尝试通过引入跨模态的一致性约束和设计自适应融合网络来增强MM-TTA的能力[22],但在处理模态丢失、噪声干扰以及复杂时序依赖建模等方面依旧存在明显短板. 由此可见,深入探究MM-TTA的算法架构与理论原理,对于推动多模态模型在开放、动态环境中的高效部署,具有重要的学术价值和广泛的工程意义.
在开放部署场景中,MM-TTA不仅要应对传统分布偏移,还需在噪声干扰、设备故障与动态环境变化的多重压力下,确保异构模态信息的高效感知与融合. 具体而言,首先,不同模态或视角在实际测试过程中的可靠度会因遮挡、传感器抖动、通信延迟或模型预测不确定性等因素大幅波动,传统的固定权重融合或简单拼接往往无法及时识别并剔除这些“失效”视图,从而引发关键语义的误判或遗漏,严重影响最终决策;其次,流式数据中经常出现过曝/欠曝帧、模糊帧、背景噪声乃至丢帧断流等低质量输入,如果将它们直接纳入在线微调,会导致有害梯度累积、模型更新方向偏离,进而削弱对其他高质量模态信号的响应能力;最后,对于视频、音频等高维时序模态,现有方法多聚焦于对齐浅层特征,忽略了帧间的上下文关联、语义演进以及长短期依赖,当面临突发场景切换或短时噪声时,模型难以维持编码的一致性,无法充分利用时序结构去过滤干扰和补全缺失信息,从而限制了持续自适应的效果. 面对这三大挑战,构建能够动态感知视图可靠度、智能筛除噪声样本并兼顾时序连贯性的MM-TTA机制,已成为实现高鲁棒性在线部署的关键所在.
为全面应对上述三大挑战,本文提出了一种置信度引导的视图裁剪与时序对比注意力(confidence-guided view pruning and temporal contrastive attention, CVPTA)框架. CVPTA在测试阶段只需进行少量在线更新便能实现跨模态可靠度自适应、低质视图智能剔除与时序一致性增强的协同优化. 具体而言,CVPTA首先利用模型对每个视图的预测置信度动态生成注意力权重,有效压制由遮挡或预测不确定性带来的低质量视图干扰;随后通过模糊度和曝光度评估自动识别并剔除低质量帧,再结合邻近高质量视图的重采样完成信息补全,切断噪声梯度的累积路径;与此同时,借助时序对比自监督策略,将同一样本的不同视图视为正样本,不同样本的视图视为负样本,强化了视频与音频流的内部时间连贯性. 上述三大模块与主任务目标及跨视图对齐损失在一次端到端优化中紧密融合,无需任何额外标注或外部先验,即可在每个测试批次上完成在线微调. 实验结果显示,CVPTA 在 Kinetics-50-C 和 VGGSound-C 两大多模态扰动基准上分别实现了超过2.3个百分点 和 0.7个百分点的准确率提升,并在极端噪声场景中依旧保持超过0.2个百分点的增益.
本文的主要贡献包括3个方面:
1)聚焦MM-TTA这一创新且尚未充分探索的研究课题. 该问题在分布偏移与多源异质信息协同等方面具有广泛的实际应用需求,但当前研究仍较为稀缺. 本文从跨模态协同、视图质量和时序一致性3个维度,全面剖析在分布偏移和噪声干扰条件下,多模态 TTA 所面临的核心挑战,为该领域提供了清晰的问题定义和研究方向.
2)提出端到端的CVPTA. CVPTA集成置信度引导动态注意力、低质量视图裁剪以及时序对比自监督三大模块,实现了在无需额外标注的测试阶段对多模态特征的自适应融合与快速校正.
3)本文在 Kinetics-50-C 和 VGGSound-C 两大扰动基准上统一采用在线更新协议评测,并通过消融研究验证了熵引导注意力、视图裁剪与时序对比三大模块的独立与协同贡献.
本研究采用递进式分析框架,系统性地梳理TTA领域的研究进展. 首先从单模态适应(1.1节)的基础范式出发,剖析其在动态环境的建模能力与持续适应稳定性方面的固有局限;进而拓展至多模态场景(1.2节),深入分析模态间协同效应与信息不一致性带来的适应挑战;在此基础上引入多视图建模(1.3节),探讨其在动态环境下的信息增强与特征对齐机制;最终聚焦对比学习(1.4节),阐释其如何为多模态TTA的多视图特征对齐提供增强手段. 通过“基础范式—扩展挑战—增强机制—优化手段”的递进式分析路径,本研究在现有方法与前沿趋势的交叉领域,创新性地提出了一个具有结构一致性与多视图融合能力的全流程多模态TTA框架,为后续方法部分的展开奠定了坚实的基础. 这一分析框架不仅清晰地勾勒出TTA领域的发展脉络,更通过4个维度的有机整合,凸显了本研究的创新点与技术突破方向.
近年来,单模态TTA技术在解决机器学习模型在测试阶段面临的域偏移问题上取得了突破性进展. 该技术的核心思想是通过在测试过程中动态调整模型参数,使预训练模型能够适应目标域的数据分布变化,从而缓解源域与目标域之间的分布差异问题. 从技术发展脉络来看,当前单模态TTA研究主要沿着3个重要方向推进:首先是以熵最小化等无监督目标为基础的在线TTA方法[23-25],这类方法通过实时更新批归一化层参数实现了计算高效和部署便捷的优势,但在处理复杂多变的分布偏移时仍存在鲁棒性不足的缺陷;其次是针对实际应用场景中特定挑战的鲁棒TTA方法[26-30],如应对标签偏移问题的样本筛选策略、处理单样本适应场景的隐空间对齐技术等,这些方法通过专门的优化设计显著提升了模型在非理想条件下的适应性能;最后是面向持续域偏移问题的持续TTA方法[31-34],通过引入模型回滚机制和记忆缓冲技术,有效解决了长期适应过程中的灾难性遗忘问题. 值得注意的是,TTA技术的应用范围已经从最初的图像分类任务扩展到语义分割、人体姿态估计等更复杂的计算机视觉任务[35-39],极大地拓展了其实际应用价值. 然而,现有单模态TTA方法仍存在2个关键性局限:一方面,仅依赖单一数据模态的特性限制了模型利用多源信息互补优势的能力;另一方面,对模态内上下文关系的动态建模能力不足,导致在开放环境中的适应性受限. 这些技术瓶颈促使研究界开始将目光转向更具挑战性但也更有前景的多模态TTA研究,通过整合视觉、语言、音频等多模态信息,有望构建出更强大、更鲁棒的自适应系统,为人工智能系统在复杂开放环境中的实际部署提供新的技术解决方案.
MM-TTA作为深度学习领域的前沿研究方向,正在重塑传统测试时自适应的技术范式,其核心挑战不仅在于处理模态内部的分布偏移,更涉及跨模态的动态协同、信息互补与特征对齐等复杂问题. 当前研究呈现出多元化发展态势,在跨模态特征对齐方面,基于联合优化的多模态归一化层参数调整和对抗训练方法虽取得一定成效[40-42],但对模态间语义层级关系的建模仍显不足;多模态提示学习充分利用视觉-语言大模型的泛化能力[43-46],通过动态调整文本或视觉提示适应分布变化,却在处理模态内部复杂上下文关系时存在局限. 与此同时,模态互补增强技术通过知识迁移和分布校准策略显著提升噪声模态鲁棒性[47-48],而动态记忆机制则通过在线记忆库存储多模态特征实现增量式适应[49-50],但都面临各自的技术瓶颈. 最新研究趋势正将自监督学习和元学习等先进技术相融合,致力于构建端到端的多模态自适应框架[51-52],但这些方法在复杂分布偏移下稳定性不足.
多视图学习作为机器学习领域的重要分支,其核心价值在于通过挖掘多源数据间的互补性和一致性来增强模型的表征能力. 传统方法主要围绕表征对齐[53-57]和表征融合[58-63] 2个方向展开,但这些方法普遍存在2个关键性局限:一方面受限于线性假设或浅层模型架构,难以捕捉数据间复杂的非线性关系;另一方面主要针对静态数据分布设计,缺乏对动态环境变化的适应能力. 随着深度学习与多模态学习的深度融合,新一代方法通过更先进的网络架构和训练机制显著提升了多视图表征的鲁棒性. 当前研究主要从3个维度推动该领域发展:解耦表征学习(如DVANet[64])通过分离视图不变特征与视图特定特征,在跨中心医疗影像分割等任务中实现了12.3%的性能提升;动态多模态融合方法(如MMNet[65])结合时序建模,有效解决了传统方法对时序信息建模不足的问题;隐私保护范式(如MUTA)则通过测试时自适应框架,避免了医疗等敏感领域的数据集中需求. 值得注意的是,尽管这些创新方法在各自应用场景中取得了显著进展,但普遍未能充分考虑分布偏移问题,这为模型在未知环境中的表现稳定性和泛化能力埋下了隐患,亟待后续研究加以解决.
对比学习作为自监督学习的重要范式,通过构建正负样本对来学习判别性表征,在表征学习领域取得了显著进展. 早期工作如SimCLR和MoCo[66]通过数据增强和动量编码器技术,证明了无监督对比学习的强大表征能力;随后发展的语义增强方法(如SupCon[67]和PCL[68])引入类别信息,在细粒度分类等任务中实现了较高的性能提升. 跨模态对比学习方面,CLIP和ALIGN[69]通过大规模图像-文本对训练,建立了跨模态对齐的嵌入空间. 为提升计算效率,BYOL[70]和SimSiam[71]探索了无需负样本的简化架构. 不同于之前的工作,我们致力于将对比学习和多视图学习进行融合,通过拉近同一样本的不同视图的特征而拉远不同样本的特征,为MM-TTA的多视图特征对齐提供增强手段.
本节将系统地阐述本文所研究的MM-TTA问题及所提出的框架CVPTA.
本文以音频模态 $ a $ 和视频模态 $ v $ 为例,定义源域有标注样本集:
$ \mathcal{D}_{\mathrm{s}}=\left\{(x_i^a,x_i^v,y_i)\right\}_{i=1}^{N\mathrm{_s}},\mathrm{ }(x_i^a,x_i^v,y_i)\sim Q(x,y), $
其中 $ {x}_{i}^{a} $ 表示音频输入,$ {x}_{i}^{v} $ 表示视频输入,标签 $ {y}_{i}\in \{1,2,… ,K\} $. 同时,无标注的目标域数据流表示为:
$ {\mathcal{D}}_{\mathrm t}=\left\{\right({x}_{j}^{a},{x}_{j}^{v}){\}}_{j=1}^{{N}_{\mathrm t}},({x}_{j}^{a},{x}_{j}^{v})\sim P(x),P\ne Q. $
目标域数据以批次方式 $ {B}_{\mathrm t}=\{({x}_{j}^{a},{x}_{j}^{v})\}_{j=1}^{N} $ 顺序到达.
在源域 $ \mathcal{D}\mathrm{_s} $ 上训练完毕后得到模型参数:
$ {\mathit{\Theta }}_{\mathrm s}=\left\{{f}_{{\mathit{\Theta }}_{\mathrm s}^{a}}^{a},{f}_{{\mathit{\Theta }}_{\mathrm s}^{v}}^{v},{f}_{{\mathit{\Theta }}_{\mathrm s}^{m}}^{m},{f}_{{\mathit{\Theta }}_{\mathrm s}^{c}}^{c}\right\}, $
其中:
1)$ {f}_{{\mathit{\Theta }}_{\mathrm s}^{a}}^{a} $:音频编码器,将原始音频信号映射到潜在特征空间;
2)$ {f}_{{\mathit{\Theta }}_{\mathrm s}^{v}}^{v} $:视频编码器,将单帧或多帧视频输入映射到潜在特征空间;
3)$ {f}_{{\mathit{\Theta }}_{\mathrm s}^{m}}^{m} $:多模态融合层,对音频与视频特征进行拼接或注意力融合;
4)$ {f}_{{\mathit{\Theta }}_{\mathrm s}^{c}}^{c} $:分类器,将融合特征映射到类别预测 logits.
在不访问源域数据 $ \mathcal{D}\mathrm{_s} $ 的前提下,仅基于无标签目标域数据流 $ \mathcal{D}\mathrm{_t} $,通过在线微调优化 $ {\mathit{\Theta }}_{\mathrm s} $ 中的可更新子模块(主要是融合层 $ {f}^{m} $ 与分类器 $ {f}^{c} $)参数,使模型能够适应目标域分布 $ P $ 的偏移并保持高精度.
CVPTA是一种面向多模态数据的在线自适应框架. 如图1所示,其设计思路是:为每个“视频片段”抽取多视图,先行剔除劣质视图;再在融合特征空间中通过多重损失进行在线更新;最后在推理阶段基于视图置信度动态加权融合得到最终预测.
1)多视图采样与低质量视图裁剪. 在测试时对每个视频片段同时抽取2帧视图,保证仅使用视觉质量高的帧以减少噪声干扰.
2)训练阶段. 无需任何额外标注,仅利用无标签目标域数据,通过最小化跨模态对齐损失 $ {{L}}_{\mathrm{a}\mathrm{l}\mathrm{i}\mathrm{g}\mathrm{n}} $、分布平衡损失 $ {{L}}_{\mathrm{b}\mathrm{a}\mathrm{l}} $、可靠性感知损失 $ {{L}}_{\mathrm{r}\mathrm{a}} $ 以及时序对比自监督损失 $ {{L}}_{\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{p}} $ 对模型可更新参数(主要是融合层 $ f\mathrm{^{\mathbf{\mathit{\mathbf{\mathbf{\mathit{m}}\mathit{\mathbf{ }}}}}}} $ 与分类器 $ f\mathrm{^{\mathit{c}}} $)进行在线更新.
3)推理阶段. 对2个视图分别计算对数概率logits 并获得置信度,再基于置信度动态线性加权融合 logits,输出最终预测 $ {y}_{j} $.
在视频流场景中,单帧图像往往无法完整表达整个片段的动态语义,且某些帧可能因运动模糊、光照过曝/欠曝或遮挡而信息稀少. 直接使用低质量帧参与在线更新,会导致有害梯度累积并损害自适应效果. 因此,本文在测试时对每段视频进行多帧采样并预先裁剪低质量帧,保证后续模块仅对“高质量视图”进行编码与优化,从源头减少噪声干扰. 具体流程分4个步骤.
1)候选视图生成
对于批次 $ B\mathrm{_t} $ 中的每个视频片段 $ {x}_{j}^{v} $,均匀抽取 $ M $ 帧候选视图:
$ \{{x}_{j}^{v}(k\left)\right|k=1,2,… ,M\}.$
同时,针对对应的音频信号 $ {x}_{j}^{a} $,本文可直接对齐2段时间窗口(对应帧时间)或使用整体音频作为2个视图输入,以保证音频和视频的同步性.
2)帧级质量评估
从2个角度对第 $ k $ 帧图像进行质量评估:
① Laplacian方差(模糊度量)
${q}_{j}^{\mathrm{l}\mathrm{a}\mathrm{p}}(k)= {V}{a}{r}({\nabla }^{2}{x}_{j}^{v}(k)), $ 用于评估细节丰富度和画面清晰度,方差越大表示采样图像越清晰、细节越丰富;
②亮度直方图偏度(曝光度量)
$ {q}_{j}^{\mathrm{l}\mathrm{u}\mathrm{m}}\left(k\right)= {s}{k}{e}{w}{n}{e}{s}{s}\left({h}{i}{s}{t}\right({x}_{j}^{v}\left(k\right)\left)\right)\mathrm{ }, $偏度值用于检测过曝(正偏度)或欠曝(负偏度)情况.
3)合并成整体质量分数
$ {q}_{j}\left(k\right)=\alpha {q}_{j}^{\mathrm{l}\mathrm{a}\mathrm{p}}\left(k\right)+\left(1-\alpha \right){q}_{j}^{\mathrm{l}\mathrm{u}\mathrm{m}}\left(k\right),\alpha \in \left[\mathrm{0,1}\right]. $
对 $ \{{q}_{j}(k)\}_{k=1}^{M} $ 做 min–max 归一化,保证最终结果落在 $ \left[\mathrm{0,1}\right] $的区间内.
4)低质视图裁剪
按降序选取质量最高的2帧索引 $ \{{k}_{1},{k}_{2}\}= \mathrm{t}\mathrm{o}\mathrm{p}2\left(\right\{{q}_{j}\left(k\right)\left\}\right) $. 检索对应视图帧:
$ {x}_{j}^{v_1}={x}_{j}^{v}\left({k}_{1}\right),{x}_{j}^{v_2}={x}_{j}^{v}\left({k}_{2}\right). $
同时选取对应的音频段,或统一使用 $ {x}_{j}^{a} $ 作为2个视图的音频输入,形成2套“高质量”的视图组合:
$ ({x}_{j}^{a_1},{x}_{j}^{v_1}),({x}_{j}^{a_2},{x}_{j}^{v_2}).$
多视图采样与低质量视图裁剪模块能够在测试阶段同步抽取2个高质量帧,既有效避免了因模糊或曝光异常的劣质帧带来的有害梯度,又保留了视频片段前后语义演进的信息,为后续对比学习与特征融合提供了更加丰富的时序样本. 此外,提前完成视图质量筛选,还能在在线更新中减少无效视图的计算开销,从而大幅提升自适应的效率和整体鲁棒性.
在 2.2.1 节得到2组“高质量”多模态视图对 $ \left\{\right({x}_{j}^{a_1},{x}_{j}^{v_1}),({x}_{j}^{a_2},{x}_{j}^{v_2}\left)\right\} $ 后,分别经编码器提取特征并融合.
1)特征提取与融合
$ \boldsymbol{z}_{j,\ell}^a=f_{\mathit{\Theta}_{\mathrm{s}}^a}^a(x_j^{a\ell}),\boldsymbol{z}_{j,\ell}^v=f_{\mathit{\Theta}_{\mathrm{s}}^v}^v(x_j^{v\ell}), $
$ \boldsymbol{z}_{j,\ell}=concat(\boldsymbol{z}_{j,\ell}^a,\boldsymbol{z}_{j,\ell}^v),\boldsymbol{z}_{j,\ell}^m=f_{\mathit{\Theta}_{\mathrm{s}}^m}^m(\boldsymbol{z}_{j,\ell}), $
其中 $ \ell \in \left\{1,2\right\} $. 最终得到2组融合特征 $ \boldsymbol{z}_{j,1}^m,\boldsymbol{z}_{j,2}^m $.
2)损失函数的计算过程
令$ \boldsymbol{p}_{j,\ell }={S}{o}{f}{t}{m}{a}{x}\left({f}_{{\mathit{\Theta }}_{\mathrm s}^{c}}^{c}\left({\boldsymbol z}_{j,\ell }^{ m}\right)\right)\mathrm{ }\in {\mathbb{R}}^{K},\ell =\mathrm{1,2}. $
定义 Jensen–Shannon 散度为:
$ {D}_{\mathrm{J}\mathrm{S}}(P\Vert Q)=\frac{1}{2}{D}_{\mathrm{K}\mathrm{L}}(P\Vert M)+\frac{1}{2}{D}_{\mathrm{K}\mathrm{L}}(Q||M), $
其中$ M=\dfrac{1}{2}(P+Q) $,则
$ {{L}}_{\mathrm{a}\mathrm{l}\mathrm{i}\mathrm{g}\mathrm{n}}=\frac{1}{\left|{B}_{\mathrm t}\right|}\sum _{j\in {B}_{\mathrm t}}{D}_{\mathrm{J}\mathrm{S}}({p}_{j,1}\Vert {p}_{j,2}). $
跨模态对齐损失能够显著缓解目标域分布偏移带来的单视图偏颇问题:当某个视图出现偏向时,该损失会强制另一视图对预测概率分布保持一致,迫使模型在更新过程中同时考虑2种视图的信息,从而避免片面偏移. 同时,通过对齐概率流,模型在融合特征时能够更好地整合来自音频和视频2种模态的互补信息,提升融合后特征的稳健性和泛化能力.
为防止模型在多模态融合后对少数类别或单个视图过度依赖,本文对整个批次 $ B\mathrm{_t} $ 中的所有视图 logits 进行归一化统计. 令批次大小为 $ B\mathrm{_t} $,共有 $ 2B\mathrm{_t} $ 个视图特征 $ \boldsymbol{z}_{j,\ell}^{{m}} $,其对应的 logits 为
$ {o}_{j,\ell }={f}_{{\mathit{\Theta }}_{{\mathrm{s}}}^{ c}}^{\mathrm c}({\boldsymbol z}_{j,\ell }^{m}),{p}_{j,\ell }={S}{o}{f}{t}{m}{a}{x}({o}_{j,\ell }). $
定义类别累积概率:
$ {p}_{\mathrm{s}\mathrm{u}\mathrm{m}}^{\left( c\right)}=\sum _{j\in {B}_{\mathrm t}}\sum _{\ell \in \{1,2\}}[{p}_{j,\ell }{]}_{ c},\; c=1,2,… ,K. $
则
$ {{L}}_{\mathrm{b}\mathrm{a}\mathrm{l}}=-\displaystyle\sum\limits_{c=1}^{K}\dfrac{{p}_{\mathrm{s}\mathrm{u}\mathrm{m}}^{\left(c\right)}}{\displaystyle\sum\limits_{k=1}^{K}{p}_{\mathrm{s}\mathrm{u}\mathrm{m}}^{\left(k\right)}}\mathrm{log}\Biggl({18}{\dfrac{{p}_{\mathrm{s}\mathrm{u}\mathrm{m}}^{(c)}}{\displaystyle\sum\limits_{k=1}^{K}{p}_{\mathrm{s}\mathrm{u}\mathrm{m}}^{\left(k\right)}}}\Biggr). $
分布约束平衡损失能够防止模型在多模态融合时对某些类别或视图过度自信,从而忽略其他有价值的信息;通过最大化整体类别概率分布的熵,它督促模型在各类别之间保持相对均衡,使得次优但仍有意义的模态信息也能得到关注,有效减缓因跨模态不平衡而导致的性能退化,从而使在线自适应过程更加稳定且具有针对性.
当某些视图因噪声或分布偏移导致预测不确定度高时,模型需要识别并放低该视图在在线更新中的权重,以免有害梯度干扰. 为此,本文设计了可靠性感知损失 $ {{L}}_{\mathrm{r}\mathrm{a}} $:
1)预测熵与可靠度计算
对每个融合特征 $ \boldsymbol{z}_{j,\ell}^{{m}} $ 计算预测分布:
$ \begin{split}&{p}_{j,\ell }=\;{Softmax}({f}_{{\mathit{\Theta }}_{\mathrm s}^{c}}^{c}({\boldsymbol z}_{j,\ell }^{m})),\\&H({p}_{j,\ell })=-\sum _{c=1}^{K} {p}_{j,\ell }[c]\mathrm{log}{p}_{j,\ell }[c],\end{split} $
其中 $ H\left(p\right) $ 为分布熵. 定义视图可靠度权重:
$ \sigma\left(p_{j,\ell}\right)=\frac{1}{1+\mathrm{e}^{-H\left(p_{j,\ell}\right)}}\in\left(\mathrm{0,1}\right). $
当 $ H\left(p\right) $ 越大(即不确定程度越高),$ \sigma \left(p\right) $ 越接近 0;当 $ H\left(p\right) $ 越小,$ \sigma \left(p\right) $ 越接近 1.
2)可靠性感知损失形式
$ L_{\mathrm{r}\mathrm{a}}=\frac{1}{2\left|B_{\mathrm{t}}\right|}\sum_{j\in B_{\mathrm{t}}}^{ }\sum_{\ell=1}^2\sigma\left(p_{j,\ell}\right)\left(H\left(p_{j,\ell}\right)+\gamma\right), $
其中 $ \gamma > 0 $ 为小常数,用于平衡熵的数量级. 在线更新阶段,可靠性感知机制能够自动识别高熵(高不确定度)视图并降低其对损失的贡献,此时 $ \sigma \left(p\right) $ 倾向于接近0,从而减少低可靠度视图的梯度权重;与之相反,当视图预测熵较低(即置信度较高)时,$ \sigma \left(p\right) $ 接近1,会突出该视图的更新信号,充分利用高质量视图信息. 通过这种动态区分视图可靠度的方式,模型在分布漂移和噪声干扰环境下能够更加稳健地进行在线自适应.
在 2.2.1 节完成“高质量”多视图采样后,得到2个融合特征 $ {\boldsymbol z}_{j,1}^{m},{\boldsymbol z}_{j,2}^{m} $;在一个批次 $ B\mathrm{_t} $ 中,其他视频样本也各自有2个融合特征 $ {\boldsymbol z}_{k,1}^{m},{\boldsymbol z}_{k,2}^{m},k\ne j $. 利用对比学习,本文定义时序对比自监督损失 $ {{L}}_{\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{p}} $ 来强化同一视频段内视图的时序连贯性,并推远不同视频样本的特征距离.
$ \begin{split}& L_{\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{p}}= \\ &-\frac{1}{2\left|B\mathrm{_t}\right|}\displaystyle\sum\limits_{j\in B_{\mathrm{t}}}^{ }\displaystyle\sum\limits_{\ell=1}^2\mathrm{l}\mathrm{o}\mathrm{g}\frac{\mathrm{e}\mathrm{x}\mathrm{p}(\boldsymbol{z}_{j,\ell}^{m\mathrm{T}}\boldsymbol{z}_{j,3-\ell}^m/\tau)}{\mathrm{e}\mathrm{x}\mathrm{p}(\boldsymbol{z}_{j,\ell}^{m\mathrm{T}}\boldsymbol{z}_{j,3-\ell}^m/\tau)+ \displaystyle\sum\limits_{(k,u)\ne(j,\ell)}^{ } \mathrm{e}\mathrm{x}\mathrm{p}(\boldsymbol{z}_{j,\ell}^{m\mathrm{T}}\boldsymbol{z}_{k,u}^m/\tau)},\end{split}$
其中 $ \tau $ 为温度超参数.
1)正样本对:同一视频段的2个融合特征 $ \boldsymbol{z}_{j,1}^m,\boldsymbol{z}_{j,2}^m $.
2)负样本对:批次中所有其他视频的视图融合特征 $ \{{\boldsymbol z}_{k,u}^{m}\mid k\ne j,u\in \{\mathrm{1,2}\left\}\right\} $.
时序对比自监督通过拉近同一视频段中2个视图的特征距离,强化了短期时序语义的一致性,即使某一帧受到噪声干扰,另一帧依然可以提供稳定信号,从而提升模型对连续语义的捕捉能力;与此同时,它又将不同视频段的视图特征相互分离,增强了模型对不同样本间差异的判别能力,避免在线更新过程中被异类信息所干扰.
在推理和微调过程中,本文对2个融合特征 $ {\boldsymbol z}_{j,1}^{m},{\boldsymbol z}_{j,2}^{m} $ 分别计算分类器输出 logits 及预测概率,并使用置信度作为视图融合的动态权重.
1)计算 logits 与预测概率:
$ \begin{split}o_{j,\ell}=f_{\mathit{\Theta}_{\mathrm{s}}^c}^c\left(\boldsymbol{z}_{j,\ell}^m\right),p_{j,\ell}=Softmax\left(o_{j,\ell}\right),\ell=1,2.\end{split} $
2)置信度定义$ {c}_{j,\ell }=\underset{k}{\mathrm{m}\mathrm{a}\mathrm{x}} {p}_{j,\ell }\left[k\right], $即第 $ \ell $ 个视图对样本 $ j $ 的最高预测概率.
3)动态加权融合
$ Logits{\tilde{o}}_{j}={c}_{j,1}{o}_{j,1}+{c}_{j,2}{o}_{j,2}, $
$ {\tilde{p}}_{j}={Softmax}({\tilde{o}}_{j}),{y}_{j}=\mathop{\mathrm{arg\;max}}\limits_k{\tilde{p}}_{j}[k].$
动态注意力机制会根据每个视图在当下帧的预测置信度动态分配权重,将更可靠的视图置于更高优先级,弱化那些因噪声干扰而置信度较低的视图,从而显著提升融合结果的稳健性,确保在测试阶段遇到分布漂移时,模型不会仅凭单一的失真视图做出错误判断.
将各子损失按权重线性组合,得到总体训练损失:
$ {{L}}_{\mathrm{t}\mathrm{r}\mathrm{a}\mathrm{i}\mathrm{n}}={{L}}_{\mathrm{a}\mathrm{l}\mathrm{i}\mathrm{g}\mathrm{n}}+{\lambda }_{\mathrm{b}\mathrm{a}\mathrm{l}}{{L}}_{\mathrm{b}\mathrm{a}\mathrm{l}}+{\lambda }_{\mathrm{r}\mathrm{a}}{{L}}_{\mathrm{r}\mathrm{a}}+{\lambda }_{\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{p}}{{L}}_{\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{p}}, $
其中
1)$ {{L}}_{\mathrm{a}\mathrm{l}\mathrm{i}\mathrm{g}\mathrm{n}} $:使用 Jensen–Shannon 散度衡量2个视图预测概率分布一致性;
2)$ {{L}}_{\mathrm{b}\mathrm{a}\mathrm{l}} $:通过最大化整体类别概率熵保证预测分布均衡,$ {\lambda }_{\mathrm{b}\mathrm{a}\mathrm{l}} $为该损失对应的权重;
3)$ {{L}}_{\mathrm{r}\mathrm{a}} $:利用预测熵与 Sigmoid 抑制高不确定度视图梯度,$ {\lambda }_{\mathrm{r}\mathrm{a}} $为该损失对应的权重;
4)$ {{L}}_{\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{p}} $:拉近同视频段不同视图特征、推远不同视频段特征,$ {\lambda }_{\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{p}} $为该损失对应的权重.
以上各项损失共同驱动融合层 $ f^{{m}} $ 和分类器 $ f^c $ 在测试阶段进行少量迭代梯度更新,保持学习率 $ \eta $ 固定.
算法1. CVPTA:多视图不确定性引导与时序对比在线自适应.
输入:源域模型$ \mathit{\Theta }_{{\mathrm{s}}}=\{{f}^{a},{f}^{v},{f}^{m},{f}^{c}\} $,无标注测试流$ {D}_{\mathrm t}={\{{x}_{j}^{a},{x}_{j}^{v}\}}_{j=1}^{{N}_{\mathrm t}} $;
输出:实时预测$ {t}_{j} $以及更新之后的$ {f}^{m},{f}^{c} $.
for ${B}_{\mathrm t} $ in ${D}_{\mathrm t} $ do:
①多视图采样与低质视图裁剪
  $ \mathrm{f}\mathrm{o}\mathrm{r}\left({x}_{j}^{a},{x}_{j}^{v}\right)\mathrm{i}\mathrm{n}\;{B}_{\mathrm t}\;\mathrm{d}\mathrm{o} $:
  抽取M帧候选视图,并计算质量分数$ {q}_{j}\left(k\right) $;
  选取质量最高的2帧作为$ {x}_{j}^{{v}_{1}},{x}_{j}^{{v}_{2}} $;
  生成对应的2组音频视图$ {x}_{j}^{{a}_{1}},{x}_{j}^{{a}_{2}} $;
  end for
②特征提取和多模态融合
 for l in {1, 2} do:
   ${\boldsymbol z}_{\{j,l\}}^{a}={f}^{a}({x}_{j}^{\{a,l\}}) $;
   ${\boldsymbol z}_{\{j,l\}}^{v}={f}^{v}({x}_{j}^{\{v,l\}}) $;
   ${\boldsymbol z}_{\{j,l\}}=concat({\boldsymbol z}_{\left\{j,l\right\}}^{a},{\boldsymbol z}_{\{j,l\}}^{v}) $;
   ${\boldsymbol z}_{\{j,l\}}^{m}={f}^{m}({\boldsymbol z}_{\{j,l\}}) $;
  end for
③$ {{L}}_{\mathrm{a}\mathrm{l}\mathrm{i}\mathrm{g}\mathrm{n}} $的计算
  ${f}^{c} $通过计算${p}_{\left\{j,1\right\}},{p}_{\left\{j,2\right\}} $;
 基于式(11)得到${{L}}_{\mathrm{a}\mathrm{l}\mathrm{i}\mathrm{g}\mathrm{n}} $;
④$ {{L}}_{\mathrm{b}\mathrm{a}\mathrm{l}} $的计算
 收集当前batch所有的${p}_{\{j,l\}} $;
 计算每个y对应的概率的和;
 基于式(14)得到$ {{L}}_{\mathrm{b}\mathrm{a}\mathrm{l}} $;
⑤$ {{L}}_{\mathrm{r}\mathrm{a}} $的计算:
 计算$H({p}_{\{j,l\}}) $和$\sigma ({p}_{\{j,l\}}) $;
 基于式(17)得到$ {{L}}_{\mathrm{r}\mathrm{a}} $;
⑥$ {{L}}_{\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{p}} $的计算:
 for j in $ B\mathrm{_t} $ do:
  for l in {1, 2} do:
   计算正对相似度;
   计算负对相似度;
   基于式(18)得到$ {{L}}_{\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{p}} $;
  end for
 end for
⑦联合优化更新基于式(22):
⑧置信度引导动态注意力融合与预测:
 for j in $ B\mathrm{_t} $ do:
  计算${o}_{\left\{j,1\right\}},{o}_{\left\{j,2\right\}} $和${p}_{\left\{j,1\right\}},{p}_{\left\{j,2\right\}} $;
   ${c}_{\left\{j,1\right\}}=\mathrm{max}{p}_{\left\{j,1\right\}} $;
   ${c}_{\left\{j,2\right\}}=\mathrm{max}{p}_{\left\{j,2\right\}} $;
   ${\tilde{o}}_{j}={c}_{\left\{j,1\right\}}\cdot{o}_{\left\{j,1\right\}}+{c}_{\left\{j,2\right\}}\cdot{o}_{\left\{j,2\right\}} $;
   $ {y}_{j}={\mathrm{arg\;max}} (softmax ({\tilde{o}}_{j})) $;
 end for
end for
本节在模态可靠性存在偏差的MM-TTA场景下,评估所提出的CVPTA框架在视听联合行为识别和事件分类任务中的性能.
为系统研究MM-TTA中由于视图质量不稳定导致的模型表现下降,本研究基于广泛使用的Kinetics-50[72]和VGGSound[73]多模态数据集构建了2个基准测试集. 参照Hendrycks & Dietterich的方法[74],为视频模态设计了11种、为音频模态设计了6种不同类型的干扰. 其中,针对视频模态设计的干扰分为噪声添加(Noise)、动态模糊(Blur)以及数码干扰(Digital)三个大方向,并在每个方向下选择了最具代表性的几种具体干扰形式,如噪声添加方向选择了高斯模糊(Gaussian)等3种干扰类型作为最终的任务设置;针对音频模态的干扰设置在思路上与视频类似,且因为音频模态本身的数据特性,只设置了Noise和天气影响(Weather)两类干扰方向,并在其中设置了以Gaussian和交通噪声(Traffic)为代表的6种经典干扰环境. 在每个具体的干扰任务中本文都按照干扰强度由低到高设置了5个强度等级,在本节中使用的所有实验结果具体使用的干扰强度都是最高等级的五级干扰强度,以此为依据最终形成包含模态损坏的Kinetics-50-C和VGGSound-C基准测试集.
本文将所提出的 CVPTA 框架与当前5种具有代表性的TTA方法(Tent, MMT, EATA, SAR, READ)进行了系统对比分析,涵盖单模态与多模态TTA的典型范式,具体说明如下:
1)Tent[75]. 在测试阶段通过最小化模型预测输出的熵值,提升模型对分布偏移样本的适应能力,广泛适用于图像分类、语义分割等任务,并在多个标准数据集上取得了优异性能.
2)MMT[76]. 基于多模态3D语义场景构建,利用互补模态间的信息融合增强自监督信号,从而提升多模态自适应分割与表示学习能力.
3)EATA[77]. 引入不确定性感知的样本选择机制,动态识别可靠样本并引导模型在稳定子集上进行测试时更新,有效缓解分布偏移引起的灾难性遗忘问题.
4)SAR[78]. 通过抑制大梯度噪声样本并引导模型优化方向靠近平坦解,实现对混合分布、非平稳环境下的鲁棒适应.
5)READ[79]. 设计模态级的可靠性评估机制,根据各模态在不同噪声条件下的表现动态调整其在模型更新中的权重,从而实现多模态噪声环境下的自适应增强.
上述方法分别代表了当前TTA研究在熵最小化、自监督增强、样本筛选、梯度平滑与模态可靠性建模等不同方向的主流策略. 后续实验将从准确率、鲁棒性与跨模态协同能力等维度对比分析上述方法与CVPTA的性能差异.
本研究提出的CVPTA是一个通用框架,以当前最先进的CAV-MAE模型[80]作为基础架构,该模型已在网络规模的视听数据上进行预训练. 本文在Kinetics-50和VGGSound训练集上对其进行微调,获得相应的源域模型. 其中,以Kinetics-50和VGGSound的训练集作为源域,而其对应的干扰版本Kinetics-50-C和VGGSound-C作为目标域. 在TTA阶段,该方法采用Adam优化器对源模型的特定参数进行更新:设置初始学习率为0.0001,批处理大小为32,单次迭代周期;置信度阈值γ固定为$ \mathrm{e}^{-1} $. 所有实验均在配备NVIDIA 2080 GPU的Ubuntu 20.04平台上完成.
参考现有的TTA方法,本文选择平均准确率(MA)作为模型的衡量指标,并独立计算每种噪声添加情况下的模型表现,每种设置下均进行多轮测试计算平均得分作为最终的结果,计算公式为:
其中$ {Accuracy}_{i} $为某一噪声添加下的模型单次测试准确率,n为测试轮次.
从表1可见,在 Kinetics-50 数据集上,CVPTA 在11种设置中的9个任务上实现了性能领先,平均准确率显著优于 Tent,MMT,EATA,SAR 和READ等主流TTA方法,展现出更强的跨模态协同能力和分布鲁棒性. 表中的”Stat.”和”Dyn.”分别是静态(statical)和动态(dynamic)的缩写, LN表示层归一化(layer normalization)机制,LF表示后融合(late fusion)机制,AF表示基于注意力的融合(attention-based fusion)机制,SAF表示自适应的基于注意力的融合(self-adaptive attention-based fusion)机制. 尤其在Gaussian噪声场景中,CVPTA达到63.2%,较当前最佳结果(READ)提高了13.8个百分点,在Shot和Impulse噪声下均领先 2.6个百分点. 在模糊扰动设置(如Glass, Zoom)中,CVPTA 在2个子任务中分别取得65.3%和68.9%的最高准确率,相较READ和Tent展现出更强的泛化能力. 与此形成对比,基于Late Fusion或伪标签生成策略的方法(如MM-TTA和READ)在模态信号质量不一致的条件下表现出明显不稳定性,难以有效利用低质量模态的信息. 在VGGSound数据集中尤为典型,由于音频模态受损严重,这类方法在Gaussian、散焦模糊(Defocus)及JPEG压缩(JPEG)场景下性能下降明显,反映出其对模态间可靠性偏差的高度敏感性. 总之,CVPTA通过构建跨模态协同优化机制和保证视图质量与时序一致性,在高噪声、高偏移环境中保持稳定性能,验证了多视图建模在MM-TTA中的有效性.
从表2可以观察到,CVPTA框架在Kinetics-50数据集的6种音频扰动设置下均取得最优性能,全面优于Tent,MMT,EATA,SAR和READ等主流TTA方法. 其中,在Gaussian 噪声场景下,CVPTA达到74.9%的准确率,较READ提高 0.8个百分点,为所有噪声设置中最高. 在Crowd与Thunder两类高噪环境中分别取得71.0%与72.4%的准确率,均显著优于现有方法. 在Traffic场景下,虽然CVPTA相较READ提升0.1个百分点,但考虑到任务本身已接近饱和,这一提升依然具有代表意义. 从整体平均准确率来看,CVPTA达到 71.6%,在所有方法中排名第一,较 READ高出0.5个百分点,相较Tent和MM-TTA等传统方法分别提升了2.0个百分点与2.2个百分点. 这一结果不仅说明本文方法在复杂音频扰动下具有更强的泛化能力,也验证了所提出的多模态协同机制在分布不确定性条件下的鲁棒性优势. 表3结果进一步验证了CVPTA框架在多模态分布扰动下的适应性优势. 在 VGGSound 数据集上的 video-corruption 实验中,CVPTA 在11个测试场景中的全部11项任务中实现了 SOTA 结果,覆盖 Noise,Blur 和 Digital 三大类扰动设定. 在部分任务中,例如 Gaussian、Defocus 和 玻璃模糊(Glass)三种典型高噪声场景,准确率相较 READ 都有一定的提升,展示了该方法对低质量模态输入的鲁棒建模能力. 相比之下,其他方法(如 Tent 和 SAR)虽在某些任务中表现接近,但整体缺乏一致性,且在模态质量严重不均衡的任务设定中易出现性能退化. 而CVPTA借助其跨模态的协同优化设计,联合构建动态注意力机制与跨模态对比损失,有效缓解了模态间可靠性偏差对测试时推理带来的影响. 综合多个任务维度的对比分析结果,CVPTA表现出稳定和显著的性能优势,说明跨模态协同优化与动态注意力机制和对比损失的协同设计是解决多模态动态适应问题的关键.
为了验证框架中每个设计的重要性和有效性,本文会对相关方法的消融实验进行分析.
1)去除熵引导的动态注意力机制. 为验证动态注意力机制的有效性,本文在Kinetics-50数据集上进行了系统对比实验. 如图2所示,包含动态注意力机制的模型在绝大多数损坏场景下均优于包含无注意力机制的对比模型,尤其在运动模糊(Motion)(71.1% vs. 70.6%)、Defocus(66.9% vs. 66.4%)和雪花噪声(Snow)(64.5% vs. 63.9%)等复杂干扰下表现突出,平均准确率提升0.5个百分点.
值得注意的是,在脉冲噪声(Impulse)这一特定损坏类型下,单视图通路略占优势(差异小于0.1个百分点),这可能由于极端噪声破坏了跨模态的协同关系,导致动态注意力机制暂时失效. 然而,在更常见的自然干扰(如Motion、Defocus、Snow等)中,动态注意力机制通过实时协同不同模态间的信息,显著提升模型鲁棒性.
2)去除低质视图裁剪与伪重采样. 为验证重采样机制的有效性,本文在多种损坏场景下进行了系统对比实验. 实验数据如图3所示,加入重采样机制的方法在绝大多数任务场景下并未优于未使用重采样机制的方法,特别是在Defocus(66.9% vs. 67.2%)、Glass(65.3% vs. 66%)和Motion(71.1% vs. 71.3%)等复杂干扰条件下表现突出,平均性能差距约0.3个百分点.
值得注意的是,在Gaussian和冲击噪声(Shot)等特定损坏类型下,2种方法表现相当接近(差异小于0.2%),这可能表明在某些特定噪声分布下,重采样机制的优化效果有限. 然而,在更广泛的自然干扰(如Defocus,Glass,Motion等)以及JPEG(70.7% vs. 70.8%)等常见图像失真情况下,重采样机制通过优化特征分布,显著提升了模型对各类干扰的鲁棒性(平均增益0.25%). 特别是在光照变化(Brightness)和像素化(Pixelate)等任务中,重采样机制展现出了稳定的性能优势,验证了其在多样化场景下的有效性.
3)去除对比损失. 为验证对比损失的有效性,本文在多种损坏场景下进行了系统对比实验. 实验数据如图4所示,加入对比损失机制的方法在多数任务场景下与未使用对比损失的方法在预测准确率上持平,但在特定干扰条件下仍展现出轻微优势,平均性能差距约0.1个百分点.
值得注意的是,在JPEG任务中,对比损失机制表现出优势(70.7% vs. 70.6%),这表明对比损失对于处理压缩伪影这类结构化噪声具有独特效果. 而在Snow(64.5% vs. 64.4%)场景下,对比损失也展现出稳定的性能提升. 虽然整体性能差异较小,但对比损失在保持模型对各种干扰的稳定处理能力方面显示出其价值,特别是在处理结构化噪声和特定类型干扰时. 这种一致性表现验证了对比损失作为辅助训练目标的可靠性,能够在不影响主要任务性能的前提下,为模型提供额外的正则化效果.
1)低质视图裁剪与伪重采样阈值敏感度分析. 为探究重采样过程中质量阈值q对模型性能的影响,本文系统分析了不同阈值设置下模型在各类损坏场景的表现.
实验数据如图5所示,随着质量阈值q从0.1逐步提升至0.9,模型整体性能保持相对稳定,但在特定干扰类型下呈现出规律性变化. 在散焦模糊Defocus等空间失真类干扰下,较高阈值(q=0.7~0.9)反而带来轻微性能提升(0.3~0.5个百分点),这表明适度放宽质量要求有助于保留更多有效训练样本. 值得注意的是,在结构化噪声场景中,不同阈值设置下的性能差异最小,反映出模型对这些干扰具有较好的鲁棒性. 特别地,当阈值设置过低(q<0.3)时,虽然在Gaussian等简单干扰下表现稳定,但在Snow等复杂干扰中出现了较大的性能波动,这可能是由于过度严格的样本筛选导致训练数据不足. 而当阈值设置过高(q>0.5)时,在极端干扰下的性能标准差明显增大(达0.5~0.6个百分点).
综合分析表明,阈值q在0.3~0.7区间能取得最佳平衡,且在不同干扰类型间展现出最稳定的性能表现. 这一发现为实际应用中的参数选择提供了重要参考.
2)时序对比自监督损失权重敏感度分析. 为探究对比损失权重d对模型性能的影响,本文系统分析了权重系数从0.1到0.9范围内模型在各类损坏场景的表现. 实验数据如图6所示,对比损失权重的变化对模型整体性能影响较小,但在特定干扰类型下仍展现出一定的规律性特征. 当对比损失权重在0.3~0.5时,模型在多数任务场景下达到最佳平衡状态. 值得注意的是,在极端干扰条件下,适当增大对比损失权重(d=0.7~0.9)可带来轻微性能提升,这表明对比学习对这些特定噪声类型具有独特的正则化效果.
而在结构化噪声场景中,不同权重设置下的性能差异最小,反映出模型对这些干扰具有较好的适应性. 特别地,在Glass空间失真类干扰下,模型性能对权重变化表现最为敏感,当权重超过0.5时出现轻微的性能波动. 而在明暗条件干扰下,模型展现出最强的稳定性,说明对比损失对这些场景的优化效果较为有限. 综合分析表明,对比损失权重为0.3~0.5能取得最佳平衡,此时各项任务的平均标准差最小(约0.5%). 这一发现为实际应用中的参数选择提供了重要参考:虽然绝对性能差异不大,但适度的对比损失权重能帮助模型在保持主要任务性能的同时,增强对特定噪声类型的鲁棒性. 值得注意的是,过高的对比损失权重(d>0.7)可能导致在部分空间失真任务中出现性能波动.
3)温度参数敏感度分析. 为探究对比学习中温度参数(Temperature)对模型性能的影响,本文系统分析了温度系数t为0.07~0.20时模型在各类损坏场景的表现. 实验数据如图7所示,温度参数的变化对模型整体性能影响较小,表明模型对温度参数的选择具有较好的鲁棒性. 在温度参数从0.07增加到0.20的过程中,各任务性能保持高度稳定,平均波动范围小于0.05个百分点. 值得注意的是,在Glass等空间失真类干扰下,不同温度设置下的性能标准差相差较大,这表明温度参数可能对这些特定干扰类型的特征对比学习产生轻微影响. 而在结构化噪声场景中,模型展现出最强的稳定性. 特别地,当温度参数设置为0.10时,模型在多数任务场景下达到最佳平衡状态,此时各项任务的平均标准差最小. 在极端噪声场景中,温度参数的调整对性能影响最为有限,这表明对比学习对这些噪声类型的特征表达具有较强的适应性.
综合分析表明,温度参数在0.07~0.15时都能取得良好效果,模型性能对温度选择表现相对不敏感. 这一发现为实际应用中的参数调优提供了重要参考:在保证温度参数处于合理范围内的前提下,可以优先考虑其他对模型性能影响更大的超参数进行优化. 值得注意的是,过低的温度参数(t<0.07)可能导致特征对比过于尖锐,而过高的温度参数(t>0.20)则可能使对比学习过于平滑,但在本实验的参数范围内均未观察到明显的性能下降.
4)跨模态对齐损失敏感度分析. 为探究跨模态对齐损失权重对模型性能的影响,本文系统分析了权重系数从1.0到0.2范围内模型在2类最具代表性的损坏场景中的表现. 实验数据如图8所示,可以看到权重系数的变化对模型整体性能影响较小,表明模型对联合损失权重的敏感度较低、鲁棒性较强. 在权重系数从1.0逐渐降低到0.2的过程中,模型性能保持高度稳定,Gaussian场景准确率波动范围仅0.01个百分点(63.6%~63.59%),Shot场景波动范围为0.03个百分点(51.79%~51.82%).
综合分析表明,联合权重系数在0.2~1.0都能取得良好效果,模型性能对联合损失权重变化的表现相对不敏感. 这一发现为实际应用中的参数调优提供了重要参考:在保证联合损失权重处于合理范围内的前提下,可以优先考虑其他对模型性能影响更大的超参数进行优化,标准差的稳定也进一步验证了该结论.
5)平衡损失权重敏感度分析. 为探究分布约束平衡损失权重对模型性能的影响,我们系统分析了权重系数从0.2到0.5范围内模型在2类典型噪声场景中的表现. 实验数据如图9所示,可以看到权重系数的变化对模型整体性能影响有限,Gaussian场景下准确率波动范围为0.13个百分点(63.54%~63.67%),Defocus场景波动为0.17个百分点(51.79%~51.96%). 在权重系数从0.2增加到0.5的过程中,模型性能呈现平缓下降趋势.
综合分析表明,平衡损失权重系数在0.2~0.5时均能维持较好效果,其中权重系数0.2时取得最佳性能(Gaussian:63.67%,Shot:51.96%). 模型对平衡损失权重变化表现较为稳健,最大性能波动不足0.2个百分点. 这一发现说明在实际应用中,可将平衡损失权重设置在0.2~0.3以获得最优性能.
6)信赖损失权重敏感度分析. 为探究信赖损失权重对模型性能的影响,本文系统分析了权重系数从1.0到0.2范围内模型在2类最具代表性的损坏场景中的表现. 实验数据如图10所示,可以看到权重系数的变化对模型整体性能影响较小,表明模型对信赖度损失权重的敏感度较低、鲁棒性较强. 在权重系数从1.0逐渐降低到0.2的过程中,模型性能保持高度稳定,平均波动范围小于0.05个百分点.
综合分析表明,信赖度权重系数在0.2~1.0时能取得良好效果,模型性能对信赖损失权重变化的表现相对不敏感. 这一发现为实际应用中的参数调优提供了重要参考:在保证信赖损失权重处于合理范围内的前提下,可以优先考虑其他对模型性能影响更大的超参数进行优化.
本文提出了一种基于跨模态协同机制的测试时自适应方法CVPTA,解决了MM-TTA中一个非常现实且极具挑战的问题:多模态数据在实际应用时面对分布偏移问题无法实现信息的充分利用和模型的稳定运行. CVPTA通过引入跨模态协同机制,强化了模态内部结构建模与模态间特征交互的鲁棒性;创新性地设计了低质量视图裁减和筛选模块,结合对比损失优化策略,从训练到测试的全流程确保模型在适应过程中具备稳定且可泛化的特征感知能力. 在此基础上,该方法基于视图不确定性集成多个视图的预测结果,通过一致性融合策略实现更稳健的模态感知与最终决策. 在2个公开数据集上的大量实验,证明了CVPTA的有效性以及全流程多视图系统的可行性.
尽管CVPTA框架在多个数据集上取得了优异成绩,但是现实世界中TTA方法需要面对的挑战还不止于此. 随着模态数量继续增加,基于多视图的全流程框架也将受到严峻的考验:跨模态协同机制需要处理更复杂的模态间关系,低质量视图裁减和筛选模块面临更高维度的计算压力,对比损失优化策略也需要适应更动态的特征空间. 如何准确衡量持续复杂化的模态可靠性,在此基础上如何构建更加完善的多模态协同体系,以及在更加开放和复杂的任务中怎么保证模型的性能,都是亟待解决的问题.
作者贡献声明:尹小龙提出了CVPTR方法流程和框架,完成实验和论文撰写;詹德川完善方法,给出了参数调整方面的建议;姜远指导论文写作,并规范了排版.

参考文献 引证文献
排序方式:
1
Gama J, Žliobaitė I, Bifet A, et al. A survey on concept drift adaptation[J]. ACM Computing Surveys, 2014, 46(4): 1−37
2
Moreno-Torres J G, Raeder T, Alaíz-Rodríguez R, et al. A unifying view on dataset shift in classification[J]. Pattern Recognition, 2012, 45(1): 521−530
3
Wang Mei, Deng Weihong. Deep visual domain adaptation: A survey[J]. Neurocomputing, 2018, 312: 135−153
4
Sun Yu, Wang Xiaolong, Liu Zhuang, et al. Test-time training with self-supervision for generalization under distribution shifts[C]//Proc of the 37th Int Conf on Machine Learning. Online, JMLR. org, 2020: 9229−9248
5
Pan S J, Yang Qiang. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345−1359
6
Zhou Kaiyang, Liu Ziwei, Qiao Yu, et al. Domain generalization: A survey[J/OL]. 2021[2022-08-08]. https://arxiv.org/abs/2103.02503
7
Liang Jian, He Ran, Tan Tieniu. A comprehensive survey on test-time adaptation under distribution shifts[J]. International Journal of Computer Vision, 2023, 133: 31−64
8
Wang Dequan, Shelhamer E, Liu Shaoteng, et al. Tent: Fully test-time adaptation by entropy minimization[C/OL]//Proc of the 9th In Int Conf on Learning Representations. Vienna: OpenReview. net, 2021[2021-03-18]. https://openreview.net/forum?id=uXl3bZLkr3c
9
Wilson G, Cook D J. A survey of unsupervised deep domain adaptation[J]. ACM Transactions on Intelligent Systems and Technology, 2020, 11(5): 1−46
10
Panagiotakopoulos T, Dovesi P L, Härenstam-Nielsen L, et al. Online domain adaptation for semantic segmentation in ever-changing conditions[C]// Proc of the 17th European Conf on Computer Vision (ECCV 2022). Berlin: Springer, 2022: 128−146
11
Yang Tao, Zhou Shenglong, Wang Yuwang, et al. Test-time batch normalization[J/OL]. 2022[2022-05-20]. https://arxiv.org/abs/2205.10210.
12
Park H, Hwang J, Mun S, et al. MedBN: Robust test-time adaptation against malicious mest mamples[C]//Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2024: 5997−6007
13
Lim H, Kim B, Choo J, et al. TTN: A domain-shift aware batch normalization in test-time adaptation[C/OL]// Proc of the 9th In Int Conf on Learning Representations. Kigali, OpenReview.net, 2023[2023-02-18]. https://arxiv.org/abs/2302.05155
14
Hong Junyuan, Lyu Lingjuan, Zhou Jiayu, et al. MECTA: Memory-economic continual test-time model adaptation[C/OL]// Proc of the 9th In Int Conf on Learning Representations. Kigali, OpenReview. net, 2023[2023-03-04]. https://openreview.net/pdf?id=N92hjSf5NNh
15
Sójka D, Cygert S, Twardowski B, et al. AR-TTA: A simple method for real-world continual test-time adaptation[C/OL]//Proc of British Machine Vision Conf. Glasgow: BMVA. org, 2024[2024-10-14]. https://arxiv.org/pdf/2309.10109
16
Yu Yongcan, Sheng Lijun, He Ran, et al. Benchmarking test-time adaptation against distribution shifts in image classification[J]. arXiv preprint, arXiv: 2307.03133, 2023
17
Chen Dian, Wang Dequan, Darrell T, et al. Contrastive test-time adaptation[C]//Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 295−305
18
Boudiaf M, Mueller R, Ayed I B, et al. Parameter-free online test-time adaptation[C]// /Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 8334−8343
19
周植, 张丁楚, 李宇峰, 等. 面向开集识别的稳健测试时适应方法. 软件学报, 2024, 35(4): 1667−1681
Zhou Zhi, Zhang Dingchu, Li Yufeng, et al. Robust test-time adaptation method for open-set recognition[J]. Journal of Software, 2024, 35(4): 1667−1681(in Chinese)
20
Baltrušaitis T, Ahuja C, Morency L. Multimodal machine learning: A survey and taxonomy[J]. IEEE transactions on pattern analysis and machine intelligence, 2018, 41(2): 423−443
21
Radford A, Kim J, Hallacy C, et al. Learning transferable visual models from natural language supervision[C/OL]// Proc of Int Conf on Machine Learning. New York: ACM, 2021[2021-02-26]. https://arxiv.org/abs/2103.00020
22
Lei Jixiang, Pernkopf F. Two-level test-time adaptation in multimodal learning[C/OL]//Proc of ICML 2024 Workshop on Foundation Models in the Wild. New York: ACM, 2024[2025-05-10]. https://icml.cc/virtual/2024/39426.
23
Mao Tingting, Li Chang, Zhao Yuchang, et al. Online test-time adaptation for patient-independent seizure prediction[J]. IEEE Sensors Journal 2023, 23(19): 23133−23144
24
Yuan Longhui, Binhui Xie, Shuang Li. Robust test-time adaptation in dynamic scenarios[C]// Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2023: 15922−15932
25
Varsavsky T, Orbes-Arteaga M, Sudre C H, et al. Test-time unsupervised domain adaptation[J]. arXiv preprint, arXiv: 2010.01926, 2020
26
Tsai Y Y , Chen F C , Chen A Y C, et al. GDA: Gneralized diffusion for robust test-time adaptation[C]//Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2024: 23242−23251
27
Zhang M, Levine S, Finn C, et al. Memo: Test time robustness via adaptation and augmentation[J]. Advances in Neural Information Processing Systems 2022, 35: 38629−38642
28
Li Shuang, Yuan Longhui, Xie Binhui, et al. Generalized robust test-time adaptation in continuous dynamic scenarios[C/OL]// Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2023[2023-05-08]. https://arxiv.org/abs/2303.13899
29
Gong T, Kim Y, Lee T, et al. Sotta: Robust test-time adaptation on noisy data streams[J]. Advances in Neural Information Processing Systems 2023, 36: 14070−14093
30
Park H, Hwang J, Mun S, et al. Medbn: Robust test-time adaptation against malicious test samples[C] // Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2024: 5997−6007
31
Wang Qin, Fink O, Gool L, et al. Continual test-time domain adaptation[C/OL] // Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022[2022-06-01]. https://openaccess.thecvf.com/content/CVPR2022/papers/Wang_Continual_Test-Time_Domain_Adaptation_CVPR_2022_paper.pdf
32
Liu Jiaming, Xu Ran, Yang Senqiao, et al. Continual-mae: Adaptive distribution masked autoencoders for continual test-time adaptation[C]// Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2024: 28653−28663
33
Döbler M, Marsden R A, Yang Bin. Robust mean teacher for continual and gradual test-time adaptation[C]// Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2023: 7704−7714
34
Song J, Lee J, Kweon I S, et al. Ecotta: Memory-efficient continual test-time adaptation via self-distilled regularization[C]// Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2023: 11920−11929
35
Tang Yushun, Chen Shuoshuo, Kan Zhehan, et al. Learning visual conditioning tokens to correct domain shift for fully test-time adaptation[J]. IEEE Transactions on Multimedia 2024, 10(1145): 6260−6269
36
Karmanov A, Guan Dayan, Lu Shijian, et al. Efficient test-time adaptation of vision-language models[C] // Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. New York: ACM, 2024: 14162−14171
37
Zhou Lihua, Ye Mao, Li Shuaifeng, et al. Bayesian test-time adaptation for vision-language models[J]. arXiv preprint, arXiv: 2503.09248, 2025
38
Döbler M, Marsden R A, Raichle T, et al. A lost opportunity for vision-language models: A comparative study of online test-time adaptation for vision-language models[J]. arXiv preprint, arXiv: 2405.14977, 2024
39
Zhang Yunbei, Mehra A, Hamm J. OT-VP: Optimal transport-guided visual prompting for test-time adaptation[C] // Proc of IEEE/CVF Winter Conf on Applications of Computer Vision. Piscataway, NJ: IEEE, 2025: 1122−1132
40
Weijler L, Mirza M J, Sick L, et al. TTT-KD: Test-time training for 3D semantic segmentation through knowledge distillation from foundation models[J]. arXiv preprint, arXiv: 2403.11691, 2024
41
Bar Y, Shaer S, Romano Y. Protected test-time adaptation via online entropy matching: A betting approach[J]. Advances in Neural Information Processing Systems 2024, 37: 85467−85499
42
Dong Hao, Liu Moru, Zhou Kaiyang, et al. Advances in multimodal adaptation and generalization: From traditional approaches to foundation models[J]. arXiv preprint, arXiv: 2501.18592, 2025
43
Shu Manli, Nie Weili, Huang Dean, et al. Test-time prompt tuning for zero-shot generalization in vision-language models[J]. Advances in Neural Information Processing Systems 2022, 35: 14274−14289
44
Xiao Zehao, Shen Jiayi, Derakhshani M M, et al. Any-shift prompting for generalization over distributions[C] // Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2024: 13849−13860
45
Sun Jiachen, Ibrahim M, Hall M, et al. VPA: Fully test-time visual prompt adaptation[C/OL]//Proc of the 31st ACM Int Conf on Multimedia. New York: ACM, 2023: 5796−5806
46
Tsai Yun-Yun, Mao Chengzhi, Yang Junfeng. [J]. Advances in Neural Information Processing Systems, 2023, 36: 27897−27921
47
Hassan J, Gani H, Hussein N, et al. Align your prompts: Test-time prompting with distribution alignment for zero-shot generalization[J]. Advances in Neural Information Processing Systems, 2023, 36: 80396−80413
48
Wang Yanshuo, Cheraghian A, Hayder Z, et al. Backpropagation-free network for 3D test-time adaptation[C]//Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2024: 23231−23241
49
Osowiechi D, Noori M, Hakim G A V, et al. WATT: Weight average test-time adaptation of clip[J]. arXiv preprint, arXiv: 2406.13875, 2024
50
Chen Ting, Kornblith S, Norouzi M, et al. A simple framework for contrastive learning of visual representations[C/OL]//Proc of Int Conf on Machine Learning. New York: ACM, 2020[2020-07-01]. https://arxiv.org/abs/2002.05709
51
Xiao Zehao, Zhen Xiantong, Liao Shengcai, et al. Energy-based test sample adaptation for domain generalization[J]. arXiv preprint, arXiv: 2302.11215, 2023
52
Xiao Zehao, Zhen Xiantong, Shao Ling, et al. Learning to generalize across domains on single test samples[J]. arXiv preprint, arXiv: 2202.08045, 2022
53
Li Yingming, Yang Ming, Zhang Zhongfei. A survey of multi-view representation learning[J]. IEEE Transactions on Knowledge and Data Engineering 2018, 31(10): 1863−1883
54
Trosten D J, Løkse S, Jenssen R, et al. Reconsidering representation alignment for multi-view clustering[C]// Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2021: 1255−1265
55
Zhang Qingheng, Sun Zequn, Hu Wei, et al. Multi-view knowledge graph embedding for entity alignment[J]. arXiv preprint, arXiv: 1906.02390, 2019
56
Wang Ren, Sun Haoliang, Ma Yuling, et al. Metaviewer: Towards a unified multi-view representation[C]// Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2023: 11590−11599
57
Guo Chenfeng, Wu Dongrui. Canonical correlation analysis (CCA) based multi-view learning: An overview[J]. arXiv preprint, arXiv: 1907.01693, 2021
58
Blum A, Mitchell T. Combining labeled and unlabeled data with co-training[C]// Proc of the 11th Annual Conf on Computational Learning Theory. New York: ACM, 1998: 92–100
59
Chen Mansheng, Lin Jiaqi, Li Xianglong, et al. Representation learning in multi-view clustering: A literature review[J]. Data Science and Engineering 2022, 7(3): 225−241
60
Qin Zequn, Chen Jingyu, Chen Chao, et al. Unifusion: Unified multi-view fusion Transformer for spatial-temporal representation in bird's-eye-view[C]// Proc of IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2023: 8656−8665
61
Ke Guanzhou, Chao Guoqing, Wang Xiaoli, et al. A clustering-guided contrastive fusion for multi-view representation learning[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 34(4): 2056−2069
62
Cap Fuyuan, Chen Xiaohui. Multi-view attributed graph clustering with shared and specific representations[J]. Journal of Software, 2025, 36(3): 1254−1267
63
Zhao Xingwang, Wang Shujun, Liu Xiaolin, et al. Joint spectral embedding for multi-view clustering via bipartite graph[J]. Journal of Software, 2024, 35(9): 4408−4424
64
Siddiqui N, Tirupattur P, Shah M. DVANet: Disentangling view and action features for multi-view action recognition[J]. Proceedings of the AAAI Conf on Artificial Intelligence, 2024, 38(5): 4873−4881
65
Bruce X B, Liu Yan, Zhang Xiang, et al. MMNet: Amodel-based multimodal network for human action recognition in RGB-D videos[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(3): 3522−3538
66
He Kaiming, Fan Haoqi, Wu Yuxin, et al. Momentum contrast for unsupervised visual representation learning[C] //Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2020: 9726−9735
67
Khosla P, Teterwak P, Wang Chen, et al. Supervised contrastive learning[J]. Advances in Neural Information Processing Systems 2020, 33: 18661−18673
68
Li Junnan, Zhou Pan, Xiong Caiming, et al. Prototypical contrastive learning of unsupervised representations[J]. arXiv preprint, arXiv: 2005.04966, 2021
69
Jia Chao, Yang Yinfei, Xia Ye, et al. Scaling up visual and vision-language representation learning with noisy text supervision[C/OL]//Proc of Int Conf on Machine Learning. New York: ACM, 2021[2021-06-11]. https://arxiv.org/abs/2102.05918
70
Grill J B, Strub F, Altché F, et al. Bootstrap your own latent-a new approach to self-supervised learning[C]// Proc of the 34th Int Conf on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2020: 21271−21284
71
Chen Xinlei, He Kaiming. Exploring simple siamese representation learning[C]// Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2021: 15745−15753
72
Kay W, Carreira J, Simonyan K, et al. The Kinetics human action video dataset[J]. arXiv preprint, arXiv: 1705.06950, 2017
73
Chen Honglie, Xie Weidi, Vedaldi A, et al. Vggsound: A large-scale audio-visual dataset[C] // Proc of IEEE Int Conf on Acoustics, Speech and Signal Processing. Piscataway, NJ: IEEE, 2020: 721−725
74
Hendrycks D, Dietterich T. Benchmarking neural network robustness to common corruptions and perturbations[J]. arXiv preprint, arXiv: 1903.12261, 2019
75
Wang Dequan, Shelhamer E, Liu Shaoteng, et al. Tent: Fully test-time adaptation by entropy minimization[C/OL]// Proc of the 9th Int Conf on Learning Representations. Vienna: OpenReview. net, 2021[2021-03-18]. https://openreview.net/forum?id=uXl3bZLkr3c
76
Shin I, Tsai Y, Zhuang Bingbing, et al. MM-TTA: Multi-modal test-time adaptation for 3D semantic segmentation[C] // Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 16907−16916
77
Niu Shuaicheng, Wu Jiaxiang, Zhang Yifan, et al. Efficient test-time model adaptation without forgetting[C/OL]//Proc of Int Conf on Machine Learning, New York: ACM, 2022[2022-05-31]. https://arxiv.org/abs/2204.02610
78
Niu Shuaicheng, Wu Jiaxiang, Zhang Yifan, et al. Towards stable test-time adaptation in dynamic wild world[J]. arXiv preprint, arXiv: 2302.12400, 2023
79
Gong Yuan, Rouditchenko A, Liu A H, et al. Contrastive audio-visual masked autoencoder[J]. arXiv preprint, arXiv: 2210.07839, 2023
80
Yang Mouxing, Li Yunfan, Zhang Changqing, et al. Test-time adaptation against multi-modal reliability bias[C/OL]// Proc of the 12th Int Conf on Learning Representations. Vienna: OpenReview. net, 2024[2024-01-16]. https://openreview.net/forum?id=TPZRq4FALB
2025年第62卷第9期
99
43
0
引用本文
BibTeX
文章信息
doi: 10.7544/issn1000-1239.202550418
  • 接收时间:2025-06-06
  • 首发时间:2026-11-10
  • 出版时间:2025-09-01
补充材料
相关文章
文章信息
作者
出版历史
  • 收稿日期:2025-06-06
  • 修回日期:2025-07-16
基金
作者信息
    南京大学人工智能学院 南京 210023
    计算机软件新技术全国重点实验室(南京大学) 南京 210023

通讯作者:

詹德川()
参考文献
分享链接
https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202550418
分享至
全文二维码

扫描看全文

引用本文
BibTeX
本文的引用情况
2种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏