收藏切换
Confidence-Guided Multimodal Data Test-Time Adaptation Framework
收藏切换
Xiaolong Yin, Dechuan Zhan*, Yuan Jiang
Journal of Computer Research and Development | 2025, 62(9) : 2166 - 2182
Less
收藏切换
Journal of Computer Research and Development | 2025, 62(9): 2166-2182
• Special Issue on Advances and Applications of Data-Centric AI •
Confidence-Guided Multimodal Data Test-Time Adaptation Framework
Full
Xiaolong Yin, Dechuan Zhan*, Yuan Jiang
Affiliations
  • School of Artificial Intelligence, Nanjing University, Nanjing 210023
  • National Key Laboratory for Novel Software Technology (Nanjing University), Nanjing 210023
  • born in 1998. PhD candidate. His main research interests include test-time adaptation and reinforcement learning

    born in 1982. PhD, professor. His main research interests include machine learning and data mining

    born in 1976. PhD, professor. Her main research interests include machine learning and data mining

About Author:

born in 1998. PhD candidate. His main research interests include test-time adaptation and reinforcement learning

born in 1982. PhD, professor. His main research interests include machine learning and data mining

born in 1976. PhD, professor. Her main research interests include machine learning and data mining

Published: 2025-09-01 doi: 10.7544/issn1000-1239.202550418
Outline
收藏切换

Test-time adaptation (TTA) seeks to dynamically recalibrate a deployed model through online fine-tuning with unlabeled or sparsely labeled data, addressing performance degradation caused by distribution shift, sensor noise, illumination changes, and other real-world challenges. It has demonstrated great promise in latency- and robustness-critical applications such as autonomous driving, remote healthcare, and video surveillance. However, existing multimodal TTA methods typically overlook the quality variance across views—low-quality or faulty views data can introduce harmful gradients—and lack mechanisms to preserve internal temporal consistency, undermining stability in dynamic scenarios. To tackle these issues, we propose CVPTA (confidence-guided view pruning and temporal contrastive attention), a unified framework comprising three core modules: 1) confidence-guided dynamic attention, which computes each view’s confidence via its predictive entropy and uses a softmax over inverted entropy scores to attenuate high-entropy views data during feature fusion; 2) view pruning, which evaluates image-modal quality using the Laplacian variance (for blur) and histogram skewness (for exposure), discards views below a quality threshold, and maintains a constant view count by interpolating adjacent high-quality views data or injecting Gaussian noise—dramatically reducing noise accumulation; 3) temporal contrastive self-supervision, which treats adjacent frames of the same view as positive pairs and non-adjacent frames as negatives, and applies a contrastive loss to enforce temporal feature consistency. We evaluate EVPTA on the public Kinetics-50-C and VGGSound-C perturbation benchmarks under the same online-update settings as prior work. Results show that EVPTA boosts Top-1 accuracy by approximately 2.3% and 0.7% on each benchmark and retains over 0.2% gains even under extreme noise conditions. Ablation studies further confirm the individual and synergistic contributions of all three modules. EVPTA requires no extra annotations, integrates seamlessly with existing multimodal systems, and delivers both efficiency and robustness—offering significant theoretical insights and practical value.

test-time adaptation  /  multimodal learning  /  multi-view learning  /  view pruning  /  temporal contrastive learning
尹小龙, 詹德川, 姜远. 置信引导的多模态数据测试时适应框架. 计算机研究与发展, 2025 , 62 (9) : 2166 -2182 . DOI: 10.7544/issn1000-1239.202550418
Xiaolong Yin, Dechuan Zhan, Yuan Jiang. Confidence-Guided Multimodal Data Test-Time Adaptation Framework[J]. Journal of Computer Research and Development, 2025 , 62 (9) : 2166 -2182 . DOI: 10.7544/issn1000-1239.202550418
Year 2025 volume 62 Issue 9
101
44
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202550418
  • Receive Date:2025-06-06
  • Online Date:2026-11-10
  • Published:2025-09-01
Article Data
Affiliations
History
  • Received:2025-06-06
  • Revised:2025-07-16
Affiliations
    School of Artificial Intelligence, Nanjing University, Nanjing 210023
    National Key Laboratory for Novel Software Technology (Nanjing University), Nanjing 210023
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202550418
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT