收藏切换
Improving Lightweight Model Robustness via Adversarial Ranking Distillation Method
收藏切换
Mingrui Lao1, Zheng Li1, Zhaoyun Ding1, Xueyi Zhang1, Xianghu Yue2, Yanming Guo1
Journal of Computer Research and Development | 2026, 63(6) : 1506 - 1521
Less
收藏切换
Journal of Computer Research and Development | 2026, 63(6): 1506-1521
• Artificial Intelligence •
Improving Lightweight Model Robustness via Adversarial Ranking Distillation Method
Full
Mingrui Lao1, Zheng Li1, Zhaoyun Ding1, Xueyi Zhang1, Xianghu Yue2, Yanming Guo1
Affiliations
  • 1College of Systems Engineering, National University of Defense Technology, Changsha 410073
  • 2College of Intelligence and Computing, Tianjing University, Tianjin 300072
  • born in 1995. PhD, associate professor. His main research interests include trustworthy AI and multimodal learning

    born in 2000. PhD candidate. His main research interests include adversarial training and multimodal learning

    born in 1982. PhD, professor. His main research interests include data mining and trustworthy AI

    born in 1998. PhD candidate. His main research interests include adversarial training and continual learning

    born in 1994. PhD, associate professor. His main research interests include multimodal learning and trustworthy AI

    born in 1989. PhD, associate professor. His main research interests include multimodal learning and trustworthy AI

About Author:

born in 1995. PhD, associate professor. His main research interests include trustworthy AI and multimodal learning

born in 2000. PhD candidate. His main research interests include adversarial training and multimodal learning

born in 1982. PhD, professor. His main research interests include data mining and trustworthy AI

born in 1998. PhD candidate. His main research interests include adversarial training and continual learning

born in 1994. PhD, associate professor. His main research interests include multimodal learning and trustworthy AI

born in 1989. PhD, associate professor. His main research interests include multimodal learning and trustworthy AI

Published: 2026-06-01 doi: 10.7544/issn1000-1239.202550339
Outline
收藏切换

With the widespread application of lightweight models in resource-constrained scenarios such as edge computing, their insufficient adversarial robustness has become increasingly prominent. While adversarial distillation serves as a primary approach to enhance lightweight models’ robustness, existing methods predominantly employ KL (Kullback-Leibler) divergence-based rigid constraints for robust knowledge transfer, which still suffer from flawed knowledge modeling and limited transfer efficiency. To address these issues, we propose an adversarial ranking distillation (ARD) method that significantly improves lightweight models’ robustness through a priority ranking constraint mechanism and a multi-level consistency framework. Specifically, the priority ranking constraint mechanism organizes the output elements of vision models on adversarial samples by importance ranking, enforces teacher-student ranking consistency, and implements differentiable loose coupling constraints by approximating discontinuous ranking processes through hyperbolic tangent functions. Building upon this, the multi-level ranking consistency distillation framework models transfer robustness knowledge from three perspectives: categorical semantic correlation, sample semantic correlation, and adversarial discrepancy correlation, enabling multi-perspective transmission of teachers’ adversarial defense capabilities while synergistically improving lightweight models’ clean sample accuracy and adversarial robustness. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets validate the effectiveness of our method, demonstrating substantial performance advantages over state-of-the-art adversarial distillation approaches. Furthermore, the proposed ARD exhibits enhanced adaptability and generalization capabilities under scenarios of limited training data and black-box attacks, providing an efficient security solution for lightweight model deployment in edge computing environments.

adversarial robustness  /  lightweight model  /  knowledge distillation  /  adversarial defense  /  deep learning
老明瑞, 李正, 丁兆云, 张雪毅, 岳祥虎, 郭延明. 提高轻量级模型鲁棒性的对抗排序蒸馏方法. 计算机研究与发展, 2026 , 63 (6) : 1506 -1521 . DOI: 10.7544/issn1000-1239.202550339
Mingrui Lao, Zheng Li, Zhaoyun Ding, Xueyi Zhang, Xianghu Yue, Yanming Guo. Improving Lightweight Model Robustness via Adversarial Ranking Distillation Method[J]. Journal of Computer Research and Development, 2026 , 63 (6) : 1506 -1521 . DOI: 10.7544/issn1000-1239.202550339
Year 2026 volume 63 Issue 6
75
36
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202550339
  • Receive Date:2025-05-23
  • Online Date:2026-11-17
  • Published:2026-06-01
Article Data
Affiliations
History
  • Received:2025-05-23
  • Revised:2026-03-04
Affiliations
    1College of Systems Engineering, National University of Defense Technology, Changsha 410073
    2College of Intelligence and Computing, Tianjing University, Tianjin 300072
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202550339
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT