Article(id=1329419901032313734, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1329419793389695520, articleNumber=null, orderNo=null, doi=10.7544/issn1000-1239.202550339, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=null, receivedDate=1747929600000, receivedDateStr=2025-05-23, revisedDate=1772553600000, revisedDateStr=2026-03-04, acceptedDate=null, acceptedDateStr=null, onlineDate=1794888003833, onlineDateStr=2026-11-17, pubDate=1780243200000, pubDateStr=2026-06-01, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1794888003833, onlineIssueDateStr=2026-11-17, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1794888003833, creator=13701087609, updateTime=1794888003833, updator=13701087609, issue=Issue{id=1329419793389695520, tenantId=1146029695717560320, journalId=1324759914146529478, year='2026', volume='63', issue='6', pageStart='1377', pageEnd='1638', issueExtLink='null', onlineDate='null', pubDate='1780243200000', pubDateStr='2026-06-01', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=1, issueType=-1, specialIssue=null, createTime=1794887978168, creator='13701087609', updateTime=1794892139295, updator='13701087609', preIssue=null, nextIssue=null, articleTotal=null, ext={EN=IssueExt(id=1329437246815580885, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1329419793389695520, language=EN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=), CN=IssueExt(id=1329437246815580886, tenantId=1146029695717560320, journalId=1324759914146529478, issueId=1329419793389695520, language=CN, specialIssueTitle=, coverIllustrator=null, specialIssueEditor=, specialIssueAbout=)}, issueFiles=null, downloadFileDto=null}, startPage=1506, endPage=1521, ext={EN=ArticleExt(id=1329419901263000455, articleId=1329419901032313734, tenantId=1146029695717560320, journalId=1324759914146529478, language=EN, title=Improving Lightweight Model Robustness via Adversarial Ranking Distillation Method, columnId=1325489891789910943, journalTitle=Journal of Computer Research and Development, columnName=Artificial Intelligence, runingTitle=null, highlight=null, articleAbstract=

With the widespread application of lightweight models in resource-constrained scenarios such as edge computing, their insufficient adversarial robustness has become increasingly prominent. While adversarial distillation serves as a primary approach to enhance lightweight models’ robustness, existing methods predominantly employ KL (Kullback-Leibler) divergence-based rigid constraints for robust knowledge transfer, which still suffer from flawed knowledge modeling and limited transfer efficiency. To address these issues, we propose an adversarial ranking distillation (ARD) method that significantly improves lightweight models’ robustness through a priority ranking constraint mechanism and a multi-level consistency framework. Specifically, the priority ranking constraint mechanism organizes the output elements of vision models on adversarial samples by importance ranking, enforces teacher-student ranking consistency, and implements differentiable loose coupling constraints by approximating discontinuous ranking processes through hyperbolic tangent functions. Building upon this, the multi-level ranking consistency distillation framework models transfer robustness knowledge from three perspectives: categorical semantic correlation, sample semantic correlation, and adversarial discrepancy correlation, enabling multi-perspective transmission of teachers’ adversarial defense capabilities while synergistically improving lightweight models’ clean sample accuracy and adversarial robustness. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets validate the effectiveness of our method, demonstrating substantial performance advantages over state-of-the-art adversarial distillation approaches. Furthermore, the proposed ARD exhibits enhanced adaptability and generalization capabilities under scenarios of limited training data and black-box attacks, providing an efficient security solution for lightweight model deployment in edge computing environments.

, authors=Mingrui Lao1, Zheng Li1, Zhaoyun Ding1, Xueyi Zhang1, Xianghu Yue2, Yanming Guo1, authorsList=Mingrui Lao, Zheng Li, Zhaoyun Ding, Xueyi Zhang, Xianghu Yue, Yanming Guo, authorCompany=null, correspAuthors=null, authorNote=

born in 1995. PhD, associate professor. His main research interests include trustworthy AI and multimodal learning

born in 2000. PhD candidate. His main research interests include adversarial training and multimodal learning

born in 1982. PhD, professor. His main research interests include data mining and trustworthy AI

born in 1998. PhD candidate. His main research interests include adversarial training and continual learning

born in 1994. PhD, associate professor. His main research interests include multimodal learning and trustworthy AI

born in 1989. PhD, associate professor. His main research interests include multimodal learning and trustworthy AI

, correspAuthorsNote=null, copyrightStatement=Copyright © 2026 Journal of Computer Research and Development. All rights reserved., copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=null, magXml=null, pdfUrl=null, pdf=null, pdfFileSize=null, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=null, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=null, mapNumber=null, fund=null), CN=ArticleExt(id=1329419904513586086, articleId=1329419901032313734, tenantId=1146029695717560320, journalId=1324759914146529478, language=CN, title=提高轻量级模型鲁棒性的对抗排序蒸馏方法, columnId=1325489892104483745, journalTitle=计算机研究与发展, columnName=人工智能, runingTitle=null, highlight=null, articleAbstract=

随着轻量级模型在边缘计算等资源受限场景中的广泛应用,其对抗鲁棒性不足的问题日益凸显。对抗蒸馏作为提升轻量级模型鲁棒性的重要手段,虽然现有方法普遍采用基于KL(Kullback-Leibler)散度的硬性约束实现鲁棒知识传递,但仍面临知识建模不完善和传递效率受限等问题。为此,该研究提出一种对抗排序蒸馏方法,通过优先级排序约束机制与多层次一致性框架,显著提升轻量级模型面向对抗攻击的抵御能力。具体地,前者旨在将视觉模型基于对抗样本的输出元素的重要性排序,进而约束教师-学生模型间的元素优先级保持一致,并通过函数近似非连续排序过程实现可导的松耦合约束。在此基础上,多层次排序一致性蒸馏框架从类别语意关联、样本语意关联以及对抗差异关联3个方面对鲁棒性进行建模与知识迁移,实现教师对抗防御能力的多视角传递,协同提升轻量级模型的干净样本准确率与对抗鲁棒性。在CIFAR-10,CIFAR-100,Tiny ImageNet数据集上实验验证了该方法的有效性,且相比于现有的先进对抗蒸馏方法有显著性能优势。同时,对抗排序蒸馏方法在训练数据受限和黑盒攻击场景下展现出更强的适应性与泛化能力,为轻量级模型在边缘部署中的安全防御提供了高效解决方案。

, authors=老明瑞1, 李正1, 丁兆云1, 张雪毅1, 岳祥虎2, 郭延明1, authorsList=老明瑞, 李正, 丁兆云, 张雪毅, 岳祥虎, 郭延明, authorCompany=null, correspAuthors=null, authorNote=

作者贡献声明

老明瑞负责方案整体设计和实验方法,并撰写论文;李正、张雪毅负责部分算法思路和实验;丁兆云、岳祥虎、郭延明提出指导意见并修改论文。岳祥虎和郭延明同为通信作者。

1995年生。博士,副教授。主要研究方向为可信人工智能、多模态学习

2000年生。博士研究生。主要研究方向为对抗训练、多模态学习

1982年生。博士,教授。 主要研究方向为数据挖掘、可信人工智能

1998年生。博士研究生。 主要研究方向为对抗训练、持续学习

1994年生。 博士,副教授。主要研究方向为多模态学习、可信人工智能

1989年生。 博士,副教授。 主要研究方向为多模态学习、可信人工智能

, correspAuthorsNote=null, copyrightStatement=版权所有 © 2026 计算机研究与发展编辑部, copyrightOwner=null, extLink=null, articleAbsUrl=null, sourceXml=lcSb70ceRff+qfFKXnuTzw==, magXml=/C/5Aauz3vc30CZqgnkIgg==, pdfUrl=null, pdf=CkalniS7b/jcMYRa7WlfkQ==, pdfFileSize=2414814, pdfExtLink=null, richHtmlUrl=null, mobilePdfUrl=null, reviewReport=null, pdfFirstPage=null, abstractGraph=NAJKB9Gq+wGQLdK/GunxLA==, abstractGraphContent=null, abstractVideo=null, citation=null, cebUrl=null, magXmlContent=+lF6+8OuATox5LndYwXUdA==, mapNumber=null, fund=null)}, authors=[Author(id=1329419905054651310, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=laomingrui17@nudt.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419905146926000, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419905054651310, language=EN, stringName=Mingrui Lao, firstName=Mingrui, middleName=null, lastName=Lao, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Systems Engineering, National University of Defense Technology, Changsha 410073, bio={"content":"

born in 1995. PhD, associate professor. His main research interests include trustworthy AI and multimodal learning

"}, bioImg=null, bioContent=

born in 1995. PhD, associate professor. His main research interests include trustworthy AI and multimodal learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419905230812081, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419905054651310, language=CN, stringName=老明瑞, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1国防科技大学系统工程学院 长沙 410073, bio={"img":"NKSZt3BLLpVUkqaBxgXhmA==","content":"

1995年生。博士,副教授。主要研究方向为可信人工智能、多模态学习

"}, bioImg=NKSZt3BLLpVUkqaBxgXhmA==, bioContent=

1995年生。博士,副教授。主要研究方向为可信人工智能、多模态学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419904823964583, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, xref=1, ext=[AuthorCompanyExt(id=1329419904836547496, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Systems Engineering, National University of Defense Technology, Changsha 410073), AuthorCompanyExt(id=1329419904853324713, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1国防科技大学系统工程学院 长沙 410073)])]), Author(id=1329419905335669683, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419905436332981, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419905335669683, language=EN, stringName=Zheng Li, firstName=Zheng, middleName=null, lastName=Li, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Systems Engineering, National University of Defense Technology, Changsha 410073, bio={"content":"

born in 2000. PhD candidate. His main research interests include adversarial training and multimodal learning

"}, bioImg=null, bioContent=

born in 2000. PhD candidate. His main research interests include adversarial training and multimodal learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419905511830454, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419905335669683, language=CN, stringName=李正, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1国防科技大学系统工程学院 长沙 410073, bio={"img":"+QZmEvtx0dM5hNkZVIyrFA==","content":"

2000年生。博士研究生。主要研究方向为对抗训练、多模态学习

"}, bioImg=+QZmEvtx0dM5hNkZVIyrFA==, bioContent=

2000年生。博士研究生。主要研究方向为对抗训练、多模态学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419904823964583, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, xref=1, ext=[AuthorCompanyExt(id=1329419904836547496, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Systems Engineering, National University of Defense Technology, Changsha 410073), AuthorCompanyExt(id=1329419904853324713, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1国防科技大学系统工程学院 长沙 410073)])]), Author(id=1329419905574745016, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419905641853882, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419905574745016, language=EN, stringName=Zhaoyun Ding, firstName=Zhaoyun, middleName=null, lastName=Ding, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Systems Engineering, National University of Defense Technology, Changsha 410073, bio={"content":"

born in 1982. PhD, professor. His main research interests include data mining and trustworthy AI

"}, bioImg=null, bioContent=

born in 1982. PhD, professor. His main research interests include data mining and trustworthy AI

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419905704768443, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419905574745016, language=CN, stringName=丁兆云, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1国防科技大学系统工程学院 长沙 410073, bio={"img":"owAZcyFPqPRuucdQCnLPUQ==","content":"

1982年生。博士,教授。 主要研究方向为数据挖掘、可信人工智能

"}, bioImg=owAZcyFPqPRuucdQCnLPUQ==, bioContent=

1982年生。博士,教授。 主要研究方向为数据挖掘、可信人工智能

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419904823964583, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, xref=1, ext=[AuthorCompanyExt(id=1329419904836547496, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Systems Engineering, National University of Defense Technology, Changsha 410073), AuthorCompanyExt(id=1329419904853324713, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1国防科技大学系统工程学院 长沙 410073)])]), Author(id=1329419905792848829, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419905880929216, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419905792848829, language=EN, stringName=Xueyi Zhang, firstName=Xueyi, middleName=null, lastName=Zhang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Systems Engineering, National University of Defense Technology, Changsha 410073, bio={"content":"

born in 1998. PhD candidate. His main research interests include adversarial training and continual learning

"}, bioImg=null, bioContent=

born in 1998. PhD candidate. His main research interests include adversarial training and continual learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419905969009601, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419905792848829, language=CN, stringName=张雪毅, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1国防科技大学系统工程学院 长沙 410073, bio={"img":"D9/PkoJyIt9Z+TvRvY0o6g==","content":"

1998年生。博士研究生。 主要研究方向为对抗训练、持续学习

"}, bioImg=D9/PkoJyIt9Z+TvRvY0o6g==, bioContent=

1998年生。博士研究生。 主要研究方向为对抗训练、持续学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419904823964583, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, xref=1, ext=[AuthorCompanyExt(id=1329419904836547496, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Systems Engineering, National University of Defense Technology, Changsha 410073), AuthorCompanyExt(id=1329419904853324713, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1国防科技大学系统工程学院 长沙 410073)])]), Author(id=1329419906040312771, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=yuexianghubit@163.com, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419906115810245, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419906040312771, language=EN, stringName=Xianghu Yue, firstName=Xianghu, middleName=null, lastName=Yue, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2College of Intelligence and Computing, Tianjing University, Tianjin 300072, bio={"content":"

born in 1994. PhD, associate professor. His main research interests include multimodal learning and trustworthy AI

"}, bioImg=null, bioContent=

born in 1994. PhD, associate professor. His main research interests include multimodal learning and trustworthy AI

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419906191307718, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419906040312771, language=CN, stringName=岳祥虎, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2天津大学智能与计算学部 天津 300072, bio={"img":"vA5u8Poo9lisdmBR7i23LA==","content":"

1994年生。 博士,副教授。主要研究方向为多模态学习、可信人工智能

"}, bioImg=vA5u8Poo9lisdmBR7i23LA==, bioContent=

1994年生。 博士,副教授。主要研究方向为多模态学习、可信人工智能

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419904945599402, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, xref=2, ext=[AuthorCompanyExt(id=1329419904953988011, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904945599402, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2College of Intelligence and Computing, Tianjing University, Tianjin 300072), AuthorCompanyExt(id=1329419904966570924, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904945599402, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2天津大学智能与计算学部 天津 300072)])]), Author(id=1329419906258416584, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, orderNo=5, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=guoyanming@nudt.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419906367468490, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419906258416584, language=EN, stringName=Yanming Guo, firstName=Yanming, middleName=null, lastName=Guo, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Systems Engineering, National University of Defense Technology, Changsha 410073, bio={"content":"

born in 1989. PhD, associate professor. His main research interests include multimodal learning and trustworthy AI

"}, bioImg=null, bioContent=

born in 1989. PhD, associate professor. His main research interests include multimodal learning and trustworthy AI

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419906468131787, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419906258416584, language=CN, stringName=郭延明, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1国防科技大学系统工程学院 长沙 410073, bio={"img":"Ym/TntEKIz+FWT5ZVcWr5w==","content":"

1989年生。 博士,副教授。 主要研究方向为多模态学习、可信人工智能

"}, bioImg=Ym/TntEKIz+FWT5ZVcWr5w==, bioContent=

1989年生。 博士,副教授。 主要研究方向为多模态学习、可信人工智能

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419904823964583, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, xref=1, ext=[AuthorCompanyExt(id=1329419904836547496, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Systems Engineering, National University of Defense Technology, Changsha 410073), AuthorCompanyExt(id=1329419904853324713, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1国防科技大学系统工程学院 长沙 410073)])])], keywords=[Keyword(id=1329419906572989388, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, orderNo=1, keyword=adversarial robustness), Keyword(id=1329419906774315981, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, orderNo=2, keyword=lightweight model), Keyword(id=1329419906870784974, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, orderNo=3, keyword=knowledge distillation), Keyword(id=1329419906937893839, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, orderNo=4, keyword=adversarial defense), Keyword(id=1329419907005002704, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, orderNo=5, keyword=deep learning), Keyword(id=1329419907067917265, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, orderNo=1, keyword=对抗鲁棒性), Keyword(id=1329419907135026130, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, orderNo=2, keyword=模型轻量化), Keyword(id=1329419907256660947, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, orderNo=3, keyword=知识蒸馏), Keyword(id=1329419907365712852, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, orderNo=4, keyword=对抗防御), Keyword(id=1329419907441210325, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, orderNo=5, keyword=深度学习)], refs=[Reference(id=1329419910201062391, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=1, rfOrder=0, authorNames=null, journalName=null, refType=null, unstructuredReference=窦慧, 张凌茗, 韩峰, 等. 卷积神经网络的可解释性研究综述[J] 软件学报, 2024, 35(1): 159−184, articleTitle=null, refAbstract=null), Reference(id=1329419910280754168, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=1, rfOrder=1, authorNames=null, journalName=null, refType=null, unstructuredReference=Dou Hui, Zhang Lingming, Han Feng, et al. Survey on convolutional neural network interpretability[J] Journal of Software, 2024, 35(1): 159–184 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1329419910360445945, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=2, rfOrder=2, authorNames=null, journalName=null, refType=null, unstructuredReference=LeCun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553): 436−444, articleTitle=null, refAbstract=null), Reference(id=1329419910414971898, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=3, rfOrder=3, authorNames=null, journalName=null, refType=null, unstructuredReference=Rawat W, Wang Zenghui. Deep convolutional neural networks for image classification: A comprehensive review[J]. Neural Computation, 2017, 29(9): 2352−2449, articleTitle=null, refAbstract=null), Reference(id=1329419910473692155, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=4, rfOrder=4, authorNames=null, journalName=null, refType=null, unstructuredReference=Zou Zhengxia, Chen Keyan, Shi Zhenwei, et al. Object detection in 20 years: A Survey[J]. Journal of the IEEE, 2023, 111(3): 257−276, articleTitle=null, refAbstract=null), Reference(id=1329419910532412412, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=5, rfOrder=5, authorNames=null, journalName=null, refType=null, unstructuredReference=Chang Yupeng, Wang Xu, Wang Jindong, et al. A survey on evaluation of large language models[J]. ACM Transactions on Intelligent Systems and Technology, 2024, 15(3): 1−45, articleTitle=null, refAbstract=null), Reference(id=1329419910591132669, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=6, rfOrder=6, authorNames=null, journalName=null, refType=null, unstructuredReference=Alabdulmohsin I, Neyshabur B, Zhai Xiaohua. Revisiting neural scaling laws in language and vision[J]. Advances in Neural Information Processing Systems, 2022, 35: 22300−22312, articleTitle=null, refAbstract=null), Reference(id=1329419910675018750, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=7, rfOrder=7, authorNames=null, journalName=null, refType=null, unstructuredReference=Chakraborty A, Alam M, Dey V, et al. A survey on adversarial attacks and defences[J]. CAAI Transactions on Intelligence Technology, 2021, 6(1): 25−45, articleTitle=null, refAbstract=null), Reference(id=1329419910750516223, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=8, rfOrder=8, authorNames=null, journalName=null, refType=null, unstructuredReference=Ye Shaokai, Xu Kaidi, Liu Sijia, et al. Adversarial robustness vs. model compression, or both?[C/OL]//Proc of the 14th IEEE/CVF Int Conf on Computer Vision. 2019[2025-09-06]. https://openaccess.thecvf.com/content_ICCV_2019/html/Ye_Adversarial_Robustness_vs._Model_Compression_or_Both_ICCV_2019_paper.html, articleTitle=null, refAbstract=null), Reference(id=1329419910817625088, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=9, rfOrder=9, authorNames=null, journalName=null, refType=null, unstructuredReference=Qian Zhuang, Huang Kaizhu, Wang Qiufeng, et al. A survey of robust adversarial training in pattern recognition: Fundamental, theory, and methodologies[J]. Pattern Recognition, 2022, 113: 1−11, articleTitle=null, refAbstract=null), Reference(id=1329419910909898752, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=10, rfOrder=10, authorNames=null, journalName=null, refType=null, unstructuredReference=Pang Tianyu, Yang Xiao, Dong Yinpeng, et al. Bag of tricks for adversarial training[J]. arXiv preprint, arXiv: 2010.00467, 2020, articleTitle=null, refAbstract=null), Reference(id=1329419911010562049, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=11, rfOrder=11, authorNames=null, journalName=null, refType=null, unstructuredReference=Wu Boxi, Chen Jinghui, Cai Deng, et al. Do wider neural networks really help adversarial robustness?[C/OL]//Proc of the 35th Neural Information ProcessingSystems. 2021[2025-09-06]. https://proceedings.neurips.cc/paper/2021/file/3937230de3c8041e4da6ac3246a888e8-Paper.pdf, articleTitle=null, refAbstract=null), Reference(id=1329419911081865218, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=12, rfOrder=12, authorNames=null, journalName=null, refType=null, unstructuredReference=Madry A, Makelov A, Schmidt L, et al. Towards deep learning models resistant to adversarial attacks[J]. arXiv preprint, arXiv: 1706.06083, 2017, articleTitle=null, refAbstract=null), Reference(id=1329419911140585475, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=13, rfOrder=13, authorNames=null, journalName=null, refType=null, unstructuredReference=Micah G, Liam F, Soheil F, et al. Adversarially robust distillation[C/OL]//Proc of the 20th AAAI Conf on Artificial Intelligence. 2020[2025-09-06]. https://ojs.aaai.org/index.php/AAAI/issue/view/252, articleTitle=null, refAbstract=null), Reference(id=1329419911203500036, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=14, rfOrder=14, authorNames=null, journalName=null, refType=null, unstructuredReference=Gou Jianping, Yu Baosheng, Stephen J M, et al. Knowledge distillation: A survey[J]. International Journal of Computer Vision, 2021, 129(6): 1789−1819, articleTitle=null, refAbstract=null), Reference(id=1329419911270608901, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=15, rfOrder=15, authorNames=null, journalName=null, refType=null, unstructuredReference=Abbasi S, Hajabdollahi M, Karimi N, et al. Modeling teacher-student techniques in deep neural networks for knowledge distillation[C]//Proc of the 2nd Int Conf on Machine Vision and Image Processing (MVIP). Piscataway, NJ: IEEE, 2020: 1−6, articleTitle=null, refAbstract=null), Reference(id=1329419911325134854, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=16, rfOrder=16, authorNames=null, journalName=null, refType=null, unstructuredReference=Zi Bojia, Zhao Shihao, Ma Xingjun, et al. Revisiting adversarial robustness distillation: Robust soft labels make student better[C]//Proc of the 16th IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2021: 16443−16452, articleTitle=null, refAbstract=null), Reference(id=1329419911396438023, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=17, rfOrder=17, authorNames=null, journalName=null, refType=null, unstructuredReference=Maroto J, Ortiz-Jiménez G, Frossard G. On the benefits of knowledge distillation for adversarial robustness[J]. arXiv preprint, arXiv: 2203.07159, 2022, articleTitle=null, refAbstract=null), Reference(id=1329419911463546889, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=18, rfOrder=18, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhao Shiji, Duan Ranjie, Wang Xizhe, et al. Improving adversarial robust fairness via anti-bias soft label distillation[C/OL]//Proc of the 38th Int Conf on Neural Information Processing Systems. 2024[2025-09-06]. https://papers.nips.cc/paper_files/paper/2024/file/a2370db7c99791ad5d9f3ef48ad6d464-Paper-Conference.pdf, articleTitle=null, refAbstract=null), Reference(id=1329419911555821579, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=19, rfOrder=19, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhao Shiji, Yu Jie, Sun Zhenlong, et al. Enhanced accuracy and robustness via multi-teacher adversarial distillation[C/OL]//Proc of the 14th European Conf on Computer Vision. 2022[2025-09-06]. https://lin k.springer.com/chapter/10.1007/978-3-031-19772-7_34, articleTitle=null, refAbstract=null), Reference(id=1329419911618736140, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=20, rfOrder=20, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhu Jianing, Yao Jiangchao, Han Bom, et al. Reliable adversarial distillation with unreliable teachers[C/OL]//Proc of the 10th Int Conf on Learning Representations. 2022[2025-09-06]. https://openreview.net/ forum?id=u6TRGdzhfip&referrer=%5BAuthor%20Console%5D(%2Fgroup%3Fid%3DICLR.cc%2F2022%2FConference%2FAuthors%23your-submissions, articleTitle=null, refAbstract=null), Reference(id=1329419911731982349, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=21, rfOrder=21, authorNames=null, journalName=null, refType=null, unstructuredReference=Huang Bo, Chen Mingyang, Wang Yi, et al. Boosting accuracy and robustness of student models via adaptive adversarial distillation[C]//Proc of the 40th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE 2023: 24668−24677, articleTitle=null, refAbstract=null), Reference(id=1329419911799091214, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=22, rfOrder=22, authorNames=null, journalName=null, refType=null, unstructuredReference=Park Hyejin, Min Dongbo. Dynamic guidance adversarial distillation with enhanced teacher knowledge[C]//Proc of the 17th European Conf on Computer Vision. Berlin: Springer, 2025: 204−219, articleTitle=null, refAbstract=null), Reference(id=1329419911862005775, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=23, rfOrder=23, authorNames=null, journalName=null, refType=null, unstructuredReference=Erven V, Harremos T. Rényi divergence and Kullback-Leibler divergence[J]. IEEE Transactions on Information Theory, 2014(7): 3797−3820, articleTitle=null, refAbstract=null), Reference(id=1329419911945891856, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=24, rfOrder=24, authorNames=null, journalName=null, refType=null, unstructuredReference=Guo Chuan, Gardner Jacob, You Yurong, et al. Simple black-box adversarial attacks[C]//Proc of the 12th Int Conf on Machine Learning. New York: PMLR, 2019: 2484-2493, articleTitle=null, refAbstract=null), Reference(id=1329419912008806417, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=25, rfOrder=25, authorNames=null, journalName=null, refType=null, unstructuredReference=Goodfellow I J, Shlens J, Szegedy C. Explaining and harnessing adversarial examples[C/OL]//Proc of the 3rd In Conf on Learning Representations. 2015[2025-09-06]. https://www.researchgate.net/publication/269935591_Explaining_and_Harnessing_Adversarial_Examples, articleTitle=null, refAbstract=null), Reference(id=1329419912067526674, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=26, rfOrder=26, authorNames=null, journalName=null, refType=null, unstructuredReference=Madry A, Makelov A, Schmidt L, et al. Towards deep learning models resistant to adversarial attacks[C/OL]//Proc of the 6th In Conf on Learning Representations. 2018[2025-09-06]. https://www.padl.ws/papers/Paper%2019.pdf, articleTitle=null, refAbstract=null), Reference(id=1329419912126246931, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=27, rfOrder=27, authorNames=null, journalName=null, refType=null, unstructuredReference=Carlini L, Wagner D. Towards evaluating the robustness of neural networks[C]//Proc of the IEEE Symp on Security and Privacy. Piscataway, NJ: IEEE, 2017: 39−57, articleTitle=null, refAbstract=null), Reference(id=1329419912189161492, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=28, rfOrder=28, authorNames=null, journalName=null, refType=null, unstructuredReference=Croce F, Hein F. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks[C/OL]//Proc of the 13th Int Conf on Machine Learning. New York: PMLR, 2020: 2206−2216, articleTitle=null, refAbstract=null), Reference(id=1329419912256270357, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=29, rfOrder=29, authorNames=null, journalName=null, refType=null, unstructuredReference=隋晨红, 王奥, 周圣文, 等. 面向鲁棒学习的对抗训练技术综述[J]. 中国图象图形学报, 2023, 28(12): 3629−3650, articleTitle=null, refAbstract=null), Reference(id=1329419912323379222, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=29, rfOrder=30, authorNames=null, journalName=null, refType=null, unstructuredReference=Sui Chenhong, Wang Ao, Zhou Shengwen, et al. A survey on adversarial training for robust learning[J]. Journal of Image and Graphics, 2023, 28(12): 3629−3650 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1329419912407265303, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=30, rfOrder=31, authorNames=null, journalName=null, refType=null, unstructuredReference=陈思宏, 沈浩靖, 王冉, 等. 预测不确定性与对抗鲁棒性的关系研究[J]. 软件学报, 2022, 33(2): 524−538, articleTitle=null, refAbstract=null), Reference(id=1329419912491151384, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=30, rfOrder=32, authorNames=null, journalName=null, refType=null, unstructuredReference=Chen Sihong, Shen Haojing, Wang Ran, et al. Relationship between prediction uncertainty and adversarial robustness[J]. Journal of Software, 2022, 33(2): 524−538 (in Chinese), articleTitle=null, refAbstract=null), Reference(id=1329419912554065945, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=31, rfOrder=33, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhao Shiji, Wang Xizhe, Wei Xingxing. Mitigating accuracy-robustness trade-off via balanced multi-teacher adversarial distillation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(12): 1−14, articleTitle=null, refAbstract=null), Reference(id=1329419912612786202, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=32, rfOrder=34, authorNames=null, journalName=null, refType=null, unstructuredReference=Yin Shenglin, Xiao Zhen, Song Mingxuan, et al. Adversarial distillation based on slack matching and attribution region alignment[C]//Proc of the 41st IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE 2024: 24605−24614, articleTitle=null, refAbstract=null), Reference(id=1329419912671506459, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=33, rfOrder=35, authorNames=null, journalName=null, refType=null, unstructuredReference=Wu Yulun, Lao Mingrui, Guo Yanming, et al. Boosting adversarial robustness distillation via hybrid decomposed knowledge[C]//Proc of the 49th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2024: 4715−471, articleTitle=null, refAbstract=null), Reference(id=1329419912730226716, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=34, rfOrder=36, authorNames=null, journalName=null, refType=null, unstructuredReference=Long Teng, Gao Qi, Xu Lili, et al. A survey on adversarial attacks in computer vision: Taxonomy, visualization and future directions[J]. Computers & Security, 2022, 121: 102847, articleTitle=null, refAbstract=null), Reference(id=1329419912793141277, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=35, rfOrder=37, authorNames=null, journalName=null, refType=null, unstructuredReference=Krizhevsky A, Hinton G. Learning multiple layers of features from Tiny Images[J/OL]. Handbook of Systemic Autoimmune Diseases. 2009[2025-09-06]. https://www.semanticscholar.org/paper/ Learning-Multiple- Lay ers-of-Features-from-Tiny-Krizhevsky/5d90f06bb70a0a3dced62 413346235c02b1aa086, articleTitle=null, refAbstract=null), Reference(id=1329419912856055838, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=36, rfOrder=38, authorNames=null, journalName=null, refType=null, unstructuredReference=Benbrahim H, Behloul A. Fine-tuned Xception for image classification on Tiny ImageNet[C/OL]//Proc of the 1st Int Conf on Artificial Intelligence for Cyber Security Systems and Privacy (AI-CSP). 2021[2025-09-06]. https://www.researchgate.net/publication/3577550 47_Fine-tuned_Xception_for_Image_Classification_on_Tiny_ImageNet, articleTitle=null, refAbstract=null), Reference(id=1329419912923164703, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=37, rfOrder=39, authorNames=null, journalName=null, refType=null, unstructuredReference=Zhang Hongyang, Yu Yaodong, Jiao Jiantao, et al. Theoretically principled trade-off between robustness and accuracy[C]//Proc of the 12th Int Conf on Machine Learning. New York: PMLR, 2019: 7472−7482, articleTitle=null, refAbstract=null), Reference(id=1329419913002856480, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, doi=null, pmid=null, pmcid=null, year=null, volume=null, issue=null, pageStart=null, pageEnd=null, url=null, language=null, rfNumber=38, rfOrder=40, authorNames=null, journalName=null, refType=null, unstructuredReference=Croce F, Andriushchenko M, Sehwag V, et al. RobustBench: A standardized adversarial robustness benchmark[J]. arXiv preprint, arXiv: 2010.09670, 2020, articleTitle=null, refAbstract=null)], funds=null, companyList=[AuthorCompany(id=1329419904823964583, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, xref=1, ext=[AuthorCompanyExt(id=1329419904836547496, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Systems Engineering, National University of Defense Technology, Changsha 410073), AuthorCompanyExt(id=1329419904853324713, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1国防科技大学系统工程学院 长沙 410073)]), AuthorCompany(id=1329419904945599402, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, xref=2, ext=[AuthorCompanyExt(id=1329419904953988011, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904945599402, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2College of Intelligence and Computing, Tianjing University, Tianjin 300072), AuthorCompanyExt(id=1329419904966570924, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904945599402, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2天津大学智能与计算学部 天津 300072)])], figs=[ArticleFig(id=1329419907575428054, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, label=Figure 1, caption=Illustration of priority ranking constraint mechanism, figureFileSmall=VsjzFaioCvfRVsFIaakqAg==, figureFileBig=NAJKB9Gq+wGQLdK/GunxLA==, tableContent=null), ArticleFig(id=1329419907663508439, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, label=图1, caption=优先级排序约束机制示意图, figureFileSmall=VsjzFaioCvfRVsFIaakqAg==, figureFileBig=NAJKB9Gq+wGQLdK/GunxLA==, tableContent=null), ArticleFig(id=1329419907890000856, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, label=Figure 2, caption=Illustration of multi-level ranking consistency distillation framework, figureFileSmall=dXNVh7FBnKixsh84NF+AgQ==, figureFileBig=V7dvGiE3Qcqq9wJbugwBkg==, tableContent=null), ArticleFig(id=1329419907961304025, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, label=图2, caption=多层次排序一致性蒸馏框架示意图, figureFileSmall=dXNVh7FBnKixsh84NF+AgQ==, figureFileBig=V7dvGiE3Qcqq9wJbugwBkg==, tableContent=null), ArticleFig(id=1329419908028412890, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, label=Figure 3, caption=Accuracy without attack of ResNet-18 student model guided by different teacher models, figureFileSmall=ZjdR8ZQtgt9WfAYqtknE2Q==, figureFileBig=wK+KuU4E4qORdzUMe6gV3Q==, tableContent=null), ArticleFig(id=1329419908099716059, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, label=图3, caption=ResNet-18学生模型在不同教师模型引导下的无攻击准确率, figureFileSmall=ZjdR8ZQtgt9WfAYqtknE2Q==, figureFileBig=wK+KuU4E4qORdzUMe6gV3Q==, tableContent=null), ArticleFig(id=1329419908166824924, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, label=Figure 4, caption=Accuracy without attack of MobileNet-v2 student model guided by different teacher models, figureFileSmall=1mm3bVZ+dH0Wlpa1GADsaQ==, figureFileBig=a5XqAePS8wpe0Lj7dSsHTg==, tableContent=null), ArticleFig(id=1329419908229739485, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, label=图4, caption=MobileNet-v2学生模型在不同教师模型引导下的无攻击准确率, figureFileSmall=1mm3bVZ+dH0Wlpa1GADsaQ==, figureFileBig=a5XqAePS8wpe0Lj7dSsHTg==, tableContent=null), ArticleFig(id=1329419908305236958, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, label=Figure 5, caption=Accuracy of ResNet-18 student model under different values of $ \lambda $ and $ \mu $, figureFileSmall=FoCq6uSxLzoHAvwTl56HQA==, figureFileBig=Mdb1CXD7LtSg3wea0O/Dxg==, tableContent=null), ArticleFig(id=1329419908376540127, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, label=图5, caption=ResNet-18学生模型在不同$ \lambda $,$ \mu $取值下的准确率, figureFileSmall=FoCq6uSxLzoHAvwTl56HQA==, figureFileBig=Mdb1CXD7LtSg3wea0O/Dxg==, tableContent=null), ArticleFig(id=1329419908464620512, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, label=Figure 6, caption=Accuracy of MobileNet-v2 student model under different values of λ and μ, figureFileSmall=y1+G2ZfYvOdCFkL1Q6+h1A==, figureFileBig=p4CJe9NuZBA/GF8ba51mpQ==, tableContent=null), ArticleFig(id=1329419908548506593, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, label=图6, caption=MobileNet-v2学生模型在不同$ \lambda ,\mu $取值下的准确率, figureFileSmall=y1+G2ZfYvOdCFkL1Q6+h1A==, figureFileBig=p4CJe9NuZBA/GF8ba51mpQ==, tableContent=null), ArticleFig(id=1329419908611421154, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, label=Figure 7, caption=Method recognition accuracy and adversarial robustness based on ResNet-18 student model in low resource scenarios, figureFileSmall=auCaIWR7kjOPuGu38hPRkw==, figureFileBig=pKVsl22IBF47RGtfgJlH8w==, tableContent=null), ArticleFig(id=1329419908686918627, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, label=图7, caption=低资源场景下基于 ResNet-18学生模型的方法识别准确率与对抗鲁棒性, figureFileSmall=auCaIWR7kjOPuGu38hPRkw==, figureFileBig=pKVsl22IBF47RGtfgJlH8w==, tableContent=null), ArticleFig(id=1329419908749833188, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, label=Figure 8, caption=Method recognition accuracy and adversarial robustness based on MobileNet-v2 student model in low resource scenarios, figureFileSmall=b8KDAwSeMJqmP+M1FS353A==, figureFileBig=lmmJXosNvquAOuLKxaa5sw==, tableContent=null), ArticleFig(id=1329419908833719269, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, label=图8, caption=低资源场景下基于 MobileNet-v2学生模型的方法识别准确率与对抗鲁棒性, figureFileSmall=b8KDAwSeMJqmP+M1FS353A==, figureFileBig=lmmJXosNvquAOuLKxaa5sw==, tableContent=null), ArticleFig(id=1329419908913411046, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, label=Figure 9, caption=Case study: Loss function and adversarial robustness variation curves during training for AdaAD and our method, figureFileSmall=YnkDWZpQ4tdixmXh+I1b7w==, figureFileBig=80yhQmEU5DgzJRCdCnP4gg==, tableContent=null), ArticleFig(id=1329419908976325607, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, label=图9, caption=案例分析:AdaAD与本文方法在训练过程中损失函数和对抗鲁棒性变化曲线, figureFileSmall=YnkDWZpQ4tdixmXh+I1b7w==, figureFileBig=80yhQmEU5DgzJRCdCnP4gg==, tableContent=null), ArticleFig(id=1329419909051823080, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, label=Table 1, caption=

Summary and Comparison of Adversarial Distillation Relative Methods

, figureFileSmall=null, figureFileBig=null, tableContent=
方法年份蒸馏改进策略蒸馏知识类型蒸馏方式
标签
改进
教师
选择
训练
优化
类别
预测
样本
关联
对抗
差异
KL
散度
排序
约束
文献[13]2020✔
文献[16]2021✔✔✔
文献[17]2022✔✔✔
文献[18]2024✔✔✔
文献[19]2022✔✔
文献[20]2022✔✔✔
文献[21]2023✔✔✔
文献[22]2025✔✔✔✔
文献[31]2024✔✔✔✔
文献[32]2024✔✔✔✔
文献[33]2024✔✔✔✔
本文✔✔✔✔✔
), ArticleFig(id=1329419909131514857, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, label=表1, caption=

对抗蒸馏相关方法总结与对比

, figureFileSmall=null, figureFileBig=null, tableContent=
方法年份蒸馏改进策略蒸馏知识类型蒸馏方式
标签
改进
教师
选择
训练
优化
类别
预测
样本
关联
对抗
差异
KL
散度
排序
约束
文献[13]2020✔
文献[16]2021✔✔✔
文献[17]2022✔✔✔
文献[18]2024✔✔✔
文献[19]2022✔✔
文献[20]2022✔✔✔
文献[21]2023✔✔✔
文献[22]2025✔✔✔✔
文献[31]2024✔✔✔✔
文献[32]2024✔✔✔✔
文献[33]2024✔✔✔✔
本文✔✔✔✔✔
), ArticleFig(id=1329419909223789546, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, label=Table 2, caption=

Method Recognition Accuracy and Adversarial Robustness Based on WideResNet-34-10 Teacher Model in CIFAR-10 Dataset %

, figureFileSmall=null, figureFileBig=null, tableContent=
学生模型方法识别准确率对抗鲁棒性
Clean(best)Clean(last)FGSM(best)FGSM(last)PGD(best)PGD(last)CW(best)CW(last)AA(best)AA(last)
ResNet-18PGD-AT82.9557.1752.8777.5647.69
TRADES83.0058.4253.1876.9249.21
ARD83.1382.9757.9457.3253.5552.9178.6378.5548.7748.32
IAD83.5083.1860.9760.8555.7155.3279.4479.3151.0150.72
RSLAD83.7883.5161.1360.7856.4556.3279.5679.3951.7151.34
MTARD84.5184.3259.8359.3154.5254.3979.2179.1950.5350.11
FAIR-ARD83.5183.4257.9857.7153.8553.7778.5078.3148.5148.50
AdaAD84.5484.1560.1459.8356.1255.9680.3880.1851.5051.34
DGAD84.7384.5160.9860.7757.3557.3180.5180.5851.8351.92
本文84.8284.7162.3362.1458.5958.4980.8580.8353.7853.72
MobileNet-v2PGD-AT77.5453.5849.9072.5444.56
TRADES79.8054.8450.5175.3145.67
ARD80.5480.1555.1654.9150.9750.8176.9376.6146.5746.51
IAD81.3481.0056.0155.7152.2151.9877.3377.2147.9347.85
RSLAD82.4481.9756.1355.9852.9152.7777.5177.5048.3148.32
MTARD83.1382.8755.8355.5151.1550.9777.2977.1647.8147.70
FAIR-ARD81.3381.2755.8355.7151.4551.3976.8176.8546.8846.81
AdaAD82.8082.5656.8156.7553.1652.8878.3178.2948.0447.51
DGAD83.8783.5858.3358.2155.2955.2879.3579.0350.1750.06
本文84.4584.2261.2060.8856.9456.8780.3980.0952.0551.84
), ArticleFig(id=1329419909303481323, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, label=表2, caption=

CIFAR-10数据集下基于WideResNet-34-10教师模型的方法识别准确率与对抗鲁棒性

, figureFileSmall=null, figureFileBig=null, tableContent=
学生模型方法识别准确率对抗鲁棒性
Clean(best)Clean(last)FGSM(best)FGSM(last)PGD(best)PGD(last)CW(best)CW(last)AA(best)AA(last)
ResNet-18PGD-AT82.9557.1752.8777.5647.69
TRADES83.0058.4253.1876.9249.21
ARD83.1382.9757.9457.3253.5552.9178.6378.5548.7748.32
IAD83.5083.1860.9760.8555.7155.3279.4479.3151.0150.72
RSLAD83.7883.5161.1360.7856.4556.3279.5679.3951.7151.34
MTARD84.5184.3259.8359.3154.5254.3979.2179.1950.5350.11
FAIR-ARD83.5183.4257.9857.7153.8553.7778.5078.3148.5148.50
AdaAD84.5484.1560.1459.8356.1255.9680.3880.1851.5051.34
DGAD84.7384.5160.9860.7757.3557.3180.5180.5851.8351.92
本文84.8284.7162.3362.1458.5958.4980.8580.8353.7853.72
MobileNet-v2PGD-AT77.5453.5849.9072.5444.56
TRADES79.8054.8450.5175.3145.67
ARD80.5480.1555.1654.9150.9750.8176.9376.6146.5746.51
IAD81.3481.0056.0155.7152.2151.9877.3377.2147.9347.85
RSLAD82.4481.9756.1355.9852.9152.7777.5177.5048.3148.32
MTARD83.1382.8755.8355.5151.1550.9777.2977.1647.8147.70
FAIR-ARD81.3381.2755.8355.7151.4551.3976.8176.8546.8846.81
AdaAD82.8082.5656.8156.7553.1652.8878.3178.2948.0447.51
DGAD83.8783.5858.3358.2155.2955.2879.3579.0350.1750.06
本文84.4584.2261.2060.8856.9456.8780.3980.0952.0551.84
), ArticleFig(id=1329419909404144620, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, label=Table 3, caption=

Method Recognition Accuracy and Adversarial Robustness Based on WideResNet-34-10 Teacher Model in CIFAR-100 Dataset

, figureFileSmall=null, figureFileBig=null, tableContent=
方法ResNet-18MobileNet-v2
CleanFGSMPGDCWAACleanFGSMPGDCWAA
PGD-AT56.2732.0829.8449.0524.9951.5529.3427.2645.7322.07
TRADES57.8232.5230.3851.3025.0253.0529.0727.4447.6221.82
ARD60.9435.3132.7253.6726.0457.1833.1330.9151.5024.20
IAD60.4335.7532.8052.7126.8456.3332.8830.1849.0024.07
RSLAD59.5535.6833.3552.8927.7756.0432.7630.2950.1424.56
AdaAD62.1935.3332.5254.6726.7461.4434.7531.9754.2125.91
DGAD63.2436.0933.6855.4727.6662.2534.9032.6454.5426.56
本文63.5839.2135.5855.6328.3261.9836.4533.7154.4626.83
), ArticleFig(id=1329419909488030701, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, label=表3, caption=

CIFAR-100数据集下基于WideResNet-34-10教师模型的方法识别准确率与对抗鲁棒性

, figureFileSmall=null, figureFileBig=null, tableContent=
方法ResNet-18MobileNet-v2
CleanFGSMPGDCWAACleanFGSMPGDCWAA
PGD-AT56.2732.0829.8449.0524.9951.5529.3427.2645.7322.07
TRADES57.8232.5230.3851.3025.0253.0529.0727.4447.6221.82
ARD60.9435.3132.7253.6726.0457.1833.1330.9151.5024.20
IAD60.4335.7532.8052.7126.8456.3332.8830.1849.0024.07
RSLAD59.5535.6833.3552.8927.7756.0432.7630.2950.1424.56
AdaAD62.1935.3332.5254.6726.7461.4434.7531.9754.2125.91
DGAD63.2436.0933.6855.4727.6662.2534.9032.6454.5426.56
本文63.5839.2135.5855.6328.3261.9836.4533.7154.4626.83
), ArticleFig(id=1329419909555139566, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, label=Table 4, caption=

Method Recognition Accuracy and Adversarial Robustness Based on WideResNet-34-10 Teacher Models in TinyImageNet Dataset

, figureFileSmall=null, figureFileBig=null, tableContent=
方法ResNet-18MobileNet-v2
CleanFGSMPGDCWAACleanFGSMPGDCWAA
ARD41.6624.4723.3037.7617.2338.7921.5122.1734.7814.74
IAD44.5723.9722.1537.5417.5840.1521.9823.1835.8815.98
RSLAD43.6223.4522.5837.1217.0541.3722.1222.9535.5316.17
AdaAD47.5424.2222.8242.7917.4143.6022.4523.1336.5716.44
DGAD47.9224.4223.0542.8117.1845.2723.1923.2536.9116.87
本文48.1524.9824.0342.8318.5745.1923.77 23.53 37.71 17.11
), ArticleFig(id=1329419909626442735, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, label=表4, caption=

TinyImageNet数据集下基于WideResNet-34-10教师模型的方法识别准确率与对抗鲁棒性

, figureFileSmall=null, figureFileBig=null, tableContent=
方法ResNet-18MobileNet-v2
CleanFGSMPGDCWAACleanFGSMPGDCWAA
ARD41.6624.4723.3037.7617.2338.7921.5122.1734.7814.74
IAD44.5723.9722.1537.5417.5840.1521.9823.1835.8815.98
RSLAD43.6223.4522.5837.1217.0541.3722.1222.9535.5316.17
AdaAD47.5424.2222.8242.7917.4143.6022.4523.1336.5716.44
DGAD47.9224.4223.0542.8117.1845.2723.1923.2536.9116.87
本文48.1524.9824.0342.8318.5745.1923.77 23.53 37.71 17.11
), ArticleFig(id=1329419909697745904, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, label=Table 5, caption=

Ablation Study Based on Adversarial Ranking Distillation

, figureFileSmall=null, figureFileBig=null, tableContent=
案例方法组件ResNet-18MobileNet-v2
KLSPCRDIRDDRDCleanFGSMPGDAACleanFGSMPGDAA
1✔$ ({T}={1}) $84.4459.9455.9751.1282.8056.8153.1648.04
2✔$ ({T}={2}) $84.2060.1056.1351.1782.6556.9553.4548.22
3✔$ ({T}={3}) $84.0159.7555.6350.9882.6256.6452.9947.93
4✔$ ({T}={1}) $✔84.5559.8755.7951.0582.5456.4952.7847.82
5✔84.7662.0958.0052.9983.9360.0355.5750.81
6✔✔84.9462.4258.2053.3084.6860.8856.3651.09
7✔✔84.8861.9757.9253.0184.0060.1255.6150.83
8✔✔84.5862.1758.3353.5183.9661.0856.5851.53
9✔✔✔84.9062.5358.4953.7284.4561.2056.9552.05
), ArticleFig(id=1329419909777437681, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, label=表5, caption=

基于对抗排序蒸馏的消融实验

, figureFileSmall=null, figureFileBig=null, tableContent=
案例方法组件ResNet-18MobileNet-v2
KLSPCRDIRDDRDCleanFGSMPGDAACleanFGSMPGDAA
1✔$ ({T}={1}) $84.4459.9455.9751.1282.8056.8153.1648.04
2✔$ ({T}={2}) $84.2060.1056.1351.1782.6556.9553.4548.22
3✔$ ({T}={3}) $84.0159.7555.6350.9882.6256.6452.9947.93
4✔$ ({T}={1}) $✔84.5559.8755.7951.0582.5456.4952.7847.82
5✔84.7662.0958.0052.9983.9360.0355.5750.81
6✔✔84.9462.4258.2053.3084.6860.8856.3651.09
7✔✔84.8861.9757.9253.0184.0060.1255.6150.83
8✔✔84.5862.1758.3353.5183.9661.0856.5851.53
9✔✔✔84.9062.5358.4953.7284.4561.2056.9552.05
), ArticleFig(id=1329419909844546546, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, label=Table 6, caption=

Method Recognition Accuracy and Adversarial Robustness in Adversarial Self-Distillation Scenario

, figureFileSmall=null, figureFileBig=null, tableContent=
方法ResNet-18MobileNet-v2
CleanFGSMPGDCWAACleanFGSMPGDCWAA
ARD80.6655.6850.9074.8746.6179.4253.1850.9073.1444.76
IAD81.3257.5452.9175.6948.2079.8154.8351.2873.5844.98
RSLAD81.9257.9453.2976.2649.0679.7854.6751.4873.4945.51
AdaAD83.1357.5453.3077.6249.6181.9055.8651.8375.8346.92
DGAD83.2658.9154.3777.9050.5482.7556.2352.1576.1746.73
本文84.0159.3355.8378.1251.6883.2458.0753.8777.2348.59
), ArticleFig(id=1329419909915849715, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, label=表6, caption=

对抗自蒸馏场景下的方法识别准确率与对抗鲁棒性

, figureFileSmall=null, figureFileBig=null, tableContent=
方法ResNet-18MobileNet-v2
CleanFGSMPGDCWAACleanFGSMPGDCWAA
ARD80.6655.6850.9074.8746.6179.4253.1850.9073.1444.76
IAD81.3257.5452.9175.6948.2079.8154.8351.2873.5844.98
RSLAD81.9257.9453.2976.2649.0679.7854.6751.4873.4945.51
AdaAD83.1357.5453.3077.6249.6181.9055.8651.8375.8346.92
DGAD83.2658.9154.3777.9050.5482.7556.2352.1576.1746.73
本文84.0159.3355.8378.1251.6883.2458.0753.8777.2348.59
), ArticleFig(id=1329419910020707316, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=EN, label=Table 7, caption=

Method Recognition Accuracy and Adversarial Robustness in Black-Box Scenario

, figureFileSmall=null, figureFileBig=null, tableContent=
黑箱模型性能指标ResNet-18MobileNet-v2
ARDIADAdaADDGAD本文ARDIADAdaADDGAD本文
ResNet34FGSM66.6366.9767.3168.1969.0664.1363.8765.4466.0167.30
PGD64.7564.5765.3365.7466.4863.7563.8164.5465.2166.17
JSMA86.2986.1387.3588.5489.3384.2385.1386.9787.1587.92

VGG16
FGSM67.2467.3168.9469.3370.4565.3765.4166.0167.5468.37
PGD66.0766.5868.1368.5470.0864.3164.3465.1966.1167.23
JSMA87.1586.9788.2289.5789.9384.9985.1486.5887.2988.10
), ArticleFig(id=1329419910096204790, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, language=CN, label=表7, caption=

黑盒场景下的方法识别准确率与对抗鲁棒

, figureFileSmall=null, figureFileBig=null, tableContent=
黑箱模型性能指标ResNet-18MobileNet-v2
ARDIADAdaADDGAD本文ARDIADAdaADDGAD本文
ResNet34FGSM66.6366.9767.3168.1969.0664.1363.8765.4466.0167.30
PGD64.7564.5765.3365.7466.4863.7563.8164.5465.2166.17
JSMA86.2986.1387.3588.5489.3384.2385.1386.9787.1587.92

VGG16
FGSM67.2467.3168.9469.3370.4565.3765.4166.0167.5468.37
PGD66.0766.5868.1368.5470.0864.3164.3465.1966.1167.23
JSMA87.1586.9788.2289.5789.9384.9985.1486.5887.2988.10
)], attaches=null, journal=Journal(id=1324759606947315908, delFlag=0, nameCn=计算机研究与发展, nameEn=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, issn=1000-1239, eissn=null, cn=11-1777/TP, coden=null, periodic=0, language=CN, oaType=null, ccby=null, superviseOffice=null, ownerOffice=null, pubOffice=null, editorOffice=null, officeType=null, aims=null, clcCode=null, officeProv=null, officeCity=null, officeAddr=null, officeZip=null, officeEmail=null, officePhone=null, editDirector=null, officeDirector=null, officeDirectorPhone=null, officeStaffNum=null, officeEmpNum=null, coverPicUrl=k2l8zgqwstuxxevFX2qxVA==, journalPrice=null, startedYear=null, abbrevIsoEn=Journal of Computer Research and Development, journalRemark=null, publicationField=null, createdTime=1793776903141, updatedTime=1793850698612, createdBy=18614031015, updatedBy=13701087609, firstLetterCn=J, firstLetterEn=J, subjectCode=Electronic Information, subjectName=null, subjectCodeEn=Electronic Information, subjectNameEn=null, picCn=k2l8zgqwstuxxevFX2qxVA==, picEn=r6k0Ry+RD+c/VXj5QarmDw==, jcr=null, cjcr=null, exts=[JournalExt(id=1325069127710286688, language=CN, name=计算机研究与发展, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698640, updatedTime=1793850698640, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""}), JournalExt(id=1325069127777395553, language=EN, name=Journal of Computer Research and Development, nameHistory1=null, nameHistory2=null, managedBy=, sponsoredBy=, publishedBy=, editorOffice=, officeProv=null, officeCity=null, officeAddr=, officeZip=, editDirector=, officeDirector=null, officePhone=null, coverPicUrl=null, journalRemark=, submitArticleUrl=null, websiteUrl=, createdTime=1793850698656, updatedTime=1793850698656, createdBy=13701087609, updatedBy=13701087609, submissionGuidelinesUrl=, submissionAuthorUrl=https://crad.ict.ac.cn/journalx/authorLogOn.action, submissionEditorUrl=https://crad.ict.ac.cn/journalx/editorLogOn.action, submissionReviewUrl=https://crad.ict.ac.cn/journalx/expertLogOn.action, submissionCeEditorUrl=https://crad.ict.ac.cn/journalx/editorInChiefLogOn.action, submissionAeEditorUrl=, option={"copyright":""})], databaseList=null, tenantJournalId=1324759914146529478, websiteList=[Website(id=1325070300139892771, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN, language=CN, createTime=1793850978169, createBy=18614031015, updateTime=1793851040259, updateBy=18614031015, name=计算机研究与发展-中文, tplId=1146099689490845704, title=计算机研究与发展, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070893751349360, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=articleTextType, value=kx, createTime=1793851119697, updateTime=1793851119697, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893730377837, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=banner, value=null, createTime=1793851119692, updateTime=1793851119692, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893772320883, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=grayFlag, value=0, createTime=1793851119702, updateTime=1793851119702, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893721989228, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851119690, updateTime=1793851119690, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893784903797, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=minRunFlag, value=0, createTime=1793851119705, updateTime=1793851119705, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893742960751, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/file/pic, createTime=1793851119695, updateTime=1793851119695, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893780709492, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=silenceFlag, value=0, createTime=1793851119704, updateTime=1793851119704, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893738766446, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_cn_619/, createTime=1793851119694, updateTime=1793851119694, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893755543665, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeColor, value=null, createTime=1793851119698, updateTime=1793851119698, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070893763932274, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300139892771, code=themeStyle, value=null, createTime=1793851119700, updateTime=1793851119700, creator=18614031015, updator=18614031015)]), Website(id=1325070300580294698, webName=null, webTitle=null, webDomain=null, webCopyrigh=null, webIpcNo=null, seoTitle=null, seoKeywords=null, seoDescription=null, tenantJournalId=null, journalId=1324759914146529478, journalNameCn=null, journalNameEn=null, grayFlag=null, tenantId=1146029695717560320, platformId=null, journalGroupId=null, journalGroupNameCn=null, journalGroupNameEn=null, type=1, domain=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN, language=EN, createTime=1793850978274, createBy=18614031015, updateTime=1793851036638, updateBy=18614031015, name=计算机研究与发展-英文, tplId=1146101810881728533, title=Journal of Computer Research and Development, delFlag=0, indexPage=/home, props=[WebsiteProps(id=1325070925288321148, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=articleTextType, value=kx, createTime=1793851127216, updateTime=1793851127216, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925263155321, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=banner, value=null, createTime=1793851127210, updateTime=1793851127210, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925309292671, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=grayFlag, value=0, createTime=1793851127221, updateTime=1793851127221, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925254766712, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=logo, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic?fileId=U27tDsy717YF7XSPA+Vwdw==, createTime=1793851127208, updateTime=1793851127208, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925326069889, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=minRunFlag, value=0, createTime=1793851127225, updateTime=1793851127225, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925279932539, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=picServerUrl, value=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/file/pic, createTime=1793851127214, updateTime=1793851127214, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925317681280, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=silenceFlag, value=0, createTime=1793851127223, updateTime=1793851127223, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925271543930, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=staticResourcePath, value=https://castjournals.cast.org.cn/joweb/cast_kjdb_en_623/, createTime=1793851127212, updateTime=1793851127212, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925292515453, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeColor, value=null, createTime=1793851127217, updateTime=1793851127217, creator=18614031015, updator=18614031015), WebsiteProps(id=1325070925300904062, tenantId=1146029695717560320, journalId=null, journalGroupId=null, siteId=1325070300580294698, code=themeStyle, value=null, createTime=1793851127219, updateTime=1793851127219, creator=18614031015, updator=18614031015)])], journalTitle=计算机研究与发展, weixinUrl=null, journalUrl=https://crad.ict.ac.cn/, iacademicId=null, status=1, seqNo=null, journalTitleEn=Journal of Computer Research and Development, journalPhotoCn=k2l8zgqwstuxxevFX2qxVA==, journalPhotoEn=r6k0Ry+RD+c/VXj5QarmDw==, journalFirstLetter=J, journalRecommend=null, journalNew=null, journalCollection=null, jcrJf=null, cjcrJf=null, jcrJfStr=null, cjcrJfStr=null, submissionFirstDecision=null, sciSubjectClassification=null, casSubjectClassification=null, citeScore=null, totalCitationFrequency=null, icpCode=null, psCode=null, advertisingLicenseCode=null, copyrightInformation=null, country=null, option=, provinceCode=null, provinceName=null, collectFlag=false, interPubPlatform=, interPubPlatformUrl=null), detailUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202550339, detailUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202550339, pdfUrlCn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/PDF/10.7544/issn1000-1239.202550339, pdfUrlEn=https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/PDF/10.7544/issn1000-1239.202550339, aliStartDate=0, aliEndDate=0, collectionFlag=false, citedCount=null, citedUrl=null, previewStatus=0, delFlag=0, hasFullText=1, orderTime=1780243200000, fullTextJson=null, articleText=null, reference=null)
收藏切换
提高轻量级模型鲁棒性的对抗排序蒸馏方法
老明瑞 1 , 李正 1 , 丁兆云 1 , 张雪毅 1 , 岳祥虎 2 , 郭延明 1
计算机研究与发展 | 人工智能 2026,63(6): 1506-1521
收起
收藏切换
计算机研究与发展 |人工智能 2026 , 63 (6) : 1506 -1521
提高轻量级模型鲁棒性的对抗排序蒸馏方法
全屏
[Author(id=1329419905054651310, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, orderNo=0, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=laomingrui17@nudt.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419905146926000, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419905054651310, language=EN, stringName=Mingrui Lao, firstName=Mingrui, middleName=null, lastName=Lao, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Systems Engineering, National University of Defense Technology, Changsha 410073, bio={"content":"

born in 1995. PhD, associate professor. His main research interests include trustworthy AI and multimodal learning

"}, bioImg=null, bioContent=

born in 1995. PhD, associate professor. His main research interests include trustworthy AI and multimodal learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419905230812081, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419905054651310, language=CN, stringName=老明瑞, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1国防科技大学系统工程学院 长沙 410073, bio={"img":"NKSZt3BLLpVUkqaBxgXhmA==","content":"

1995年生。博士,副教授。主要研究方向为可信人工智能、多模态学习

"}, bioImg=NKSZt3BLLpVUkqaBxgXhmA==, bioContent=

1995年生。博士,副教授。主要研究方向为可信人工智能、多模态学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419904823964583, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, xref=1, ext=[AuthorCompanyExt(id=1329419904836547496, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Systems Engineering, National University of Defense Technology, Changsha 410073), AuthorCompanyExt(id=1329419904853324713, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1国防科技大学系统工程学院 长沙 410073)])]), Author(id=1329419905335669683, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, orderNo=1, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419905436332981, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419905335669683, language=EN, stringName=Zheng Li, firstName=Zheng, middleName=null, lastName=Li, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Systems Engineering, National University of Defense Technology, Changsha 410073, bio={"content":"

born in 2000. PhD candidate. His main research interests include adversarial training and multimodal learning

"}, bioImg=null, bioContent=

born in 2000. PhD candidate. His main research interests include adversarial training and multimodal learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419905511830454, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419905335669683, language=CN, stringName=李正, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1国防科技大学系统工程学院 长沙 410073, bio={"img":"+QZmEvtx0dM5hNkZVIyrFA==","content":"

2000年生。博士研究生。主要研究方向为对抗训练、多模态学习

"}, bioImg=+QZmEvtx0dM5hNkZVIyrFA==, bioContent=

2000年生。博士研究生。主要研究方向为对抗训练、多模态学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419904823964583, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, xref=1, ext=[AuthorCompanyExt(id=1329419904836547496, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Systems Engineering, National University of Defense Technology, Changsha 410073), AuthorCompanyExt(id=1329419904853324713, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1国防科技大学系统工程学院 长沙 410073)])]), Author(id=1329419905574745016, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, orderNo=2, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419905641853882, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419905574745016, language=EN, stringName=Zhaoyun Ding, firstName=Zhaoyun, middleName=null, lastName=Ding, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Systems Engineering, National University of Defense Technology, Changsha 410073, bio={"content":"

born in 1982. PhD, professor. His main research interests include data mining and trustworthy AI

"}, bioImg=null, bioContent=

born in 1982. PhD, professor. His main research interests include data mining and trustworthy AI

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419905704768443, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419905574745016, language=CN, stringName=丁兆云, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1国防科技大学系统工程学院 长沙 410073, bio={"img":"owAZcyFPqPRuucdQCnLPUQ==","content":"

1982年生。博士,教授。 主要研究方向为数据挖掘、可信人工智能

"}, bioImg=owAZcyFPqPRuucdQCnLPUQ==, bioContent=

1982年生。博士,教授。 主要研究方向为数据挖掘、可信人工智能

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419904823964583, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, xref=1, ext=[AuthorCompanyExt(id=1329419904836547496, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Systems Engineering, National University of Defense Technology, Changsha 410073), AuthorCompanyExt(id=1329419904853324713, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1国防科技大学系统工程学院 长沙 410073)])]), Author(id=1329419905792848829, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, orderNo=3, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=null, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419905880929216, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419905792848829, language=EN, stringName=Xueyi Zhang, firstName=Xueyi, middleName=null, lastName=Zhang, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Systems Engineering, National University of Defense Technology, Changsha 410073, bio={"content":"

born in 1998. PhD candidate. His main research interests include adversarial training and continual learning

"}, bioImg=null, bioContent=

born in 1998. PhD candidate. His main research interests include adversarial training and continual learning

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419905969009601, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419905792848829, language=CN, stringName=张雪毅, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1国防科技大学系统工程学院 长沙 410073, bio={"img":"D9/PkoJyIt9Z+TvRvY0o6g==","content":"

1998年生。博士研究生。 主要研究方向为对抗训练、持续学习

"}, bioImg=D9/PkoJyIt9Z+TvRvY0o6g==, bioContent=

1998年生。博士研究生。 主要研究方向为对抗训练、持续学习

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419904823964583, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, xref=1, ext=[AuthorCompanyExt(id=1329419904836547496, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Systems Engineering, National University of Defense Technology, Changsha 410073), AuthorCompanyExt(id=1329419904853324713, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1国防科技大学系统工程学院 长沙 410073)])]), Author(id=1329419906040312771, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, orderNo=4, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=yuexianghubit@163.com, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419906115810245, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419906040312771, language=EN, stringName=Xianghu Yue, firstName=Xianghu, middleName=null, lastName=Yue, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2College of Intelligence and Computing, Tianjing University, Tianjin 300072, bio={"content":"

born in 1994. PhD, associate professor. His main research interests include multimodal learning and trustworthy AI

"}, bioImg=null, bioContent=

born in 1994. PhD, associate professor. His main research interests include multimodal learning and trustworthy AI

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419906191307718, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419906040312771, language=CN, stringName=岳祥虎, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=2, address=2天津大学智能与计算学部 天津 300072, bio={"img":"vA5u8Poo9lisdmBR7i23LA==","content":"

1994年生。 博士,副教授。主要研究方向为多模态学习、可信人工智能

"}, bioImg=vA5u8Poo9lisdmBR7i23LA==, bioContent=

1994年生。 博士,副教授。主要研究方向为多模态学习、可信人工智能

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419904945599402, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, xref=2, ext=[AuthorCompanyExt(id=1329419904953988011, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904945599402, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2College of Intelligence and Computing, Tianjing University, Tianjin 300072), AuthorCompanyExt(id=1329419904966570924, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904945599402, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=2天津大学智能与计算学部 天津 300072)])]), Author(id=1329419906258416584, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, orderNo=5, firstName=null, middleName=null, lastName=null, nameCn=null, orcid=null, stid=null, country=null, authorPic=null, dead=0, email=guoyanming@nudt.edu.cn, emailSecond=null, emailThird=null, correspondingAuthor=0, authorType=1, ext={EN=AuthorExt(id=1329419906367468490, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419906258416584, language=EN, stringName=Yanming Guo, firstName=Yanming, middleName=null, lastName=Guo, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1College of Systems Engineering, National University of Defense Technology, Changsha 410073, bio={"content":"

born in 1989. PhD, associate professor. His main research interests include multimodal learning and trustworthy AI

"}, bioImg=null, bioContent=

born in 1989. PhD, associate professor. His main research interests include multimodal learning and trustworthy AI

, aboutCorrespAuthor=null), CN=AuthorExt(id=1329419906468131787, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, authorId=1329419906258416584, language=CN, stringName=郭延明, firstName=null, middleName=null, lastName=null, prefix=null, suffix=null, authorComment=null, nameInitials=null, affiliation=null, department=null, xref=1, address=1国防科技大学系统工程学院 长沙 410073, bio={"img":"Ym/TntEKIz+FWT5ZVcWr5w==","content":"

1989年生。 博士,副教授。 主要研究方向为多模态学习、可信人工智能

"}, bioImg=Ym/TntEKIz+FWT5ZVcWr5w==, bioContent=

1989年生。 博士,副教授。 主要研究方向为多模态学习、可信人工智能

, aboutCorrespAuthor=null)}, companyList=[AuthorCompany(id=1329419904823964583, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, xref=1, ext=[AuthorCompanyExt(id=1329419904836547496, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=EN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1College of Systems Engineering, National University of Defense Technology, Changsha 410073), AuthorCompanyExt(id=1329419904853324713, tenantId=1146029695717560320, journalId=1324759914146529478, articleId=1329419901032313734, companyId=1329419904823964583, language=CN, country=null, province=null, city=null, postcode=null, companyName=null, departmentName=null, remark=1国防科技大学系统工程学院 长沙 410073)])])]
老明瑞1 , 李正1, 丁兆云1, 张雪毅1, 岳祥虎2 , 郭延明1
作者信息
  • 1国防科技大学系统工程学院 长沙 410073
  • 2天津大学智能与计算学部 天津 300072
作者简介:

作者贡献声明

老明瑞负责方案整体设计和实验方法,并撰写论文;李正、张雪毅负责部分算法思路和实验;丁兆云、岳祥虎、郭延明提出指导意见并修改论文。岳祥虎和郭延明同为通信作者。

1995年生。博士,副教授。主要研究方向为可信人工智能、多模态学习

2000年生。博士研究生。主要研究方向为对抗训练、多模态学习

1982年生。博士,教授。 主要研究方向为数据挖掘、可信人工智能

1998年生。博士研究生。 主要研究方向为对抗训练、持续学习

1994年生。 博士,副教授。主要研究方向为多模态学习、可信人工智能

1989年生。 博士,副教授。 主要研究方向为多模态学习、可信人工智能

Improving Lightweight Model Robustness via Adversarial Ranking Distillation Method
Mingrui Lao1 , Zheng Li1, Zhaoyun Ding1, Xueyi Zhang1, Xianghu Yue2 , Yanming Guo1
Affiliations
  • 1College of Systems Engineering, National University of Defense Technology, Changsha 410073
  • 2College of Intelligence and Computing, Tianjing University, Tianjin 300072
  • born in 1995. PhD, associate professor. His main research interests include trustworthy AI and multimodal learning

    born in 2000. PhD candidate. His main research interests include adversarial training and multimodal learning

    born in 1982. PhD, professor. His main research interests include data mining and trustworthy AI

    born in 1998. PhD candidate. His main research interests include adversarial training and continual learning

    born in 1994. PhD, associate professor. His main research interests include multimodal learning and trustworthy AI

    born in 1989. PhD, associate professor. His main research interests include multimodal learning and trustworthy AI

出版时间: 2026-06-01 doi: 10.7544/issn1000-1239.202550339
文章导航
收藏切换

随着轻量级模型在边缘计算等资源受限场景中的广泛应用,其对抗鲁棒性不足的问题日益凸显。对抗蒸馏作为提升轻量级模型鲁棒性的重要手段,虽然现有方法普遍采用基于KL(Kullback-Leibler)散度的硬性约束实现鲁棒知识传递,但仍面临知识建模不完善和传递效率受限等问题。为此,该研究提出一种对抗排序蒸馏方法,通过优先级排序约束机制与多层次一致性框架,显著提升轻量级模型面向对抗攻击的抵御能力。具体地,前者旨在将视觉模型基于对抗样本的输出元素的重要性排序,进而约束教师-学生模型间的元素优先级保持一致,并通过函数近似非连续排序过程实现可导的松耦合约束。在此基础上,多层次排序一致性蒸馏框架从类别语意关联、样本语意关联以及对抗差异关联3个方面对鲁棒性进行建模与知识迁移,实现教师对抗防御能力的多视角传递,协同提升轻量级模型的干净样本准确率与对抗鲁棒性。在CIFAR-10,CIFAR-100,Tiny ImageNet数据集上实验验证了该方法的有效性,且相比于现有的先进对抗蒸馏方法有显著性能优势。同时,对抗排序蒸馏方法在训练数据受限和黑盒攻击场景下展现出更强的适应性与泛化能力,为轻量级模型在边缘部署中的安全防御提供了高效解决方案。

对抗鲁棒性  /  模型轻量化  /  知识蒸馏  /  对抗防御  /  深度学习

With the widespread application of lightweight models in resource-constrained scenarios such as edge computing, their insufficient adversarial robustness has become increasingly prominent. While adversarial distillation serves as a primary approach to enhance lightweight models’ robustness, existing methods predominantly employ KL (Kullback-Leibler) divergence-based rigid constraints for robust knowledge transfer, which still suffer from flawed knowledge modeling and limited transfer efficiency. To address these issues, we propose an adversarial ranking distillation (ARD) method that significantly improves lightweight models’ robustness through a priority ranking constraint mechanism and a multi-level consistency framework. Specifically, the priority ranking constraint mechanism organizes the output elements of vision models on adversarial samples by importance ranking, enforces teacher-student ranking consistency, and implements differentiable loose coupling constraints by approximating discontinuous ranking processes through hyperbolic tangent functions. Building upon this, the multi-level ranking consistency distillation framework models transfer robustness knowledge from three perspectives: categorical semantic correlation, sample semantic correlation, and adversarial discrepancy correlation, enabling multi-perspective transmission of teachers’ adversarial defense capabilities while synergistically improving lightweight models’ clean sample accuracy and adversarial robustness. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets validate the effectiveness of our method, demonstrating substantial performance advantages over state-of-the-art adversarial distillation approaches. Furthermore, the proposed ARD exhibits enhanced adaptability and generalization capabilities under scenarios of limited training data and black-box attacks, providing an efficient security solution for lightweight model deployment in edge computing environments.

adversarial robustness  /  lightweight model  /  knowledge distillation  /  adversarial defense  /  deep learning
老明瑞, 李正, 丁兆云, 张雪毅, 岳祥虎, 郭延明. 提高轻量级模型鲁棒性的对抗排序蒸馏方法. 计算机研究与发展, 2026 , 63 (6) : 1506 -1521 . DOI: 10.7544/issn1000-1239.202550339
Mingrui Lao, Zheng Li, Zhaoyun Ding, Xueyi Zhang, Xianghu Yue, Yanming Guo. Improving Lightweight Model Robustness via Adversarial Ranking Distillation Method[J]. Journal of Computer Research and Development, 2026 , 63 (6) : 1506 -1521 . DOI: 10.7544/issn1000-1239.202550339
随着神经网络[1]和深度学习[2]技术的快速发展,以图像分类[3]和目标识别[4]为代表的计算机视觉研究在多个应用领域取得了显著进展。受益于大模型技术[5]及扩展法则(scaling law)[6]的影响,视觉模型逐步朝着大规模网络框架与结构演进。然而,在实际应用场景中,视觉识别系统通常部署在资源受限的用户端或边缘设备上。因此,开发参数量级较小且性能优越的轻量级模型显得尤为必要。此外,由于轻量级模型在应用中用户交互和边缘设备部署的特点,相较于大规模视觉模型,它们更易受到攻击者的恶意攻击[7],进而影响模型的鲁棒性和可靠性[8]。因此,提升轻量级模型的对抗鲁棒性具有重要的研究价值。
在众多对抗防御的方法中,对抗训练[9]是运用最广泛且高效的策略,其原理是通过在训练过程中引入对抗样本,使模型在面对潜在攻击时能够提高其对抗鲁棒性。从数学形式化角度而言,对抗训练过程可建模为一个双层优化问题,其本质表现为最小-最大优化框架。在该框架下,内层优化通过最大化对抗扰动以生成最具攻击性的对抗样本,而外层优化则通过最小化模型在这些对抗样本上的经验风险来实现鲁棒性提升。然而,对抗训练在小模型的对抗防御场景中仍存在一定的局限性[10-12]:一方面,轻量级模型由于参数量受限,其对抗训练的表达能力和样本学习能力较弱,难以捕捉对抗样本的复杂特征,训练过程也相对不稳定;另一方面,轻量级模型在同时处理噪声扰动和干净样本时缺乏足够的灵活性与适应性,难以在对抗训练中生成高效的对抗样本。近年来,对抗鲁棒蒸馏[13]逐渐成为提升小模型对抗训练能力的关键研究技术。其旨在基于教师-学生网络的知识迁移框架[14-15],充分利用经过对抗训练的大模型的知识,以引导轻量级模型进行有效的对抗训练。通过这种方式,轻量级模型能够借鉴与模仿大模型面向对抗样本的抵御和判别能力,提升其在实际应用中应对恶意攻击的安全性与可靠性。
自面向轻量级模型的对抗鲁棒性蒸馏范式[13]被提出以来,大量工作对其进行了深入探索和优化,其核心研究点主要聚焦在3个方面:1)标签改进技术。通过如鲁棒软标签[16]、标签混合[17]、标签公平性[18]等手段增强轻量级模型对大模型的对抗鲁棒性的模仿能力。2)教师选择方法。通过结合干净样本训练教师[19]、学生模型自蒸馏[20]等手段多教师联合引导轻量级模型的鲁棒性学习。3)训练改进策略。通过对内层/外层优化[21-22]的持续改进提升轻量级模型的对抗样本生成质量与防御能力。尽管这些方法一定程度提升了对抗鲁棒蒸馏的性能,但目前几乎所有先进方法在蒸馏模式上都贯彻了一致的思路:基于给定教师模型的响应(如优化后的软标签),利用KL(Kullback-Leibler)散度[23]实现大模型向轻量级模型的对抗防御知识传递,其本质是约束教师模型与学生模型间的响应/预测值保持一致。
然而,本文认为,基于KL散度对教师-学生模型预测值施加的硬性约束在对抗鲁棒蒸馏中仍存在一定局限性:首先,与干净样本不同,对抗样本为了实现强欺骗性,其攻击方向通常指向与正确答案高度相关的类别[24],而KL散度下的类别预测值难以准确捕捉这种攻击方向上的类别语义关系,从而限制了对抗鲁棒性知识的传递;其次,由于教师模型与学生模型的参数量存在显著差异,两者在对抗样本的特征学习与表征能力上差距明显,对预测值进行硬约束可能进一步增加轻量级模型提升对抗鲁棒性的难度。
针对这一问题,本文提出了一种新颖且高效的轻量级模型鲁棒性提升方法——对抗排序蒸馏。具体而言,与现有方法通过对教师-学生模型的输出元素(如类别预测值)施加刚性约束的蒸馏模式不同,本研究提出的优先级排序约束机制通过以下方式实现鲁棒知识迁移:首先对视觉模型基于对抗样本的输出元素进行重要性排序,然后强制保持教师模型与学生模型间的排序一致性。受冒泡排序启发,该方法构建优先级对比函数实现两两元素间重要性的比较(重要则输出0,反之输出1),并整合所有元素对的函数输出得到每个元素的优先级排序。然而,这种理想化的优先级对比函数是非连续的,其无法直接通过梯度反向传播方法来优化学生模型进行训练。针对这一问题,我们提出用双曲正切函数近似排序过程,使整个约束过程具备可导性。我们所提出的优先级排序约束机制在对抗鲁棒性蒸馏范式中有2个方面优势:1)面向元素(如类别预测)的排序相比于其输出数值能够更有效地建模与表达对抗鲁棒性知识,如基于特定对抗样本2种不同预测类别的重要程度及对抗样本在类别攻击方向上发生的偏移等;2)基于优先级排序的约束在教师-学生网络间具有松耦合性且易于实现,能够在在学生网络参数量与表征能力受限的情况下,显著降低轻量级网络学习对抗鲁棒性知识的难度。
受益于优先级排序约束机制的对抗鲁棒性建模与高效知识传递能力,对抗排序蒸馏方法构建了一个多层次排序一致性蒸馏框架,旨在从类别语义关联、样本语义关联以及对抗差异关联3个方面对鲁棒性进行建模与知识迁移。1)类感知的排序一致性蒸馏。基于特定对抗样本输入,根据模型对不同类别预测的重要性进行优先级排序,其中排序元素是所有预测类别。2)实例感知的排序一致性蒸馏。基于特定预测类别,根据一个训练批次内样本与该类别的预测相似性进行排序,此时排序元素是训练批次中的全部对抗样本。3)差异感知的排序一致性蒸馏。基于特定训练样本,根据模型对其干净样本与对抗样本的预测差异进行类别排序(如对“飞机”预测的变化幅度大于“汽车”的变化幅度),此排序反映了模型预测面向对抗攻击的鲁棒性偏移。不同于主流方法局限于类别预测的知识迁移,本文提出的对抗排序蒸馏方法能够从多视角传递教师模型应对恶意攻击的防御能力,协同提升小模型的干净样本准确率与对抗鲁棒性。总体而言,本文贡献有3个方面。
1)从对抗蒸馏的鲁棒性知识建模以及学生模型的鲁棒性学习难度出发,首次提出以响应元素排序的视角实现教师-学生模型间对抗防御能力的高效传递。
2)提出了一种用于提升轻量级模型鲁棒性的全新策略——对抗排序蒸馏,具体包括2部分:优先级排序约束机制构建了可导的输出元素排序策略,实现教师-学生网络响应的松耦合约束;多层次排序一致性则基于前者的对抗鲁棒性表达能力,实现从类别语义关联、样本语义关联以及对抗差异关联3个维度的多视角鲁棒知识蒸馏。
3)为了可靠地评估对抗排序蒸馏在轻量级模型对抗防御上的有效性,本文在对抗训练领域广泛使用的CIFAR-10,CIFAR-100,Tiny ImageNet数据集上进行了大量实验分析。结果表明,相比已有方法,对抗排序蒸馏能够在稳定维持轻量级模型干净样本准确率的同时,显著提升对抗鲁棒性能。同时,在训练样本数量受限以及不同外部最小化函数设定的环境中,相比于刚性约束的对抗鲁棒蒸馏策略,本文方法具备更强的适应力与通用性。
深度神经网络在多个研究应用领域展现出卓越性能,而轻量级模型凭借其高效的计算性能和边缘部署优势获得了广泛关注。然而,研究表明深度神经网络存在显著的对抗脆弱性问题[7]。具体而言,攻击者通过在原始数据中添加人眼难以察觉的微小扰动生成对抗样本,这些样本会导致神经网络产生错误输出,从而引发实际应用中的安全威胁。值得注意的是,由于参数规模有限和表征能力不足,轻量级模型的对抗脆弱性问题更为突出。
为提升深度神经网络的对抗鲁棒性,对抗训练[9]作为一种典型的启发式解决方案,采用攻击与防御协同优化的框架。该策略通过以下2个关键机制实现鲁棒性提升:首先,利用对抗攻击方法生成具有强干扰性的对抗样本;其次,将这些对抗样本纳入训练过程,优化模型使其能够正确识别对抗样本,从而增强模型对对抗攻击的防御能力。这种“以毒攻毒”的训练范式已被证明能有效提升模型的鲁棒性。针对对抗训练的持续改进和优化方法主要聚焦在对抗样本生成以及模型防御训练2个方面:对抗样本生成聚焦于在对抗训练的攻击阶段,生成难以感知攻击隐蔽性强的图像空间攻击样本,主要方法包括FGSM[25],PGD[26],CW[27],AA[28]等方法;模型防御训练则关注对抗训练的防御阶段,其基本问题是利用已生成的对抗样本进行训练,并通过分类或检测损失最小化来更新模型,其通常结合参数自适应、模型增强、损失正则项以及早期停止等技术进一步提升对抗鲁棒性。然而,由于对抗样本生成质量以及数据拟合能力的差异,研究表明[29]对抗训练方法在容量大、高度参数化网络中的表现显著强于轻量级模型,限制了对抗防御技术在移动端与边缘设备的应用发展。
受知识蒸馏在模型压缩领域成功应用的启发,基于教师-学生架构的对抗鲁棒蒸馏已成为提升轻量级模型对抗防御能力的重要范式。该范式采用经过对抗训练的大型模型作为教师模型,通过知识迁移机制,在保持干净样本预测精度的同时,指导轻量级学生模型学习对抗鲁棒性特征[30]。后续对抗鲁棒蒸馏的相关工作主要从3个方面进行创新:1)标签优化策略,旨在教师-学生知识传递中产生更为高效的训练标签来指导学生进行鲁棒性强化。AKD方法[17]较早探索了标签混合对于鲁棒性转移的影响,而RSLAD[16]提出使用鲁棒软标签作为监督信号,能够更好地捕捉教师模型的鲁棒行为。Fair-ARD 方法[18]则将鲁棒公平性引入对抗蒸馏,并提出了基于几何视角的最少对抗步数度量方法,以对标签类别权重进行动态重加权。2)教师选择策略,旨在筛选多样化教师进行学生网络鲁棒蒸馏。由于教师模型在学生生成对抗样本上表现出的不稳定现象,IAD方法[20]提出教师蒸馏与自蒸馏结合的多教师蒸馏策略。不局限于单个鲁棒教师模型,Zhao等人[19]提出额外引入一个干净样本训练下的教师模型来指导对抗蒸馏,通过结合2个教师模型各自的优势,来综合改善学生模型的性能。B-MTARD[31]则在此基础上通过增加一个额外的熵平衡模块,来平衡不同教师模型之间的知识规模。3)训练改进方法,旨在从对抗蒸馏内层/外层优化的训练模式上进行持续提升。Huang 等人[21]将教师模型引入内层优化过程,自适应地搜索学生模型和教师模型在扰动邻域中的最大预测差异点以改进蒸馏效果。DGAD 方法[22]提出了动态指导的对抗蒸馏框架,通过动态调整样本权重优化知识传递,并解决教师模型的误分类问题。SmaraAD[32]则设计了一种归因区域对齐机制,从缓解训练干扰以及注意力行为理解不足的视角持续提高学生模型鲁棒性。Wu等人[33]提出了一种基于混合解耦蒸馏的小容量模型对抗防御方法,通过高效提取预训练教师模型中多层次、细粒度的鲁棒性知识,显著提升小容量模型的对抗鲁棒性。相关工作及方法特点总结如表1所示。
不同于常规基于KL散度的刚性蒸馏方法,本文提出一种松弛性更高的元素排序策略来实现对抗鲁棒蒸馏,得益于在鲁棒性建模与知识传递方面的高效性,本文方法获得使轻量级学生模型能够从3个视角协同继承教师大模型抵御对抗攻击的能力:面向特定样本的类别预测、面向特定类别的样本语义关联以及面向干净样本与对抗样本的类别预测差异。
本文方法主要基于对抗训练和对抗蒸馏理论,下面就相关概念和基本知识予以介绍。
为抵御对抗样本的恶意攻击,对抗训练结合对抗样本生成与防御性训练技术,进而生成鲁棒视觉识别模型。这个过程可以被定义为一个最小最大优化问题,表示如下
$ \underset{\theta }{\min }\; {E}_{\left(x,y\right)\sim \mathcal{D}}\left[\underset{\delta \in \mathit{\varOmega }}{\max } {\mathcal{L}}_{\text{ce}}\left(f\left({x}_{\text{cle}}+\delta ;\theta \right),y\right)\,\right]\,\;, $
其中$ f(\cdot ;\theta ) $代表一个权重$ \theta $的深度神经网络,$ \mathcal{D} $是包括了干净训练样本$x_{\rm cle} $及其对应的类别标签y的数据分布,${\mathcal{L}}_{\text{ce}} $为交叉熵损失函数。δ是对抗扰动,而$\varOmega $是一个定义为$\varOmega = \{\delta: \|\delta\| \leq \epsilon\} $的界限,其中最大扰动范围为ϵ。对抗样本则表示为干净样本与对抗扰动的叠加$x_{\rm adv}=x_{\rm cle}+\delta $。为了在式(1)中的内部最大化中生成对抗样本,对抗训练普遍采用PGD(projected gradient descent)方法通过多次迭代逐步优化扰动δ。
$ {\delta }^{n+1}={{\varPi }}_{{\varOmega }}\left({\delta }^{n}+\alpha \cdot \text{sgn} \left({\nabla }_{x}\mathcal{L}\left(f\left({x}_{\text{cle}}+{\delta }^{n};\theta \right),y\right)\right)\right)\;, $
其中${\delta }^{n} $是第n次优化步骤的对抗扰动,而$\varPi(\cdot) $和$\alpha $分别代表映射函数和步长。最终的对抗扰动$\delta^* $是通过初始化$\delta^0 $为均匀分布并在PGD算法中进行N次总步数的迭代获得的。
标准知识蒸馏范式通过最小化教师模型和学生模型预测之间的分布差异,将知识从预先训练好的教师模型转移到学生模型。形式上,对于学生和教师网络输出逻辑$S(x)\in \mathbb R^{N^{\rm b}\times C} $和$T(x)\in \mathbb R^{N^{\rm b}\times C}$,其中$N^{\rm b} $和C分别表示训练批次大小和预测总类别数,知识蒸馏的损失表示为
$ \begin{split} \mathcal L_{\rm kd}(T(x);S(x;\theta))=&\dfrac{\tau^2}{N^{\rm b}}\sum_{i=1}^{N^{\rm b}}KL(\sigma(T(x)_{i,:},\tau);\sigma(S(x)_{i,:},\tau))=\\&\sum_{i=1}^{N^{\rm b}}\sum_{j=1}^{C}\ln \left(\dfrac{\sigma(T(x)_{i,j},\tau)}{\sigma(S(x)_{i,j},\tau)}\right)\;,\\[-1pt]\end{split}$
其中$ KL $($ \cdot ;\cdot $)指KL散度,用于衡量学生模型与教师模型输出分布的相似性。参与蒸馏学生及教师模型的预测分布为
$ \begin{aligned} & \sigma(S(x)_{i,:},\tau)=\mathit{softmax}\left(\dfrac{S\left(x\right)_{i,:}}{\tau}\right)\; , \\ & \sigma(T(x)_{i,:},\tau)=\mathit{softmax}\left(\dfrac{T\left(x\right)_{i,:}}{\tau}\right)\; ,\end{aligned} $
其中$ \tau $是用于控制预测平滑程度的温度因子。在对抗防御中,学生模型不仅要继承教师模型对自然样本预测的准确性,还要模仿其抵御恶意攻击样本的对抗鲁棒性。因此,作为知识蒸馏在对抗防御领域的扩展,对抗蒸馏旨在结合对抗训练技术,通过教师模型的指导提升学生模型在对抗样本上的鲁棒性表现。其核心思想是引入对抗样本作为蒸馏过程中的重要组成部分,使学生模型能够模仿教师模型在对抗样本上的输出分布,从而学习到教师模型的鲁棒性表现。对抗蒸馏方法的优化目标可以写为
$ \begin{split} \mathop{\arg\;\min}\limits_{\theta}&(1-\beta)\cdot \mathcal L_{\rm ce}(S(x_{\rm cle}),y)+\beta \cdot\mathcal L_{\rm kd}(S(x_{\rm adv};\theta),T(x_{\rm cle})) ,\\& x_{\rm adv}=\mathop{\arg\;\max}\limits_{\|\delta\|_p<\epsilon}\mathcal L_{\rm ce}(S_\theta(x_{\rm cle}+\delta),y)\;,\\[-1pt]\end{split}$
其中$ {x}_{\text{adv}} $,$ {x}_{\text{cle}} $分别指代干净与对抗样本,$ \beta $为任务和蒸馏损失间的权重调整参数。此外,为了充分增强学生模型生成样本在教师模型中的适应性,自适应对抗蒸馏技术(AdaAD)将教师模型融入对抗样本的生成过程,即在最大化学生模型与教师模型在对抗扰动输入上的预测差异,具体公式可表达为
$ \begin{aligned} & \mathop{\arg\;\min}\limits_{\theta}(1-\beta)\cdot \mathcal L_{\rm kd}(S_\theta(x_{\rm cle})\|T(x_{\rm cle}))+\\&\quad\;\;\; \beta\cdot\mathcal L_{\rm kd}(S_\theta(x_{\rm adv})\|T(x_{\rm adv}))\;, \\& x_{\rm adv}=\mathop{\arg\;\max}\limits_{\|\delta\|_p<\epsilon}\mathcal L_{\rm kd}(S_\theta(x_{\rm cle}+\delta),T(x_{\rm cle}+\delta))\;。\end{aligned} $
鉴于当前对抗蒸馏方法普遍忽视了教师-学生网络间刚性约束的局限性,本文提出了一种新颖且有效的对抗排序蒸馏方法,该方法包括2个核心部分:优先级排序约束机制以及多层次排序一致性蒸馏框架。前者面向对抗鲁棒性构建了松耦合且高效的知识建模与传递机制,而后者在此基础上分别从类别语意关联、样本语意关联以及对抗差异关联3种不同层次挖掘教师模型对抗鲁棒性,并引导其协同提升轻量级模型的干净样本下的准确率以及对抗攻击下的韧性。
尽管现有对抗蒸馏研究在监督标签改进、教师模型选择等方面上进行了广泛研究与优化,但几乎所有这些方法在教师-学生网络间蒸馏模式上均采用基于KL散度的硬性约束,即监督学生模型的输出元素值(例如响应)与教师模型保持一致。然而,值得注意的是,相较于干净样本,对抗样本的预测数值普遍因对抗攻击的方向而发生偏移,此时类别间的优先级关系往往比预测数值本身更为重要。其次,由于大模型与小模型在参数量及表征能力上的显著差异,学生模型在模拟教师模型对抗样本响应时面临的难度明显高于模拟干净样本,这导致对抗训练的难度显著增加。
针对上述问题,本文提出了一种优先级排序约束机制,如图1所示,不同于KL散度对教师模型与学生模型输出响应值的分布约束,该机制关注响应元素优先级排序并以排序一致性作为约束。
具体地,假设教师网络与学生网络的输出元素序列分别为$ {\boldsymbol Z}^{\mathrm{t}}=(z_{1}^{\mathrm{t}},z_{2}^{\mathrm{t}},… ,z_{N}^{\mathrm{t}})\in {\mathbb{R}}^{d} $以及$ {\boldsymbol Z}^{\mathrm{s}}=(z_{1}^{\mathrm{s}},z_{2}^{\mathrm{s}},… , z_{N}^{\mathrm{s}})\in {\mathbb{R}}^{d} $,其中$ d $为元素序列维度(即元素个数)。首先需要对响应值$ {\boldsymbol Z}^{\mathrm{t}},{\boldsymbol Z}^{\mathrm{s}} $内元素的重要性排序,受冒泡排序策略启发,构建优先级对比矩阵$ {\boldsymbol{D}}^{\mathrm{t}},{\boldsymbol{D}}^{\mathrm{s}}\in {\mathbb{R}}^{N\times N} $可表达为
$ {\boldsymbol{D}}^{\mathrm{t}}=I\left(\left(\begin{matrix}z_{1}^{\mathrm{t}} & … & z_{N}^{\mathrm{t}}\\ \vdots & & \vdots \\ z_{1}^{\mathrm{t}} & … & z_{N}^{\mathrm{t}}\\ \end{matrix}\right)-\left(\begin{matrix}z_{1}^{\mathrm{t}} & … & {z}_{1}^{\mathrm{t}}\\ \vdots & & \vdots \\ z_{N}^{\mathrm{t}} & … & {z}_{N}^{\mathrm{t}}\\ \end{matrix}\right)\right)\;, $
$ I\left(x\right)=\left\{\begin{aligned}&1,\;\;\mathrm{if}\; x \lt \mathrm{0}\;,\\& 0,\;\;\mathrm{if}\; x \geq \mathrm{0}\;, \end{aligned}\right.$
其中$ I(\cdot ) $为二值示性函数,$ D_{ij}^{\mathrm{t}}=I(z_{i}^{\mathrm{t}}-z_{j}^{\mathrm{t}}) $表示在教师模型响应中第$ i $个元素与第$ j $个元素的优先级比较,根据优先级矩阵$ {\boldsymbol{D}}^{\mathrm{t}} $可定义$ {z}_{i}^{\mathrm{t}} $在元素队列$ ({z}_{1}^{\mathrm{t}},{z}_{2}^{\mathrm{t}},… ,{z}_{N}^{\mathrm{t}})\in {\mathbb{R}}^{d} $中的优先级排序$ L_{i}^{\mathrm{t}} $为
$ L_{i}^{\mathrm{t}}=1+\displaystyle\sum_{k=1,k\neq i}^{d}\,I\left(\boldsymbol{D}_{i}^{\mathrm{t}} \lt \boldsymbol{D}_{k}^{\mathrm{t}}\right)\;, $
其中$ k $代表元素队列中第$ k $个不同于待排序$ i $的元素。虽然式(8)能够有效对元素排序值进行构建,其所涉及的二值示性函数$ I(\cdot ) $是非连续不具备可导属性的,并不适用于梯度回传优化模式下的对抗蒸馏任务。针对这一问题,优先级排序约束机制提出使用双曲正切函数变体$ \mathbb{A}(\cdot ) $来近似示性函数$ I(\cdot ) $的优先级对比功能
$ I\left(z_{i}^{\mathrm{t}}-z_{j}^{\mathrm{t}}\right)\approx \mathbb{A}\left(z_{i}^{\rm t}z_{j}^{\rm t}\right)=\dfrac{1}{2}\left(\tanh \left(z_{i}^{\mathrm{t}}-z_{j}^{\mathrm{t}}\right)+1\right)\;。 $
由此,式(9)可求导为
$ \dfrac{\partial \mathbb{A}}{\partial z_{i}^{\mathrm{t}}}=1-\dfrac{1}{2}{\left(\tanh \left(z_{i}^{\mathrm{t}}-z_{j}^{\mathrm{t}}\right)\right)}^{2}\;, $
则$ z_{i}^{\mathrm{t}} $在元素队列$ (z_{1}^{\mathrm{t}},z_{2}^{\mathrm{t}},… ,z_{N}^{\mathrm{t}})\in {\mathbb{R}}^{d} $中的优先级排序$ L_{i}^{\mathrm{t}} $可改进为
$ L_{i}^{\mathrm{t}}\approx 1+\displaystyle\sum_{k=1,k\neq i}^{d}\,\dfrac{1}{2}\left(\tanh \left(z_{i}^{\mathrm{t}}-z_{k}^{\mathrm{t}}\right)+1\right)\;。 $
基于式(11),所提出的优先级排序约束机制可进一步使用绝对值函数$ |\cdot | $对教师模型与学生模型间响应排序的一致性进行监督。对于包含$ {N}^{\mathrm{b}} $个样本的训练批次,其训练损失函数$ {\mathcal{L}}_{\text{rp}} $可表达为
$ {\mathcal{L}}_{\text{rp}}=\dfrac{1}{{N}^{\mathrm{b}}}\displaystyle\sum_{i=1}^{{N}^{\mathrm{b}}}\,\displaystyle\sum_{j=1}^{d}\,\left| L_{ij}^{\mathrm{t}}-L_{ij}^{\mathrm{s}}\right| \;。 $
与传统的知识蒸馏机制(式(3))进行对比,优先级排序约束机制能够实现教师-学生间对抗鲁棒性的高效传递。其中,优先级对比矩阵实现逐对元素间重要性比较,为对抗鲁棒知识进行高效建模;排序一致性约束则充分利用松耦合监督促进能力受限小模型对大模型鲁棒性行为的模仿。排序约束在训练阶段计算复杂度为$ O(N{^{2}}) $,高于KL散度的$ O(N) $,但在必要情况下可通过$ \text{Top} $-$ K $采样策略将开销控制在一定范围内;测试阶段,该方法的推理过程与计算复杂度与常规方法无异。
优先级排序约束机制聚焦于教师-学生网络间的对抗样本知识建模与传递的模式,而多层次排序一致性蒸馏框架则注重将不同视角的对抗鲁棒性从大模型充分传递给小模型,具体可分为类感知排序一致性蒸馏、实例感知排序一致性蒸馏,以及差异感知排序一致性蒸馏,方式示意如图2所示。
1)类感知排序一致性蒸馏。与主流对抗蒸馏方法视角类似,类感知排序一致性蒸馏$ {\mathcal{L}}_{\text{crd}} $关注给定对抗样本下教师-学生网络预测间的知识蒸馏, 此时待排序元素是全部候选预测类别。以学生模型$ S(\cdot) $中类感知排序为例,网络对第$ i $个对抗样本的输出逻辑为$ {S({x}_{\text{adv}})}_{i: }\in {\mathbb{R}}^{C} $,其中$ C $为候选预测类别数量,则第$ j $个预测类别的优先级排序$ L_{ij}^{\text{sc}} $可表示为
$ L_{ij}^{\text{sc}}=1+\displaystyle\sum_{k=1,k\neq j}^{C}\,\dfrac{1}{2}\left(\tanh \left({S({x}_{\text{adv}})}_{ij}-{S({x}_{\text{adv}})}_{ik}\right)+1\right)\;。 $
同理可得教师模型$ T(\cdot) $下的优先级排序$ L_{ij}^{\text{tc}} $。基于3.1节提出排序一致性监督损失(式(12)),类感知排序一致性蒸馏损失$ {\mathcal{L}}_{\text{crd}} $为每个类别加入了额外类感知排序权重$ W_{i: }^{\mathrm{c}}=\sigma ({T\left({x}_{\text{cle}}\right)}_{i: },\tau ) $,其中$ \tau $为温度系数,可以表示为
$ {\mathcal{L}}_{\text{crd}}=\dfrac{1}{{N}^{\rm b}}\displaystyle\sum_{i=1}^{{N}^{{\rm b}}}\,\displaystyle\sum_{j=1}^{C}W_{ij}^{\mathrm{c}}\cdot \left| L_{ij}^{\text{tc}}-L_{ij}^{\text{sc}}\right| \;, $
其中类感知排序权重$ W_{i: }^{\mathrm{c}} $为教师模型对干净样本$ i $的类别预测,体现了在该标签样本下不同类别的语义关联。相关研究[34]指出,如果错误候选类别与标签类别具有强语义关系,这些类别普遍将成为对抗样本的扰动方向,而$ W_{i: }^{\mathrm{c}} $为类感知排序一致性蒸馏$ {\mathcal{L}}_{\text{crd}} $提供了鲁棒性排序学习的重要类别选择能力。
2)实例感知排序一致性蒸馏。区别于主流对抗蒸馏方法仅关注类别知识,实例感知排序一致性蒸馏$ {\mathcal{L}}_{\text{ird}} $聚焦不同对抗样本实例与特定类别的语义相似性排序,此时待排序元素是一个训练批次$ {N}^{\mathrm{b}} $内全部的对抗样本。以学生模型$ S(\cdot) $中实例感知排序为例,模型对$ {N}^{\mathrm{b}} $个训练样本在类别$ j $上的响应可表示为$ {S({x}_{\text{adv}})}_{: j}\in {\mathbb{R}}^{{{N}^{\mathrm{b}}}} $,则第$ i $个对抗样本的优先级排序$ L_{ij}^{\text{sI}} $可表示为
$ L_{ij}^{\text{sI}}=1+\displaystyle\sum_{k=1,k\neq i}^{{N}^{\mathrm{b}}}\,\dfrac{1}{2}\left(\tanh \left({S({x}_{\text{adv}})}_{ij}-{S({x}_{\text{adv}})}_{kj}\right)+1\right)\;。 $
计算得到$ L_{ij}^{\text{sI}} $和$ L_{ij}^{\text{tI}} $,与$ {\mathcal{L}}_{\text{crd}} $类似我们额外加入了干净样本引导的实例感知权重$ W_{:j}^{\mathrm{I}}=\sigma ({T\left({x}_{\text{cle}}\right)}_{: j},\tau ) $以区分不同对抗样本在特定类别排序计算中的语义关联性,最后的实例感知排序一致性蒸馏损失$ {\mathcal{L}}_{\text{ird}} $表示为
$ {\mathcal{L}}_{\text{ird}}=\dfrac{1}{{N}^{\mathrm{b}}}\displaystyle\sum_{i=1}^{{N}^{\mathrm{b}}}\,\displaystyle\sum_{j=1}^{C}W_{ij}^{\mathrm{I}}\cdot \left| L_{ij}^{\text{tI}}-L_{ij}^{\text{sI}}\right| \;。 $
3)差异感知排序一致性蒸馏。现有对抗蒸馏方法依赖于对抗样本作为输入进行显示鲁棒知识传递,普遍忽略了对抗和干净样本之间的模型响应差异,如模型面向同一干净和对抗样本之间的类别预测偏移。本文认为这种预测差异能够对对抗样本攻击方向、模型抵御能力等隐式鲁棒知识性知识进行建模。针对这一点,差异感知排序一致性蒸馏$ {\mathcal{L}}_{\text{drd}} $聚焦于不同类别在对抗或干净样本下预测差异的重要性排序,将隐式鲁棒知识从大模型传递给小模型。具体地,先构建学生模型针对样本$ i $的鲁棒性-准确率差异$ {{{D}^{\mathrm{s}}}\left(x\right)}_{i: }= {S({x}_{\text{adv}})}_{i: }-{S\left({x}_{\text{cle}}\right)}_{i: } $,同理可得教师模型差异$ {{{D}^{\mathrm{t}}}\left(x\right)}_{i: } $。差异感知排序$ L_{ij}^{\text{sD}} $关注模型从预测干净样本到对抗样本的过程中,不同类别预测变化的程度并进行排序,其排序$ L_{ij}^{\text{sD}} $以及一致性蒸馏损失$ {\mathcal{L}}_{\text{drd}} $可表达为
$ \begin{aligned} &L_{ij}^{\text{sD}}= 1+\displaystyle\sum_{k=1,k\neq j}^{C}\,\dfrac{1}{2}\left(\tanh \left({{{D}^{\mathrm{s}}}\left(x\right)}_{ij}-{{{D}^{\mathrm{t}}}\left(x\right)}_{ik}\right)+1\right)\;,\\ &{\mathcal{L}}_{\text{drd}}= \dfrac{1}{{N}^{\mathrm{b}}}\displaystyle\sum_{i=1}^{{N}^{\mathrm{b}}}\,\displaystyle\sum_{j=1}^{d}\,\left| L_{ij}^{\text{tD}}-L_{ij}^{\text{sD}}\right| \;。\end{aligned} $
基于上述介绍的约束机制和蒸馏框架,最终的训练损失函数综合考虑3种层次的知识蒸馏,可表示为
$ \mathcal{L}_{\rm{total}}=\mathcal{L}_{\rm{crd}}+ {\lambda }\cdot \mathcal{L}_{\rm{ird}}+ {\mu }\cdot \mathcal{L}_{\rm{drd}}\;, $
其中$ \lambda $,$ \mu $为权重因子,用于动态调整不同层次的一致性约束在对抗排序蒸馏中的贡献,通过第4节实验证明,最终对抗排序损失函数的简单取值为$ \lambda =\mu =1 $时效能达到最优。
本节从梯度幅值上界以及梯度幅值衰减2个方面,对优先级排序约束相较于KL散度约束的优越性进行数学证明。其中,显式的上界约束确保参数更新量可控,防止优化过程因梯度幅值过大而震荡或发散;而具备梯度幅值衰减能力可促使模型减弱远离决策边界对抗样本对梯度的贡献,提升模型的鲁棒性。
定理1. 排序约束的梯度幅值上界。
排序约束的梯度幅值满足以下不等式:
$ \left\|\dfrac{\partial {\mathcal{L}}_{\text{crd}}}{\partial z_{k}^{\mathrm{s}}}\right\|\leq \dfrac{1}{2}\left(C-1\right),\;\;\forall k\in \left\{1{,}2,… ,C\right\}\;。 $
证明.
对任一类别的输出$ z_{k}^{\mathrm{s}} $, 比较对贡献计算得到
$ \dfrac{\partial {\mathcal{L}}_{\text{crd}}}{\partial z_{k}^{\mathrm{s}}}=\dfrac{1}{2}\displaystyle\sum_{j\neq k}\,\mathrm{sgn}\left(z_{k}^{\mathrm{t}}-z_{j}^{\mathrm{t}}-z_{k}^{\mathrm{s}}+z_{j}^{\mathrm{s}}\right)\\ \cdot {sech}^{2} (z_{k}^{\mathrm{s}}-z_{j}^{\mathrm{s}}) \;。 $
由于$ {sech}^{2}(x)\leq 1 $对所有实数x成立,梯度幅值满足:
$ \left| \dfrac{\partial {\mathcal{L}}_{\text{crd}}}{\partial z_{k}^{\mathrm{s}}}\right| \leq \dfrac{1}{2}\displaystyle\sum_{j\neq k}\,1=\dfrac{1}{2}\left(C-1\right)\;。 $
证毕.
推论1. 总梯度范数满足
$ { \|\nabla {\mathcal{L}}_{\text{crd}} \|}_{2}\leq \dfrac{1}{2}C\sqrt{C-1}\;。 $
定理2. KL散度的梯度幅值上界。
KL散度的梯度幅值满足
$ \left\|\dfrac{\partial {L}_{\text{KL}}}{\partial z_{k}^{\mathrm{s}}}\right\|\leq 1-\dfrac{1}{C},\;\;\forall k\in \left\{1{,}2,… ,C\right\}\;。 $
证明.
KL散度的梯度为
$ \dfrac{\partial {L}_{\text{KL}}}{\partial z_{k}^{\mathrm{s}}}={\mathbb{E}}_{x}\left[\sigma {\left({\boldsymbol Z}^{\mathrm{s}}\right)}_{k}-\sigma { ({\boldsymbol Z}^{\mathrm{t}})}_{k}\right]\;。 $
由于$ \sigma {\left({\boldsymbol Z}^{\mathrm{s}}\right)}_{k}\in (0{,}1) $且$ \displaystyle\sum_{k=1}^{C}\,\sigma {\left({\boldsymbol Z}^{\mathrm{s}}\right)}_{k}=1 $,当类别数C较大时,$ \sigma {\left({\boldsymbol Z}^{\mathrm{s}}\right)}_{k}\approx \dfrac{1}{C} $。因此
$ \left| \dfrac{\partial {L}_{\text{KL}}}{\partial z_{k}^{\mathrm{s}}}\right| \leq \max \left(\sigma {\left({\boldsymbol Z}^{\mathrm{s}}\right)}_{k},\sigma { ({\boldsymbol Z}^{\mathrm{t}})}_{k}\right)\leq 1-\dfrac{1}{C} \;。$
证毕.
梯度幅值上限结论:排序约束的梯度幅值上界严格低于KL散度,训练更具稳定性,即
$ \dfrac{1}{2}\left(C-1\right) \lt C\left(1-\dfrac{1}{C}\right)\;。 $
定理3. KL散度的梯度幅值无衰减性。
当学生模型的某一类别逻辑值$ z_{k}^{\mathrm{s}} $显著高于其他类别(即 $ z_{k}^{\mathrm{s}}\gg z_{j}^{\mathrm{s}},\forall j\neq k $)时,KL散度的梯度幅值不随$ z_{k}^{\mathrm{s}}-z_{j}^{\mathrm{s}} $的增大而衰减。
证明.
假设$ z_{k}^{\mathrm{s}}\rightarrow +\mathrm{\infty } $,而其他 $ z_{j}^{\mathrm{s}} $固定或有限,则Softmax概率近似为
$ \sigma {\left({\boldsymbol Z}^{\mathrm{s}}\right)}_{k}\approx 1,\;\;\sigma {\left({\boldsymbol Z}^{\mathrm{s}}\right)}_{j}\approx 0\left(\forall j\neq k\right)\;。 $
此时,梯度表达式简化为
$ \dfrac{\partial {L}_{\text{KL}}}{\partial z_{k}^{\mathrm{s}}}\approx {\mathbb{E}}_{x}\left[1-\sigma {\left({\boldsymbol Z}^{\mathrm{t}}\left(x\right)\right)}_{k}\right]\;, $
梯度幅值为
$ \left| \dfrac{\partial {L}_{\text{KL}}}{\partial z_{k}^{\mathrm{s}}}\right| \approx \left| 1-{\mathbb{E}}_{x}\left[\sigma {\left({\boldsymbol Z}^{\mathrm{t}}\left(x\right)\right)}_{k}\right]\right| \;, $
该幅值仅取决于教师模型的概率分布$ \sigma {\left({\boldsymbol Z}^{t}\right)}_{k} $,与学生模型输出逻辑值的绝对差异$ z_{k}^{\mathrm{s}}-z_{j}^{\mathrm{s}} $无关。因此,KL散度的梯度幅值在$ z_{k}^{\mathrm{s}}\gg z_{j}^{\mathrm{s}} $时不会自动衰减。
证毕.
定理4. 排序约束的梯度指数衰减性。
当学生模型的某一类别逻辑值$ z_{k}^{\mathrm{s}} $显著高于其他类别(即$ z_{k}^{\mathrm{s}}\gg z_{j}^{\mathrm{s}} $)时,排序约束的梯度幅值随$ z_{k}^{\mathrm{s}}-z_{j}^{\mathrm{s}} $的增大而指数衰减:
$ \left| \dfrac{\partial {\mathcal{L}}_{\text{crd}}}{\partial z_{k}^{\mathrm{s}}}\right| \leq \dfrac{\gamma }{2}\displaystyle\sum_{j\neq k}\,4{\mathrm{e}}^{-2\gamma (z_{k}^{\mathrm{s}}-z_{j}^{\mathrm{s}} )}\;。 $
证明.
由双曲正割函数的性质$ {sech}^{2}(x)\leq 4{\mathrm{e}}^{-2|x|} $,当$ z_{k}^{\mathrm{s}}\gg z_{j}^{\mathrm{s}} $时
$ {sech}^{2} (\gamma (z_{k}^{\mathrm{s}}-z_{j}^{\mathrm{s}}) )\leq 4{\mathrm{e}}^{-2\gamma (z_{k}^{\mathrm{s}}-z_{j}^{\mathrm{s}})} \;。$
因此,梯度幅值满足
$ \left| \dfrac{\partial {\mathcal{L}}_{\text{crd}}}{\partial z_{k}^{\mathrm{s}}}\right| \leq \dfrac{\gamma }{2}\displaystyle\sum_{j\neq k}\,4{\mathrm{e}}^{-2\gamma (z_{k}^{\mathrm{s}}-z_{j}^{\mathrm{s}})}=2\gamma \displaystyle\sum_{j\neq k}\,{\mathrm{e}}^{-2\gamma (z_{k}^{\mathrm{s}}-z_{j}^{\mathrm{s}})}\;, $
当$ z_{k}^{\mathrm{s}}-z_{j}^{\mathrm{s}}\rightarrow +\mathrm{\infty } $时,指数项$ {\mathrm{e}}^{-2\gamma \left(z_{k}^{s}-z_{j}^{s}\right)}\rightarrow 0 $,梯度幅值指数衰减至0。
证毕.
梯度衰减性结论:梯度幅值上界低是训练稳定性的基础,但梯度衰减性直接决定收敛率,对训练的影响程度更高。常规KL散度约束梯度幅值不随学生模型输出差异增大而衰减,导致优化过程易受远离决策边界样本的干扰;而排序化约束通过双曲正割函数的指数衰减特性,梯度幅值在输出差异显著时自动收缩,提升训练稳定性与效率。
本节主要研究6个问题:
1)对抗排序蒸馏对小模型鲁棒性的提升效果如何。
2)对抗排序蒸馏的每个主要组件是否对小模型鲁棒性提升做出贡献。
3)对抗排序蒸馏的主要参数对小模型鲁棒性产生了怎样的影响。
4)对抗排序蒸馏在教师-学生模型一致的对抗自蒸馏场景下能否保持提升效果。
5)对抗排序蒸馏在目标模型未知的白盒攻击场景下能否保持提升效果。
6)对抗排序蒸馏在训练数据受限场景下能否保持提升效果。
为讨论以上6个研究问题,本节首先在4.1节分别介绍数据集、对比方法、网络架构、训练配置、评价指标等相关实验设置,并在4.2~4.6节分别对这些研究问题展开针对性实验分析。
实验数据选择为在对抗防御研究中常用的CIFAR10[35],CIFAR100[35],Tiny ImageNet[36]数据集,对应候选类别分别是10,100,200,这3个数据集中图像的像素范围均被归一化至 [0,1] 的区间内。在此基础上我们选取了9种广泛使用且高效的小模型对抗防御方法,包括对抗训练代表方法PGD-AT[36]和TRADES[37],以及对抗蒸馏代表方法ARD[13],IAD[20],RSLAD[16],AdaAD[21],MTARD[19],Fair-ARD[18],DGAD[22]。所有对比方法的实验配置均参照其原文推荐的最优参数,并在其公开代码基础上实现。
1)基础网络架构。根据现有对抗蒸馏工作[21-22]中的设置,在CIFAR10数据集中选择WideResNet-34-10(参数量为66.5×106)、WideResNet-34-20 (参数量为125×106)以及WideResNet-28-10(36.5×106)作为教师模型,而在CIFAR100,Tiny ImageNet中教师模型选择为WideResNet-34-10,所有教师模型事先通过TRADES进行预先训练来构建对抗鲁棒能力。学生模型方面,3个数据集中均使用ResNet-18(11.7×106)和MobileNet-v2(3.4×106)作为轻量级模型进行对抗训练。
2)基本训练配置。我们使用SGD动量优化器训练模型,其中初始学习率设置为0.1,动量为0.9,权重衰减为5E−4。遵照主流对抗蒸馏工作[21-22],对抗训练的数据批次大小设置为128。对于PGD-AT,我们采用110个训练周期,并使用提前停止策略[31,35];而对于TRADES和其他自适应对抗方法,我们使用200个训练周期,并且在第100和第150个周期时将学习率缩小为原来的1/10。除非另有说明,内部优化的迭代次数设置为10,步长为2/255,总扰动界限在L∞约束下为8/255。用于对比的对抗蒸馏方法,实验结果依据相关公开代码,并遵从其文章中所推荐的最优参数进行设置。所有实验通过配置有2张NVIDIA RTX4090 显卡的服务器实现,并基于Pytorch框架和advertorch库。
3)评价指标。除了识别准确率(Clean)外,实验还使用了3种对抗防御研究中普遍使用的典型对抗攻击方法FGSM, PGD, CW作为鲁棒性评估指标。同时,我们还采用了一种RobustBench[38]中的评估工具,一种强攻击策略AutoAttack(AA)深入评估对抗鲁棒性。对于FGSM,PGD,AA,最大扰动大小设置为8/255,而PGD采用10个步骤,步长为2/255。CW受到L2范数约束,其平衡常数设置为0.1。除非另有说明,最终模型参数选择基于训练中在PGD-10准确率最高的检查点上的结果。
本节针对在同一教师模型下面向不同对抗防御方法的准确率-鲁棒性效能对比分析。具体地,以WideResNet-34-10为教师模型,ResNet-18和MobileNet-v2为学生模型,所提出的对抗排序蒸馏方法和9种典型对抗防御方法的实验结果如表2所示。
首先,8种对抗蒸馏策略的准确率-鲁棒性效能显著优于PGD-AT和TRADES两种典型对抗训练方法,这说明教师-学生模型知识传递策略是提升小模型鲁棒性的高效范式。其次,我们所提出的对抗排序蒸馏方法在准确率和全部对抗指标达到最优,其中典型鲁棒性指标PGD和AA相比于当前先进的AdaAD和DGAD方法能够取得压倒性优势,这种性能优势可归因于对抗排序蒸馏在鲁棒性知识建模以及传递的高效设计;最后,相比于其他对抗蒸馏策略,本文方法在PGD-10最高检查点PGD(best)和训练结束检查点PGD(last)间测试结果差异较小,这说明对抗排序蒸馏中松耦合约束机制能够有效提升小模型对抗训练过程的稳定性,缓解潜在的对抗过拟合问题。
图3与图4体现了在不同教师-学生模型组合下对抗蒸馏方法的性能比较,其中横坐标为学生-教师模型参数比例(数值越小说明大小模型能力差距越大),而纵坐标为准确率和鲁棒性指标(PGD,AA)。总体而言,对抗排序蒸馏策略在6种教师-学生模型组合下均实现了最优结果,体现了其性能高效以及场景泛用性。同时, 相比于其他对抗蒸馏策略,对抗排序蒸馏方法随教师模型参数增大呈现明显能力上升趋势,说明该方法能够有效缓解因大小模型能力差异导致的学生网络学习难度提高问题,充分发挥教师模型能力优势。最后,表3与表4从不同实验数据集视角对本文提出的第1个研究问题进行分析,本文方法仍有显著的鲁棒性能优势,彰显了该方法的泛用性和性能稳定性。
本节聚焦研究问题2和问题3,即验证所提出对抗排序蒸馏方法中组件的有效性,以及超参数的影响。首先针对问题2,基于CIFAR10数据集以及WideResNet-34-10教师模型,分别对2种不同学生模型下方法的重要组件有效性进行探索。实验结果如表5所示,其中方法组件KL为常规对抗蒸馏方法使用的KL散度约束,SP为典型样本关系感知的知识蒸馏策略,CRD,IRD,DRD分别为本文方法中所涉及的3个核心组件模块,即类感知、实例感知、差异感知排序一致性蒸馏。
首先,由表5中案例1,2,3与案例5的实验结果进行对比可知,相比于主流知识蒸馏的刚性KL散度约束,对抗排序蒸馏中基于优先级排序约束机制下的类感知一致性蒸馏在准确率,以及3种不同对抗攻击下的鲁棒性表现上均呈现压倒性性能优势。尽管KL散度可以通过调整温度系数T进行松弛调整,但各方面效果仍与本文提出的排序约束机制有明显差距。其次,通过案例4和案例7的实验结果可知,引入典型的实例关系蒸馏方法SP对轻量级模型鲁棒性提升十分有限。相反,同样是对训练批次中样本关系进行知识传递,本文中所提出的实例感知排序一致性蒸馏(案例6)充分发挥排序约束的松耦合优势,在类感知的基础上全面提升了小模型准确率和鲁棒性。最后,案例9引入差异感知排序一致性蒸馏后,完整的对抗排序蒸馏方法能够在维持准确率的基础上,充分挖掘教师模型在干净/对抗样本响应间的差异性知识,全面提升学生模型鲁棒性效能。
针对问题3,图5(ResNet-18)和图6(MobileNet-v2)共同讨论了对抗排序蒸馏总训练损失(式(18))中权重调整因子λ,μ的不同取值对于该方法性能的影响,其中将λ,μ分别在{0, 0.25, 0.5, 1, 2, 4}的合理区间内进行动态调整取值。由实验结果可得,当差异感知排序蒸馏参数μ取值在[0, 1]区间时能够维持准确率的前提下大幅提高PGD,AA这2种鲁棒性指标性能,而取值为2时容易降低干净样本准确率,而实例感知蒸馏参数同样在λ=1取得最好实验表现。由此,在进行简单超参数探索并取值λ=μ=1时,本文所提出方法得到最佳效果。
除了轻量级模型鲁棒性,知识蒸馏在教师-学生模型网络结构一致的对抗自蒸馏场景下普遍也被认为是提升对抗训练效果的关键技术,因此本节讨论所提出对抗排序蒸馏方法在对抗自蒸馏任务下是否能保持其优越性能(问题6)。此过程首先用对抗训练得到教师模型,并使用同模型结构的学生网络进行对抗鲁棒知识迁移。本节采用ResNet-18和MobileNet-v2的学生网络框架在CIFAR10数据集上进行自蒸馏对比实验,结果如表6所示。
由于对抗自蒸馏弱化了教师-学生模型间网络参数以及表征能力差距,因此相较于表2实验结果,本文方法在性能方面有所减弱,但准确率和鲁棒性表现仍明显强于AdaAD和DGAD等先进方法,其中PGD,AA指标稳定提升1.5个百分点和1个百分点以上。
由于现实应用中攻击目标模型的网络结构普遍未知其特点,本节从黑盒攻击场景深入探索本文方法的性能优越性和实用性(问题5)。具体地,针对在教师模型WideResNet-34-10上进行对抗蒸馏得到的学生模型ResNet-18以及MobileNet-v2,我们以ResNet34 和 VGG16网络作为元模型对本文方法和相关策略进行迁移攻击,实验结果如表7所示。总体而言,与白盒攻击策略下的实验结果趋势一致,所提出的对抗排序蒸馏方法在3种不同鲁棒性评估指标下均取得了优越性能,PGD性能稳定优于DGAD方法1个百分点以上。
在边缘计算等实际应用中,除了设备模型参数轻量化需求外,训练数据资源受限也是方法设计需考虑的现实问题。为探索对抗排序蒸馏方法在低训练资源场景下的实用性(问题6),本节在学生网络训练数据集仅有原数据集5%,10%,25%样本比例下,以WideResNet-34-10为教师模型,对学生模型ResNet-18以及MobileNet-v2继续训练,实现结果分别如图7与图8所示。
总体而言,尽管训练数据规模受限,相比其他先进对抗蒸馏策略,本文方法在准确率和鲁棒性效能上仍呈现压倒性优势。特别地,本文方法在数据比例5%场景下FGSM,PGD,AA的对抗鲁棒性性能,甚至优于ARD,IAD,RSLAD等方法在10%数据量上的表现。这种显著的鲁棒性优势可归因于2个方面:一是本文方法采用了优先级排序的松耦合且高效的约束机制,能够在模型参数/数据规模差异明显的条件下实现高效的鲁棒知识传递;二是本文方法在优先级排序约束机制的基础上,从类别、实例关系、干净/对抗样本预测差异3个维度协同进行样本鲁棒性知识挖掘,充分提升小模型应对恶意攻击的韧性。
为映证3.3节数学理论分析中对抗排序蒸馏方法在梯度幅值上界和梯度幅值衰减上训练可控的优势,本节以在CIFAR-10数据集上训练ResNet-18学生模型为案例,记录了训练过程中基于常规KL散度的AdaAD和本文方法在训练过程中损失函数衰减曲线,以及PGD对抗鲁棒性上升曲线。如图9所示,本文方法梯度幅值下降更快,总体趋势更平稳,且在后期能够维持较低水平;而比较之下,基于KL散度的AdaAD方法整体训练损失较高,且对抗鲁棒性存在明显劣势,映证了本文方法在数学证明上训练收敛的优越性。
对抗蒸馏方法作为提升轻量级模型对抗防御能力的关键技术,普遍忽视了对抗样本预测类别间关系的挖掘以及大小模型间鲁棒性学习能力的差异。为此,本文提出了一种新颖且高效的轻量级模型鲁棒性提升方法——对抗排序蒸馏,具体涉及2个重要组件:优先级排序约束机制提出了一种基于元素间对比排序实现大小模型间松耦合约束的通用策略;多层次排序一致性蒸馏框架则在此基础上,分别从类别语义关联、样本语义关联以及对抗差异关联3个方面进行高效对抗鲁棒性建模与知识迁移。实验结果表明, 相比于现有先进的对抗蒸馏策略,本文方法能够在稳定维持轻量级模型干净样本准确率的同时显著提升对抗鲁棒性能。同时在训练样本受限、内部优化函数变化以及对抗自蒸馏的实验场景中,对抗排序蒸馏方法具备更强的适应力与泛用性。
下一步工作集中于理论创新方面,所提出的多层次排序一致性蒸馏框架采用动态搜索的方法对类别语义关联、样本语义关联以及对抗差异关联3种不同对抗鲁棒性知识传递进行动态赋权,尽管探索过程简单但仍缺乏部署灵活性,在接下来的研究工作中将进一步考虑设计自适应权重计算策略协同调整3种异质知识蒸馏的贡献度;应用实践方面,当前轻量级模型鲁棒性研究仅聚焦在视觉与图像识别应用领域,接下来会进一步将本文的工作扩展到轻量化本文识别以及语音分析模型的对抗鲁棒性研究中。

参考文献 引证文献
排序方式:
1
窦慧, 张凌茗, 韩峰, 等. 卷积神经网络的可解释性研究综述[J] 软件学报, 2024, 35(1): 159−184
Dou Hui, Zhang Lingming, Han Feng, et al. Survey on convolutional neural network interpretability[J] Journal of Software, 2024, 35(1): 159–184 (in Chinese)
2
LeCun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553): 436−444
3
Rawat W, Wang Zenghui. Deep convolutional neural networks for image classification: A comprehensive review[J]. Neural Computation, 2017, 29(9): 2352−2449
4
Zou Zhengxia, Chen Keyan, Shi Zhenwei, et al. Object detection in 20 years: A Survey[J]. Journal of the IEEE, 2023, 111(3): 257−276
5
Chang Yupeng, Wang Xu, Wang Jindong, et al. A survey on evaluation of large language models[J]. ACM Transactions on Intelligent Systems and Technology, 2024, 15(3): 1−45
6
Alabdulmohsin I, Neyshabur B, Zhai Xiaohua. Revisiting neural scaling laws in language and vision[J]. Advances in Neural Information Processing Systems, 2022, 35: 22300−22312
7
Chakraborty A, Alam M, Dey V, et al. A survey on adversarial attacks and defences[J]. CAAI Transactions on Intelligence Technology, 2021, 6(1): 25−45
8
Ye Shaokai, Xu Kaidi, Liu Sijia, et al. Adversarial robustness vs. model compression, or both?[C/OL]//Proc of the 14th IEEE/CVF Int Conf on Computer Vision. 2019[2025-09-06]. https://openaccess.thecvf.com/content_ICCV_2019/html/Ye_Adversarial_Robustness_vs._Model_Compression_or_Both_ICCV_2019_paper.html
9
Qian Zhuang, Huang Kaizhu, Wang Qiufeng, et al. A survey of robust adversarial training in pattern recognition: Fundamental, theory, and methodologies[J]. Pattern Recognition, 2022, 113: 1−11
10
Pang Tianyu, Yang Xiao, Dong Yinpeng, et al. Bag of tricks for adversarial training[J]. arXiv preprint, arXiv: 2010.00467, 2020
11
Wu Boxi, Chen Jinghui, Cai Deng, et al. Do wider neural networks really help adversarial robustness?[C/OL]//Proc of the 35th Neural Information ProcessingSystems. 2021[2025-09-06]. https://proceedings.neurips.cc/paper/2021/file/3937230de3c8041e4da6ac3246a888e8-Paper.pdf
12
Madry A, Makelov A, Schmidt L, et al. Towards deep learning models resistant to adversarial attacks[J]. arXiv preprint, arXiv: 1706.06083, 2017
13
Micah G, Liam F, Soheil F, et al. Adversarially robust distillation[C/OL]//Proc of the 20th AAAI Conf on Artificial Intelligence. 2020[2025-09-06]. https://ojs.aaai.org/index.php/AAAI/issue/view/252
14
Gou Jianping, Yu Baosheng, Stephen J M, et al. Knowledge distillation: A survey[J]. International Journal of Computer Vision, 2021, 129(6): 1789−1819
15
Abbasi S, Hajabdollahi M, Karimi N, et al. Modeling teacher-student techniques in deep neural networks for knowledge distillation[C]//Proc of the 2nd Int Conf on Machine Vision and Image Processing (MVIP). Piscataway, NJ: IEEE, 2020: 1−6
16
Zi Bojia, Zhao Shihao, Ma Xingjun, et al. Revisiting adversarial robustness distillation: Robust soft labels make student better[C]//Proc of the 16th IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2021: 16443−16452
17
Maroto J, Ortiz-Jiménez G, Frossard G. On the benefits of knowledge distillation for adversarial robustness[J]. arXiv preprint, arXiv: 2203.07159, 2022
18
Zhao Shiji, Duan Ranjie, Wang Xizhe, et al. Improving adversarial robust fairness via anti-bias soft label distillation[C/OL]//Proc of the 38th Int Conf on Neural Information Processing Systems. 2024[2025-09-06]. https://papers.nips.cc/paper_files/paper/2024/file/a2370db7c99791ad5d9f3ef48ad6d464-Paper-Conference.pdf
19
Zhao Shiji, Yu Jie, Sun Zhenlong, et al. Enhanced accuracy and robustness via multi-teacher adversarial distillation[C/OL]//Proc of the 14th European Conf on Computer Vision. 2022[2025-09-06]. https://lin k.springer.com/chapter/10.1007/978-3-031-19772-7_34
20
Zhu Jianing, Yao Jiangchao, Han Bom, et al. Reliable adversarial distillation with unreliable teachers[C/OL]//Proc of the 10th Int Conf on Learning Representations. 2022[2025-09-06]. https://openreview.net/ forum?id=u6TRGdzhfip&amp;referrer=%5BAuthor%20Console%5D(%2Fgroup%3Fid%3DICLR.cc%2F2022%2FConference%2FAuthors%23your-submissions
21
Huang Bo, Chen Mingyang, Wang Yi, et al. Boosting accuracy and robustness of student models via adaptive adversarial distillation[C]//Proc of the 40th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE 2023: 24668−24677
22
Park Hyejin, Min Dongbo. Dynamic guidance adversarial distillation with enhanced teacher knowledge[C]//Proc of the 17th European Conf on Computer Vision. Berlin: Springer, 2025: 204−219
23
Erven V, Harremos T. Rényi divergence and Kullback-Leibler divergence[J]. IEEE Transactions on Information Theory, 2014(7): 3797−3820
24
Guo Chuan, Gardner Jacob, You Yurong, et al. Simple black-box adversarial attacks[C]//Proc of the 12th Int Conf on Machine Learning. New York: PMLR, 2019: 2484-2493
25
Goodfellow I J, Shlens J, Szegedy C. Explaining and harnessing adversarial examples[C/OL]//Proc of the 3rd In Conf on Learning Representations. 2015[2025-09-06]. https://www.researchgate.net/publication/269935591_Explaining_and_Harnessing_Adversarial_Examples
26
Madry A, Makelov A, Schmidt L, et al. Towards deep learning models resistant to adversarial attacks[C/OL]//Proc of the 6th In Conf on Learning Representations. 2018[2025-09-06]. https://www.padl.ws/papers/Paper%2019.pdf
27
Carlini L, Wagner D. Towards evaluating the robustness of neural networks[C]//Proc of the IEEE Symp on Security and Privacy. Piscataway, NJ: IEEE, 2017: 39−57
28
Croce F, Hein F. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks[C/OL]//Proc of the 13th Int Conf on Machine Learning. New York: PMLR, 2020: 2206−2216
29
隋晨红, 王奥, 周圣文, 等. 面向鲁棒学习的对抗训练技术综述[J]. 中国图象图形学报, 2023, 28(12): 3629−3650
Sui Chenhong, Wang Ao, Zhou Shengwen, et al. A survey on adversarial training for robust learning[J]. Journal of Image and Graphics, 2023, 28(12): 3629−3650 (in Chinese)
30
陈思宏, 沈浩靖, 王冉, 等. 预测不确定性与对抗鲁棒性的关系研究[J]. 软件学报, 2022, 33(2): 524−538
Chen Sihong, Shen Haojing, Wang Ran, et al. Relationship between prediction uncertainty and adversarial robustness[J]. Journal of Software, 2022, 33(2): 524−538 (in Chinese)
31
Zhao Shiji, Wang Xizhe, Wei Xingxing. Mitigating accuracy-robustness trade-off via balanced multi-teacher adversarial distillation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(12): 1−14
32
Yin Shenglin, Xiao Zhen, Song Mingxuan, et al. Adversarial distillation based on slack matching and attribution region alignment[C]//Proc of the 41st IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE 2024: 24605−24614
33
Wu Yulun, Lao Mingrui, Guo Yanming, et al. Boosting adversarial robustness distillation via hybrid decomposed knowledge[C]//Proc of the 49th IEEE Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2024: 4715−471
34
Long Teng, Gao Qi, Xu Lili, et al. A survey on adversarial attacks in computer vision: Taxonomy, visualization and future directions[J]. Computers & Security, 2022, 121: 102847
35
Krizhevsky A, Hinton G. Learning multiple layers of features from Tiny Images[J/OL]. Handbook of Systemic Autoimmune Diseases. 2009[2025-09-06]. https://www.semanticscholar.org/paper/ Learning-Multiple- Lay ers-of-Features-from-Tiny-Krizhevsky/5d90f06bb70a0a3dced62 413346235c02b1aa086
36
Benbrahim H, Behloul A. Fine-tuned Xception for image classification on Tiny ImageNet[C/OL]//Proc of the 1st Int Conf on Artificial Intelligence for Cyber Security Systems and Privacy (AI-CSP). 2021[2025-09-06]. https://www.researchgate.net/publication/3577550 47_Fine-tuned_Xception_for_Image_Classification_on_Tiny_ImageNet
37
Zhang Hongyang, Yu Yaodong, Jiao Jiantao, et al. Theoretically principled trade-off between robustness and accuracy[C]//Proc of the 12th Int Conf on Machine Learning. New York: PMLR, 2019: 7472−7482
38
Croce F, Andriushchenko M, Sehwag V, et al. RobustBench: A standardized adversarial robustness benchmark[J]. arXiv preprint, arXiv: 2010.09670, 2020
2026年第63卷第6期
76
36
0
引用本文
BibTeX
文章信息
doi: 10.7544/issn1000-1239.202550339
  • 接收时间:2025-05-23
  • 首发时间:2026-11-17
  • 出版时间:2026-06-01
补充材料
相关文章
文章信息
作者
出版历史
  • 收稿日期:2025-05-23
  • 修回日期:2026-03-04
基金
作者信息
    1国防科技大学系统工程学院 长沙 410073
    2天津大学智能与计算学部 天津 300072
参考文献
分享链接
https://castjournals.cast.org.cn/joweb/jsjyjyfz/CN/10.7544/issn1000-1239.202550339
分享至
全文二维码

扫描看全文

引用本文
BibTeX
本文的引用情况
2种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏