Science and Technology Foresight

类别	定义	应用场景	可解释性	满足难度
分类鲁棒性	正确分类的样本经过扰动后仍可被神经网络正确分类	分类神经网络	高	容易
标准鲁棒性	输入扰动后，网络输出的变化范围在用户指定的范围内	一般神经网络	中	一般
利普希茨鲁棒性	神经网络输出的变化范围与输入的扰动范围存在常数约束	一般神经网络	低	困难

类别	定义	应用场景	可解释性	满足难度
分类鲁棒性	正确分类的样本经过扰动后仍可被神经网络正确分类	分类神经网络	高	容易
标准鲁棒性	输入扰动后，网络输出的变化范围在用户指定的范围内	一般神经网络	中	一般
利普希茨鲁棒性	神经网络输出的变化范围与输入的扰动范围存在常数约束	一般神经网络	低	困难

方法	核心思想	训练要素	适用性	实际效果	方法评价
基于数据增强训练	采样	数据集	高	差	简单易行，但采样得到的对抗样本比例低，该方法实际可行性差
基于攻击样本训练	网络攻击	数据集	高	一般	更高效地寻找对抗样本，但需要的采样（迭代）次数多，效率较低
基于网络验证训练	边界过近似	损失函数	中	好	训练过程同时优化网络准确性和最差鲁棒性违背情况，可行性强
利普希茨鲁棒性训练	网络参数约束	损失函数	低	一般	针对利普希茨鲁棒性，损失函数添加正则化项约束利普希茨常数

方法	核心思想	训练要素	适用性	实际效果	方法评价
基于数据增强训练	采样	数据集	高	差	简单易行，但采样得到的对抗样本比例低，该方法实际可行性差
基于攻击样本训练	网络攻击	数据集	高	一般	更高效地寻找对抗样本，但需要的采样（迭代）次数多，效率较低
基于网络验证训练	边界过近似	损失函数	中	好	训练过程同时优化网络准确性和最差鲁棒性违背情况，可行性强
利普希茨鲁棒性训练	网络参数约束	损失函数	低	一般	针对利普希茨鲁棒性，损失函数添加正则化项约束利普希茨常数

Advances and Prospects of Training Methods for Robust Neural Networks

PDF Download

Zhen LIANG ¹ , Wanwei LIU ²^,^† , Taoran WU ³^,⁴ , Dejin REN ³^,⁴ , Bai XUE ³

Science and Technology Foresight | Review and Commentary 2023,2(1): 78-89

fold up

Science and Technology Foresight | Review and Commentary 2023, 2(1): 78-89

Advances and Prospects of Training Methods for Robust Neural Networks

Full

Zhen LIANG¹, Wanwei LIU²^,^†, Taoran WU³^,⁴, Dejin REN³^,⁴, Bai XUE³

Authors

1. Institute of Quantum Information & State Key Laboratory of High Performance Computing, National University of Defense Technology, Changsha 410073, China

2. College of Computer Science and Technology, National University of Defense Technology, Changsha 410073, China

3. State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing 100190, China

4. School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 100190, China

Corresponding author:

^†

Advances and Prospects of Training Methods for Robust Neural Networks

Zhen LIANG¹, Wanwei LIU²^,^†, Taoran WU³^,⁴, Dejin REN³^,⁴, Bai XUE³

Affiliations

1. Institute of Quantum Information & State Key Laboratory of High Performance Computing, National University of Defense Technology, Changsha 410073, China

2. College of Computer Science and Technology, National University of Defense Technology, Changsha 410073, China

3. State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing 100190, China

4. School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 100190, China

Published: 2023-03-20 doi: 10.3981/j.issn.2097-0781.2023.01.006

Outline

Abstract

fold up

In recent years, deep neural networks have developed into important computing models for deep learning, whose robustness is essential for their deployment in safety-critical areas. Therefore, the way to train robust neural networks is a popular issue that has attracted the attention of academia and industry. In this paper, three mainstream classes of training methods for robust neural networks are introduced, i.e., the method based on data enhancement, that based on adversarial training, and the Lipschitz robust training method. Meanwhile, their core ideas, representative work, and the application scope are introduced. Then, the advantages and disadvantages of the training methods in recent years are compared, and in-depth analysis and comparison are carried out when they correspond to key elements in neural network training. The robustness evaluation metrics of neural networks obtained by these training methods are introduced and compared. Finally, hotspots and challenges of robust neural network training are analyzed, and the possible future directions and some suggestions are briefly summarized.

keyword

deep neural network / robustness / neural network training

Abstract

fold up

Key words

deep neural network / robustness / neural network training

Cited Article

梁震, 刘万伟, 吴陶然, 任德金, 薛白. 鲁棒神经网络的训练方法研究进展与前景[J]. 前瞻科技, 2023 , 2 (1) : 5 -143 . DOI: 10.3981/j.issn.2097-0781.2023.01.006

Zhen LIANG, Wanwei LIU, Taoran WU, Dejin REN, Bai XUE. Advances and Prospects of Training Methods for Robust Neural Networks[J]. Science and Technology Foresight, 2023 , 2 (1) : 5 -143 . DOI: 10.3981/j.issn.2097-0781.2023.01.006

fund

fold up

References

fold up

literature

fold up

References Cited literature

Sorting Method：

Year 2023 Volume 2 Issue 1

PDF Download

1977

1139

Cite this article

BibTeX

Article Information

doi: 10.3981/j.issn.2097-0781.2023.01.006

Received：2022-12-24
Published：2023-03-20
Release：2023-03-27

Supplementary materials

Related Articles

文章信息

作者

出版历史

收稿日期：2022-12-24
修回日期：2023-02-01

基金

国家自然科学基金(61872371)

国家自然科学基金(61836005)

国家自然科学基金(62032024)

Authors

1. Institute of Quantum Information & State Key Laboratory of High Performance Computing, National University of Defense Technology, Changsha 410073, China

2. College of Computer Science and Technology, National University of Defense Technology, Changsha 410073, China

3. State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, Beijing 100190, China

4. School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 100190, China

通讯作者:

^†

参考文献

分享链接

https://castjournals.cast.org.cn/joweb/qzkj/EN/10.3981/j.issn.2097-0781.2023.01.006

分享至

全文二维码

扫描看全文

引用本文

BibTeX

本文的引用情况

梁震, 刘万伟, 吴陶然, 任德金, 薛白. 鲁棒神经网络的训练方法研究进展与前景[J]. 前瞻科技, 2023 , 2 (1) : 5 -143 . DOI: 10.3981/j.issn.2097-0781.2023.01.006

表12种不同金属材料的力学参数

科 Family	属数 Number of genus	种数 Number of species	占总种数比例 Percentage of total species (%)	属 Genus	种数 Number of species	占总种数比例 Percentage of total species (%)
鹅膏菌科Amanitaceae	2	11	5.26	鹅膏菌属 Amanita	10	4.78
小菇科 Mycenaceae	2	12	5.74	丝盖伞属 Inocybe	5	2.39
多孔菌科 Polyporaceae	8	14	6.70	蜡蘑属 Laccaria	5	2.39
红菇科 Russulaceae	3	23	11.00	小皮伞属 Marasmius	6	2.87
				小菇属 Mycena	11	5.26
				光柄菇属 Pluteus	5	2.39
				红菇属 Russula	17	8.13
				栓菌属 Trametes	5	2.39

Close Full

BibTeX
EndNote
RefWorks
TxT

Access Statistics

Links

Contact Us

WeChat