收藏切换
Feature-Oriented and Decoupled Network Structure Based Filter Pruning Method
收藏切换
Ruiwen Shi1, Guanghui Li1, Chenglong Dai1, Feifei Zhang2
Journal of Computer Research and Development | 2024, 61(7) : 1836 - 1849
Less
收藏切换
Journal of Computer Research and Development | 2024, 61(7): 1836-1849
• Artificial Intelligence •
Feature-Oriented and Decoupled Network Structure Based Filter Pruning Method
Full
Ruiwen Shi1, Guanghui Li1, Chenglong Dai1, Feifei Zhang2
Affiliations
  • 1School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, Jiangsu 214122
  • 2Jiangsu Bangrong Microelectronics Co., Ltd., Kunshan, Jiangsu 215300
  • born in 1999. Master. His main research interests include model compression and deep learning

    born in 1970. PhD, professor and PhD supervisor. Senior member of CCF. His main research interests include wireless sensor network, model compression and intelligent nondestructive detection technology

    born in 1992. Lecturer. His main research interests include electroencephalogram processing, electroencephalogram analyzing, and model compression

    born in 1982. Master. His main research interests include the hardware acceleration implementation of image processing algorithm and SoC chip design

About Author:

born in 1999. Master. His main research interests include model compression and deep learning

born in 1970. PhD, professor and PhD supervisor. Senior member of CCF. His main research interests include wireless sensor network, model compression and intelligent nondestructive detection technology

born in 1992. Lecturer. His main research interests include electroencephalogram processing, electroencephalogram analyzing, and model compression

born in 1982. Master. His main research interests include the hardware acceleration implementation of image processing algorithm and SoC chip design

Published: 2024-07-01 doi: 10.7544/issn1000-1239.202330085
Outline
收藏切换

Many existing pruning methods for deep neural network models require modifying the loss function or embedding additional variables in the network, thus they can’t benefit from the pre-trained network directly, and complicate the forward inference and training process. So far, most of the feature-oriented pruning work only use the intra-channel information to analyze the importance of filters, which makes it impossible to use the potential connections among channels during the pruning process. To address these issues, we consider the feature-oriented filter pruning task from an inter-channel perspective. The proposed method uses geometric distance to measure the potential correlation among channels, defines filter pruning as an optimization problem, and applies a greedy strategy to find an approximate solution to the optimal solution. The method achieves the decoupling of pruning from network and pruning from training, thus simplifying the pruning task. Extensive experiments demonstrate that the proposed pruning method achieves high performance for various network structures, for example, on CIFAR-10 dataset, the number of parameters and floating point operations of VGG-16 are reduced by 87.1% and 63.7%, respectively, while still has an accuracy of 93.81%. We also evaluate the proposed method using MobileFaceNets, a lightweight network, on CASIA-WebFace large dataset, and the evaluation results show that, when the number of parameters and floating-point operations are reduced by 58.0% and 63.6%, respectively, MobileFaceNets achieves an accuracy of 99.02% on LFW dataset without loss of inference accuracy (The code is available at: https://github.com/SSriven/FOAD).

deep learning  /  model compression  /  model pruning  /  neural network acceleration  /  geometric distance
施瑞文, 李光辉, 代成龙, 张飞飞. 一种基于特征导向解耦网络结构的滤波器修剪方法. 计算机研究与发展, 2024 , 61 (7) : 1836 -1849 . DOI: 10.7544/issn1000-1239.202330085
Ruiwen Shi, Guanghui Li, Chenglong Dai, Feifei Zhang. Feature-Oriented and Decoupled Network Structure Based Filter Pruning Method[J]. Journal of Computer Research and Development, 2024 , 61 (7) : 1836 -1849 . DOI: 10.7544/issn1000-1239.202330085
Year 2024 volume 61 Issue 7
59
28
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202330085
  • Receive Date:2023-02-16
  • Online Date:2026-11-17
  • Published:2024-07-01
Article Data
Affiliations
History
  • Received:2023-02-16
  • Revised:2023-10-12
Affiliations
    1School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, Jiangsu 214122
    2Jiangsu Bangrong Microelectronics Co., Ltd., Kunshan, Jiangsu 215300
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202330085
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT