收藏切换
A Low-Precision Loss Quantification Method for Self-Attention Module
收藏切换
Delü Lin, Kun He*
Journal of Computer Research and Development | 2026, 63(1) : 162 - 175
Less
收藏切换
Journal of Computer Research and Development | 2026, 63(1): 162-175
• Artificial Intelligence •
A Low-Precision Loss Quantification Method for Self-Attention Module
Full
Delü Lin, Kun He*
Affiliations
  • School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan 430074
  • born in 1998. Master. His main research interests include high performance computing, model quantization, inference acceleration for large language models and autonomous driving models

    born in 1972. PhD, professor. Distinguished member of CCF. Her main research interests include adversarial learning, deep representation learning, graph machine learning, and combinatorial optimization

About Author:

born in 1998. Master. His main research interests include high performance computing, model quantization, inference acceleration for large language models and autonomous driving models

born in 1972. PhD, professor. Distinguished member of CCF. Her main research interests include adversarial learning, deep representation learning, graph machine learning, and combinatorial optimization

Published: 2026-01-01 doi: 10.7544/issn1000-1239.202440804
Outline
收藏切换

With the rapid progress of deep learning technology and the continuous exploration of massive datasets, the self-attention module has been widely applied in various fields such as natural language processing, computer vision, and large language models. Although the self-attention module significantly improves the detection accuracy of deep learning models, its huge computational demand makes it particularly difficult to deploy on computing devices with limited computing power. Integer quantization, as one of the key technologies for deploying models on low-power computing chips, faces the problem of high precision loss caused by the structural characteristics of the self-attention module. To address this issue, a thorough analysis of the integer quantization error in the self-attention module is conducted, and two methods, pseudo-softmax vector quantization and block-wise pseudo-softmax vector quantization, are proposed. These two methods aim to significantly improve inference speed while effectively reducing the error caused by integer quantization by performing special integer quantization on the softmax vectors in the self-attention module. Experimental results show that compared with traditional direct quantization methods, the pseudo-softmax vector quantization method can reduce the quantization accuracy loss by 50%, while the block-wise pseudo-softmax vector quantization method can further reduce the accuracy loss by approximately 90%. These results fully demonstrate the effectiveness of the two quantization methods in reducing precision loss, providing strong support for the efficient deployment of the self-attention module on devices with limited computing power.

model quantization  /  self-attention module  /  low-precision loss  /  inference acceleration  /  divide and conquer
林德铝, 何琨. 一种自注意力模块的低精度损失量化方法. 计算机研究与发展, 2026 , 63 (1) : 162 -175 . DOI: 10.7544/issn1000-1239.202440804
Delü Lin, Kun He. A Low-Precision Loss Quantification Method for Self-Attention Module[J]. Journal of Computer Research and Development, 2026 , 63 (1) : 162 -175 . DOI: 10.7544/issn1000-1239.202440804
Year 2026 volume 63 Issue 1
174
72
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202440804
  • Receive Date:2024-10-19
  • Online Date:2026-11-06
  • Published:2026-01-01
Article Data
Affiliations
History
  • Received:2024-10-19
  • Revised:2025-05-19
  • Accepted:2025-06-11
Affiliations
    School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan 430074
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202440804
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT