收藏切换
SW-Inf: Efficient Transformer Inference Method for Sunway AI Accelerator Card
收藏切换
Wenyuan Shen1, Lingtao Tang1, Haotian He1, Tiancheng Fei1, Yulong Zhao1, Lufei Zhang1, Xiaojun Qin1, Xin Liu2, *, Zuoning Chen2
Journal of Computer Research and Development | 2026, 63(3) : 782 - 797
Less
收藏切换
Journal of Computer Research and Development | 2026, 63(3): 782-797
• High Performance Computing •
SW-Inf: Efficient Transformer Inference Method for Sunway AI Accelerator Card
Full
Wenyuan Shen1, Lingtao Tang1, Haotian He1, Tiancheng Fei1, Yulong Zhao1, Lufei Zhang1, Xiaojun Qin1, Xin Liu2, *, Zuoning Chen2
Affiliations
  • 1State Key Laboratory of Mathematical Engineering and Advanced Computing, Wuxi, Jiangsu 214125
  • 2National Parallel Computer Engineering Technology Research Center, Beijing 100083
  • born in 2000. Undergraduate. His main research interests include high performance computing and operating systems

    born in 1994. PhD. His main research interests include machine learning, privacy protection, and intelligent operating systems

    born in 1997. Master. His main research interests include high performance computing and operating systems

    born in 1999. Master. His main research interests include high performance computing and operating systems

    born in 1984. PhD candidate. His main research interests include artificial intelligence and operating system

    born in 1986. PhD, senior engineer. His main research interests include high-performance computing, operating system, and machine learning

    born in 1975. PhD, senior engineer. His main research interests include system security, software vulnerability analysis, and artificial intelligence

    born in 1979. PhD, professor. Her main research interest includes parallel algorithms and applications

    born in 1957. Academician of Chinese Academy of Engineering. Her main research interests include software theory, operating systems, and information security

About Author:

born in 2000. Undergraduate. His main research interests include high performance computing and operating systems

born in 1994. PhD. His main research interests include machine learning, privacy protection, and intelligent operating systems

born in 1997. Master. His main research interests include high performance computing and operating systems

born in 1999. Master. His main research interests include high performance computing and operating systems

born in 1984. PhD candidate. His main research interests include artificial intelligence and operating system

born in 1986. PhD, senior engineer. His main research interests include high-performance computing, operating system, and machine learning

born in 1975. PhD, senior engineer. His main research interests include system security, software vulnerability analysis, and artificial intelligence

born in 1979. PhD, professor. Her main research interest includes parallel algorithms and applications

born in 1957. Academician of Chinese Academy of Engineering. Her main research interests include software theory, operating systems, and information security

Published: 2026-03-01 doi: 10.7544/issn1000-1239.202550153
Outline
收藏切换

Transformer has fundamentally revolutionized various fields of natural language processing through its self-attention mechanism that effectively models long-range contextual relationships. It overcomes traditional neural networks’ limitations in handling long sequences, and has emerged as the mainstream architecture for modern large language models. Currently, Transformer inference systems generally face a shortage of computing power and are commonly confronted with challenges like substantial scheduling latency, concurrency execution difficulties, and severe memory access limitations during deployment. The Sunway AI accelerator card, featuring a domestically-developed next-generation intelligent processor, has successfully empowered diverse edge inference deployments. However, conventional inference frameworks struggle to effectively explore the architectural potential in three critical dimensions: robust system control capabilities, multi-level parallel computing, and rich memory hierarchy. We propose SW-Inf: a high-efficiency Transformer inference method based on the Sunway AI accelerator card. First, an on-chip management processing element (MPE)-orchestrated computation graph scheduling mechanism is proposed to minimize kernel launch latency and inter-process data transfer overhead through graph aggregation and MPE coordination. Second, an LDM-prioritized heuristic tiling strategy is introduced to enable load-balanced parallel computation across processing elements. Finally, a hardware-aware tightly-coupled attention computation implementation is developed to eliminate redundant intermediate data movement between registers and main memory. We implement an end-to-end inference system on the Sunway AI accelerator card. Experimental results demonstrate that compared with the vLLM inference system on NVIDIA V100S which has higher memory bandwidth than the Sunway AI accelerator card, SW-Inf achieves 15% higher throughput and 20% better memory utilization rate.

Sunway AI accelerator card  /  large model inference  /  scheduling optimization  /  parallel strategy  /  operator fusion
沈文渊, 汤凌韬, 何昊天, 费天成, 赵玉龙, 张鲁飞, 秦晓军, 刘鑫, 陈左宁. SW-Inf:基于申威智能加速卡的Transformer高效推理方法. 计算机研究与发展, 2026 , 63 (3) : 782 -797 . DOI: 10.7544/issn1000-1239.202550153
Wenyuan Shen, Lingtao Tang, Haotian He, Tiancheng Fei, Yulong Zhao, Lufei Zhang, Xiaojun Qin, Xin Liu, Zuoning Chen. SW-Inf: Efficient Transformer Inference Method for Sunway AI Accelerator Card[J]. Journal of Computer Research and Development, 2026 , 63 (3) : 782 -797 . DOI: 10.7544/issn1000-1239.202550153
Year 2026 volume 63 Issue 3
213
100
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202550153
  • Receive Date:2025-03-01
  • Online Date:2026-11-06
  • Published:2026-03-01
Article Data
Affiliations
History
  • Received:2025-03-01
  • Revised:2025-09-24
  • Accepted:2025-10-11
Affiliations
    1State Key Laboratory of Mathematical Engineering and Advanced Computing, Wuxi, Jiangsu 214125
    2National Parallel Computer Engineering Technology Research Center, Beijing 100083
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202550153
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT