收藏切换
SW-IntraCC: A Collective Communication Mechanism for the Internals of Sunway AI Acceleration
收藏切换
Yulong Zhao1, Yanqing Gu1, Songtao Tian1, Chunzhi Wu2, Lingtao Tang1, Lufei Zhang1, Xiaojun Qin1, Xin Liu3, Zuoning Chen3
Journal of Computer Research and Development | 2025, 62(6) : 1333 - 1346
Less
收藏切换
Journal of Computer Research and Development | 2025, 62(6): 1333-1346
• High Performance Computing System •
SW-IntraCC: A Collective Communication Mechanism for the Internals of Sunway AI Acceleration
Full
Yulong Zhao1, Yanqing Gu1, Songtao Tian1, Chunzhi Wu2, Lingtao Tang1, Lufei Zhang1, Xiaojun Qin1, Xin Liu3, Zuoning Chen3
Affiliations
  • 1State Key Laboratory of Mathematical Engineering and Advanced Computing, Wuxi, Jiangsu 214125
  • 2Space Engineering University, Beijing 100004
  • 3National Parallel Computer Engineering Technology Research Center, Beijing 100083
  • born in 1984. PhD candidate. His main research interest includes artificial intelligence infrastructure software

    born in 1992. Master. His main research interests include high-performance computing and operating systems

    born in 1983. Bachelor. His main research interests include high-performance computing and operating systems

    born in 1991. PhD candidate. His main research interests include space launch equipment assurance and deep learning algorithms

    born in 1994. PhD. His main research interests include machine learning, privacy protection, and intelligent operating systems

    born in 1986. PhD, senior engineer. His main research interests include high-performance computing, operating systems, and machine learning

    born in 1975. PhD. Full senior engineer. His main research interests include system security, software vulnerability analysis, and artificial intelligence

    born in 1979. PhD, professor. Her main research interest includes parallel algorithms and applications

    born in 1957. PhD, PhD supervisor, Academician of Chinese Academy of Engineering. Her main research interests include software theory, operating systems, and information security

About Author:

born in 1984. PhD candidate. His main research interest includes artificial intelligence infrastructure software

born in 1992. Master. His main research interests include high-performance computing and operating systems

born in 1983. Bachelor. His main research interests include high-performance computing and operating systems

born in 1991. PhD candidate. His main research interests include space launch equipment assurance and deep learning algorithms

born in 1994. PhD. His main research interests include machine learning, privacy protection, and intelligent operating systems

born in 1986. PhD, senior engineer. His main research interests include high-performance computing, operating systems, and machine learning

born in 1975. PhD. Full senior engineer. His main research interests include system security, software vulnerability analysis, and artificial intelligence

born in 1979. PhD, professor. Her main research interest includes parallel algorithms and applications

born in 1957. PhD, PhD supervisor, Academician of Chinese Academy of Engineering. Her main research interests include software theory, operating systems, and information security

Published: 2025-06-02 doi: 10.7544/issn1000-1239.202550143
Outline
收藏切换

The number of large-scale language model parameters is growing exponentially, which puts forward higher requirements on the arithmetic density and communication efficiency of the acceleration card, and promotes the rapid development of new architectures, such as single-card multi-core, multi-chip and multi-communication entities. Sunway AI acceleration card adopts four-core group on-chip Ring bus architecture, but in the large model training, the data communication volume is large, and the traditional Ring collection communication method faces the core bottlenecks such as the double limitation of single-core group memory capacity and transmission bandwidth, low collection communication efficiency, and the inability to overlap the communication and computation. In this paper, the optimization framework of SW-IntraCC (Sunway-intra collective communication) is proposed by adopting the concept of software-hardware collaborative design to break through the above limitations through the three-tier storage architecture. First, the three-tier storage architecture is constructed based on on-chip high-speed Ring network, which expands the memory capacity of a single core group by up to four times and increases the host-accelerator card transmission bandwidth by 2.5 times; Second, an intra-chip cross shared communication (CSC) algorithm is designed with interleaved memory access patterns, implementing CSC-AG (CSC-AllGather) and CSC-RS (CSC-ReduceScatter) operators optimized for large model training. Benchmark results demonstrate that CSC achieves 2.15 times higher communication efficiency compared with conventional collective primitives. Finally, a bidirectional operator fusion strategy is proposed to enable communication-computation overlap, yielding a 59% improvement in communication performance after optimization.

collective communications  /  Sunway AI acceleration card  /  SW-IntraCC  /  communications optimization  /  ring network
赵玉龙, 顾燕卿, 田松涛, 吴春志, 汤凌韬, 张鲁飞, 秦晓军, 刘鑫, 陈左宁. SW-IntraCC:一种面向申威智能加速卡内部的集合通信机制. 计算机研究与发展, 2025 , 62 (6) : 1333 -1346 . DOI: 10.7544/issn1000-1239.202550143
Yulong Zhao, Yanqing Gu, Songtao Tian, Chunzhi Wu, Lingtao Tang, Lufei Zhang, Xiaojun Qin, Xin Liu, Zuoning Chen. SW-IntraCC: A Collective Communication Mechanism for the Internals of Sunway AI Acceleration[J]. Journal of Computer Research and Development, 2025 , 62 (6) : 1333 -1346 . DOI: 10.7544/issn1000-1239.202550143
Year 2025 volume 62 Issue 6
116
55
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202550143
  • Receive Date:2025-03-01
  • Online Date:2026-11-10
  • Published:2025-06-02
Article Data
Affiliations
History
  • Received:2025-03-01
  • Revised:2025-04-08
Affiliations
    1State Key Laboratory of Mathematical Engineering and Advanced Computing, Wuxi, Jiangsu 214125
    2Space Engineering University, Beijing 100004
    3National Parallel Computer Engineering Technology Research Center, Beijing 100083
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202550143
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT