收藏切换
Performance Modeling and Optimization for Large-Scale Heterogeneous Consistency Integrated Computing System
收藏切换
Rengang Li1, Yinan Tang2, *, Zhenhua Guo2, Li Wang2, Zan Zong1, Guangwen Yang1
Journal of Computer Research and Development | 2025, 62(6) : 1363 - 1379
Less
收藏切换
Journal of Computer Research and Development | 2025, 62(6): 1363-1379
• High Performance Computing System •
Performance Modeling and Optimization for Large-Scale Heterogeneous Consistency Integrated Computing System
Full
Rengang Li1, Yinan Tang2, *, Zhenhua Guo2, Li Wang2, Zan Zong1, Guangwen Yang1
Affiliations
  • 1Department of Computer Science and Technology, Tsinghua University, Beijing 100084
  • 2IEIT SYSTEMS Co., Ltd., Jinan 250101
  • born in 1980. PhD candidate, senior engineer. His main research interest includes heterogeneous computing

    born in 1993. PhD. His main research interests include heterogeneous computing and simulation system

    born in 1988. PhD. His main research interests include computer system architecture and heterogeneous computing

    born in 1989. Master. Her main research interests include heterogeneous computing and artificial intelligence

    born in 1994. PhD. His research interest includes performance acceleration of distributed deep learning systems and large-scale data processing systems

    born in 1963. PhD, professor, PhD supervisor. His main research interest includes high performance computing

About Author:

born in 1980. PhD candidate, senior engineer. His main research interest includes heterogeneous computing

born in 1993. PhD. His main research interests include heterogeneous computing and simulation system

born in 1988. PhD. His main research interests include computer system architecture and heterogeneous computing

born in 1989. Master. Her main research interests include heterogeneous computing and artificial intelligence

born in 1994. PhD. His research interest includes performance acceleration of distributed deep learning systems and large-scale data processing systems

born in 1963. PhD, professor, PhD supervisor. His main research interest includes high performance computing

Published: 2025-06-02 doi: 10.7544/issn1000-1239.202550120
Outline
收藏切换

With the widespread adoption and development of large-scale artificial intelligence applications, the demand for computing power in artificial intelligence from both industry and academia is increasing. Heterogeneous consistency integrated computing systems, which combine heterogeneous computing technology with cache consistency technology, are gradually becoming an important solution for building intelligent computing centers in the future. However, due to the immaturity of heterogeneous computing and consistency interconnect technologies, it is hard for existing research to model the performance of such systems, making it difficult for researchers to evaluate construction schemes, predict computing performance, and assess system optimization methods at a low cost. We propose HCSim, a performance modeling tool for heterogeneous consistency integrated computing systems, addressing challenges in modeling system topology and inaccuracies in workload modeling within consistency systems. HCSim provides researchers with a flexible, low-cost, and efficient modeling and simulation tool for evaluating interconnect topologies and AI computing tasks. Using HCSim, we model a heterogeneous consistency integrated computing system with thousands of accelerators, and simulate the data-parallel distributed training task of the LLAMA2-13B large language model (LLM) on this system, exploring the impact of variables such as heterogeneous computing power distribution, bandwidth, latency, and task scale on system performance and task execution efficiency. Furthermore, we also design optimization strategies for the communication issues in heterogeneous consistency integrated computing systems and validate the effectiveness of these strategies using HCSim. The simulation results show that HCSim not only meets the performance modeling needs of heterogeneous consistency integrated computing systems, but can also be applied to evaluate and verify optimization strategies for such systems.

artificial intelligence  /  high performance computing  /  modeling and simulation  /  distributed computing  /  cache consistency  /  heterogeneous computing
李仁刚, 唐轶男, 郭振华, 王丽, 宗瓒, 杨广文. 大规模异构一致性融合计算系统的性能建模与优化. 计算机研究与发展, 2025 , 62 (6) : 1363 -1379 . DOI: 10.7544/issn1000-1239.202550120
Rengang Li, Yinan Tang, Zhenhua Guo, Li Wang, Zan Zong, Guangwen Yang. Performance Modeling and Optimization for Large-Scale Heterogeneous Consistency Integrated Computing System[J]. Journal of Computer Research and Development, 2025 , 62 (6) : 1363 -1379 . DOI: 10.7544/issn1000-1239.202550120
Year 2025 volume 62 Issue 6
127
58
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202550120
  • Receive Date:2025-03-01
  • Online Date:2026-11-10
  • Published:2025-06-02
Article Data
Affiliations
History
  • Received:2025-03-01
  • Revised:2025-04-08
Affiliations
    1Department of Computer Science and Technology, Tsinghua University, Beijing 100084
    2IEIT SYSTEMS Co., Ltd., Jinan 250101
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202550120
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT