Latest ArticlesThe number of large-scale language model parameters is growing exponentially, which puts forward higher requirements on the arithmetic density and communication efficiency of the acceleration card, and promotes the rapid development of new architectures, such as single-card multi-core, multi-chip and multi-communication entities. Sunway AI acceleration card adopts four-core group on-chip Ring bus architecture, but in the large model training, the data communication volume is large, and the traditional Ring collection communication method faces the core bottlenecks such as the double limitation of single-core group memory capacity and transmission bandwidth, low collection communication efficiency, and the inability to overlap the communication and computation. In this paper, the optimization framework of SW-IntraCC (Sunway-intra collective communication) is proposed by adopting the concept of software-hardware collaborative design to break through the above limitations through the three-tier storage architecture. First, the three-tier storage architecture is constructed based on on-chip high-speed Ring network, which expands the memory capacity of a single core group by up to four times and increases the host-accelerator card transmission bandwidth by 2.5 times; Second, an intra-chip cross shared communication (CSC) algorithm is designed with interleaved memory access patterns, implementing CSC-AG (CSC-AllGather) and CSC-RS (CSC-ReduceScatter) operators optimized for large model training. Benchmark results demonstrate that CSC achieves 2.15 times higher communication efficiency compared with conventional collective primitives. Finally, a bidirectional operator fusion strategy is proposed to enable communication-computation overlap, yielding a 59% improvement in communication performance after optimization.
The exact diagonalization (ED) method, a key numerical technique in quantum and condensed matter physics, directly computes the ground state of quantum systems. This study leverages Hamiltonian matrix symmetry, matrix-free methods, hierarchical communication models, and a data-level parallel algorithm optimized for the MT-3000 architecture. We propose a heterogeneous parallel algorithm for large-scale sparse Hamiltonian matrix-vector multiplication on the new generation Tianhe supercomputing system, enabling large-scale exact diagonalization for the one-dimensional Hubbard model. Tests on the Tianhe new generation system show a strong scaling efficiency of 55.27% when scaling from 256 to 8192 processes for a 140-billion-dimension matrix. Weak scaling efficiency remains above 51.25% when scaling from 64 to 13 740 processes for a 730-billion-dimension matrix.
With the widespread adoption and development of large-scale artificial intelligence applications, the demand for computing power in artificial intelligence from both industry and academia is increasing. Heterogeneous consistency integrated computing systems, which combine heterogeneous computing technology with cache consistency technology, are gradually becoming an important solution for building intelligent computing centers in the future. However, due to the immaturity of heterogeneous computing and consistency interconnect technologies, it is hard for existing research to model the performance of such systems, making it difficult for researchers to evaluate construction schemes, predict computing performance, and assess system optimization methods at a low cost. We propose HCSim, a performance modeling tool for heterogeneous consistency integrated computing systems, addressing challenges in modeling system topology and inaccuracies in workload modeling within consistency systems. HCSim provides researchers with a flexible, low-cost, and efficient modeling and simulation tool for evaluating interconnect topologies and AI computing tasks. Using HCSim, we model a heterogeneous consistency integrated computing system with thousands of accelerators, and simulate the data-parallel distributed training task of the LLAMA2-13B large language model (LLM) on this system, exploring the impact of variables such as heterogeneous computing power distribution, bandwidth, latency, and task scale on system performance and task execution efficiency. Furthermore, we also design optimization strategies for the communication issues in heterogeneous consistency integrated computing systems and validate the effectiveness of these strategies using HCSim. The simulation results show that HCSim not only meets the performance modeling needs of heterogeneous consistency integrated computing systems, but can also be applied to evaluate and verify optimization strategies for such systems.
Large language models with hundreds of billions of parameters are driving rapid technology innovations and business model transformations in artificial intelligence and heterogeneous computing. However, training such models requires prolonged occupation of extensive hardware resources, thus often incurring diverse high-frequency software/hardware failures. These failures not only are challenging to diagnose, but also lead to much longer training time due to unwanted computation waste and slow training convergence. Resilio, an elastic fault-tolerant system for training large language models is proposed, to provide an efficient automated fault recovery mechanism. It is designed to target multiple typical failure scenarios during training processes, such as network interruptions, node crashes, and process failures. Leveraging the characteristics of the parallel model training strategies and underlying hierarchical storage architectures, Resilio implements multi-layer optimizations on checkpoint read/write operations and Just-In-Time (JIT) recovery mechanisms. For models with 100 billion scale parameters, Resilio reduces the end-to-end recovery time under 10 minutes, while reducing the re-started computation after interruptions to the cost of a single training iteration. Upon variations of the computation resources, Resilio can quickly identify the cluster configurations to enable optimal parallel training strategies. Combined with the fault-tolerant scheduling capability, the system ensures adaptive and elastic resource allocations to greatly improve training efficiency and boost GPU utilization across large-scale computing clusters.
Using edge workers to train deep learning recommendation model (DLRM) in edge intelligent computing system brings several benefits, particularly in terms of data privacy protection, low latency and personalization. However, due to the huge size of embedding tables, typical DLRM training frameworks adopt one or more parameter servers to maintain global embedding tables, while leveraging several edge workers to cache part of them. This incurs significant transmission cost for embedding transmissions between workers and parameter servers, which can dominate the training cycle. In this paper, we investigate how to dispatch input embedding samples to appropriate edge workers to minimize the total embedding transmission cost when facing edge-specific challenges such as heterogeneous networks and limited resources. We develop ESD, a novel mechanism that optimizes the dispatching of input embedding samples to edge workers based on expected embedding transmission cost. We propose HybridDis as the dispatch decision method within ESD, which combines a resource-intensive optimal algorithm and a heuristic algorithm to balance decision quality and resource consumption. We implement a prototype of ESD using C++ and Python and compare it with state-of-the-art mechanisms on real-world workloads. Extensive experimental results show that ESD reduces the embedding transmission cost by up to 36.76% and achieves up to 1.74 times speedup in end-to-end DLRM training.
Traditional hierarchical federated learning (HFL) encounters significant challenges in real world due to device heterogeneity, data heterogeneity (e.g., variations in data volume and feature distribution), and communication resource constraints. Device heterogeneity results in inefficient cross-device collaboration during model training, whereas data heterogeneity induces accuracy degradation and diminished generalization capabilities in the global model. To address these limitations while maximizing the utilization of computation, communication, and data resources in the heterogeneous edge networks, we propose an adaptive synergistic method for hierarchical federated learning. This method synergistically integrates model partitioning and client selection under hardware resource constraints, communication bottlenecks, and non-independent and identically distributed (Non-IID) data conditions to accelerate federated learning training while enhancing model accuracy and adaptability across heterogeneous environments. To quantify the influence of local datasets on global model convergence, a data contribution metric is introduced to evaluate the consistency of client contributions. Furthermore, by integrating deep reinforcement learning (DRL) with real-time resource monitoring and data contribution quantification, the DRL agent dynamically optimizes client selection and edge-cloud collaborative model partitioning strategies prior to each training iteration. This adaptive mechanism leverages system resource availability (e.g., bandwidth, device status) and local data contribution scores to derive optimal policies, thereby accelerating training convergence and enhancing global model accuracy. Simulation results demonstrate that the proposed method achieves significant improvements in model accuracy and training efficiency compared with baseline methods, while exhibiting robust adaptability across diverse heterogeneous environment configurations.
Many high performance computing (HPC) applications such as solving sparse linear systems involve the calculation of sequences of sparse matrix-vector multiplications (SpMV) like Ax, A2x, …, Asx, which is known as the sparse matrix power kernel (MPK). Since MPK calls SpMV using the same sparse matrix, reusing the matrix elements in cache, instead of repeatedly loading them from main memory, can potentially alleviate the memory-constrained problem for SpMV and enhance the performance of MPK. However, reusing the matrix introduces data dependencies between subsequent SpMVs. Prior work mainly either focuses on optimizing an individual SpMV invocation, or introduces significant overheads for cache data reuse in MPK. We propose a cache-aware MPK (Ca-MPK) to optimize MPK via cache data reuse. Based on the dependency graph of a sparse matrix, an architecture-aware recursive partitioning of the graph is designed to obtain subgraphs/submatrices which are fit into cache. Separating subgraphs (i.e., separators) are constructed to decouple data dependencies among subgraphs. Then cache data reuse is achieved by executing sequences of SpMVs on subgraphs with a specified order. Performance evaluation demonstrates that Ca-MPK outperforms the MPK implementations based on the Intel OneMKL library and the state-of-the-art approach, with an average speedup of up to about 1.57 times and 1.40 times, respectively.
Sparse matrix-vector multiplication (SpMV) is a core operation in numerical computations, widely used in scientific computing, engineering simulations, and machine learning. The performance optimization of SpMV is mainly constrained by irregular sparse patterns, with traditional optimizations relying on manually designed storage formats, computation strategies, and memory access patterns. Existing tensor compilers such as TACO and TVM use domain-specific languages (DSL) to generate high-performance operators, alleviating developers from the burdensome manual optimization tasks. However, the optimization support for sparse computations remains insufficient, making it challenging to adaptively optimize performance for different sparse patterns. To address these issues, we propose a sparse compiler framework called SparseMode, which generates efficient vectorized code for SpMV computations based on the matrix’s sparse pattern and adaptively adjusts optimization strategies according to the characteristics of the hardware platform. The compiler framework first designs a domain-specific language, SpMV-DSL, which efficiently and concisely expresses sparse matrices and computational operations in SpMV. Then, a sparse pattern-aware method is introduced, which dynamically selects computation strategies based on the matrix storage format and non-zero element distribution defined in SpMV-DSL. Finally, efficient parallel SpMV operator code is generated through sparse pattern analysis and scheduling optimizations, fully utilizing SIMD instructions to enhance performance. Experimental results on different hardware platforms show that SpMV operator code generated by SparseMode achieves up to a 2.44 times speedup compared with the existing TACO and TVM tensor compilers.
Sequential task flow (STF) represents the access to shared data as dependencies between tasks. The STF runtime system achieves asynchronous parallelism through task construction, dependency analysis, and task dependence graph (TDG) generation, as well as task scheduling. The overhead of these three steps directly affects the performance of parallel programs. The current AceMesh runtime system, with STF at its core, employs a single master core architecture and multiple slave cores for execution on SW39000 processor. However, the discrete memory access performance of SW39000 processor is weak, and the composition of fine-grained tasks increases the discrete memory access, making the composition more likely to become a bottleneck. In this regard, we propose an algorithm that uses multiple auxiliary cores to assist the main core in patterning. First, we analyze the parallelism in the dependency analysis and TDG generation process, and implement a multi-core assisted parallel graph construction algorithm, parallelized fatTDG building algorithm with helpers (PFBH), based on the fat task dependency graph fatTDG on SW39000 processor, and optimize it. Secondly, in response to the problem of main memory resource contention among threads, a method for adjusting the resources of the subordinate cores and parameter selection during parallel graph construction and execution is proposed. Finally, experiments are conducted under five typical applications: compared with a single-core serial graph construction system, the acceleration ratio could achieve up to 1.75 times in fine-grained task scenarios; compared with the OpenACC model on SW39000 processor, AceMesh could achieve up to 2 times acceleration.
RISC-V, renowned for its open-source nature and modular design, has achieved remarkable success in embedded systems and is progressively expanding into the high-performance computing (HPC) domain. While RISC-V hardware tailored for HPC, such as the Sophon SG2042 multi-core processors, has demonstrated performance comparable to x86/ARM counterparts, its underdeveloped software ecosystem remains a critical barrier to broader adoption. To address this challenge, we develop RVBT, a process-level dynamic binary translator for RISC-V, designed to bridge the software gap by efficiently porting the mature x86 ecosystem to RISC-V platforms, thereby accelerating RISC-V’s integration into HPC applications. Focusing on the pervasive use of SIMD instructions in HPC programs, we tackle the inefficiencies arising from fundamental differences in programming models between x86 SIMD and RISC-V vector (RVV) extensions. Specifically, x86 SIMD hardcodes data types within opcodes, whereas RVV dynamically configures vtype and mask registers, leading to redundant operations during direct translation. To overcome this, we propose three innovative optimizations to achieve: 1) Redundancy elimination via data type locality. By leveraging the locality of data types in adjacent SIMD operations, we statically analyze and remove redundant configurations of vtype (achieving 100% dynamic elimination rates for csrr and vsetvl, and 56.31% for vsetvli) and mask settings (74.66% elimination rate in floating-point benchmarks). 2) Hybrid translation with on-demand synchronization. We decouple scalar and vectorized floating-point operations, translating x86 SIMD scalar double-precision instructions to RISC-V’s floating-point extensions and reserving RVV for vectorized operations. Data synchronization between scalar and vector registers is optimized through def-use analysis, achieving a 67.35% dynamic synchronization reduction in floating-point benchmarks. Experimental results on SPEC CPU 2006 demonstrate significant improvements on the optimized RVBT, achieving 47.39% and 40.06% of native execution efficiency for integer and floating-point benchmarks, respectively, representing speedups of 1.21 times and 8.31 times over the unoptimized version. RVBT vastly outperforms QEMU (18.84% and 4.81% for integer and floating-point), with floating-point efficiency surpassing QEMU by 8.33 times, highlighting its potential for deployment in specific HPC scenarios. Crucially, these optimizations are architecture-agnostic: The methodology of exploiting data type locality, hybrid instruction translation, and adaptive synchronization apply equally to ARM SIMD (e.g., NEON) to RVV translation, offering a universal framework for cross-ISA binary compatibility. This work provides a pivotal technical foundation for breaking the software ecosystem deadlock and advancing RISC-V’s role in HPC.