Latest ArticlesWafer-scale computer integrates multiple chiplets through advanced packaging technologies, overcoming traditional chip area limitations to achieve computational power scaling. However, existing domain-specific designs struggle to meet generalized computing requirements. In this study, we propose Yingtian-Lake, which is a wafer-scale general-purpose computer targeting workload characteristics of high-performance computing and intelligent computing scenarios. First, a decoupled computing module-interposer architecture design with standardized I/O interfaces enables multi-modal computing module compatibility. Second, a reconfigurable wafer-scale network employing dynamic topology adaptation technology accommodates diverse traffic patterns. Third, a fault-aware tolerant routing algorithm ensures service continuity during computing unit failures. Experimental results demonstrate that the proposed reconfigurable network achieves second-level topology switching latency. The prototyped 16-module system fabricated with TSMC 28 nm process shows 1.45 times and 1.78 times energy efficiency improvements in high-performance linear algebra computations and deep learning inference tasks respectively, while delivering petaflops-level performance on a single wafer. This breakthrough architecture validates the technical feasibility of universal wafer-scale systems, establishing a scalable hardware foundation for next-generation heterogeneous computing platforms.
Reinforcement learning with human feedback (RLHF) has been proven effective in aligning large language models (LLMs) with human preferences. The most costly part of RLHF is proximal policy optimization (PPO), which consists of three dependent steps. Different PPO steps in RLHF exhibit different computation modes, simply employing the same parallelization strategy to accelerate all steps that involve multiple model variants, as done in existing frameworks, will lead to poor performance in the PPO generation step due to insufficient utilization of computational resources. Thus, we introduce Pipe-RLHF, a parallelism framework for RLHF fine-tuning, which adaptively employs distinct parallelization strategies for different steps according to the computation mode. Specifically, we first investigate the characteristics of various computation modes to explore their best-fit parallelization approach. And then, we present a novel delayed inter-batch pipeline parallelization approach specifically designed for the PPO generation step, enabling the sufficient utilization of computational resources. Subsequently, based on the proposed inter-batch pipeline parallelization approach, we define a hierarchical parallel plan space for distributed RLHF fine-tuning. Finally, we present optimization algorithms to find the optimal parallelization plan from the defined hierarchical parallel plan space to minimize the overall time consumption. Implementation and evaluation across multiple LLMs demonstrates that the proposed Pipe-RLHF achieves 3.7 times speedup compared with existing methods while achieving near-linear scalability.
Vision Transformers (ViTs) have demonstrated remarkable success in computer vision tasks, but their complex architecture and computational demands hinder deployment on edge devices. While post-training quantization (PTQ) is widely adopted for model compression, existing PTQ methods exhibit severe performance degradation in 4-bit ultra-low-bitwidth scenarios. This work systematically addresses two fundamental limitations: 1) spatial mismatch between quantization-sensitive layers (e.g., Softmax) and compute-intensive layers (e.g., linear projections), where quantizing Softmax causes 80% accuracy loss despite contributing merely 8% computational load; 2) non-Gaussian activation distributions with hidden Gaussian-like clustering properties (97% values less than three times z-score). We propose DAQ (divide-and-conquer and adaptive quantization), a hardware-friendly PTQ method. DAQ adpots z-score-driven dynamic partitioning algorithm to separate data into normal-range and abnormal-range groups and quantizes the two groups with connected parameter. DAQ further explores hardware accelerated kernel such as tensor core to speed up quantization ViT models. Experimental results demonstrate that DAQ achieves a maximum improvement of 4.37% in ImageNet Top-1 accuracy under 4-bit quantization. In object detection tasks, its average error margin remains below 0.4% compared with the baseline and achieves a maximum improvement of 8.2%, even surpassing the full-precision model by 0.1% in specific cases, thereby realizing near-lossless low-bit-width quantization. Through hardware implementation optimization, DAQ achieves 43%~86% computational acceleration without significantly increasing computational overhead. This approach establishes a synergistic algorithm-hardware co-optimized quantization deployment paradigm for resource-constrained scenarios, effectively balancing model efficiency and precision retention.
Fully homomorphic encryption (FHE), which enables computation on encrypted data without decryption throughout the entire processing flow, offers a promising solution for privacy preservation in cloud computing and other distributed environments. However, the practical deployment of FHE remains significantly constrained by its high computational complexity, poor data locality, and limited parallelism. Among the core operations in FHE, the number theoretic transform (NTT) plays a pivotal role in determining overall system performance. We target the butterfly computation pattern, which is central to the NTT algorithm, and propose a high-efficiency NTT accelerator architecture based on a dataflow computing model. First, we design an RVFHE extension instruction set tailored for NTT butterfly operations, incorporating custom modular multiplication and modular addition/subtraction units to enhance the efficiency of modular arithmetic. Second, we introduce a novel NTT data reordering scheme, combined with a structured butterfly address generation strategy, to reduce the control complexity and access conflicts associated with cross-row and cross-column data exchanges. Finally, we develop a dataflow-driven NTT accelerator architecture that leverages data dependency-triggered execution to enable efficient on-chip scheduling and data reuse, thereby exploiting instruction-level parallelism to the fullest extent. Experimental results demonstrate that, compared with NVIDIA GPU, the proposed architecture achieves up to 8.96 times speedup and 8.53 times improvement in energy efficiency. Furthermore, compared with state-of-the-art dedicated NTT accelerators, our design delivers a 1.37 times performance gain.
High-performance and intelligent computing applications require massive data. The transfer and storage of data pose challenges for computer systems. Data compression algorithms reduce storage and transmission costs, making themselves crucial for improving system efficiency. Domain-specific hardware design is an effective way to accelerate data compression algorithms. The emerging data compression software, Zstandard, significantly enhances throughput and compression ratio. Zstandard is based on LZ77 compression algorithm, but it has a larger sliding window that increases on-chip storage overhead. Also, it has complex data dependencies and control flow. These features limit the effect of hardware acceleration. To improve the throughput while achieving a similar compression ratio for data compression accelerator in the context of large sliding windows, we propose a cross-layer optimization approach based on algorithm-architecture co-design to develop a novel data compression acceleration architecture, BeeZip2. First, we introduce the MetaHistory match method into the design of the large sliding window parallel Hash table, offering regular parallelism and addressing control flow data dependency. Then, we propose the shared match PE architecture, distributing the large sliding window across multiple processing units to share on-chip memory and reduce overhead. In addition, the Lazy match strategy and corresponding architecture help to fully leverage the resources for a higher compression ratio. Experimental results show BeeZip2 achieves 13.13 GB/s throughput while maintaining the software compression ratio. Compared with single-core and 36-core CPU software implementations, throughput increases by 29.2 times and 3.35 times, respectively. Compared with the baseline accelerator BeeZip, BeeZip2 achieves a 1.26 times throughput improvement and a 2.02 times throughput-per-area enhancement under the constraint of maintaining a higher compression ratio than its software counterpart.
The rapid development of artificial intelligence (AI) and Internet of things (IoT) technologies has increased demands for the computing energy efficiency, and terminal devices are also facing great challenges in terms of hardware resource overhead. In order to solve the problem of energy efficiency, new designs of approximate computing units with low power have been widely studied. In application scenarios such as digital signal processing and image processing, there are a large number of floating-point operations. These applications consume a lot of hardware resources. However, they have certain fault tolerance characteristics and do not have to perform accurate calculations. In this paper, we propose a 32-bit single-precision reconfigurable approximate floating-point multiplier design method based on multiplication to shift addition (MTA) and asymmetric truncation. Firstly, we utilize an efficient and low-power approximate algorithm that transforms multiplication into shift-addition for partial operands. Secondly, to ensure a balance between accuracy and cost, we propose asymmetric truncation processing for the most significant bits (MSBs) and perform precise calculations on the truncated retained parts. The design achieves high precision and low power. A wide design space is generated by combining the MTA algorithm with different bit widths and varying the number of rows in the partial product array after truncation. This allows for various trade-offs between accuracy and cost. Compared with the exact floating-point multiplier, MTA5T5 has an accuracy loss of mean relative error distance (MRED) of only about 0.32%, a power reduction of 85.80%, and an area reduction of about 79.53%. The accuracy loss of MTA3T3 with lower accuracy is about 1.92%, while the power and area decrease to 90.55% and 85.80%, respectively. Ultimately, we test the application of FIR filtering and image processing, and the results show that the proposed design has significant advantages in terms of accuracy.
Network packet processing is a fundamental function of network devices, involving tasks such as packet modification, checksum and Hash computation, mirroring, filtering, and packet metering. As a domain-specific processor, network processor (NP) can provide line-rate performance and programmability for network packet processing. However, due to different design requirement, architecture of NP differs, including single-phase NP and multi-phase NP, posing challenges for NP designers. Existing simulation methods mainly target single NP or single architecture and are not available to explore both of the architectures. We propose Neptune, an analyzing framework for generic network processor microarchitecture modeling and performance simulation. Based on detailed analysis, Neptune adopts multi-phase NP architecture as the hardware model while providing ability to simulate single-phase architecture. Besides, Neptune employs event list mechanism and inter-core queues to support simulation of different data paths and various scheduling strategies in multi-phase NP. Furthermore, Neptune utilizes bulk synchronous parallel graph computing mechanism and takes advantage of both event-driven and time-driven simulation, ensuring accuracy and efficiency. Our experiment shows that Neptune achieves over 95% accuracy in simulating both of the architectures and simulates network processors at a performance of 3.31 MIPS, achieving an order of magnitude improvement over PFPSim. We illustrate the universality and capability of the Neptune simulation framework through three specific cases. Firstly, we evaluate multi-phase and single-phase NP, showing that single-phase NP can achieve up to a 1.167 times performance improvement. Secondly, we optimize the packet parsing module using a programmable pipeline and analyze its performance differences. Finally, we use Neptune to test the performance of the network packet processing engine under different thread counts, providing insights for software and hardware multi-threading optimization.
The Chiplet-integrated chip based on advanced packaging technology offers a number of advantages in terms of manufacturing cost, design efficiency and special customization, etc. We represent a new and effective method of maintaining the performance growth of the chip in the post-Moore era. As an important method of quantitative analysis of architectural design, design space exploration (DSE) can assist designers in comprehending and evaluating the intricate interrelationships between design parameters. However, when applying the traditional DSE method directly to the Chiplet design, it gives rise to issues such as incomplete evaluation, inaccurate simulation, and low efficiency. The solution to these problems is FireLink, an evaluation framework for Chiplet design space exploration. FireLink supports the modelling and simulation of Chiplet microarchitectures and interconnection networks, and is capable of efficiently evaluating performance, power, area and cost metrics. Furthermore, experiments are conducted using ID3 (iterative dichotomiser 3) machine learning algorithm in this framework, which has been demonstrated to effectively improve the efficiency of DSE. In comparison with existing DSE methodologies, FireLink exhibits notable advantages in comprehensiveness of evaluating, completeness of modeling and efficiency of DSE, therefore designers can explore a wider range of design space in a shorter time, so as to select a better Chiplet design scheme.
In the information stage, the importance of data storage lies in ensuring the reliability, consistency, security, and real-time accessibility of information. Erasure codes (EC) play a crucial role in data storage systems due to their ability to minimize storage overhead and handle multiple component failures. However, the process of encoding and decoding EC involves intensive computation, impacting storage system efficiency. This paper focuses on optimizing EC, with a special emphasis on the Galois field (GF) multiplication within multi-layer loops, a time-consuming aspect of EC. We first evaluate the pros and cons of three methods for GF multiplication calculation: the log table searching method, the complete multiplication table searching method, and the shift decomposition method. Subsequently, a 4 b splitting (SP) method is proposed to reduce memory access overhead during table searching in GF(28). We delve into the SP’s analysis and leverage the 64 b modern processor architecture and vector instruction set characteristics to introduce data-level parallelism in multi-layer loops. This involves amplifying data access granularity and implementing single instruction multiple data (SIMD) vectorization. Based on the open-source Intel storage acceleration library (ISA-L), all optimization methods are implemented and tested on Sunway processor and x86 processor. The experimental results show the effectiveness of proposed optimization in improving EC performance across different data scalability scenarios. When compared with the original ISA-L, our optimizations exhibit an average performance speedup of 3.28× on Sunway, 2.36× on x86.
LoongArch ISA (instruction set architecture) introduces new memory accessing instructions with bound-checking to decrease the overhead of memory security check. However, as a new type of memory accessing instruction, the existing GCC (GNU compiler collection) compiler tools cannot support it and thus LoongArch based hardware remains underutilized. Therefore, in this paper, we revise the GCC compiler with the LoongArch memory accessing instructions to optimize the memory security check. Specifically, our work is divided into three parts: 1) designing built-in functions for the memory accessing instructions; 2) improving the RTL (register transfer language) optimizer of GCC to recognize two kinds of semantic patterns of memory accessing instructions with bound-checking, which are non-exception handling and exception handling; 3) implementing a new exception signal SIGBCE for the bound check exception BCE that is raised by CPU in Linux kernel, and implementing the corresponding signal handling function in glibc (GNU C library) to deal with the bound check exception. The experiments on GCC 12.2.0 and Loongson 3C5000L server show that the revised compiler is able to correctly employ the new memory accessing instructions and bring an acceleration of approximately 20% in some security routines. Our work improves the ecosystem of LoongArch and boosts the development of LoongArch ISA. It will also be referential to GCC optimization for the specialized instructions.