Home Archive
Archive
2024 Volume 61 Issue 3  Published: 2024-03-02
    Research Highlights
  • Zhenkun Yang , Chuanhui Yang , Fusheng Han , Guoping Wang , Zhifeng Yang , Xiaojun Cheng
    doi: 10.7544/issn1000-1239.202330835

    Relational database is the key information infrastructure of today’s society. The Internet and digitization have brought high concurrency and massive data. Due to their centralized architectures, the processing power and storage capacity of traditional relational databases are stretched. OceanBase is a distributed relational database based on commodity PC servers. It achieves online horizontal scalability, automatic lossless disaster recovery from data center failure and high-ratio data compression. It has been used in finance, government affairs, telecommunication systems, Internet, etc. We introduce the architecture and some key technologies of OceanBase, including distributed transaction processing, LSM-tree-based storage system and distributed SQL optimizer. In addition, we explain in detail the high availability and data consistency of OceanBase, which can ensure that RPO is 0 and RTO is less than 8 seconds. At the same time, it also introduces OceanBase’s multi-tenant mechanism, which adopts a native multi-tenant design within the cluster to implement multiple independent database services in the cluster. Based on the Sysbench and TPC-H evaluation benchmarks, comparative experimental results show that 1) in a stand-alone mode, the performance of OceanBase is 1.27 times to over 2 times that of MySQL; 2) in a single-master mode, the performance of OceanBase is 1.25 times to nearly 2 times that of MySQL; 3) in a multi-master mode, the performance of OceanBase is 1.09 to 3.1 times that of MySQL, and for complex OLAP queries, the performance of OceanBase is 6 to 327 times that of MySQL.

  • Special Issue on Intelligent Storage and Data Analysis for the Digital Economy
  • Zirui Wang , Dejun Jiang
    doi: 10.7544/issn1000-1239.202330538

    With the rapid increase of memory-intensive applications, memory capacity is playing an increasingly prominent role in application requirements. However, particle density puts constraints on the DRAM memory capacity scalability. The swapping mechanism, known as a common memory-expansion technology, is to temporarily store less-used memory pages in devices to expand memory. In the past, the disk’s read/write speed was the main limit to prevent the wide adoption of the swapping mechanism. In recent years, with the rapid development of ultra-low latency SSDs, the swapping mechanism can take advantage of its low-latency read and write characteristics to improve the efficiency of swapping. The I/O stack of the swapping approach, however, has a significant software overhead with low I/O latency. We analyze and evaluate the Linux swapping mechanism using ultra-low latency SSDs and design Ultraswap, a swapping mechanism based on ultra-low latency SSDs. Ultraswap adds the processing of polling requests to the Linux I/O stack and reduces the I/O merging and scheduling overhead to achieve a lightweight I/O stack. Based on Ultraswap’s I/O stack, the swap-in and swap-out paths of the kernel swapping mechanism are further optimized. By optimizing the handling of faulted pages and direct memory recycling, the time overhead on the critical path of the swapping mechanism is reduced. The results show that Ultraswap can improve the average performance by 19% compared with Linux swapping mechanism; with 20% of local memory, Ultraswap can achieve a 33% performance improvement, effectively reducing the time overhead on the critical path of the swapping path.

  • Special Issue on Intelligent Storage and Data Analysis for the Digital Economy
  • Kaixin Zhang , Yijie Wang , Han Bao , Junhui Kan
    doi: 10.7544/issn1000-1239.202330541

    Nowadays, the increasing demand for cross-cloud collaborative scheduling of storage and computation puts high demands on cross-cloud data access speed. Therefore, cross-cloud data access methods based on data redundancy techniques (erasure coding and multiple-duplicate) with high cross-cloud data access speed are gaining attention. Among them, the cross-cloud data access method based on erasure coding has become a hot research topic because of its low storage overhead and high fault tolerance. In order to improve the data access speed by shortening the transmission time of coded blocks, existing cross-cloud data access methods based on erasure coding introduce caching techniques and optimize the coded data access scheme. However, due to the coarse granularity of cache management and the lack of coordinated optimization of cache management and coded data access scheme, the existing methods suffer from low cache hits, low cache hit efficiency, and high access volume of coded blocks with low transmission speed, which prolong the coded block transmission time. To this end, we first propose an IPFS-based cross-cloud storage system framework (IBCS) that can realize fine-grained cache management based on IPFS data slice management mechanism, and thus can improve cache hits. Then, we propose an adaptive erasure-coded data access method for cross-cloud collaborative scheduling of storage and computation (AECAM) that evaluates the transmission speed of each coded block during data access based on the distribution of coded blocks (including cached coded blocks) and data access nodes, and accordingly formulates a coded data access scheme that can avoid accessing low transmission speed coded blocks. In addition, AECAM identifies coded blocks that are easily selected in the coded data access scheme and have low transmission speed, and caches them near the data access nodes, thus improving both cache hits and hit efficiency. We build a cross-cloud storage system for collaborative scheduling of storage and computation (C2S2) based on IBCS and AECAM. Compared with existing erasure-coded storage systems that introduce caching, experiments in a cross-cloud environment show that C2S2 can improve data access speed by 75.22%−81.29%.

  • Special Issue on Intelligent Storage and Data Analysis for the Digital Economy
  • Minhui Xie , Youyou Lu , Yangyang Feng , Jiwu Shu
    doi: 10.7544/issn1000-1239.202330402

    Emerging deep learning recommendation models (DLRM) have been widely used in modern recommendation systems. The unique embedding layer of DLRM, commonly with tens of trillions of parameters, induces massive irregular access to storage resources, which becomes the performance bottleneck of model inference. Existing inference systems rely on CPU access to embedding parameters on DRAM and SSD. However, we find that this architecture suffers from excessive CPU-GPU communication overhead and redundant memory copies, resulting in increased latency of embedding layers and limited inference performance. In this paper, we propose GDRec, a recommendation model inference system based on the architecture of GPU direct storage access. The core idea of GDRec is to eliminate the CPU from the access path of embedding parameters and let the GPU directly access storage resources with the paradigm of zero copy. For direct access to DRAM, GDRec retrofits the unified virtual addressing feature of CUDA, to allow GPU kernels to issue fine-grained access to host DRAM. GDRec further introduces two optimizations, access coalescing and access aligning, to fully unleash the performance of DRAM access. For direct access to SSD, GDRec implements a lightweight NVMe driver on GPU, allowing GPU to submit I/O commands to read data from SSD to GPU memory directly, without extra copies on DRAM. GDRec also leverages the massive parallelism of GPU to shorten the submission time of I/O commands. Experiments on three public datasets show that GDRec can improve inference throughput by 1.9 times, compared with a highly-optimized recommendation model inference system, NVIDIA HugeCTR.

  • Special Issue on Intelligent Storage and Data Analysis for the Digital Economy
  • Haotian Fang , Chunhua Li , Qing Wang , Ke Zhou
    doi: 10.7544/issn1000-1239.202330543

    Microservice architecture is increasingly favored by cloud applications due to its good scalability and maintainability. Meanwhile, the complex interactions among microservices make it more difficult to detect performance anomalies in the system. Existing methods cannot adequately establish the complex relationship among microservices cross different call paths and their corresponding response time, resulting in low accuracy of anomaly detection and inaccurate root cause positioning. In this paper, we propose a Transformer based microservice performance anomaly detection and root cause positioning method TTEDA (Transformer trace explore data analysis), which constructs a call chain with microservice call sequence and its response time series, then captures the call relationship among microservices via self-attention mechanism, and the correlation between the response time of microservice and its call path is established through an encoder-decoder architecture, thus the normal response time distribution of microservice across different call chains is obtained. Based on the learned normal pattern, TTEDA can achieve accurate call chain anomaly detection and pinpoint the anomalies at the microservice level. Further, TTEDA uses the relationships among microservices and the propagation of anomalies to perform reverse topological sorting on abnormal microservices, achieving accurate and fast root cause localization. The effectiveness of TTEDA is evaluated on the dataset of the open source benchmark microservice system Train-Ticket and AIops Challenge dataset. Compared with similar methods AEVB, Multi-LSTM, and TraceAnomaly, TTEDA has an average precision improvement of 48.6%, 30.2%, and 3.5%, and an average recall improvement of 34.7%, 11.1%, and 4.1%. Compared with the root localization algorithms MonitorRank and TraceAnomaly, the accuracy of root localization is improved by 35.4% and 6.1%.

  • Special Issue on Intelligent Storage and Data Analysis for the Digital Economy
  • Shuai Liu , Ying Qiao , Xiongfei Luo , Yijing Zhao , Hong’an Wang
    doi: 10.7544/issn1000-1239.202330536

    With the continuous development of the industrial Internet of things (IIoT), an increasing number of devices and sensors are being connected to networks, resulting in a massive influx of time series data. The explosive growth of time series data presents new challenges for database management systems: continuous high-throughput data ingestion, low-latency multidimensional data queries, high-performance time series indexing, and cost-effective data storage. In recent years, time series database technology has become a hot research topic in the field of databases. Some scholars have conducted in-depth research on time series database technology, while specialized time series databases have emerged for managing time series data and have been applied in various fields. These databases have become essential components in IIoT. The existing reviews of time series databases primarily focus on the comparison of functionalities and performance, as well as providing selection recommendations for specific domains. There is a lack of research on key technologies such as data persistence, querying, computation, and indexing in time series stores. Additionally, these reviews appeared earlier and lacked research on modern time series database technologies. We conduct a comprehensive investigation and research analysis of both academic research on time series data storage and industrial time series databases. We take a deep dive into four key technologies in time series databases: 1) time series index optimization techniques; 2) in-memory data organization techniques; 3) high-throughput data ingestion and low-latency data query techniques; 4) cost-effective storage techniques for massive historical data. We also analyze and summarize existing TSDB benchmarks. Finally, we present future development directions for the key technologies in time series databases.

  • Artificial Intelligence
  • Cuizhu Bao , Kai Ding , Jianfeng Dong , Xun Yang , Mande Xie , Xun Wang
    doi: 10.7544/issn1000-1239.202220294

    VideoQA (video question answering), which automatically answers natural language question according to the content of videos, is a relatively new research direction in the field of visual language and has attracted extensive attention in recent years. The solution of videoQA task is of great significance for human-computer interaction, intelligent education, intelligent transportation, scenario analysis, video retrieval, and other fields. VideoQA is a challenging task because it requires a model to understand semantic information of the video and the question to generate the answer. In this work, we analyze the difference between VideoQA and ImageQA (image question answering), and summarize four challenges faced by VideoQA relative to ImageQA. Then, the existing VideoQA models are carefully classified according to the research method around these challenges. Following the classifications, we introduce the generation background and focus on the implementation of models and the relationship between different models. After that, the benchmark datasets commonly used in VideoQA are summarized, the performances of current mainstream algorithms on some datasets are introduced in detail, and the comparison, analysis and summary are carried out. Finally, the future challenges and research trends in this field are discussed, which will provide some ideas for further research in the future.

  • Artificial Intelligence
  • Gewei Zhao , Shengquan Xu , Juanying Xie
    doi: 10.7544/issn1000-1239.202220860

    There are tens of thousands of butterfly species. Each butterfly species is closely related to a specific type of plants. It is significant to study butterfly species automatic identification. However, it is very challenging to study butterfly species recognition via the images taken in the field environments. One reason is that there are small number of butterfly species in the existing datasets compared with the reported species in the world. The other reason is that the number of samples (images) of each butterfly species is limited in the datasets. These situations make it challengeable to train a general system for butterfly species identification via machine learning algorithms. In addition, butterfly wings are always folded in the images taken in the field environments, which make it challengeable to learn butterfly classification features, which further make it difficult to study butterfly species recognition using machine learning techniques. Therefore, meta-learning is introduced to address the challenges, and DL-MAML (deep learning advanced model-agnostic meta-learning) algorithm is proposed for identifying the butterfly species using available images taken in the field environment. DL-MAML introduces a L2 regularization term to the objective function and the parameter updating of the meta-learning MAML while introducing ResNet34 deep learning model to extract butterfly features and plus two more convolution layers to the MAML model to avoid over-fitting. The extensive experiments including ablation experiments and comparison with available meta-learning models demonstrate that the initial model parameters obtained by DL-MAML algorithm are effective to identify new species butterflies. DL-MAML is effective in identifying butterfly species using images taken in the field environments, which make it possible to construct the general and complete butterfly species identification system using available butterfly images from limited categories.

  • Artificial Intelligence
  • Songshen Han , Songhui Guo , Kaiyong Xu , Bo Yang , Miao Yu
    doi: 10.7544/issn1000-1239.202221034

    At present, adversarial attacks on speech recognition models have typically involved adding noise to the entire speech signal, resulting in a wide perturbation range and introducing high-frequency noise. Existing research has attempted to reduce the perturbation range by designing optimization targets. However, controlling the transcription result requires adding perturbations to each frame, thus limiting further reduction in perturbation range. To address this issue, we propose a novel approach that examines the feature extraction process of speech recognition systems from a frame structure perspective. The study finds that framing and windowing determine the distribution of critical regions within the frame structure. Specifically, the weight of adding perturbation to each sampling point within the frame is influenced by its location. Based on the results of perturbation analysis on input features, we partition regions with shared attributes. Then we propose the adversarial example space measurement method and evaluation index to quantify the weight of sampling points for adversarial examples generation. We conduct cross-experiments by adding perturbations at different intervals within the frame, which enables us to identify key regions for perturbation addition. Our experiments on multiple models demonstrate that adding adversarial perturbation to vital regions can narrow the perturbation range, and provide a new perspective for generating high-quality audio adversarial examples.

  • Artificial Intelligence
  • Ao Li , Yongxin Ge , Huijun Liu , Chunhua Yang , Xiuzhuang Zhou
    doi: 10.7544/issn1000-1239.202220795

    To address the challenges of using high-resolution pavement images as input for existing convolutional neural network models and the inability of existing preprocessing algorithms to effectively perceive and retain information from low-ratio distress regions in original pavement images, a novel architectural unit called adaptive perception module (APM) paying greater attention to pavement distress region is proposed with the help of visual interpretation techniques, which achieves a rapid and accurate detection of pavement distress in high-resolution images and could be used to build a software system for automatic detection of pavement distress based on computer vision. Firstly, big kernel convolution and residual operations are used to reduce the origin image resolution and get the low-level but rich feature representation. Secondly, attention mechanism is developed to perceive and activate the region of pavement distress and filter the irrelevant background pixel noise. By means of joint learning, APM training could be completed without additional cost. After the visual interpretation method is used to aid the selection and design of the specific structure of APM, experimental results on the latest public dataset CQU-BPMDD show that the proposed APM significantly improves the classification accuracy, up to 84.47%. Experiments across different datasets CQU-BPDD demonstrate the generalization and robustness of APM. Code is available on https://github.com/Li-Ao-Git/apm.

  • Artificial Intelligence
  • Chunhong Liu , Weili Li , Jie Jiao , Jingxiong Wang , Junna Zhang
    doi: 10.7544/issn1000-1239.202220796

    Based on feature selection and model interpretable method, a cloud platform task termination state prediction model with strong interpretability is constructed. The model visualizes the mapping relationship between static and dynamic attributes of tasks/jobs and termination status, then finds out the mapping mechanism between the load characteristics and the task termination states. The workload monitoring log published by Google is used, and the task dynamic information in the cloud platform is added. Shapley Additive explain (SHAP) is used to find out the importance of the influence of static and dynamic attributes on the termination state, and the results of task termination states prediction model modeling are explained by using the importance of variables combined with SHAP value and XGBoost model. Visualization technology is used to show how load characteristics affect the model’s prediction of different task termination states. The average value of the absolute value of SHAP is used to measure the importance of features, and the global visualization of the importance of features in different termination states is realized. According to the results, 20 variables that have great influence on the prediction model of task termination states are selected as the basis of feature selection. How the change of characteristics affects the different termination states of tasks is visualized. From the visualization results, it can be seen that in the process of task running, different eigenvalues of each feature have influences on the termination states of the task, and different eigenvalues have different influences on the termination states. Feature selection combined with model interpretable method is applied to the construction process of task termination states prediction model, which can help to build a task termination states prediction model with high classification performance and easy understanding. By exploring the mapping mechanism between load characteristics and task termination status, the scheduling mechanism of cloud platform can be optimized.

  • Graphics and Image Processing
  • Wenfa Qi , Yuxin Liu , Zongming Guo
    doi: 10.7544/issn1000-1239.202220797

    Nowadays, digital cameras and smart phones have played an increasingly important role in daily lives, and have become the main tools for people to perceive the world, record information and communicate with each other. When these devices are used to shoot electronic screens, the irregular Moiré patterns in the image are produced due to the overlap of the display devices and digital grids of the camera sensor, which seriously affects the visual quality of the captured images. Therefore, the removal of Moiré pattern is of great significance for the post-processing of captured images. In this paper, recent research about Moiré removal is reviewed in detail, and the existing methods are classified into two categories according to different application scenarios and technical implementations: prior knowledge based Moiré removal and deep learning based methods respectively. According to different training data acquisition and alignment methods, deep learning based Moiré removal techniques can be divided into convolutional neural network (CNN) based methods and generative adversarial network (GAN) based methods. Then, based on the same public dataset, we implement the mainstream deep learning based Moiré removal algorithms, compare and analyze their performance, and summarize the advantages and disadvantages of various methods. Finally, possible directions for future research are discussed.

  • Graphics and Image Processing
  • Zhenyan Wang , Shengcheng Jiang , Qihong Song , Bo Liu , Xiuli Bi , Bin Xiao
    doi: 10.7544/issn1000-1239.202220623

    Cultural relics are prone to partial losses of structure or texture due to improper preservation. In order to restore the original image of cultural relics, we propose a new method for restoring cultural relics by using Transformer’s global structure understanding ability to restore the coherent structure of cultural relics and using convolutional neural networks’ local texture understanding ability to restore the delicate texture of cultural relics. To achieve this goal, the restoration work is divided into two steps: the first step is to use Transformer to reconstruct the rough image and restore the coherent structure; the second step is to use a convolutional neural network to enlarge the rough image and restore the fine texture of the missing area. Considering that there is no high-quality, large-scale heritage database in China and abroad, a new heritage image database is also proposed. The experimental results show that the restoration results outperform the current image restoration algorithms in both subjective and objective evaluations in both breakage restoration experiments and large-area breakage restoration experiments that match realistic scenes. At the same time, the proposed method supports diversified output, which provides diverse references for restorers and improves the restoration efficiency.

  • Graphics and Image Processing
  • Qidong Zhang , Jing Chi , Yuyan Chen , Caiming Zhang
    doi: 10.7544/issn1000-1239.202220812

    In the field of image defogging, it is difficult for most defogging models to maintain a balance between accuracy and efficiency. Specifically, high-precision models are often accompanied by complex network structures, and simple network structures often lead to low-quality results. To address the problem, we propose a multi-branch defogging network based on fog concentration classification and dark and bright channel priors. The model uses the defogging networks with different complexity to handle the images with different fog concentrations, which significantly raises the computational efficiency under ensuring the defogging precision. The model is composed of a lightweight foggy image classifier and a multi-branch defogging network. The classifier divides the foggy images into light, medium and dense foggy images and outputs the fog concentration labels. The multi-branch network contains three branches with the same structure but different widths that process three types of fog images separately. We propose a new fog concentration classification method and a new fog concentration classification loss function. The function combines the dark channel characteristics and defogging difficulty of the foggy image with the defogging precision and computational efficiency of the model, so as to obtain a reasonable fog concentration classification, and consequently achieve a good balance of defogging quality and computing power requirements. We propose a new dark channel prior loss function and a new bright channel prior loss function to constrain the multi-branch defogging network, which effectively enhances the defogging precision. Extensive experiments show that the model is beneficial to get better defogging effect with lower network parameters and complexity.

  • Computer Architecture
  • Hao Kong , Wenyan Lu , Yan Chen , Guihai Yan , Xiaowei Li
    doi: 10.7544/issn1000-1239.202220789

    For sort acceleration on FPGA, the selection and optimization of various performance metrics, such as latency, throughput, power efficiency, hardware utilization and bandwidth efficiency, etc., are of critical importance. We compare the evolution of performance-driven sort acceleration, with advances in larger data size, more data types, more algorithm support, hardware-software cooperation and new hardware-based design; we analyze the problems and optimization strategies faced at different stages of design, implementation, testing and so on. Among the numerous sorting algorithms, merge sort becomes mainstream due to its excellent hardware parallelism, scalability and simple control logic. Sort acceleration is an architectural design that is deeply tied to specific application scenarios. We analyze the architectural adjustments made from the perspective of database system acceleration for resource competition, data arrangement, unique operations and diversity of user requests problems faced in databases. At last, to address the problems and shortcomings of existing studies, we provide an outlook on future directions in terms of distributed sort acceleration for very large data scale, the introduction of new hardware devices such as data processing unit, and the improvement of auxiliary tool chains such as high level synthesis to drive the iterative update of sort acceleration design.

  • Software Technology
  • Junlu Wang , Guiyue Zhang , Likuan Du , Su Li , Tingwei Chen
    doi: 10.7544/issn1000-1239.202220739

    Master-slave blockchain is a novel information processing technology that is domain-oriented and uses efficient cryptography principles for trustworthy communication and storage of big data. With the exponential growth of the scale of domain data, the existing master-slave blockchain system has increasingly serious problems such as low query efficiency and long traceability time. To address these issues, we propose a multi-level index construction method for master-slave blockchain (MSMLI). Firstly, MSMLI introduces a weight matrix and partitions the entire master-slave blockchain based on the master chain structure, and the weight of each partition is assigned. Secondly, for the master blockchain in each partition, a master chain index construction method based on jump consistent Hash (JHMI) is proposed, which takes the key value of the nodes and the number of index slots as input and outputs the master chain index. Finally, a Bloom filter is introduced to improve the column-based selection function and a secondary composite index on the subordinate blockchain corresponding to each master block is built. Experimental results on three constraint conditions and two types of datasets demonstrate that the proposed method reduces the index construction time by an average of 9.28%, improves the query efficiency by 12.07%, and reduces the memory overhead by 24.4%.