Home Archive
Archive
2024 Volume 61 Issue 4  Published: 2024-04-01
    Architecture
  • Yangyang Feng , Qing Wang , Minhui Xie , Jiwu Shu
    doi: 10.7544/issn1000-1239.202330554

    The large models represented by ChatGPT have attracted a lot of attention from industry and academia for their excellent performance on text generation and semantic understanding tasks. The number of large model parameters has increased tens of thousands of times in three years and is still growing, which brings new challenges to storage systems. First, we analyze the storage challenges of large model training, pointing out that large model training has unique computation patterns, storage access patterns, and data characteristics, which makes traditional storage techniques inefficient in handling large model training tasks. Then, we describe three types of storage acceleration techniques and two types of fault-tolerant techniques. The storage acceleration techniques for large model training include: 1) distributed storage technique based on large model computation patterns designs the partitioning, storage, and transferring strategies of model data in distributed clusters based on the partitioning of large model computation tasks and the dependencies between computation tasks; 2) heterogeneous storage access pattern-aware technique for large model training develops data prefetching and transferring strategies among heterogeneous devices with the predictability of storage access patterns in large model training; 3) large model data reduction technique reduces the data size in the model training process according to the characteristics of large model data. The storage fault-tolerant techniques for large model training include: 1) parameter checkpointing technique stores the large model parameters to persistent storage devices; 2) redundant computation technique computes the same version of parameters repeatedly in multiple GPUs. Finally, we give the summary and suggestions for future research.

  • Architecture
  • Bingzhang Chen , Wei Liu , Xiaoyu Yu
    doi: 10.7544/issn1000-1239.202220818

    Graph application is an important branch in the field of big data. Although graph analysis has more significant performance advantages than traditional relational databases in displaying the relationship between entities, the irregular memory access pattern caused by a large number of random accesses in graph processing destroys the time and space locality of memory access, thus causing great performance pressure on the off-chip memory system. Therefore, how to correctly measure the performance of graph application in memory system is crucial for efficient graph application architecture optimization. As an extension of average memory access time (AMAT), concurrent average memory access time (C-AMAT) takes into account the locality and concurrency of memory access, and can more accurately evaluate and analyze the performance of modern processors in the storage system. However, the C-AMAT model ignores the fact that the lower-level cache layer of the processor accesses serially, which will lead to the inaccuracy of the calculation. At the same time, it is difficult to obtain the parameters required for the calculation due to the “pure miss cycle” and other reasons, which also makes it difficult for C-AMAT to be applied in practice. In order to match the computing model of C-AMAT with the memory access mode in modern computers, we propose parallel C-AMAT (PC-AMAT) and serial C-AMAT (SC-AMAT) based on C-AMAT. PC-AMAT and SC-AMAT respectively extend and characterize the computing model of C-AMAT from the parallel and serial access modes of cache. On this basis, we design and implement a “pure miss cycle” extraction algorithm to avoid the huge hardware overhead caused by direct measurement. The experimental results show that the correlation between PC-AMAT and SC-AMAT, and IPC is stronger than that of C-AMAT in single-core and multi-core mode. Finally, PC-AMAT and SC-AMAT are used to measure and analyze the memory performance of graph application, based on which the optimization strategy of graph application access is proposed.

  • Architecture
  • Yuhong Song , Hsing-Mean Sha Edwin , Qingfeng Zhuge , Rui Xu , Han Wang
    doi: 10.7544/issn1000-1239.202220738

    With the development of AI democratization, deep neural networks (DNNs) have been widely applied to edge devices, such as smart phones and automated driving, etc. Stochastic computing (SC) as a promising technique performs fundamental machine learning (ML) tasks using simple logic gates instead of complicated binary arithmetic circuits. SC has advantages of low-power and low-cost DNNs execution on edge devices with constrained resources (e.g., energy, computation and memory units, etc.). However, previous SC work only designs one group of setting for fixed hardware implementation, ignoring the dynamic hardware resources (e.g., battery), which leads to low hardware efficiency and short battery life. In order to save energy for battery-powered edge devices, dynamic voltage and frequency scaling (DVFS) technique is widely used for hardware reconfiguration to prolong battery life. In this paper, we creatively propose a run-time reconfigurable framework, namely RR-SC, for SC-based DNNs and first attempt to combine hardware and software reconfigurations to satisfy the time constraint of inference and maximally save energy. RR-SC using reinforcement learning (RL) can generate multiple groups of model settings at one time, which can satisfy the accuracy constraints under different hardware settings (i.e., different voltage/frequency levels). The solution has the best accuracy and hardware efficiency trade-off. Meanwhile, the model settings are switched on a backbone model at run-time, which enables lightweight software reconfiguration. Experimental results show that RR-SC can switch the lightweight settings within 110 ms to guarantee the required real-time constraint at different hardware levels. Meanwhile, it can achieve up to 7.6 times improvement for the number of model inference with only 1% accuracy loss.

  • Artificial Intelligence
  • Jieyi Yang , Yihong Dong , Jiangbo Qian
    doi: 10.7544/issn1000-1239.202220933

    Few-shot learning (FSL) aims to learn to get a problem-solving model using a small number of samples. Under the trend of training models with big data, deep learning has gained success in many fields, but realistic scenarios often lack sufficient samples or labeled samples. Therefore, FSL becomes a promising research direction at present. Graph neural networks (GNN) have attracted great attention due to their excellent performance in many applications. In view of this, many methods try to use GNN for FSL. Currently there are few review researches related to FSL methods based on GNN, and there is a lack of division system and introductory work on this type of methods. We systematically compose the current work related to FSL based on GNN. The work outlines the basis and concepts of graph methods for FSL, broadly classifies them into four categories of methods based on node-based feature, edge-based feature, node-pair-based feature and class-level-based feature according to the basic ideas of the models. The research progress of the four methods is introduced as well. Then the experimental results of the commonly used few-shot datasets and representative models on these datasets are summarized, as well as the advantages and disadvantages of each type of methods. Finally, current status and challenges of the graph methods for FSL are introduced, and their future directions are prospected.

  • Artificial Intelligence
  • Ting Li , Fusheng Jin , Ronghua Li , Guoren Wang , Huanzhong Duan , Yanxiong Lu
    doi: 10.7544/issn1000-1239.202220643

    Graph neural network and hypergraph neural network (HGNN) have become research hotspots in the field of collaborative filtering recommendation. However, the interaction between users and projects in actual scenarios is very complex. As a result, there are high-order complex relationships among users, while ordinary graph structures can only express simple paired relationships. Stacking network structures easily leads to excessive smoothness of middle-tier representations, and the ability of user modeling, user similarity discovery and mining in sparse scenarios is weak. At the same time, the complex structure of heterogeneous hypergraph neural network makes the training efficiency of the model low. In the highly sparse data scene represented by WeChat “Search and Search” and other content platforms, the existing models have poor recommendation effect and weak interpretability of user representation for the circle content recommendation task based on the portrait of the user’s group. Therefore, for this kind of task, a new lightweight homogeneous hypergraph neural network model is proposed, which includes three parts: the transformation of user interaction data into hypergraph, the generation of user representation sequence by convolution, and the calculation and filtering of user representation. Firstly, the user-item interaction data is transformed into a homogeneous hypergraph with only user nodes, and the initial value of user representation decoupling sequence is calculated. Then information propagation and sequence values are iteratively generated according to the Laplacian filter matrix of hypergraph, the model structure is simplified by convolution method without activation layer, and the weight matrix is generated for each sequence value according to the proposed mean difference JK attention mechanism. Finally, the coding of user representation is realized by weighted summation and filtering of decoupled sequences, and experiments on real data sets verify the relatively better effect of the proposed model.

  • Artificial Intelligence
  • Mingyang Liu , Ruomei Wang , Fan Zhou , Ge Lin
    doi: 10.7544/issn1000-1239.202221008

    Video question answering requires models to understand, fuse, and reason about the multimodal data in videos to assist people in quickly retrieving, analyzing, and summarizing complex scenes in videos, becoming a hot research topic in artificial intelligence. However, existing methods lack abilities of obtaining the motion details of visual objects in feature extraction, which may lead to false causality. In addition, in data fusion and reasoning, existing methods lack effective active learning ability, making it difficult to obtain prior knowledge beyond feature extraction, which affects the model’s deep understanding of multimodal content. To address these issues, we propose a multimodal knowledge-based active learning video question answering solution. The solution acquires the semantic correlation of visual targets in image sequences and the dynamic relationship with the surrounding environment to establish the motion trajectory of each visual target. Further, static content is supplemented with dynamic content to provide more accurate video feature expression for data fusion and reasoning. Then, the solution achieves self-improvement and logical thinking focus of multimodal information understanding through knowledge auto-enhancement multimodal data fusion and reasoning model, filling the gap in deep understanding of multimodal content. Experimental results show that the performance of our scheme is better than the most advanced video question answering algorithm, and a large number of ablation and visualization experiments also verify the rationality of this solution.

  • Artificial Intelligence
  • Lixiang Xu , Wei Ge , Enhong Chen , Bin Luo
    doi: 10.7544/issn1000-1239.202221004

    Graph representation learning has become a research hotspot in the field of graph deep learning. Most graph neural networks suffer from oversmoothing, and these methods focus on graph node features and pay little attention to the structural features of graphs. In order to improve the representation of graph structural features, we propose a graph classification method based on graph kernel homomorphic network, namely KerGIN. The method first encodes the node features of the graph through graph isomorphism network(GIN), and then uses the graph kernel method to encode the graph structure. The Nyström method is further used to reduce the dimension of the graph kernel matrix. The graph kernel matrix is aligned with the graph feature matrix with the help of MLP, and the feature encoding and structure encoding of the graph are adaptively weighted and fused through the attention mechanism to obtain the final feature representation of the graph, which enhances the ability to express the structural feature information of the graph. Finally, the model is experimentally evaluated on seven publicly available graph classification datasets: compared with the existing graph representation models, KerGIN model is able to improve the graph classification accuracy substantially, and it can enhance the ability of GIN to represent the graph structural feature information.

  • Artificial Intelligence
  • Jiacheng Wang , Kai Wang , Haofen Wang , Wen Du , Zhidong He , Tong Ruan , Jingping Liu
    doi: 10.7544/issn1000-1239.202220999

    On distantly supervised named entity recognition (NER), there are many reinforcement learning based approaches, which exploit the powerful decision-making ability of reinforcement learning to detect noise from the automatically labeled data generated by distant supervision. However, the structures of the policy network models used are typically simple, which results in a weak ability to recognize noisy instances. Furthermore, correct instances are identified at sentence level, resulting in part of the useful information in the sentence being discarded. In this paper, we propose a new reinforcement learning based method for distantly supervised NER, named RLTL-DSNER, which can detect correct instances at token level from noisy data generated by distant supervision, proposing to reduce the negative impact of noisy instances on distantly supervised NER model. Specifically, we introduce a tag confidence function to identify correct instances accurately. In addition, we propose a novel pretraining strategy for the NER model. This strategy can provide accurate state representations and effective reward values for the initial training of the reinforcement learning model. The pre-training strategy can help guide it to update in the right direction. We conduct experiments on four datasets to verify the superiority of the RLTL-DSNER method, gaining 4.28% F1 improvement on NEWS dataset over state-of-the-art methods.

  • Software Technology
  • Sichao Lai , Xiaoying Wu , Yuwei Peng , Zhiyong Peng
    doi: 10.7544/issn1000-1239.202220931

    Index tuning is an important problem in database performance tuning and has been studied consistently by worldwide researchers. Due to the theoretical complexity of index tuning as well as the advent of the big data era, manual tuning by DBA is no longer feasible for modern database systems, hence automated and intelligent solutions have been developed. With the development of machine learning techniques, more and more index tuning solutions have integrated with machine learning techniques for better performance and significant progress has been made recently. In this survey, we formulate the problem of index tuning under a search-based paradigm, and under this context, we analyze the main tasks and challenges of this problem. We categorize relevant studies into three main components of the search-based paradigm, namely the generation of the index configurations’ search space, the evaluation of specific index configurations, and the enumeration or the search of index configurations. Then we discuss and compare the related work in each category. We further identify and analyze new challenges for the online index tuning problem where the workload is ad hoc, dynamic, and shifting. We summarize the existing solutions under the online feedback control loop framework. Finally, we discuss the state-of-the-art index tuning tools. Hopefully, with the thorough discussion and evaluation of current research, this survey can provide insights to interested researchers and conclude with future research directions for index tuning.

  • Software Technology
  • Zheng Wu , Xu Jin , Hong An
    doi: 10.7544/issn1000-1239.202220787

    As a critical component, convolution is frequently applied in deep learning. The parallel algorithms of convolution have always been a popular research topic in high-performance computing. With the rapid development of Chinese homegrown ShenWei-26010 many-core processor in artificial intelligence, there is an urgent demand for high-performance convolutional algorithms on the processor. We propose an efficient convolutional design, that is, the fused Winograd-based convolutional algorithm, towarding ShenWei-26010 architectural characteristics and the computational features of Winograd-based convolution. Unlike the traditional Winograd-based convolutional algorithm that depends on the official GEMM (general matrix multiplication) library interface, the proposed algorithm owns the customized matrix multiplication implementation. The feature makes the execution process of the proposed algorithm visible, which can better adapt to common convolutions in reality. The proposed algorithm is composed of four parts: input Winograd transformation, filter Winograd transformation, core operation, and output Winograd inverse transformation. The four parts are fused together instead of executing each part separately. The core operation can gain the required transformed data in real time. Subsequently, the computational results are transformed inversely to the final output immediately. The fused execution improves the data locality of the proposed algorithm to reduce the memory access overhead significantly. Moreover, We design other optimization methods to enhance the performance of the proposed algorithm, such as merged Winograd-transformed mode, DMA (direct memory access) double buffering, the enhanced usage of on-chip storage, the elastic processing of output data tiles, and instruction reordering. The experiments show the performance of the proposed algorithm is 7.8 times that of the traditional Winograd-based convolutional algorithm on VGG network model. Moreover, we extract the common convolution from multiple typical convolutional neural networks to measure the hardware efficiency. The results show the proposed algorithm can significantly overperform the traditional Winograd-based convolutional algorithm for all the convolution cases. The best performance of the proposed algorithm is 116.21% of the theoretical peak performance of ShenWei-26010 processor, and the average one can reach 93.14%.

  • Software Technology
  • Ruihan Li , Yefan Hou , Yuhui Li , Zhaoyuan Liu , Haihong Zhang , Jingang Liang
    doi: 10.7544/issn1000-1239.202221032

    Dispersed particle fuel is a new type of nuclear fuel that is in shape of small spheres and dispersed in a matrix. It has been widely used in advanced reactors such as high-temperature gas-cooled reactors (HTRs), space reactors and fluoride-salt-cooled high-temperature reactors. This study, taking an HTR and a space reactor as examples, develops a virtual lattice method based on the open-source Monte Carlo code OpenMC to speed up dispersed particle fuel criticality simulation. Parallel tests on the scale of 100000 cores are carried out on Shanhe supercomputing platform. The keff result of the HTR model agrees well with Shidao-Bay nuclear power plant experiment, indicating that the code is of high accuracy. As for the performance of the code, results show that the virtual lattice model is of less memory footprint and higher computational efficiency than the original physical lattice model. The memory-consumption and time-consumption of the HTR virtual lattice model are 0.2% and 82% that of the physical lattice model respectively. And thanks to the simplification of geometry, the virtual lattice model is of higher parallel efficiency. For strong scalability, the parallel efficiency of the virtual lattice model with 10752 cores is 83.4% while that of the physical lattice model is 63.6%. And for weak scalability, the parallel efficiency of the virtual lattice model with 131600 cores is 83.1% while that of the physical lattice model is 66.1%.

  • Software Technology
  • Zhongyi Pei , Lin Liu , Chen Wang , Jianmin Wang
    doi: 10.7544/issn1000-1239.202220794

    Data-driven intelligent software based on machine learning technology is an important means to realize industrial digital transformation. The research and development processes of data-driven intelligent software require the combined use of software requirements engineering, data and domain knowledge engineering, machine learning and so on. This process involves many subjects and roles, making it extremely challenging to clearly explain why and how the domain knowledge, business logic and data semantics relate to each other. Hence, a systematic requirements engineering approach is needed to explicitly address the explainability requirements issues of data-driven intelligence applications. It is still a fast-evolving research field which requires the proper embedding of various domain models and end-to-end machine learning technology fused into a given business processes. A key research question is how to deal with explainability as a core requirement for safety-critical scenarios in industrial, medical and other applications. We provide a research overview on requirements engineering for machine learning applications, in relation to explainability. First, the research status quo, research foci and representative research progress are reviewed. Then, an explainability-centric requirements analysis framework for machine learning applications is proposed, and some open important issues are put forward. Finally, based on the proposed framework, a case study of industrial intelligence application is discussed to illustrate the proposed requirements analysis methodological framework.

  • Software Technology
  • Hongjing Guo , Chuanqi Tao , Zhiqiu Huang
    doi: 10.7544/issn1000-1239.202220745

    Due to the complexity and uncertainty of deep neural network (DNN) models, generating test inputs to comprehensively test general and corner case behaviors of DNN models is of great significance for ensuring model quality. Current research primarily focuses on designing coverage criteria and utilizing fuzzing testing technique to generate test inputs, thereby improving test adequacy. However, few studies have taken into consideration the diversity and individual fault-revealing ability of test inputs. Surprise adequacy quantifies the neuron activation differences between a test input and the training set. It is an important metric to measure test adequacy, which has not been leveraged for test input generation. Therefore, we propose a surprise adequacy-guided test input generation approach. Firstly, the approach selects important neurons that contribute more to decision-making. Activation values of these neurons are used as features to improve the surprise adequacy metric. Then, seed test inputs are selected with error-revealing capability based on the improved surprise adequacy measurements. Finally, the approach utilizes the idea of coverage-guided fuzzing testing to jointly optimize the surprise adequacy value of test inputs and the prediction probability differences among classes. The gradient ascent algorithm is adopted to calculate the perturbation and iteratively generate test inputs. Empirical studies on 5 DNN models covering 4 different image datasets demonstrate that the improved surprise adequacy metric effectively captures surprising test inputs and reduces the time cost of the calculation. Concerning test input generation, compared with DeepGini and RobOT, the follow-up test set generated by using the proposed seed input selection strategy exhibits the highest surprise coverage improvement of 5.9% and 15.9%, respectively. Compared with DLFuzz and DeepXplore, the proposed approach achieves the highest surprise coverage improvement of 26.5% and 33.7%, respectively.

  • Network and Information Security
  • Yunhua He , Mingshun Luo , Qing Hu , Bin Wu , Chao Wang , Ke Xiao
    doi: 10.7544/issn1000-1239.202220892

    Driven by the carbon peaking and carbon neutrality goals of China, the digital transformation of the energy industry is imperative. With the application and development of blockchain in the digital transformation of the energy industry, the concept of energy blockchain has gradually formed a consensus. Energy blockchain is a new industrial form of the deep integration of the blockchain technology and the energy Internet, which can facilitate efficient collaboration between energy entities and provide technical support and services for innovative business models such as green and low-carbon business. The large-scale development of energy blockchain is inseparable from the breakthrough of multi-level cross-chain technology, but the cross-chain service of energy blockchain still faces many problems. We divide the current research status of energy blockchain into five categories, namely energy blockchain architecture, smart contract applications, cross-chain technology, blockchain node management and blockchain privacy protection, and we summarize the related research work in these five directions respectively, sort out the principles, advantages and disadvantages of each research scheme in detail; then, in order to promote the development of the cross-chain service security technology of the energy blockchain, we combine the supervision mechanism and the consensus mechanism to propose a multi-level cross-chain collaborative supervision of the energy blockchain architecture, according to the actual needs of the energy blockchain; finally, we summarize problems that need to be solved in the energy blockchain cross-chain service security technology, and put forward the research prospect of blockchain in the energy field.

  • Network and Information Security
  • Donghui Yang , Bin Zeng , Zhenyu Li
    doi: 10.7544/issn1000-1239.202220846

    Since ICANN initiated the delegation of new generic top-level domains (new gTLDs) in 2013, more than a thousand of new gTLDs have been added to the domain name system (DNS). Previous work has shown that while new gTLD domains bring flexibility to registrants, they are also commonly used for malicious behavior because of their low registration costs, and it is important to identify malicious new gTLD domains. However, because of the unique characteristics (e.g., domain length) of new gTLD domains, the accuracy is low when applying existing malicious domain identification methods to malicious new gTLD domain identification. To address this issue, we first characterize the resolution behavior of new gTLD domains based on massive domain name resolution data from five aspects including the number of associated SLDs per new gTLD, query volume, query failure rate, content replication and hosting infrastructure sharing. Then we analyze the resolution behavior of malicious new gTLD domains and find their unique behavioral characteristics in terms of content hosting infrastructure concentration, the number of FQDNs per SLD, the number of queries, the distribution of end users’ network footprints, and the distribution of the length of SLDs. Finally, according to these features, we design a malicious new gTLD domain identification method based on random forest. The results of the experiment show that the proposed method achieves 94% accuracy, which is better than the existing malicious domain identification methods.

  • Network and Information Security
  • Zhichuang Liang , Jieyu Zheng , Yunlei Zhao
    doi: 10.7544/issn1000-1239.202220980

    Constructing post-quantum key encapsulation mechanism based on NTRU lattice is one of the popular research fields in lattice-based cryptography. To reduce the ciphertext size, some current schemes compress the ciphertext with the aid of extra hardness assumptions and error correction codes, which leads to idealistic underlying assumption and complicated implementation. To address the issues, an efficient and compact key encapsulation mechanism, named LTRU, is proposed. LTRU is only based on NTRU one-wayness assumption and enables ciphertext compression without using any error correction codes. The performance-balanced parameter set of LTRU is provided, featuring 128 b quantum security level along with the matching and negligible error probability, and smaller public key size and ciphertext size. LTRU is based on the NTT-friendly polynomial ring. To compute the polynomial operations of LTRU, an efficient mixed-radix NTT is presented. At last, both C implementation and AVX2 implementation of LTRU are provided. When compared with NIST Round 3 finalist NTRU-HRSS, the classical and quantum security of LTRU are strengthened by 6 b and 5 b, respectively. LTRU reduces the public key size, ciphertext size and total bandwidth by 14.6%, 26.0% and 20.3%, respectively. LTRU is 10.9 times faster in key generation and 1.7 faster in decapsulation with respect to AVX2 implementation, respectively.

  • Network and Information Security
  • Hu Xiong , Ye Lin , Ting Yao
    doi: 10.7544/issn1000-1239.202220809

    The identity-based encryption with equality test (IBEET) scheme solves the problem of certificate management in traditional equality test schemes and gets wide attention. However, the existing IBEET systems are difficult to resist penetration attacks and based on foreign cipher algorithm designs without independent intellectual property rights. To deal with this challenge, we propose a SM9 identity-based encryption scheme with equality test and cryptographic reverse firewalls (SM9-IBEET-CRF). The cipher reverse firewalls (CRF) which are deployed in the upstream channel between users and cloud server can re-randomize the information to protect against penetration attacks. This scheme expands SM9 identity-based encryption algorithm to IBEET, improves its efficiency and enriches the research of secret algorithm in cloud computing. We give the definition of SM9-IBEET-CRF and corresponding security models. In random oracle model, the scheme formalizes the IBE-IND-CCA and IBE-OW-CCA security into the BDH difficulty assumption by considering two different opponents. At the same time, we demonstrate that CRF deployment provides functionality-maintaining, security-preserving and exfiltration-resistant by considering the third opponent. The experimental simulation and analysis results show the effectiveness of the scheme.