ArchiveAs the “last mile” of Internet content delivery, terminal networks seem rather simple but in fact constitute 99% of the performance bottlenecks. Classic design is usually oriented to typical devices and regular environments, thus making it difficult to accommodate and reproduce diversified scenarios and resulting in severe performance degradation. By comprehensively gathering and deeply diagnosing the anomalies of large-scale complex terminal networks at the cloud, we have revealed several important defects of the classic design for terminal networks in three key dimensions—availability, reliability and credibility. In order to fix these defects effectively and efficiently, the cross-layer and cross-technology collaboratively reinforced design methodology is adopted (e.g., the time-inhomogeneous 4G/5G dual connectivity management method is utilized to minimize the probability of network disconnection), so as to fulfill self-regulation mechanism design for ubiquitous scenarios. The research achievements have been applied to the high-speed network of the Ministry of Public Security, 17 million UUSpeedTest App users, 70 million Xiaomi mobile phones, 100 million Baidu PhoneGuard users, and 900 million WiFi devices. In recent years, we have also conducted forward-looking network design based on cloud-hosted emulators to discover and fix potential defects without real-world user engagement, thus making the design of terminal networks “born in the cloud and grow in the cloud”. The research achievements have been applied to Huawei DevEco Studio IDE (Integrated Development Environment), Tencent App Market, Google Android Emulator, and multiple popular Apps (like Douyin and Toutiao) of ByteDance.
In recent years, some progress has been made in the key technologies of the architecture and systems software for graph processing. Large-scale graph processing has also been widely used in many fields, including scientific computing, machine learning, social networks, intelligent transportation, bioinformatics, etc. However, most real-world graph computations have characteristics such as dynamic changes and complex and diverse application requirements. This poses new demands and challenges for graph processing in terms of basic theory, architecture, and key technologies of systems software. To address these challenges, researchers have proposed a series of graph processing systems and accelerators, which optimize the graph processing process through technologies such as high-performance computing and parallel computing. Furthermore, in order to meet the demands of practical application scenarios, various graph processing frameworks and algorithms are constantly being innovated and optimized, thus enhancing the practical value of graph processing in terms of processing large-scale graph data and improving computational efficiency. We review the research and development status of key technologies in graph processing architecture and systems software, and summarize, compare, analyze the latest progress of research at home and abroad, and select fields closely related to national economy and people’s livelihood in combination with national development strategies and major application requirements. The industry progress of graph processing-related technologies is analyzed and summarized from typical applications. Finally, the future technical challenges and research directions are prospected.
The huge cost of investment and low resource utilization in the datacenter has long been a great concern to cloud providers. To address this issue, a straightforward way is co-locating more applications on the same hardware to improve resource efficiency. However, the shared resource contention caused by co-located applications leads to performance interference, affecting the application’s performance, quality of service (QoS) and user satisfaction. Therefore, how to guarantee the performance of the co-located application has been a key issue in the colocation scenario. We introduce the researches of guaranteeing the performance of co-located applications, including the background of co-location, challenges, and key technologies. The related work is summarized from four aspects: application and cluster characterization (basic), interference detection (premise), server-level resource allocation (micro-level policy), and cluster-level job scheduling (macro-level policy). In addition, due to the diverse characteristics of co-located applications and clusters, the research of guaranteeing the performance faces different challenges and problem complexity in the different co-located scenarios. For example, the number of co-located applications deployed on a unit resource will directly affect the time cost of searching resource space, and the running mode of applications will affect the competition intensity of shared resources. Therefore, from the perspective of problem complexity, we discuss and analyze the challenges of research work from three dimensions, cluster and application characteristics, resource interference dimension, and the number of co-located applications. At the end of this paper, we discuss the future research directions and the challenges in the high deployment density scenario. We conclude that the software/hardware co-designed full-stack approach is the trend to guarantee the performance in high deployment density clusters, and this approach can help to provide predictable performance and high resource efficiency in the datacenter.
High-performance password recovery system is one of the important application scenarios of the Sunway many-core processor. Many popular password recovery systems and tools adopt rule processing as a mainstream password generation method due to its relatively high hit rate compared with the dictionary/mask based password generation methods. However, current researches lack optimization for the rule processing algorithm on the Sunway processor, which makes the rule-based password generation speed become the bottleneck of the password recovery systems. By analyzing the parallelism of the rule processing algorithm at different levels, we propose several optimization techniques for the rule processing on the Sunway processor. For the thread-level optimizations, we explore the optimal scheme to parallelize the rule processing algorithm, which includes the optimal task mapping technique, the optimal local data memory allocation technique, the load balancing technique, and the variable-length rule storage technique. For the data-level optimizations, we analyze the computing patterns of the rule functions and leverage the Sunway SIMD instructions to vectorize the rule functions and reduce the execution time. The experimental results based on the SW26010 processor show that the proposed optimization techniques effectively eliminate the performance bottleneck of rule processing and the rule-based password recovery speed is increased by 30 to 101 times.
In order to meet the needs of scientific research and engineering applications, the performance of high-performance computers has been continuously improved, the system scale has continued to increase, and the error rate of systems and applications has inevitably continued to increase. Quickly discovering and locating system and application-level errors and providing high-quality services to users have become issues that need to be considered urgently in the design and development of supercomputer systems. Hardware failures and exceptions, software program errors, etc. in supercomputer systems will cause users to hang up and exit large-scale parallel applications. How to quickly and accurately locate the fault site, so that administrators or users can view abnormal faults for high-precision and high-efficiency diagnosis based on this, is an important basis for maintaining the reliability of high-performance computing systems. According to the architecture of “New Generation Sunway Supercomputer” and the characteristics of SW26010-Pro many-core processor, a runtime fault location method is proposed, including fault correlation analysis based on message passing, online comprehensive analysis and diagnosis based on global aggregated information, The key technologies such as abnormal thread filtering method for Shenwei many-core processors are expounded on how to effectively detect, collect, and process abnormal information of a large number of system resources and parallel processes, so as to deal with the problem of efficient fault location system in ultra-large-scale high-performance computing in the future.
Real-time embedded system (RTES) needs to guarantee not only logical correctness of the calculation results, but also temporal correctness of the interaction with outside world, so low-level programs must be able to accurately express time behavior in upper-level models. TTI instruction set (time-triggered instruction set) is proposed to try to solve the problem of the lack of timing semantics at the computer instruction set architecture level, and the realization of real-time processing unit (RPU) based on TTI instruction set proves the feasibility and effectiveness of TTI instruction set. However, the current work lacks research on design and analysis method of TTI program. Therefore, based on TTI instruction set and RPU, we propose four types of timing semantics that TTI instruction set can express, give the design paradigm of TTI program. Then, we define the representation of TTI program time behavior — TFG+, which is an extension of TFG. TFG+ distinguishes timing semantic instructions and the common code segments in the TTI program, and TFG+ can represent the control flow information of TTI program, the time behavior specified by the user, and the time attributes related to hardware platform. Finally, we propose time analysis method of TTI program and time safety checking method, which provide the basis for the design and deployment of TTI program.
Domain adaptation relaxes the assumption that test data follows the same distribution as training data in traditional machine learning, and transfers knowledge from label-rich source domain to label-scarce target domain when domain bias exists. However, most domain adaptation methods make assumptions about the relative relationship between the label space of source domain and target domain which limits the application of these methods in reality. Therefore, universal domain adaptation is proposed to label target data without prior information of label spaces of source domain and target domain. Generally, a universal domain adaptation model needs to solve several problems. Firstly, it is desired to judge whether a target sample belongs to classes in source domain. Then it ought to transfer knowledge of common classes shared by source domain and target domain to label target samples, as well as dealing with the domain bias and potential category shift. Due to its practicability, universal domain adaptation has gained more and more attention now. Many branches of universal domain adaptation have been proposed, and researchers are trying to apply it onto datasets different from images. In this review, we conduct an exhaustive survey of universal domain adaptation methods till now from the point of problem setting and strategies. Then we compare typical methods and analyze the difficulties in the universal domain adaptation. After that, we introduce current applications of universal domain adaptation. Meanwhile, we analyze relevant research areas briefly. Lastly, we discuss the problems of universal domain adaptation that can be studied further.
Decisions are often made in complex environment without exact information in many real-world occasions. Hence the capability of making proper decisions is expected for artificial intelligence agents. As abstractions of the real world, games provoke interests of researchers with the benefits of well-defined game structure and the facility to evaluate various algorithms. Among these games, GuanDan poker games are typical games with large action space and huge information set size, which exacerbates the problem and increases the difficulty to solve these games. In this work, we propose a novel soft deep Monte Carlo(SDMC) method to overcome the above-mentioned difficulties. By considering how the expert strategy acts in the training process, SDMC can better utilize the expert knowledge and accelerate the convergence of training process. Meanwhile, SDMC applies an action sample strategy in real time playing to confuse the opponents and prohibits the potentional exploitation of them, which could also lead to significant improvement of the performance against different agents. SDMC agent was the champion of the 2nd Chinese Artificial Intelligence Game Algorithm competition. Comprehensive experiments that evaluate the training time and final performance are conducted in this work, showing superior performance of SDMC against other agents such as the champion of 1st competition.
As the non-invasive analyzing media, electroencephalogram (EEG) signals are widely applied in brain-computer interfaces, dysfunctional disorder diagnosis and rehabilitation. However, the techniques used in such applications are supervised and completely require EEG labels, like classification. Meanwhile, with the ever-increasing of unlabeled EEG emerged in these applications, traditional supervised techniques are becoming inapplicable, which probably degrades the development of this new-type unlabeled EEG in the emerging potential fields. To deal with the issue of unsupervised analysis for unlabeled EEG signals, we propose a multiple regularization constrained pseudo label propagation optimization model, which integrates the pseudo label propagation learning, EEG similarity adjacency matrix approximation, and label classifier learning. Subsequently, to pursuit the goal of EEG clustering with the proposed model, we transform the model to a multi-objective optimization function and propose a gradient descent-based algorithm named EEGapc (electroencephalogram clustering with pseudo label propagation) to solve it. EEGapc not only can make best use of messages passing through pairwise EEG signals in EEG-constructed graph, but can also quickly converge to its local optima. Experimental results by comparing EEGapc with 8 different types of state-of-the-art clustering algorithms on 14 real-world EEG data sets clearly demonstrate the superiority of EEGapc, and its performances with respect to average NMI (normalized mutual information), ARI (adjusted rand index), F-score and kappa are at least improved by 86.88%, 58.01%, 6.29%, 61.17%, respectively.
Concept drift is an important and challenging problem in streaming data mining field. However, most existing methods can only deal with linear or simple nonlinear mappings. In spite of the ability of fitting nonlinear functions, neural network models have difficulty in adjusting dynamically according to the changing data streaming because only one sample or one batch of samples is available at a time for model training in the context of streaming data mining task. In order to solve above problem, the thought of gradient boosting algorithm is introduced to solve the problem of streaming data mining task with concept drift and a concept drift convergence method based on adaptive deep ensemble networks (CD_ADEN) is proposed. The proposed model combines several shallow neural networks as base leaner, and subsequent base learner corrects the output of precedent base learner to make the final output achieve high real-time generalization performance. Besides, because of the high convergence speed of shallow neural network, the proposed model will quickly recover from accuracy decrease caused by concept drift. The experimental results on multiple datasets show that the average real-time accuracy of the proposed CD_ADEN method is significantly improved compared with the comparative methods, the average real-time accuracy is improved by 1%−5%, and the average ordinal value ranks first in the comparison of several algorithms. It shows that the proposed model can correct the error of the pre-order output, and the learning model can quickly recover from the accuracy drop caused by concept drift, which improves the real-time generalization performance of the online learning model.
Event detection aims to detect triggers in sentences and classify them into pre-defined event types. The key factors lie in appropriately representing triggers. Existing representation-based methods learn the semantic representation of candidate triggers through complex deep neural networks to improve the performance of models. However, these methods ignore two important problems: 1) affected by sentence context, the same trigger can trigger different event types; 2) due to the diversity of natural language expression, different triggers can trigger the same event type. Inspired by hidden variables in the variational auto-encoder (VAE) and hierarchical structure in other natural language processing (NLP) tasks, we propose a hierarchical latent semantic-driven network (HLSD) for event detection to address the above two problems through latent semantic information of sentences and words. The model reduces the dimension from the text representation space to the new latent semantic space and explores the more essential influence information in the macro and micro context of events. Firstly, we get the representation of a sentence and the words through BERT. Secondly, a dual latent semantic mechanism is designed, and VAE is used to mine the latent semantic information at the sentence and word levels. Finally, from the perspective of different granularity contexts, a hierarchical structure from coarse to fine is proposed to make full use of the latent semantic information of sentences and words, to improve the performance of the model. The experimental results on ACE2005 corpus show that the F1 performance of the proposed method achieves 77.9%. In addition, we quantitatively analyze the above two problems in the experiment, which proves the effectiveness of our method.
Federated learning is a distributed machine learning framework, which enables clients to conduct model training without transmitting their data to the servers. It is used to solve the dilemma of data silos and data privacy. It can work well on clients with similar data characteristics and distribution. However, in many scenarios, the differences of data distribution cause difficulties in global model training. Therefore, personalized federated learning is proposed as a new federated learning paradigm. It aims to guarantee the effectiveness of client personalized models through the collaboration between clients and the servers. Intuitively, providing a tighter collaboration for clients with similar data characteristics and distribution can facilitate the construction of personalized models. However, due to the invisibility of client data, it is a challenge to extract client features at a fine-grained level and define the collaborative relationships between clients. In this paper, we design an attention-enhanced meta-learning network (AMN) to address this issue. AMN can utilize model parameters as features and train the meta-learning network to provide an additional meta-model for each client to automatically analyze client feature similarity. According to two-layers framework of AMN, a trade-off between clients’ personality and commonality can be reasonably achieved, and a hybrid model with useful information from all clients is provided. Considering the need to maintain both the meta-model and the client’s local model during the training process, we design an alternative training strategy to perform the training in an end-to-end manner. To demonstrate the effectiveness of our method, we conduct extensive experiments on two benchmark datasets and eight baseline methods. Compared with the existing best-performing personalized federated learning methods, our method improves the accuracy rate by 3.39% and 2.45% on average in two datasets.
In recent years, deep neural networks (DNN) have been widely used in many fields, even replacing human to make decisions in some safety-critical systems, such as autonomous driving and smart healthcare, which requires higher reliability of DNN. It is difficult to understand the internal prediction mechanism and debug because of the complex multi-layer nonlinear network structure of DNN. The existing DNN debugging work mainly improves the performance by adjusting the parameters and augments the training set to optimize DNN. However, it is difficult to control the modification range of adjusting parameters directly, and probably make the model lose the ability of fitting the training set. And unguided augmentation of training set will dramatically increase training costs. To address this problem, a DNN optimization method OptDIA (optimizing DNN based on interpretability analysis) is proposed. Interpretability analysis is conducted on the training process and the decision-making behavior of DNN. According to the interpretability analysis results, the original training data is split into different partitions to evaluate their influences on decision-making results of DNN. After that, the partitions of original training data are transformed with different probabilities to generate new training data, which are used to retrain DNN to improve the performance of the model. The experiments on nine DNNs trained by three datasets shows that OptDIA can improve the accuracy of DNNs by 0.39% to 2.15% and F1-score of DNNs by 0.11% to 2.03%.
In the deep learning community, lots of efforts have been made to enhance the robustness and the reliability of deep neural networks (DNNs). Previous research mainly analyzed the fragility of DNN from the perspective of adversarial attack, and researchers designed numerous adversarial attack and defense methods. However, with the wide application of pre-trained models (PTMs), a new security threat against DNN especially PTM, called backdoor attack is emerging. Backdoor attack aims at injecting hidden backdoors into DNN, such that the backdoored model behaves properly on normal inputs but produces attacker-specified malicious outputs on the poisoned inputs embedded with special triggers. Backdoor attack poses a severe threat against DNN based systems like spam filter or hate speech detector. Compared with the textual adversarial attack and defense which has been widely studied, textual backdoor attack and defense has not been thoroughly investigated and requires a systematic review. In this paper, we present a comprehensive survey of backdoor attack and defense methods in the text domain. Specifically, we first summarize and categorize the textual backdoor attack and defense methods from different perspectives, then we introduce typical work and analyze their pros and cons. We also enumerate widely adopted benchmark datasets and evaluation metrics in the current literatures. Moreover, we respectively compare the backdoor attack with two relevant threats (i.e., adversarial attack and data poisoning). Finally, we discuss existing challenges of backdoor attack and defense in the text domain and present several promising future directions in this emerging and rapidly growing research area.
Trusted execution environment (TEE) technologies are widely developed in the current computer systems along with the user’s serious concerns about privacy protection, secure computing, etc. in network services. Generally, TEEs provide an isolated execution environment for the managers and users for privacy and confidential computing even if the underlying operating systems are compromised. To build the TEEs, the device manufacturers like Intel, Arm update the hardware foundation by adding the external processor mode, memory control, cryptography engine, etc. In addition, they provide corresponding interfaces in the system following the application requirements. Except that, researchers further design compatible TEE models for various goals with the above hardware or firmware assistance. We comprehensively analyze the technical characteristics of TEE technology in mainstream system architecture (including x86, Arm, RISC-V, heterogeneous computing unit), including infrastructure and hardware facilities design, software interface definition, security boundary, etc., and explore the feasible application scenarios of TEE technology. At the same time, we analyze the challenges of current TEE technologies and discuss the limitations and the security risks, e.g., side-channel attacks. Finally, we summarize the advantages and disadvantages of various TEE technologies from the aspects of security and functionality, and consider the future development of TEE.
Side channel attack based on deep learning needs to model and train each key byte of the cryptographic algorithm, which costs a lot of data acquisition and model training. To solve this problem, a side channel attack method based on multi-source data aggregation neural network is proposed. In order to screen the leaked data of key byte with good generalization quality for data aggregation, taking AES-128 algorithm as an example, firstly 16 single key byte models are trained based on the leaked data of 16 key bytes, and models are used to recover 16 key bytes respectively. Secondly, a scoring mechanism is designed to evaluate the generalization effect of each single key byte model, and models with the best recovery effect for each key byte are selected according to score sorting. Finally, a multi-source data aggregation model is constructed based on the key byte leaked data sets corresponding to the selected models to realize key recovery. The tested results show that the multi-source data aggregation model has good generalization effect, effectively improves the accuracy and efficiency of key recovery, reduces the number of traces used to recover the key, and the model also has good attack effect in the case of less traces.