Latest ArticlesNamed entity recognition (NER) is a traditional task in natural language processing. The mainstream approach for nested named entity recognition is the span-based classification method, which involves concatenating representations of entity boundaries to obtain spans in general. However, long entities tend to lead to weakened semantic associations between two entity boundaries. In addition, single-scale spans cannot completely capture how entities behave in different contexts. To address this, an entity semantic enhancement method based on multi-scale box fusion is proposed. The method represents spans as boxes with boundary position information. At first, multi-scale boxes will be obtained by fusing features of different scales to enhance the semantic features in the boxes and make the boxes more context-dependent. Then, the boundary positions of the boxes are further refined by a position-weighted attention mechanism to make the box information more accurate. Finally, the entity category of the boxes and the offset of the boxes relative to the true entities are simultaneously predicted to effectively support the recognition and localization of nested named entities. The method achieves a new state-of-the-art F1 scores of 88.63% on the ACE2004 English dataset, 88.53% on ACE2005 English dataset and 73.86% on Weibo Chinese dataset, which demonstrates the effectiveness of the model.
Pretraining data detection aims to determine whether a piece of text belongs to the pretraining data of a large language model (LLM) when the model’s pretraining data are not publicly disclosed, which can be used to audit whether the pretraining data usage complies with legal regulations. Existing methods generally assume that an LLM tends to assign higher token probabilities to training texts compared with non-training texts, and thus identify texts with high probabilities as training texts. However, due to the significant overlap in fragments between training texts and non-training texts, an LLM may also assign relatively high token probabilities to non-training texts, which makes existing methods prone to misclassifying non-training texts as training texts. Inspired by research on the memorization capabilities of LLMs, we propose a novel method to mitigate this issue by comparing the token probabilities given the full context with those given the short-range context, and the contribution of long-range context to token probability increase is then computed, with a higher contribution indicating a greater likelihood of the text being part of the pretraining data. The key idea is that when an LLM predicts the token probabilities for training texts, the contribution of long-range context to token probability increase is greater than that for non-training texts. Experimental results on multiple datasets demonstrate the effectiveness of the proposed method. The code is available at https://github.com/zhang-wei-chao/Long-Range-Context-for-PDD.
Federated learning, as an emerging distributed neural network training paradigm in edge computing, faces a key challenge of data heterogeneity over clients, which significantly impacts global model performance. While clustering federated learning, serving as a promising solution, has shown improvements in model accuracy, its effectiveness is limited by the insufficient utilization of local data distribution statistics. To address this issue, we propose a novel federated learning framework named HS-CFA (hierarchical Sinkhorn distance-based clustering federated algorithm), which is designed to optimize the aggregation weights of local models under the heterogeneous data environments, thereby enhancing the performance of the global model. It utilizes the entropy-regularized optimal transport cost to capture the characteristics of local data distribution and dynamically adjusts the aggregation weights of local models with a hierarchical clustering strategy. Specially, HS-CFA employs the Sinkhorn distance as a lightweight optimal transport cost metric to measure distributional dissimilarities across clients. Furthermore, it adopts a hierarchical two-layer clustering mechanism, combining density-based spatial clustering and average aggregation during the server training phase, facilitating the dynamic and adaptive adjustment of local model aggregation weights. Experimental results on multiple benchmark datasets demonstrate that HS-CFA significantly enhances the accuracy and robustness of the global model in scenarios with highly heterogeneous distribution settings.
Online mental health forums have become an important carrier of mental health services. Detecting psychological distress from a vast number of posts is the basis for psychological intervention. Fully utilizing the social relationships of seekers is conducive to judging their mental health status. However, most existing methods rely on explicit social relationships. The existing models fail to pay attention to the psychological support relationships between patients and doctors (seekers and supporters). These relationships are based on the patient’s personal experiences, symptom causes, self-cognition, and psychological support expertise. Thus, we take suicidal ideation as the detection target and propose post-user psychological support heterogeneous graph (PU-PSHG). PU-PSHG is used to represent the semantics of posts and the doctor-patient relationships between seekers and supporters in online mental health forums. According to PU-PSHG, graph-enhanced suicide ideation detection (GSID) model is proposed. Firstly, based on the definition of psychological support relationships, the semantics of two meta-paths (user-to-user and user-to-post) are defined, and PU-PSHG containing users and posts are constructed. DeepWalk algorithm is used to learn doctor-patient relationships or community relationships from PU-PSHG. Then, the representation of psychological support relationships is learned through relational features. Besides the post semantics and doctor-patient relationships are fused based on heterogeneous relationships. Finally, suicide ideation detection is performed based on the representation of posts. Experimental results on CLPsych2017 shared task show that GSID has better performance compared with existing methods. Compared with C-GraphSAGE, GSID are improved by 7.8%, 4.8%,1.4% in Non-green F1, All F1, All Acc, respectively. Ablation experiments find that removing three different types of relationships (the reply relationship between posts, the psychological support relationship between users and posts, the psychological support relationship between users and users) from PU-PSAG leads to decreases in Non-green F1 of 3.04%, 3.80%, 6.17% respectively.
Existing methods for skeleton-based human action recognition always ignore motion domain knowledge, resulting in the lack of interpretability of logical decision-making that human can understand. In this paper, we propose a novel skeleton-based human action recognition method by fusion of domain knowledge and adaptive spatio-temporal Transformer, to improve recognition performance and interpretability. Firstly, inspired by the short-term motion knowledge, a temporal multi-branch structure is designed to learn and capture the characteristics of short-term sub-acitons. Secondly, a dynamic information fusion module is proposed to learn the weight vectors of different temporal branches, and then fuse multiscale short-term motion features. Finally, to learn the relationship between different sub-actions and facilitate the motion information interaction between skeleton joints, a multiscale temporal convolution feature fusion module is proposed to capture the long-term motion correlations, by integrating the domain knowledge of the long-term motion. Experimental evaluations are conducted on four large action datasets, including NTU RGB+D, NTU RGB+D 120, FineGym, and InHARD. The experimental results show that the recognition performance of the proposed method is superior to several data-driven methods, effectively improving the modelling ability of short-term motion feature learning and information interaction between skeleton joints, with the interpretability.
Researchers have proposed and implemented many distributed runtime systems to help users build distributed applications. These distributed runtime systems are usually only good at processing certain types of loads in specific scenarios. However, in the things-edge-cloud scenario, the components of things-edge-cloud collaborative applications have heterogeneous quality requirements, runtime environments, and heterogeneous communication protocols, making it difficult to use one runtime to build high-performance and robust things-edge-cloud collaborative applications. Deploying application components independently to different runtimes will increase the difficulty of application management and lack of unified performance and fault-tolerance support. The Grip system is proposed to address the problem. The Grip system supports the unified access and utilization of multiple runtimes by introducing a virtual runtime adapter layer and a virtual runtime API layer. These two virtual layers specify the interfaces that need to be implemented when accessing a runtime. The Grip system supports the unified management of multi-runtime applications through Griplet and Grip abstractions. It utilizes ownership methods to provide mechanisms for supporting user-defined fault tolerance and scaling policy. Experiments show that in the things-edge-cloud environment, compared with using a single runtime such as Ray, Docker, and Kubernetes, the Grip system reduces the average end-to-end latency by 31% to 77%, the 90th percentile tail latency by 25% to 78%, and the 95th percentile tail latency by 22% to 78%.
As the number of parameters in deep learning models continues to increase, the cost of training also keeps rising. To reduce training costs, using spot instances provided by cloud service providers has become a viable solution. Spot instances are priced at only 30% of normal instances, which can significantly lower training costs. However, despite the low cost of elastic instances, there is a risk of them being reclaimed at any time, posing new challenges to the stability of the model training system. To address the fault tolerance issue in the context of elastic instances, existing work mainly falls into two categories: checkpoint-based fault tolerance and redundancy-based fault tolerance. Checkpoint-based solutions incur substantial overhead, while redundancy-based solutions impose certain limitations on the model’s parallelism strategy, leading to suboptimal training efficiency. We propose a solution for training in the context of spot instances that leverages the grace period of spot instances to back up training progress, thereby reducing fault tolerance overhead. It also employs the bottleneck alleviation approach to adjust the parallelism strategy, maximizing the use of available cluster resources and enhancing training efficiency. Experimental results show that this solution not only achieves low-cost fault tolerance but also ensures training efficiency, allowing for efficient completion of model training tasks in the context of spot instances and reducing overall training costs.
In order to resist attacks of quantum computing and protect the privacy and data security of underwater nodes, a multi-party key encapsulation mechanism based on the Internet of underwater things is proposed using the difficulty assumption of NTRU cryptographic system. Firstly, the pseudo-identities of device serial numbers are generated by combining ocean sensor acoustic sequences and underwater acoustic waveform factorization, and a verifiable identity ocean acoustic message code is designed. Secondly, a key generation algorithm suitable for underwater communication is designed using orthogonal frequency division multiplexing (OFDM) frequency domain oversampling technique and number theoretic transform (NTT).On this basis, a multi-party public key encryption algorithm with indistinguishability and anonymity under chosen plaintext attack (IND-Anon-CPA) security is constructed using identity-bound hybrid encryption and ocean noise-based obfuscation operation, Thirdly, a SeaFO transform based on ocean noise is introduced to develop a multi-party key encapsulation algorithm with indistinguishability and anonymity under chosen ciphertext attack (IND-Anon-CCA) security without full re-encryption process. Finally, a novel session key update mechanism is devised where autonomous underwater vehicles verify ciphertext components, and assess pseudo-identity and timestamp validity using OFDM subcarriers. The new session keys are decapsulated using SeaFO transform, which not only achieve implicit rejection in multi-party environments and but also thwart adversaries’ adaptive corruption on SeaNTRU. Security analysis demonstrates that SeaNTRU has the characteristics of resistance to key replacement attack, replay attack, and man-in-the-middle attack. Experimental results show that SeaNTRU has lower computational cost and communication overhead than the existing schemes.
Anonymous credentials are a privacy-preserving authentication method that verifies the holder’s qualifications or attributes without revealing their identity, widely used in digital identity management, e-government, and digital banking. Anonymous credentials that comply with Chinese commercial cryptography standards have also garnered significant attention. However, existing anonymous credential schemes often depend on centralized issuing authorities, which not only restrict their applicability in decentralized networks but also introduce risks of system failures and privacy breaches caused by single points of failure. To address these issues, we propose a decentralized anonymous credential scheme based on SM2 commercial cryptography. The proposed scheme leverages blockchain networks to replace credential issuing authorities and employs zero-knowledge proofs to ensure the secure and reliable distribution of credentials. Additionally, the scheme enables users to selectively disclose attributes according to their needs, thereby effectively avoiding the excessive exposure of private information. We also explore the construction methods of zero-knowledge proofs that comply with Chinese commercial cryptography standards and propose a set membership proof scheme based on SM2, providing a foundational tool for designing the SM2-based decentralized anonymous credential. Security analysis proves that the proposed SM2-based decentralized anonymous credential scheme satisfies both unforgeability and anonymity requirements. Performance experiments further show that the scheme maintains efficiency suitable for real-world deployment.
Vehicular ad hoc network is the foundation of the intelligent transportation system, which can effectively improve traffic efficiency and enhance road safety. However, due to the openness of its wireless communication environment, the edge nodes in vehicular ad hoc networks are vulnerable to various external attacks. In order to balance privacy protection and traceability, also ensure secure authentication, an effective method is to adopt the conditional privacy-preserving authentication scheme in vehicular ad hoc networks. Although various privacy-preserving authentication schemes have been proposed successively, the security of these schemes almost all relies on the difficult assumptions of traditional number theory, and it is difficult to have post-quantum security characteristics. Therefore, aiming at the security and efficiency problems of existing lattice-based conditional privacy-preserving authentication schemes in vehicular ad hoc networks, and in order to solve the key escrowing problem, by leveraging the decisional learning with errors over rings (RLWE) hardness assumption and the ring-learning with errors (RLWE) small integer solution (SIS) hardness assumption on structured lattices, and combining with the rejection sampling technique, a lattice-based sampling-free conditional privacy preserving aggregate authentication scheme is designed. It is proved that the proposed scheme can achieve adaptive security in the random oracle. In addition, the performance evaluation shows that the proposed scheme is superior to the scheme based on the general lattice in terms of computation cost.