Home Latest Articles
Latest Articles
  • Mi Zhang, Xudong Pan, Min Yang
    Journal of Computer Research and Development. 2024, 61(5): 1113-1127.

    We propose a universal safety testing benchmark for large language models (LLMs), JADE-DB. The benchmark is automatically constructed via the targeted mutation approach, which is able to convert test questions that are manually crafted by experienced LLM testers and multidisciplinary experts to highly threatening universal test questions. The converted questions still preserve the naturalness of human language without changing the core semantics of the original question, and in the meantime are able to consistently break over ten widely-used LLMs. Based on the incremental linguistic complexity, JADE-DB incorporates three levels of LLM safety testing, namely, basic, advanced and dangerous, which accounts for thousands of test questions covering 4 major unsafe generation categories, i.e., crime, tort, bias and core values, spanning over 30 unsafe topics. Specifically, we construct three dangerous safety benchmarks respectively for the three groups of LLMs, i.e., eight open-sourced Chinese, six commercial Chinese and four commercial English LLMs. The benchmarks simultaneously trigger harmful generation of multiple LLMs, with an average unsafe generation ratio of 70%. The results indicate that, due to the complexity of human language, most of the current best LLMs can hardly learn the infinite number of different syntactic structures of human language and thus recognize the invariant evil therein.

  • Xuanting Chen, Junjie Ye, Can Zu, Nuo Xu, Tao Gui, Qi Zhang
    Journal of Computer Research and Development. 2024, 61(5): 1128-1142.

    The GPT models have demonstrated impressive performance in various natural language processing (NLP) tasks. However, their robustness and abilities to handle various complexities of the open world have not yet to be well explored, which is especially crucial in assessing the stability of models and is a key aspect of trustworthy AI. In this study, we perform a comprehensive experimental analysis of GPT-3 and GPT-3.5 series models, exploring their performance and robustness using 15 datasets (about 147000 original test samples) with 61 robust probing transformations from TextFlint covering 9 popular NLP tasks. Additionally, we analyze the model’s robustness across different transformation levels, including character, word, and sentence. Our findings reveal that while GPT models exhibit competitive performance in certain tasks like sentiment analysis, semantic matching, and reading comprehension, they exhibit severe confusion regarding information extraction tasks. For instance, GPT models exhibit severe confusion in relation extraction and even exhibit “hallucination” phenomena. Moreover, they experience significant degradation in robustness in terms of tasks and transformations, especially in classification tasks and sentence-level transformations. Furthermore, we validate the impact of the quantity and the form of demonstrations on performance and robustness. These findings reveal that GPT models are still not fully proficient in handling common NLP tasks, and highlight the difficulty in addressing robustness challenges through enhancing model performance or altering prompt content. By comparing the performance and robustness of the updated version of gpt-3.5-turbo, gpt-4, LLaMA2-7B and LLaMA2-13B, we further validate the experimental findings. Future studies on large language models should strive to enhance their capacities in information extraction and semantic understanding, while simultaneously bolstering overall robustness.

  • Mengru Wang, Yunzhi Yao, Zekun Xi, Jintian Zhang, Peng Wang, Ziwen Xu, Ningyu Zhang
    Journal of Computer Research and Development. 2024, 61(5): 1143-1155.

    Although large language models (LLMs) have achieved remarkable success, they still face security problems in practical applications, and it is easy to generate toxic and harmful content under malicious induction. Existing methods to mitigate the unsafe behavior of LLMs often demand significant computational resources and incur high costs associated with secure data collection. Knowledge editing offers a novel approach to constrain the model’s behavior precisely for specific inputs without the need for retraining, thus saving substantial resources. This approach provides a new feasible avenue for optimizing large models to generate secure content. Nevertheless, existing datasets for mitigating the unsafe behavior of LLMs do not encompass all unsafe scenarios. Moreover, the toxicity issues in these datasets are nearly insurmountable for post-alignment LLMs’ security defenses, hindering the optimization of safety concerns in post-alignment LLMs. In light of these challenges, we introduce a new dataset called SafeGen and propose a novel evaluation framework to analyze the potential of knowledge editing in optimizing the generation of secure content by LLMs. Extensive experiments reveal that knowledge editing demonstrates broad applications in rectifying unsafe behaviors exhibited by LLMs, and editing parameters can enhance the internal safety beliefs of LLMs. However, the fluency of text generated by knowledge editing falls short of expectations, indicating the inherent difficulty of this task. We hope that our work provides insights for the large model security community.

  • Nan Li, Yidong Ding, Haoyu Jiang, Jiafei Niu, Ping Yi
    Journal of Computer Research and Development. 2024, 61(5): 1156-1181.

    In recent years, large language models (LLMs) have been widely applied in a range of downstream tasks and have demonstrated remarkable text understanding, generation, and reasoning capabilities in various fields. However, jailbreak attacks are emerging as a new threat to LLMs. Jailbreak attacks can bypass the security mechanisms of LLMs, weaken the influence of safety alignment, and induce harmful outputs from aligned LLMs. Issues such as abuse, hijacking and leakage caused by jailbreak attacks have posed serious threats to both dialogue systems and applications based on LLMs. We present a systematic review of jailbreak attacks in recent years, categorize these attacks into three distinct types based on their underlying mechanism: manually designed attacks, LLM-generated attacks, and optimization-based attacks. We provide a comprehensive summary of the core principles, implementation methods, and research findings derived from relevant studies, thoroughly examine the evolutionary trajectory of jailbreak attacks on LLMs, offering a valuable reference for future research endeavors. Moreover, a concise overview of the existing security measures is offered. It introduces pertinent techniques from the perspectives of internal defense and external defense, which aim to mitigate jailbreak attacks and enhance the content security of LLM generation. Finally, we delve into the existing challenges and frontier directions in the field of jailbreak attacks on LLMs, examine the potential of multimodal approaches, model editing, and multi-agent methodologies in tackling jailbreak attacks, providing valuable insights and research prospects to further advance the field of LLM security.

  • Suxia Zhu, Jinyin Wang, Guanglu Sun
    Journal of Computer Research and Development. 2024, 61(5): 1182-1192.

    Deep learning models are vulnerable to backdoor attacks and behave normally when processing clean data, but they will exhibit malicious behavior when processing toxic samples with trigger patterns. However, most backdoor attacks currently produce backdoor images that are easily perceived by the human eye, resulting in insufficient stealthiness of backdoor attacks. Therefore, a multi-objective optimized covert image backdoor attack method based on perceptual similarity is proposed. Firstly, the visual difference between the backdoor image and the original image is reduced using a perceptual similarity loss function. Secondly, a multi-objective optimization method is used to solve the problem of inter-task conflict on the poisoning model, thus ensuring stable performance of the model after poisoning. Finally, a two-stage training method is adopted to automate the generation of trigger patterns and improve the training efficiency. The final experimental results show that it is difficult for human eye to distinguish the generated backdoor image from the original image without any degradation in clean accuracy. Meanwhile, the backdoor attack is successfully performed on the target classification model, and the attack success rate reaches 100% for all experimental datasets under the all-to-one attack strategy. Compared with other steganographic backdoor attack methods, our method has better stealthiness.

  • Di Wu, Yanyan Zhao, Bing Qin
    Journal of Computer Research and Development. 2024, 61(5): 1193-1205.

    With the rapid development of large language models, their safety has become a growing concern among researchers and the public. To prevent potential harm in collaboration, aligning these models’ judgments with human moral values in daily scenarios is essential. A key challenge is ensuring that large language models can adaptively adjust or reassess rules in moral judgment, like humans, to maintain consistency with human morals in various contexts. Inspired by psychological and cognitive science research on the emotional and cognitive influences on human moral judgments, this study leverages the strengths of large language models in cognitive reasoning and emotional analysis. We develop an approach that emulates the interaction between emotional and cognitive judgment in human moral reasoning, thus enhancing these models’ moral judgment capabilities. Experimental results demonstrate the effectiveness of our approach in this task. Overall, this study not only presents an innovative approach to the moral judgment of large language models but also highlights the importance of integrating psychological and cognitive science theories in this field, setting a foundation for future research.

  • Meng Lin, Chengwei Dai, Tao Guo
    Journal of Computer Research and Development. 2024, 61(5): 1206-1217.

    With the advancement of 5G technology, offensive speech has increasingly proliferated across social networks in the form of multimodal memes. Consequently, the detection and interpretive generation of offensive memes play a crucial role in enhancing content moderation effectiveness and maintaining a harmonious and healthy public discourse environment. Existing studies on the interpretive generation of offensive memes focus solely on the targets and content of offense, neglecting the societal background knowledge and metaphorical expressions embedded in memes. This oversight limits the ability to comprehensively and accurately interpret the meaning of offensive memes, thus constraining the applicability of interpretations. To address this challenge, we propose a method based on multimodal large language model for generating interpretations of offensive memes. By augmenting elements such as offense targets, the content of the offense, and metaphor recognition into the instruction tuning data, we can effectively improve the multimodal large model’s proficiency in interpretively generating offensive memes through instruction tuning. The experimental outcomes validate three key strengths of our method: first, it achieves a notable 19% enhancement in the BERTScore evaluation metric over baseline models; second, it incorporates comprehensive background knowledge pertinent to offensive metaphorical expressions within its interpretations; third, it exhibits strong generalization capabilities when handling previously unseen meme data.

  • Weixin Liu, Yewei Guan, Jiarong Huo, Yuanchao Ding, Hua Guo, Bo Li
    Journal of Computer Research and Development. 2024, 61(5): 1218-1229.

    Transformer has been widely used in many fields such as natural language processing and computer vision, and has outstanding performance. The users’ data will be leaked to the Transformer model provider during inference. With the increasing public attention on data privacy, the above data leakage problem has triggered researchers’ study on secure Transformer inference. Implementing secure Transformer inference with secure multi-party computation (MPC) is today’s hot topic. Due to the widely existence of non-linear functions in Transformer, it is hard to use MPC to implement secure Transformer inference, which leads to huge computation and communication cost. We focus on Softmax attention, bottleneck in secure Transformer inference, and propose two kinds of MPC-friendly attention mechanism, Softmax freeDiv Attention and 2Quad freeDiv Attention. By replacing the Softmax attention in Transformer with the MPC-friendly attention mechanism proposed, combining with the replacement of activation function GeLU and knowledge distillation, we propose an MPC-friendly Transformer convert framework, which can convert Transformer model to an MPC-friendly one, so as to improve the performance of secure Transformer inference later. Based on the proposed MPC-friendly Transformer convert framework , we perform secure Bert-Base inference on SST-2 in the LAN setting, using privacy computing protocols provided by secure processing unit (SPU). The result shows that the secure inference achieves 2.26 times speedup while maintaining the accuracy with non-approximation model.

  • Rui Wang, Changjiang Yang, Xiangdong Deng, Yuan Liu, Zhihong Tian
    Journal of Computer Research and Development. 2024, 61(5): 1230-1249.

    Deception defense, as the most promising technology in proactive defense, aids defenders in facing highly covert and unknown threats, turning passivity into proactivity, and breaking the inherent imbalance between offense and defense. In the face of potential threat scenarios, how to effectively use deception defense technology to help defenders anticipate threats, perceive threats, and entrap threats, is a key issue that currently need to be addressed. Game theory and attack graph models provide strong support in formulating active defense strategies and analyzing potential risks. We summarize and review the recent work of both in the realm of deception defense. With the rapid development of large language model technology and its increasingly close integration with the field of cybersecurity, we review traditional deception defense technology and propose a large language model-based intelligent external network HoneyPoint generation technique. Experimental analysis validates the effectiveness of external network HoneyPoint in capturing network threats, showing improvements over traditional Web honeypots in aspects like simulation, stability, and flexibility. To enhance the collaborative cooperation between HoneyPoints and improve the capabilities for threatening exploration and perception, the concept of Honey-Landscape is introduced. We provide an outlook on how to utilize HoneyPoint and Honey-Landscape technologies to construct an integrated active defense mechanism that includes threat prediction, threat perception, and threat entrapment.

  • Jing Ke, Zheyong Xie, Tong Xu, Yuhao Chen, Xiangwen Liao, Enhong Chen
    Journal of Computer Research and Development. 2024, 61(5): 1250-1260.

    The advancement of generative artificial intelligence technology has significantly contributed to the progress in various fields. However, this technological development has also inadvertently facilitated the creation and widespread dissemination of misinformation. Prior research has concentrated on addressing grammatical issues, inflammatory content, and other pertinent features by employing deep learning models to characterize and model deceptive elements within fake news content. These approaches not only are lack of the capability to assess the content itself, but also fall short in elucidating the reasons behind the model’s classification. Based on the above problems, we propose a fine-grained fake news detection method with implicit semantic enhancement. This method fully utilizes the summarization and reasoning capabilities of the existing generative large language model. The method employs inference based on major events, fine-grained minor events, and implicit information to systematically evaluate the authenticity of news content. This method strategically leverages the full potential of the model by decomposing tasks, thereby not only optimizing its proficiency but also significantly enhancing its prowess in capturing instances of fake news. Simultaneously, it is designed to be interpretable, providing a solid foundation for detection. With its inherent ability, this method not only ensures reliable identification but also holds vast potential for diverse applications.