Home Archive
Archive
2024 Volume 61 Issue 5  Published: 2024-05-01
    Special Issue on Large Models and Safety
  • Huimin Chen , Zhiyuan Liu , Maosong Sun
    doi: 10.7544/issn1000-1239.202330700

    Large language models such as ChatGPT have recently gained widespread attention in society due to their high level of generality and intelligence. This technological advancement has sparked not only a technological revolution, but also a social revolution. Against this backdrop, exploring the evolution and characteristics of large language models based on the underlying technology of ChatGPT, as well as their potential opportunities and challenges for society, has become a crucial topic in contemporary times. We first review the technological development of large language models, then summarize the revolution into three stages and discuss their key technical features, respectively. Subsequently, from the perspective of the interaction between technology and society, we explore the social opportunities brought about by large language models: serving as a personal universal information assistant, turning into a new paradigm of scientific research, contributing to industrial infrastructure, and becoming a powerful tool for modernizing national governance capabilities. What’s more, we address potential challenges that arise from large language models. These include issues of information pollution, distribution of social power, ethical and legal concerns, and ideological security. Finally, we conclude with reflections and discussions on the future development of large language models and their relationship with societal progress.

  • Special Issue on Large Models and Safety
  • Xiaochen Wang , Kun Zhang , Peng Zhang
    doi: 10.7544/issn1000-1239.202330955

    With the widespread application of big models in the field of artificial intelligence, the security issues of large models, especially large language models, have received attention. As an emerging technology, the security situation analysis of large models and the construction of security systems need further exploration. We analyze the overall trend of large model security from two perspectives: society and technology application. Based on the characteristics of large models, we sort out the practice of large model security building, and provide a reference plan for building a security system for large model development and large model application construction. The large model security practice plan introduced in this article includes three parts: security benchmark construction, model values alignment method, and model online service security system construction.

  • Special Issue on Large Models and Safety
  • Mi Zhang , Xudong Pan , Min Yang
    doi: 10.7544/issn1000-1239.202330959

    We propose a universal safety testing benchmark for large language models (LLMs), JADE-DB. The benchmark is automatically constructed via the targeted mutation approach, which is able to convert test questions that are manually crafted by experienced LLM testers and multidisciplinary experts to highly threatening universal test questions. The converted questions still preserve the naturalness of human language without changing the core semantics of the original question, and in the meantime are able to consistently break over ten widely-used LLMs. Based on the incremental linguistic complexity, JADE-DB incorporates three levels of LLM safety testing, namely, basic, advanced and dangerous, which accounts for thousands of test questions covering 4 major unsafe generation categories, i.e., crime, tort, bias and core values, spanning over 30 unsafe topics. Specifically, we construct three dangerous safety benchmarks respectively for the three groups of LLMs, i.e., eight open-sourced Chinese, six commercial Chinese and four commercial English LLMs. The benchmarks simultaneously trigger harmful generation of multiple LLMs, with an average unsafe generation ratio of 70%. The results indicate that, due to the complexity of human language, most of the current best LLMs can hardly learn the infinite number of different syntactic structures of human language and thus recognize the invariant evil therein.

  • Special Issue on Large Models and Safety
  • Xuanting Chen , Junjie Ye , Can Zu , Nuo Xu , Tao Gui , Qi Zhang
    doi: 10.7544/issn1000-1239.202330801

    The GPT models have demonstrated impressive performance in various natural language processing (NLP) tasks. However, their robustness and abilities to handle various complexities of the open world have not yet to be well explored, which is especially crucial in assessing the stability of models and is a key aspect of trustworthy AI. In this study, we perform a comprehensive experimental analysis of GPT-3 and GPT-3.5 series models, exploring their performance and robustness using 15 datasets (about 147000 original test samples) with 61 robust probing transformations from TextFlint covering 9 popular NLP tasks. Additionally, we analyze the model’s robustness across different transformation levels, including character, word, and sentence. Our findings reveal that while GPT models exhibit competitive performance in certain tasks like sentiment analysis, semantic matching, and reading comprehension, they exhibit severe confusion regarding information extraction tasks. For instance, GPT models exhibit severe confusion in relation extraction and even exhibit “hallucination” phenomena. Moreover, they experience significant degradation in robustness in terms of tasks and transformations, especially in classification tasks and sentence-level transformations. Furthermore, we validate the impact of the quantity and the form of demonstrations on performance and robustness. These findings reveal that GPT models are still not fully proficient in handling common NLP tasks, and highlight the difficulty in addressing robustness challenges through enhancing model performance or altering prompt content. By comparing the performance and robustness of the updated version of gpt-3.5-turbo, gpt-4, LLaMA2-7B and LLaMA2-13B, we further validate the experimental findings. Future studies on large language models should strive to enhance their capacities in information extraction and semantic understanding, while simultaneously bolstering overall robustness.

  • Special Issue on Large Models and Safety
  • Mengru Wang , Yunzhi Yao , Zekun Xi , Jintian Zhang , Peng Wang , Ziwen Xu , Ningyu Zhang
    doi: 10.7544/issn1000-1239.202330965

    Although large language models (LLMs) have achieved remarkable success, they still face security problems in practical applications, and it is easy to generate toxic and harmful content under malicious induction. Existing methods to mitigate the unsafe behavior of LLMs often demand significant computational resources and incur high costs associated with secure data collection. Knowledge editing offers a novel approach to constrain the model’s behavior precisely for specific inputs without the need for retraining, thus saving substantial resources. This approach provides a new feasible avenue for optimizing large models to generate secure content. Nevertheless, existing datasets for mitigating the unsafe behavior of LLMs do not encompass all unsafe scenarios. Moreover, the toxicity issues in these datasets are nearly insurmountable for post-alignment LLMs’ security defenses, hindering the optimization of safety concerns in post-alignment LLMs. In light of these challenges, we introduce a new dataset called SafeGen and propose a novel evaluation framework to analyze the potential of knowledge editing in optimizing the generation of secure content by LLMs. Extensive experiments reveal that knowledge editing demonstrates broad applications in rectifying unsafe behaviors exhibited by LLMs, and editing parameters can enhance the internal safety beliefs of LLMs. However, the fluency of text generated by knowledge editing falls short of expectations, indicating the inherent difficulty of this task. We hope that our work provides insights for the large model security community.

  • Special Issue on Large Models and Safety
  • Nan Li , Yidong Ding , Haoyu Jiang , Jiafei Niu , Ping Yi
    doi: 10.7544/issn1000-1239.202330962

    In recent years, large language models (LLMs) have been widely applied in a range of downstream tasks and have demonstrated remarkable text understanding, generation, and reasoning capabilities in various fields. However, jailbreak attacks are emerging as a new threat to LLMs. Jailbreak attacks can bypass the security mechanisms of LLMs, weaken the influence of safety alignment, and induce harmful outputs from aligned LLMs. Issues such as abuse, hijacking and leakage caused by jailbreak attacks have posed serious threats to both dialogue systems and applications based on LLMs. We present a systematic review of jailbreak attacks in recent years, categorize these attacks into three distinct types based on their underlying mechanism: manually designed attacks, LLM-generated attacks, and optimization-based attacks. We provide a comprehensive summary of the core principles, implementation methods, and research findings derived from relevant studies, thoroughly examine the evolutionary trajectory of jailbreak attacks on LLMs, offering a valuable reference for future research endeavors. Moreover, a concise overview of the existing security measures is offered. It introduces pertinent techniques from the perspectives of internal defense and external defense, which aim to mitigate jailbreak attacks and enhance the content security of LLM generation. Finally, we delve into the existing challenges and frontier directions in the field of jailbreak attacks on LLMs, examine the potential of multimodal approaches, model editing, and multi-agent methodologies in tackling jailbreak attacks, providing valuable insights and research prospects to further advance the field of LLM security.

  • Special Issue on Large Models and Safety
  • Suxia Zhu , Jinyin Wang , Guanglu Sun
    doi: 10.7544/issn1000-1239.202330521

    Deep learning models are vulnerable to backdoor attacks and behave normally when processing clean data, but they will exhibit malicious behavior when processing toxic samples with trigger patterns. However, most backdoor attacks currently produce backdoor images that are easily perceived by the human eye, resulting in insufficient stealthiness of backdoor attacks. Therefore, a multi-objective optimized covert image backdoor attack method based on perceptual similarity is proposed. Firstly, the visual difference between the backdoor image and the original image is reduced using a perceptual similarity loss function. Secondly, a multi-objective optimization method is used to solve the problem of inter-task conflict on the poisoning model, thus ensuring stable performance of the model after poisoning. Finally, a two-stage training method is adopted to automate the generation of trigger patterns and improve the training efficiency. The final experimental results show that it is difficult for human eye to distinguish the generated backdoor image from the original image without any degradation in clean accuracy. Meanwhile, the backdoor attack is successfully performed on the target classification model, and the attack success rate reaches 100% for all experimental datasets under the all-to-one attack strategy. Compared with other steganographic backdoor attack methods, our method has better stealthiness.

  • Special Issue on Large Models and Safety
  • Di Wu , Yanyan Zhao , Bing Qin
    doi: 10.7544/issn1000-1239.202330812

    With the rapid development of large language models, their safety has become a growing concern among researchers and the public. To prevent potential harm in collaboration, aligning these models’ judgments with human moral values in daily scenarios is essential. A key challenge is ensuring that large language models can adaptively adjust or reassess rules in moral judgment, like humans, to maintain consistency with human morals in various contexts. Inspired by psychological and cognitive science research on the emotional and cognitive influences on human moral judgments, this study leverages the strengths of large language models in cognitive reasoning and emotional analysis. We develop an approach that emulates the interaction between emotional and cognitive judgment in human moral reasoning, thus enhancing these models’ moral judgment capabilities. Experimental results demonstrate the effectiveness of our approach in this task. Overall, this study not only presents an innovative approach to the moral judgment of large language models but also highlights the importance of integrating psychological and cognitive science theories in this field, setting a foundation for future research.

  • Special Issue on Large Models and Safety
  • Meng Lin , Chengwei Dai , Tao Guo
    doi: 10.7544/issn1000-1239.202330960

    With the advancement of 5G technology, offensive speech has increasingly proliferated across social networks in the form of multimodal memes. Consequently, the detection and interpretive generation of offensive memes play a crucial role in enhancing content moderation effectiveness and maintaining a harmonious and healthy public discourse environment. Existing studies on the interpretive generation of offensive memes focus solely on the targets and content of offense, neglecting the societal background knowledge and metaphorical expressions embedded in memes. This oversight limits the ability to comprehensively and accurately interpret the meaning of offensive memes, thus constraining the applicability of interpretations. To address this challenge, we propose a method based on multimodal large language model for generating interpretations of offensive memes. By augmenting elements such as offense targets, the content of the offense, and metaphor recognition into the instruction tuning data, we can effectively improve the multimodal large model’s proficiency in interpretively generating offensive memes through instruction tuning. The experimental outcomes validate three key strengths of our method: first, it achieves a notable 19% enhancement in the BERTScore evaluation metric over baseline models; second, it incorporates comprehensive background knowledge pertinent to offensive metaphorical expressions within its interpretations; third, it exhibits strong generalization capabilities when handling previously unseen meme data.

  • Special Issue on Large Models and Safety
  • Weixin Liu , Yewei Guan , Jiarong Huo , Yuanchao Ding , Hua Guo , Bo Li
    doi: 10.7544/issn1000-1239.202330966

    Transformer has been widely used in many fields such as natural language processing and computer vision, and has outstanding performance. The users’ data will be leaked to the Transformer model provider during inference. With the increasing public attention on data privacy, the above data leakage problem has triggered researchers’ study on secure Transformer inference. Implementing secure Transformer inference with secure multi-party computation (MPC) is today’s hot topic. Due to the widely existence of non-linear functions in Transformer, it is hard to use MPC to implement secure Transformer inference, which leads to huge computation and communication cost. We focus on Softmax attention, bottleneck in secure Transformer inference, and propose two kinds of MPC-friendly attention mechanism, Softmax freeDiv Attention and 2Quad freeDiv Attention. By replacing the Softmax attention in Transformer with the MPC-friendly attention mechanism proposed, combining with the replacement of activation function GeLU and knowledge distillation, we propose an MPC-friendly Transformer convert framework, which can convert Transformer model to an MPC-friendly one, so as to improve the performance of secure Transformer inference later. Based on the proposed MPC-friendly Transformer convert framework , we perform secure Bert-Base inference on SST-2 in the LAN setting, using privacy computing protocols provided by secure processing unit (SPU). The result shows that the secure inference achieves 2.26 times speedup while maintaining the accuracy with non-approximation model.

  • Special Issue on Large Models and Safety
  • Rui Wang , Changjiang Yang , Xiangdong Deng , Yuan Liu , Zhihong Tian
    doi: 10.7544/issn1000-1239.202330961

    Deception defense, as the most promising technology in proactive defense, aids defenders in facing highly covert and unknown threats, turning passivity into proactivity, and breaking the inherent imbalance between offense and defense. In the face of potential threat scenarios, how to effectively use deception defense technology to help defenders anticipate threats, perceive threats, and entrap threats, is a key issue that currently need to be addressed. Game theory and attack graph models provide strong support in formulating active defense strategies and analyzing potential risks. We summarize and review the recent work of both in the realm of deception defense. With the rapid development of large language model technology and its increasingly close integration with the field of cybersecurity, we review traditional deception defense technology and propose a large language model-based intelligent external network HoneyPoint generation technique. Experimental analysis validates the effectiveness of external network HoneyPoint in capturing network threats, showing improvements over traditional Web honeypots in aspects like simulation, stability, and flexibility. To enhance the collaborative cooperation between HoneyPoints and improve the capabilities for threatening exploration and perception, the concept of Honey-Landscape is introduced. We provide an outlook on how to utilize HoneyPoint and Honey-Landscape technologies to construct an integrated active defense mechanism that includes threat prediction, threat perception, and threat entrapment.

  • Special Issue on Large Models and Safety
  • Jing Ke , Zheyong Xie , Tong Xu , Yuhao Chen , Xiangwen Liao , Enhong Chen
    doi: 10.7544/issn1000-1239.202330967

    The advancement of generative artificial intelligence technology has significantly contributed to the progress in various fields. However, this technological development has also inadvertently facilitated the creation and widespread dissemination of misinformation. Prior research has concentrated on addressing grammatical issues, inflammatory content, and other pertinent features by employing deep learning models to characterize and model deceptive elements within fake news content. These approaches not only are lack of the capability to assess the content itself, but also fall short in elucidating the reasons behind the model’s classification. Based on the above problems, we propose a fine-grained fake news detection method with implicit semantic enhancement. This method fully utilizes the summarization and reasoning capabilities of the existing generative large language model. The method employs inference based on major events, fine-grained minor events, and implicit information to systematically evaluate the authenticity of news content. This method strategically leverages the full potential of the model by decomposing tasks, thereby not only optimizing its proficiency but also significantly enhancing its prowess in capturing instances of fake news. Simultaneously, it is designed to be interpretable, providing a solid foundation for detection. With its inherent ability, this method not only ensures reliable identification but also holds vast potential for diverse applications.

  • Artificial Intelligence
  • Zishu Wei , Yue Han , Sihao Liu , Shengyu Zhang , Fei Wu
    doi: 10.7544/issn1000-1239.202440063

    In current era, marked by advancements and achievements made in digital and intelligent fields, artificial intelligence (AI) has emerged as a pivotal engine driving technological innovation, which indicates encapsulating and examining the latest trends and future trajectories in AI research makes sense on the development of future AI research. This can be implemented by collecting the research outcomes during recent three years from top-tier international conferences and journals in the field of AI that are recommended by the China Computer Federation (CCF-A category), introducing keyword-centric analyses based on a bibliometric methodologies, and analyzing research hotspots based on high-frequency keywords, discerning emerging trends through newly-added keywords, identifying high-impact studies using citation-weighted keyword analysis. The result of these analyses, which contains significant information about trends in AI research, can enable the principal directions of AI research to be delineated and the interconnections and integrative fusion within mainstream AI research directions to be unveiled. Moreover, an in-depth exploration of the current hot topics, such as large language models (LLMs), AI-driven scientific research (AI for Science) and visual generation technologies, would help us reveal the underlying scientific theories and application prospects behind these technological innovations, thereby the latest trends and future trajectories in AI field get demonstrated more adequately and concretely.

  • Artificial Intelligence
  • Ying Sun , Yuting Zhang , Fuzhen Zhuang , Hengshu Zhu , Qing He , Hui Xiong
    doi: 10.7544/issn1000-1239.202330133

    Accurately quantifying the relationship between skills and salary is essential to improve reasonable job salary setting and promote talent attraction and retention. However, the relationship between skills and salary is complex because it involves modeling set utility in a high-dimensional space with massive possible elements. Deep neural networks offer a new solution for complex fitting problems. However, for skill-based fine-grained salary prediction, there still lacks interpretable neural networks that can effectively model set utility under the influence of complex variables. To address this issue, we propose a marginal contribution-based incremental set utility network (MCISUN). MCISUN models the marginal contribution of elements when they are added to the set. In this way, the set utility can be naturally obtained in a flexible and interpretable way. In particular, rather than relying on pooling structures to ensure permutation invariance, MCISUN constructs order-sensitive intermediate results through recurrent attention neural networks and takes advantage of the sets’ permutation invariance property to achieve data augmentation, thus improving the model’s robustness. We conduct extensive experiments on a real-world large-scale salary dataset. The experimental results show that MCISUN outperforms state-of-the-art models by 30% for skill-based salary prediction. Qualitative experiments show that our model can recognize reasonable skill contribution values and capture the relationship between skills.

  • Artificial Intelligence
  • Ting Bai , Xuanning Liu , Bin Wu , Zibin Zhang , Zhiyuan Xu , Kangyi Lin
    doi: 10.7544/issn1000-1239.202220943

    Learning effective high-order feature interactions is crucial for click through rate (CTR) prediction in recommender systems. Existing methods that learn meaningful high-order feature combinations by reassembling low-order feature combinations, i.e., 2-order feature interaction, suffer from high computational costs to calculate the interaction weight of all pairwise feature interactions. Some deep neural network-based methods can be seen as universal function approximators to potentially learn all kinds of feature interactions. However, it had been proved to be inefficient to approximate the low-order interactions, i.e., 2-order or 3rd-order feature interactions, which may influence the accuracy of CTR prediction task. Based on the above consideration, we propose a multi-granularity based feature interaction pruning network (FeatNet) for CTR prediction task. Firstly, FeatNet generates different subsets with a threshold pruning operation to select the meaningful feature combinations on the explicit feature granularity, which enables FeatNet to keep the diversity of different feature combinations, and reduce the complexity of high-order feature interactions. Based on the pruned feature subsets, implicit high-order feature interactions are further conducted on the granularity of feature elements, which automatically filters out the invalid feature interactions. Extensive experiments are conducted on two real-world datasets, showing the superiority of FeatNet in CTR prediction.

  • Artificial Intelligence
  • Jinming Du , Yuanyuan Sun , Hongfei Lin , Liang Yang
    doi: 10.7544/issn1000-1239.202220951

    Conversational emotion recognition is the task of classifying emotions based on conversations. The conversation data are characterized by colloquial language and a wide range of topics, with semantic similarities among labels. Colloquial language exhibits issues such as word ambiguity and the omission of semantic information, emphasizing the importance of common sense and grammatical knowledge in conversational emotion recognition tasks, and these factors enable the model to accurately capture semantic information. Moreover, the current challenge lies in the variations in text richness and the frequency of emotion transfer across different dialogue scenarios, which result in suboptimal classification performance. We propose CK-ERC model to address these challenges. In the pre-training phase, CK-ERC model extracts structured data to incorporate common sense knowledge graphs and grammatical knowledge graphs, aiding the model in accurately capturing colloquial information. In the fine-tuning phase, a supervised contrast learning task is introduced to help the model identify similar emotional labels. Furthermore, a dynamic threshold-based curriculum learning strategy is designed for training and optimizing the model based on text richness (from high to low) and emotion transfer frequency (from low to high). CK-ERC model demonstrates superior performance in various conversation modes, including two-person conversation, multi-person conversation, simulated conversation, and daily conversation. Particularly, CK-ERC model achieves the best performance on MELD and EmoryNLP datasets.

  • Artificial Intelligence
  • Xuejian Huang , Tinghuai Ma , Gensheng Wang
    doi: 10.7544/issn1000-1239.202330722

    Multimodal machine learning represents a novel paradigm in artificial intelligence, leveraging various modalities and intelligent processing algorithms to achieve enhanced performance. Multimodal representation and fusion are two pivotal tasks in multimodal machine learning. Currently, most multimodal representation methods pay little attention to inter-sample collaboration, leading to a lack of robustness in feature representation. Additionally, most multimodal feature fusion methods exhibit sensitivity to noisy data. Therefore, in the realm of multimodal representation, an approach based on both intra-sample and inter-sample multimodal collaboration is proposed to facilitate a comprehensive understanding of interactions within and between modalities, ultimately enhancing the robustness of feature representation. Firstly, text, speech, and visual features are individually extracted based on pre-trained models such as BERT, Wav2vec 2.0, and Faster R-CNN. Subsequently, considering the complementarity and consistency of multimodal data, two categories of encoders, namely modality-specific and modality-shared, are constructed to learn both modality-specific and shared feature representations. Furthermore, intra-sample collaboration loss functions are formulated using central moment differences and orthogonality, while inter-sample collaboration loss functions are established using contrastive learning. Lastly, a representation learning function is designed based on intra-sample collaboration, inter-sample collaboration, and sample reconstruction errors. Regarding multimodal fusion, an adaptive multimodal feature fusion method is designed, accounting for the possibility that each modality may exhibit varying types of effects and levels of noise at different times, using attention mechanisms and gated neural networks. Experimental results on the multimodal intent recognition dataset MIntRec and emotion datasets CMU-MOSI and CMU-MOSEI demonstrate that this multimodal learning approach outperforms baseline methods across multiple evaluation metrics.

  • Artificial Intelligence
  • Jianwei Ma , Tiexin Wang , Hong Jiang , Tao Chen , Chao Zhang , Bohan Li
    doi: 10.7544/issn1000-1239.202330691

    Police dossier, as one of the main records handled by the police department, contains massive and crucial policing information. As an important means, efficient information extraction from police dossier is helpful for case analysis, crime trend prediction, and the improvement of the public security management. However, the text of police dossier is written by police officers using natural language, which makes it difficult to extract crucial information. Traditional information extraction of police dossier heavily relies on manual effort and predefined templates, resulting in low efficiency and poor generality. Considering the particularity of police dossier, in this paper, a knowledge extraction method based on deep semantics analysis is proposed. This method consists of two core tasks: named entity recognition and relation extraction. Focusing on Chinese text, we propose a named entity recognition method that integrates structural and glyph features of Chinese characters. On the basis of entity recognition results, with the help of a specially constructed policing thesaurus, a relationship extraction method combining rule based and trigger word is proposed. Both on a publicly available Weibo dataset and a real dossier dataset provided by our partner a local police department, compared with several baseline named entity recognition models, our proposed method shows better performance in classifying exact entities and finding more potential entities. The automatically extracted relationships have also been verified and committed by the police department branch. A particular information system has been used in practice.

  • Artificial Intelligence
  • Xiaogang Song , Haoyue Hu , Jingyu Ning , Li Liang , Xiaofeng Lu , Xinhong Hei
    doi: 10.7544/issn1000-1239.202330485

    In this paper, the mutually beneficial relationship between depth estimation and semantic segmentation is investigated, and a self-supervised monocular depth estimation method for joint semantic segmentation USegDepth is proposed. The shared encoder for semantic segmentation and depth estimation is implemented to achieve semantic guidance. To further improve the across multiple tasks performance of the encoder, a multi-task feature extraction module is designed. The module is stacked to generate the shared encoder, solving the poor feature representation problem of the model due to limited receptive field and lack of cross-channel interaction, and the model accuracy is improved further. And a cross-task interaction module is proposed for bidirectional cross-domain information interaction to refine the depth features, improving depth estimation performance, especially in weak texture regions and object boundaries with limited luminosity consistency supervision. Through training and evaluation on KITTI dataset, the experimental results show that the mean square relative error of USegDepth is reduced by 0.176 percentage points compared with that of SGDepth, and the threshold accuracy reaches 98.4% at a threshold value of 1.253, proving the high accuracy of USegDepth in depth prediction.