收藏切换
Jailbreak Attack for Large Language Models: A Survey
收藏切换
Nan Li, Yidong Ding, Haoyu Jiang, Jiafei Niu, Ping Yi*
Journal of Computer Research and Development | 2024, 61(5) : 1156 - 1181
Less
收藏切换
Journal of Computer Research and Development | 2024, 61(5): 1156-1181
• Special Issue on Large Models and Safety •
Jailbreak Attack for Large Language Models: A Survey
Full
Nan Li, Yidong Ding, Haoyu Jiang, Jiafei Niu, Ping Yi*
Affiliations
  • School of Cyber Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240
  • born in 2002. Master candidate. His main research interests include artificial intelligence backdoor attack and large language model security

    born in 2001. Master candidate. His main research interests include artificial intelligence backdoor attack and defense, and large language models

    born in 1999. Master candidate. His main research interests include artificial intelligence backdoor attack and graph neural network

    born in 2001. Master candidate. His main research interests include artificial intelligence backdoors and large language model security

    born in 1969. PhD, associate professor. Senior member of CCF. His main research interests include security for artificial intelligence and system security

About Author:

born in 2002. Master candidate. His main research interests include artificial intelligence backdoor attack and large language model security

born in 2001. Master candidate. His main research interests include artificial intelligence backdoor attack and defense, and large language models

born in 1999. Master candidate. His main research interests include artificial intelligence backdoor attack and graph neural network

born in 2001. Master candidate. His main research interests include artificial intelligence backdoors and large language model security

born in 1969. PhD, associate professor. Senior member of CCF. His main research interests include security for artificial intelligence and system security

Published: 2024-05-01 doi: 10.7544/issn1000-1239.202330962
Outline
收藏切换

In recent years, large language models (LLMs) have been widely applied in a range of downstream tasks and have demonstrated remarkable text understanding, generation, and reasoning capabilities in various fields. However, jailbreak attacks are emerging as a new threat to LLMs. Jailbreak attacks can bypass the security mechanisms of LLMs, weaken the influence of safety alignment, and induce harmful outputs from aligned LLMs. Issues such as abuse, hijacking and leakage caused by jailbreak attacks have posed serious threats to both dialogue systems and applications based on LLMs. We present a systematic review of jailbreak attacks in recent years, categorize these attacks into three distinct types based on their underlying mechanism: manually designed attacks, LLM-generated attacks, and optimization-based attacks. We provide a comprehensive summary of the core principles, implementation methods, and research findings derived from relevant studies, thoroughly examine the evolutionary trajectory of jailbreak attacks on LLMs, offering a valuable reference for future research endeavors. Moreover, a concise overview of the existing security measures is offered. It introduces pertinent techniques from the perspectives of internal defense and external defense, which aim to mitigate jailbreak attacks and enhance the content security of LLM generation. Finally, we delve into the existing challenges and frontier directions in the field of jailbreak attacks on LLMs, examine the potential of multimodal approaches, model editing, and multi-agent methodologies in tackling jailbreak attacks, providing valuable insights and research prospects to further advance the field of LLM security.

generative artificial intelligence  /  jailbreak attack  /  large language model (LLM)  /  natural language processing (NLP)  /  cyber security
李南, 丁益东, 江浩宇, 牛佳飞, 易平. 面向大语言模型的越狱攻击综述. 计算机研究与发展, 2024 , 61 (5) : 1156 -1181 . DOI: 10.7544/issn1000-1239.202330962
Nan Li, Yidong Ding, Haoyu Jiang, Jiafei Niu, Ping Yi. Jailbreak Attack for Large Language Models: A Survey[J]. Journal of Computer Research and Development, 2024 , 61 (5) : 1156 -1181 . DOI: 10.7544/issn1000-1239.202330962
Year 2024 volume 61 Issue 5
100
49
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202330962
  • Receive Date:2023-11-30
  • Online Date:2026-11-17
  • Published:2024-05-01
Article Data
Affiliations
History
  • Received:2023-11-30
  • Revised:2024-01-31
Affiliations
    School of Cyber Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202330962
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT