收藏切换
A Trusted Evaluation System for Safe Deployment of Large Language Models
收藏切换
Wentao Ye1, Jiaqi Hu1, Haobo Wang2, *, Gang Chen1, Junbo Zhao1
Journal of Computer Research and Development | 2025, 62(7) : 1668 - 1684
Less
收藏切换
Journal of Computer Research and Development | 2025, 62(7): 1668-1684
• Special Issue on Generative AI-driven Information Systems •
A Trusted Evaluation System for Safe Deployment of Large Language Models
Full
Wentao Ye1, Jiaqi Hu1, Haobo Wang2, *, Gang Chen1, Junbo Zhao1
Affiliations
  • 1College of Computer Science and Technology, Zhejiang University, Hangzhou 310027
  • 2School of Software Technology, Zhejiang University, Ningbo, Zhejiang 315048
  • born in 2001. PhD candidate. His main research interests include large language models, artificial intelligence safety, and natural language processing

    born in 2002. Undergraduate. His main research interests include natural language processing and large language models

    born in 1996. PhD, professor, PhD supervisor. His main research interests include machine learning and data mining, especially on weakly-supervised learning and large language models

    born in 1973. PhD, professor, PhD supervisor. Member of IEEE, ACM, standing member of the CCF Database Professional Committee. His main research interests include database management technology, intelligent computing based big data, and massive Internet systems

    born in 1992. PhD, professor, PhD supervisor. His main research interests include large language models, table pre-training, machine learning, and AI+X

About Author:

born in 2001. PhD candidate. His main research interests include large language models, artificial intelligence safety, and natural language processing

born in 2002. Undergraduate. His main research interests include natural language processing and large language models

born in 1996. PhD, professor, PhD supervisor. His main research interests include machine learning and data mining, especially on weakly-supervised learning and large language models

born in 1973. PhD, professor, PhD supervisor. Member of IEEE, ACM, standing member of the CCF Database Professional Committee. His main research interests include database management technology, intelligent computing based big data, and massive Internet systems

born in 1992. PhD, professor, PhD supervisor. His main research interests include large language models, table pre-training, machine learning, and AI+X

Published: 2025-07-01 doi: 10.7544/issn1000-1239.202440566
Outline
收藏切换

The recent popularity of large language models (LLMs) has brought a significant impact to boundless fields, particularly through their open-ended ecosystem such as the APIs, open-sourced models, and plugins. However, with their widespread deployment, there is a general lack of research that thoroughly discusses and analyzes the potential risks concealed. In that case, we intend to conduct a preliminary but pioneering study covering the robustness, consistency, and credibility of LLMs systems. With most of the related literature in the era of LLMs uncharted, we propose an automated workflow that copes with an upscaled number of queries/responses. Overall, we conduct over a million queries to the mainstream LLMs including ChatGPT, LLaMA, and OPT. Core to our workflow consists of a data primitive, followed by an automated interpreter that evaluates these LLMs under different adversarial metrical systems. As a result, we draw several and perhaps unfortunate conclusions that are quite uncommon from this trendy community. Briefly, they are: 1) the minor but inevitable error occurrence in the user-generated query input may, by chance, cause the LLM to respond unexpectedly; 2) LLMs possess poor consistency when processing semantically similar query input. In addition, as a side finding, we find that ChatGPT is still capable to yield the correct answer even when the input is polluted at an extreme level. While this phenomenon demonstrates the powerful memorization of the LLMs, it raises serious concerns about using such data for LLM-involved evaluation in academic development. To deal with it, we propose a novel index associated with a dataset that roughly decides the feasibility of using such data for LLM-involved evaluation. Extensive empirical studies are tagged to support the aforementioned claims.

large language models  /  scalable deployment  /  robustness  /  automated workflow  /  trusted evaluation
叶文涛, 胡家齐, 王皓波, 陈刚, 赵俊博. 面向大语言模型安全部署的可信评估体系. 计算机研究与发展, 2025 , 62 (7) : 1668 -1684 . DOI: 10.7544/issn1000-1239.202440566
Wentao Ye, Jiaqi Hu, Haobo Wang, Gang Chen, Junbo Zhao. A Trusted Evaluation System for Safe Deployment of Large Language Models[J]. Journal of Computer Research and Development, 2025 , 62 (7) : 1668 -1684 . DOI: 10.7544/issn1000-1239.202440566
Year 2025 volume 62 Issue 7
155
72
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202440566
  • Receive Date:2024-06-21
  • Online Date:2026-11-10
  • Published:2025-07-01
Article Data
Affiliations
History
  • Received:2024-06-21
  • Revised:2025-03-03
  • Accepted:2025-04-03
Affiliations
    1College of Computer Science and Technology, Zhejiang University, Hangzhou 310027
    2School of Software Technology, Zhejiang University, Ningbo, Zhejiang 315048
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202440566
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT