Full
Outline
To evaluate the knowledge and clinical capabilities of large language models in the field of traditional Chinese medicine.
Using literature research and experimental research methods，9 different categories of large language models were selected，a data set of knowledge ability and clinical ability was constructed，and traditional Chinese medicine professional ability prompts were designed.The automatic evaluation method and expert scoring method were used to evaluate the large language models，and the variance was used.Analysis and multiple comparison methods were used to comparatively analyze the medical professional ability level of large language models.
In the evaluation of knowledge ability，Lingyi Wanwu（78.93），Zhipu Qingyan（77.91）and Tongyi Qianwen（77.22）performed best.There were significant differences in the knowledge ability scores of different models，different subjects and different types of models（P<0.05）.In the clinical capability evaluation，the responses generated by each model had high legibility（81.00），but low accuracy（74.86）.Among them，the average score of Wenxin Yiyan was the highest at 85.40，and all five-dimension scores reached excellent levels（≥80）.The difference in clinical ability scores of different types of models was significant.
The general large language model has advantages in generalization，legibility and security.The medical large language model performs well overall in the field of traditional Chinese medicine specialization and traditional Chinese medicine consultation.The application of large language models in the field of traditional Chinese medicine has broad development prospects in the future.
38
17
| 科 Family | 属数 Number of genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) | 属 Genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) |
|---|---|---|---|---|---|---|
| 鹅膏菌科Amanitaceae | 2 | 11 | 5.26 | 鹅膏菌属 Amanita | 10 | 4.78 |
| 小菇科 Mycenaceae | 2 | 12 | 5.74 | 丝盖伞属 Inocybe | 5 | 2.39 |
| 多孔菌科 Polyporaceae | 8 | 14 | 6.70 | 蜡蘑属 Laccaria | 5 | 2.39 |
| 红菇科 Russulaceae | 3 | 23 | 11.00 | 小皮伞属 Marasmius | 6 | 2.87 |
| 小菇属 Mycena | 11 | 5.26 | ||||
| 光柄菇属 Pluteus | 5 | 2.39 | ||||
| 红菇属 Russula | 17 | 8.13 | ||||
| 栓菌属 Trametes | 5 | 2.39 |
关闭全屏
No. 86 Xueyuan South Road, Haidian District, Beijing
010-62662699(Operation and Technical Support)
qkjq@cast.org.cn
Copyright © 2025 China Association for Science and Technology. All rights reserved. For all open access content, the relevant licensing terms apply.
Sponsored by the Office of the Leading Group for Cybersecurity and Informatization of CAST, and supported by Science and Technology Review Publishing House
