收藏切换
Conversation Generation Based on Variational Attention Knowledge Selection and Pre-trained Language Model
收藏切换
Naizhou Zhang1, Wei Cao1, Xiaojian Zhang1, Shijun Li2
Journal of Computer Research and Development | 2025, 62(8) : 1902 - 1917
Less
收藏切换
Journal of Computer Research and Development | 2025, 62(8): 1902-1917
• Artificial Intelligence •
Conversation Generation Based on Variational Attention Knowledge Selection and Pre-trained Language Model
Full
Naizhou Zhang1, Wei Cao1, Xiaojian Zhang1, Shijun Li2
Affiliations
  • 1College of Computer and Information Engineering, Henan University of Economics and Law, Zhengzhou 450046
  • 2School of Computer Science, Wuhan University, Wuhan 430072
  • born in 1970. PhD,associate professor. Member of CCF. His main research interests include Web search, data mining, and machine learning

    born in 1977. Master, associate professor. Her main research interests include machine learning and Web data mining

    born in 1980. PhD, professor, master supervisor. His main research interests include differential privacy, privacy analysis in language model, and graph data management

    born in 1964. PhD, professor, PhD supervisor. His main research interests include database theory, Web search engine, and Web mining

About Author:

born in 1970. PhD,associate professor. Member of CCF. His main research interests include Web search, data mining, and machine learning

born in 1977. Master, associate professor. Her main research interests include machine learning and Web data mining

born in 1980. PhD, professor, master supervisor. His main research interests include differential privacy, privacy analysis in language model, and graph data management

born in 1964. PhD, professor, PhD supervisor. His main research interests include database theory, Web search engine, and Web mining

Published: 2025-08-01 doi: 10.7544/issn1000-1239.202440551
Outline
收藏切换

Research on knowledge-grounded dialogue often suffers from the problem of external knowledge containing redundant or even noisy information irrelevant to the conversation topic, which leads to a degradation in the performance of the dialogue system. Knowledge selection becomes an important approach to solve this issue. However, existing work has not yet investigated in depth some issues involving it such as how to design a knowledge selector, how to exploit the selected knowledge, what are the suitable scenarios for the knowledge selection conversation methods, etc. In this paper, we propose a new neural conversation method based on conditional variational attention knowledge selection and a pre-trained language model. This method employs a knowledge selection algorithm based on conditional variational autoencoder (CVAE) and a multi-layer attention mechanism to pick up the most relevant textual knowledge collection to the current conversation, which effectively exploits the dialogue response in training data to improve the efficiency of knowledge selection. Our novel model adopts the pre-trained language model Bart as encoder-decoder architecture and incorporates selected textual knowledge into the Bart model to fine-tune it during the training process. The experimental results show that the model proposed, in contrast to the current representative dialog models, can generate more diverse and coherent dialogue responses with higher accuracy.

knowledge-grounded dialogue  /  knowledge selection  /  pre-trained language model  /  CVAE  /  attention mechanism  /  memory network
张乃洲, 曹薇, 张啸剑, 李石君. 基于变分注意力知识选择和预训练语言模型的对话生成. 计算机研究与发展, 2025 , 62 (8) : 1902 -1917 . DOI: 10.7544/issn1000-1239.202440551
Naizhou Zhang, Wei Cao, Xiaojian Zhang, Shijun Li. Conversation Generation Based on Variational Attention Knowledge Selection and Pre-trained Language Model[J]. Journal of Computer Research and Development, 2025 , 62 (8) : 1902 -1917 . DOI: 10.7544/issn1000-1239.202440551
Year 2025 volume 62 Issue 8
104
36
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202440551
  • Receive Date:2024-06-20
  • Online Date:2026-11-10
  • Published:2025-08-01
Article Data
Affiliations
History
  • Received:2024-06-20
  • Revised:2024-10-22
  • Accepted:2024-11-12
Affiliations
    1College of Computer and Information Engineering, Henan University of Economics and Law, Zhengzhou 450046
    2School of Computer Science, Wuhan University, Wuhan 430072
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202440551
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT