收藏切换
Enhancing Keyframe Selection Method via LLM-Generated Pseudo-Labels for Long-Form Video Question Answering
收藏切换
Zijie Song1, Xin Guan2, Zhenzhen Hu2, *, Jia Li2, Richang Hong2
Journal of Computer Research and Development | 2026, 63(6) : 1563 - 1573
Less
收藏切换
Journal of Computer Research and Development | 2026, 63(6): 1563-1573
• Artificial Intelligence •
Enhancing Keyframe Selection Method via LLM-Generated Pseudo-Labels for Long-Form Video Question Answering
Full
Zijie Song1, Xin Guan2, Zhenzhen Hu2, *, Jia Li2, Richang Hong2
Affiliations
  • 1School of Big Data and Statistics, Anhui University, Hefei 230601
  • 2School of Computer Science and Information Engineering, Hefei University of Technology, Hefei 230601
  • born in 1997. PhD, lecturer. His main research interest includes cross-media analysis and reasoning

    born in 1997. Master. His main research interest includes video question answering

    born in 1986. PhD, professor, PhD supervisor. Her main research interests include multimedia computing, computer vision, and image processing

    born in 1993. PhD, associate professor, master supervisor. His main research interests include affective computing, computer vision, and multimodal learning

    born in 1981. PhD, professor, PhD supervisor. His main research interests include multimedia content analysis and social media

About Author:

born in 1997. PhD, lecturer. His main research interest includes cross-media analysis and reasoning

born in 1997. Master. His main research interest includes video question answering

born in 1986. PhD, professor, PhD supervisor. Her main research interests include multimedia computing, computer vision, and image processing

born in 1993. PhD, associate professor, master supervisor. His main research interests include affective computing, computer vision, and multimodal learning

born in 1981. PhD, professor, PhD supervisor. His main research interests include multimedia content analysis and social media

Published: 2026-06-01 doi: 10.7544/issn1000-1239.202550537
Outline
收藏切换

Keyframe selection is an important method for long-form video question answering, as it accurately identifies key content from redundant information and establishes an interpretable reasoning path. However, existing keyframe selection methods face challenges with insufficient semantic sensitivity during end-to-end training, leading to the introduction of a significant amount of irrelevant frame noise, which affects the model’s accuracy and interpretability. To address this, we propose a pseudo-labels guided keyframe selection (PGKS) model. This model first utilizes large language models (LLMs) to semantically integrate questions and answers, generating global descriptive text. It then employs a multimodal alignment model to compute semantic similarity scores between the description and video sampled frames, thereby constructing frame-level pseudo-labels. By guiding the calculation of frame-level similarity scores with these pseudo-labels, the model effectively suppresses irrelevant frame noise, enhancing both the accuracy and interpretability of keyframe selection. The decoupling between the utilization of LLMs and the training of the backbone network is architecturally designed to provide a cost-effective solution for leveraging LLMs in resource-constrained scenarios. Furthermore, our method balances differentiable and non-differentiable sorting results, and introduces a sliding time window mechanism to further improve the model’s understanding of temporal relationships in videos. Experimental results demonstrate that PGKS model achieves an accuracy of 62.67% on the long-form video question answering dataset NExT-QA, outperforming existing methods of comparable size and improving by 8.51% over the baseline without pseudo-label guidance.

video question answering  /  keyframe selection  /  pseudo-labels  /  large language model  /  similarity score
宋子杰, 关鑫, 胡珍珍, 李佳, 洪日昌. 基于大语言模型伪标签生成的长视频问答关键帧选择增强方法. 计算机研究与发展, 2026 , 63 (6) : 1563 -1573 . DOI: 10.7544/issn1000-1239.202550537
Zijie Song, Xin Guan, Zhenzhen Hu, Jia Li, Richang Hong. Enhancing Keyframe Selection Method via LLM-Generated Pseudo-Labels for Long-Form Video Question Answering[J]. Journal of Computer Research and Development, 2026 , 63 (6) : 1563 -1573 . DOI: 10.7544/issn1000-1239.202550537
Year 2026 volume 63 Issue 6
100
49
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202550537
  • Receive Date:2025-07-20
  • Online Date:2026-11-17
  • Published:2026-06-01
Article Data
Affiliations
History
  • Received:2025-07-20
  • Revised:2025-12-15
  • Accepted:2026-01-29
Affiliations
    1School of Big Data and Statistics, Anhui University, Hefei 230601
    2School of Computer Science and Information Engineering, Hefei University of Technology, Hefei 230601
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202550537
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT