收藏切换
A Cross-Modal Entity Linking Model Based on Contrastive Learning
收藏切换
Yuanzheng Wang, Wenxiang Sun, Yixing Fan*, Huaming Liao, Jiafeng Guo
Journal of Computer Research and Development | 2025, 62(3) : 662 - 671
Less
收藏切换
Journal of Computer Research and Development | 2025, 62(3): 662-671
• Artificial Intelligence •
A Cross-Modal Entity Linking Model Based on Contrastive Learning
Full
Yuanzheng Wang, Wenxiang Sun, Yixing Fan*, Huaming Liao, Jiafeng Guo
Affiliations
  • CAS Key Laboratory of Network Data Science & Technology (Institute of Computing Technology, Chinese Academy of Sciences), Beijing 100190
  • University of Chinese Academy of Science, Beijing 100049
  • born in 1997. PhD candidate. His main research interests include information retrieval and natural language processing

    born in 1998. Master. His main research interests include information retrieval and natural language processing

    born in 1990. Associate professor. His main research interests include information retrieval and natural language understanding

    born in 1972. Associate professor. Her main research interests include big data application, information retrieval, and distributed data processing

    born in 1980. Professor. His main research interests include representation learning, and neural models for information retrieval and filtering

About Author:

born in 1997. PhD candidate. His main research interests include information retrieval and natural language processing

born in 1998. Master. His main research interests include information retrieval and natural language processing

born in 1990. Associate professor. His main research interests include information retrieval and natural language understanding

born in 1972. Associate professor. Her main research interests include big data application, information retrieval, and distributed data processing

born in 1980. Professor. His main research interests include representation learning, and neural models for information retrieval and filtering

Published: 2025-03-01 doi: 10.7544/issn1000-1239.202330731
Outline
收藏切换

Image-text cross-modal entity linking is an extension of traditional named entity linking. The inputs are images containing entities, which are linked to textual entities in the knowledge base. Existing models usually adopt a dual-encoder architecture which encodes entities of visual and textual modality into separate vectors, then calculates their similarities using dot product, and links the image entities to the most similar text entities. The training process usually adopts the cross-modal contrastive learning task. For a given modality of entity vectors, this task pulls closer the vector of another modality that corresponds to itself, and pushes away the vector of another modality corresponding to other entities. However, this approach overlooks the differences in representation difficulty within the two modalities: visually similar entities are often more difficult to distinguish than textual similar entities, resulting in the incorrect linking of the former ones. To solve this problem, we propose two new contrastive learning tasks, which can enhance the discriminative power of the vectors. The first is self-contrastive learning, which aims to improve the distinction between visual vectors. The second is hard-negative contrastive learning, which helps a textual vectors to distinguish similar visual vectors. We conduct experiments on the open-source dataset WikiPerson. With a knowledge base of 120000 entities, our model achieves an accuracy improvement of 4.5% compared with the previous state-of-the-art model.

entity linking model  /  multi-modal  /  cross-modal  /  contrastive learning  /  visual information
王苑铮, 孙文祥, 范意兴, 廖华明, 郭嘉丰. 基于对比学习的跨模态实体链接模型. 计算机研究与发展, 2025 , 62 (3) : 662 -671 . DOI: 10.7544/issn1000-1239.202330731
Yuanzheng Wang, Wenxiang Sun, Yixing Fan, Huaming Liao, Jiafeng Guo. A Cross-Modal Entity Linking Model Based on Contrastive Learning[J]. Journal of Computer Research and Development, 2025 , 62 (3) : 662 -671 . DOI: 10.7544/issn1000-1239.202330731
Year 2025 volume 62 Issue 3
101
43
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202330731
  • Receive Date:2023-09-12
  • Online Date:2026-11-10
  • Published:2025-03-01
Article Data
Affiliations
History
  • Received:2023-09-12
  • Revised:2024-07-02
  • Accepted:2024-08-09
Affiliations
    CAS Key Laboratory of Network Data Science & Technology (Institute of Computing Technology, Chinese Academy of Sciences), Beijing 100190
    University of Chinese Academy of Science, Beijing 100049
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202330731
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT