计算机辅助设计与图形学学报

融合局部和长距离信息的图像特征点提取

PDF下载

唐付林 ¹⁾ , 马朋举 ²⁾ , 段鸿杰 ²⁾ , 郑彦 ²⁾ , 计洪图 ²⁾ , 郭云龙 ²⁾ , 吴毅红 ¹⁾^,^*

计算机辅助设计与图形学学报 | 图像视频处理 2025,37(12): 2147-2157

收起

计算机辅助设计与图形学学报 |图像视频处理 2025 , 37 (12) : 2147 -2157

融合局部和长距离信息的图像特征点提取

全屏

唐付林¹⁾, 马朋举²⁾, 段鸿杰²⁾, 郑彦²⁾, 计洪图²⁾, 郭云龙²⁾, 吴毅红¹⁾^,^*

作者信息

¹⁾中国科学院自动化研究所多模态人工智能系统全国重点实验室　北京　100190

²⁾中国石油化工股份有限公司胜利油田分公司　东营　257000

通讯作者:

（yihong.wu@ia.ac.cn）

作者简介:

唐付林（1992—），男，博士，副研究员，主要研究方向为基于视觉的同时定位与建图、多传感器融合的实时定位与建图、增强现实

马朋举（1979—），男，硕士，高级工程师，主要研究方向为图像匹配

段鸿杰（1972—），男，博士，教授级高级工程师，主要研究方向为三维重建

郑彦（1970—），男，学士，高级政工师，主要研究方向为相机标定

计洪图（1983—），男，硕士，高级工程师，主要研究方向为视觉定位

郭云龙（1989—），男，硕士，工程师，主要研究方向为三维重建

吴毅红（1973—），女，博士，研究员，博士生导师，论文通信作者，主要研究方向为摄像机定标、摄像机定位、图像匹配、三维重建、多视几何学、SLAM、虚拟现实/增强现实.

Fusing Local and Long-Range Information for Image Feature Point Extraction

Fulin Tang¹⁾, Pengju Ma²⁾, Hongjie Duan²⁾, Yan Zheng²⁾, Hongtu Ji²⁾, Yunlong Guo²⁾, Yihong Wu¹⁾^,^*

Affiliations

¹⁾State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing　100190

²⁾China Petroleum and Chemical Corporation, SHENGLI OILFIELD, Dongying　257000

出版时间: 2025-12-20 doi: 10.3724/SP.J.1089.2023-00743

文章导航

摘要

收起

现有的基于学习的图像特征点提取方法忽略了图像中的长距离上下文信息，在诸如石油生产场景等重复纹理和弱纹理场景中表现不佳，为油田生产的安全视频监控分析和数字孪生平台的构建带来了较大困难. 为了解决上述问题，提出一种融合局部和长距离信息的图像特征点提取方法. 首先利用高效的大卷积核捕捉长距离信息，并将长距离信息与普通卷积捕捉到的局部信息进行有效的融合，得到判别性更强的图像局部描述子；为了进一步提升图像特征点的匹配精度，提出一种全图像范围内自适应最困难描述子三元组构建方法，将描述子三元组的构造范围由共视区域扩展至整幅图像，提升训练过程和实际应用场景的相似度. 在HPatches数据集上，所提方法的平均匹配准确率高出对比方法2.89%. 在Aachen Day-Night v1.1数据集上，所提方法的视觉定位准确率高出对比方法2.10%. 在InLoc数据集上，所提方法的视觉定位准确率高出对比方法3.70%. 在富含重复纹理和弱纹理区域的石油生产场景数据集中，所提方法的平均特征点匹配内点数高出对比方法20.37%.

关键词

图像匹配 / 局部描述子 / 特征提取 / 最困难描述子三元组

Abstract

收起

Existing learning-based feature point extraction methods neglect long-range context information. Thus, existing learning-based feature point extraction methods perform poorly in repetitive-texture and low-texture scenes such as oil production scenes. This problem poses difficulties for the analysis of safety video surveillance and the construction of digital twin platforms in oil field production. To solve this problem, we propose a novel and efficient feature point extraction network that fuses local and long-range information. Firstly, the proposed network utilizes efficient large-kernel convolutional layers to capture long-range information. Then, the proposed network can extract local descriptors with higher discriminative ability by fusing the long-range information and the local information captured by vanilla convolutional layers. To further enhance the matching accuracy of feature points, we propose a novel method of adaptively constructing hardest descriptor triplets within entire images. The proposed method extends the scope of constructing triplets from covisible areas to entire images, which raises the similarity between the training process and real application scenarios. On the HPatches dataset, the mean matching accuracy of the proposed method is 2.89% higher than those of the comparison methods. On the Aachen Day-Night v1.1 dataset, the visual localization accuracy of the proposed method is 2.10% higher than those of the comparison methods. On the InLoc dataset, the visual localization accuracy of the proposed method is 3.70% higher than those of the comparison methods. On the dataset of oil production scenes with repetitive and weak texture regions, the average number of feature point matching inliers of the proposed method is 20.37% higher than those of the comparison methods.

Key words

image matching / local descriptors / feature extraction / hardest descriptor triplet construction

引用本文

唐付林, 马朋举, 段鸿杰, 郑彦, 计洪图, 郭云龙, 吴毅红. 融合局部和长距离信息的图像特征点提取. 计算机辅助设计与图形学学报, 2025 , 37 (12) : 2147 -2157 . DOI: 10.3724/SP.J.1089.2023-00743

Fulin Tang, Pengju Ma, Hongjie Duan, Yan Zheng, Hongtu Ji, Yunlong Guo, Yihong Wu. Fusing Local and Long-Range Information for Image Feature Point Extraction[J]. Journal of Computer-Aided Design & Computer Graphics, 2025 , 37 (12) : 2147 -2157 . DOI: 10.3724/SP.J.1089.2023-00743

基金

收起

国家自然科学基金(62202468)
中国石油化工股份有限公司“油田生产应急处置应用研究”项目

参考文献

收起

文献

收起

参考文献引证文献

排序方式：

[1]

Balntas V, Lenc K, Vedaldi A, et al. HPatches: a benchmark and evaluation of handcrafted and learned local descriptors[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2017: 3852-3861

[2]

Schönberger J L, Frahm J M. Structure-from-motion revisited[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2016: 4104-4113

[3]

Zhang Z C, Sattler T, Scaramuzza D. Reference pose generation for long-term visual localization via learned features and view synthesis[J]. International Journal of Computer Vision, 2021, 129(4): 821-844

[4]

Campos C, Elvira R, Rodríguez J J G, et al. ORB-SLAM3: an accurate open-source library for visual, visual-inertial, and multimap SLAM[J]. IEEE Transactions on Robotics, 2021, 37(6): 1874-1890

[5]

Lowe D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110

[6]

Harris C, Stephens M. A combined corner and edge detector[OL]. [2023-11-19]. https://web.stanford.edu/class/cs231m/references/harris-stephens.pdf

[7]

Rosten E, Drummond T. Machine learning for high-speed corner detection[C]//Proceedings of the 9th European Conference on Computer Vision. Heidelberg: Springer, 2006: 430-443

[8]

Alcantarilla P F, Bartoli A, Davison A J. KAZE features[C]//Proceedings of the 12th European Conference on Computer Vision. Heidelberg: Springer, 2012: 214-227

[9]

Rublee E, Rabaud V, Konolige K, et al. ORB: an efficient alternative to SIFT or SURF[C]//Proceedings of the International Conference on Computer Vision. Los Alamitos: IEEE Computer Society Press, 2011: 2564-2571

[10]

Calonder M, Lepetit V, Strecha C, et al. BRIEF: binary robust independent elementary features[C]//Proceedings of the 11th European Conference on Computer Vision. Heidelberg: Springer, 2010: 778-792

[11]

DeTone D, Malisiewicz T, Rabinovich A. SuperPoint: self-supervised interest point detection and description[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Los Alamitos: IEEE Computer Society Press, 2018: 337-349

[12]

Dusmanu M, Rocco I, Pajdla T, et al. D2-Net: a trainable CNN for joint description and detection of local features[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2019: 8084-8093

[13]

Luo Z X, Zhou L, Bai X Y, et al. ASLFeat: learning local features of accurate shape and localization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2020: 6588-6597

[14]

Revaud J, Weinzaepfel P, De Souza C, et al. R2D2: repeatable and Reliable detector and descriptor[C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc, 2019: Article No.1113

[15]

Zhao X M, Wu X M, Miao J Y, et al. ALIKE: accurate and lightweight keypoint detection and descriptor extraction[J]. IEEE Transactions on Multimedia, 2023, 25: 3101-3112

[16]

Dusmanu M, Miksik O, Schönberger J L, et al. Cross-descriptor visual localization and mapping[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Los Alamitos: IEEE Computer Society Press, 2021: 6038-6047

[17]

Shen X L, Wang C, Li X, et al. Learning scale awareness in keypoint extraction and description[J]. Pattern Recognition, 2022, 121: Article No.108221

[18]

Fan B, Yang Y Z, Feng W S, et al. Seeing through darkness: visual localization at night via weakly supervised learning of domain invariant features[J]. IEEE Transactions on Multimedia, 2023, 25: 1713-1726

[19]

Wang Z C, Wu C X, Yang Y F, et al. Learning transformation-predictive representations for detection and description of local features[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2023: 11464-11473

[20]

Wang X J, Liu Z Y, Hu Y, et al. FeatureBooster: boosting feature descriptors with a lightweight neural network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2023: 7630-7639

[21]

Guo M H, Lu C Z, Liu Z N, et al. Visual attention network[J]. Computational Visual Media, 2023, 9(4): 733-752

[22]

Dong Rongsheng, Liu Yi, Ma Yuqi, et al. Lightweight network with convolutional attention feature fusion for real-time semantic segmentation[J]. Journal of Computer-Aided Design & Computer Graphics, 2023, 35(6): 935-943 (in Chinese) (董荣胜, 刘意, 马雨琪, 等. 轻量化卷积注意力特征融合网络的实时语义分割[J]. 计算机辅助设计与图形学学报, 2023, 35(6): 935-943)

[23]

Wei Wenxiao, Liu Jieyu, Xu Junhui, et al. Small target detection algorithm based on receptive field amplification feature fusion[J]. Journal of Computer-Aided Design & Computer Graphics, 2023, 35(1): 48-54 (in Chinese) (魏文晓, 刘洁瑜, 徐军辉, 等. 扩增感受野特征融合的小目标检测算法[J]. 计算机辅助设计与图形学学报, 2023, 35(1): 48-54)

[24]

Li Z Q, Snavely N. MegaDepth: learning single-view depth prediction from internet photos[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2018: 2041-2050

[25]

Taira H, Okutomi M, Sattler T, et al. InLoc: indoor visual localization with dense matching and view synthesis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(4): 1293-1307

[26]

Szegedy C, Liu W, Jia Y Q, et al. Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2015: 1-9

[27]

Arandjelović R, Zisserman A. Three things everyone should know to improve object retrieval[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2012: 2911-2918

[28]

Sarlin P E, Cadena C, Siegwart R, et al. From coarse to fine: robust hierarchical localization at large scale[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2019: 12708-12717

2025年第37卷第12期

PDF下载

引用本文

BibTeX

文章信息

doi: 10.3724/SP.J.1089.2023-00743

接收时间：2023-11-19
首发时间：2026-10-10
出版时间：2025-12-20

补充材料

相关文章

文章信息

作者

出版历史

收稿日期：2023-11-19
修回日期：2024-04-24

基金

国家自然科学基金(62202468)

中国石油化工股份有限公司“油田生产应急处置应用研究”项目

作者信息

¹⁾中国科学院自动化研究所多模态人工智能系统全国重点实验室　北京　100190

²⁾中国石油化工股份有限公司胜利油田分公司　东营　257000

通讯作者:

（yihong.wu@ia.ac.cn）

参考文献

分享链接

https://castjournals.cast.org.cn/joweb/jsjfzsj/CN/10.3724/SP.J.1089.2023-00743

分享至

全文二维码

扫描看全文

引用本文

BibTeX

本文的引用情况

2种不同金属材料的力学参数

科 Family	属数 Number of genus	种数 Number of species	占总种数比例 Percentage of total species (%)	属 Genus	种数 Number of species	占总种数比例 Percentage of total species (%)
鹅膏菌科Amanitaceae	2	11	5.26	鹅膏菌属 Amanita	10	4.78
小菇科 Mycenaceae	2	12	5.74	丝盖伞属 Inocybe	5	2.39
多孔菌科 Polyporaceae	8	14	6.70	蜡蘑属 Laccaria	5	2.39
红菇科 Russulaceae	3	23	11.00	小皮伞属 Marasmius	6	2.87
				小菇属 Mycena	11	5.26
				光柄菇属 Pluteus	5	2.39
				红菇属 Russula	17	8.13
				栓菌属 Trametes	5	2.39

关闭全屏

BibTeX
EndNote
RefWorks
TxT