收藏切换
A Method for Infrared and Visible Image Fusion Based on CNN-Mamba Feature Extraction Combined with Information Selection
收藏切换
Xiaocong WU1, Xin FENG1, Meina HUANG1, Bin ZHAO1, Kaiqun HU1, Yu SHEN2
Acta Photonica Sinica | 2026, 55(1) : 144 - 160
Less
收藏切换
Acta Photonica Sinica | 2026, 55(1): 144-160
• Image Processing •
A Method for Infrared and Visible Image Fusion Based on CNN-Mamba Feature Extraction Combined with Information Selection
Full
Xiaocong WU1, Xin FENG1, Meina HUANG1, Bin ZHAO1, Kaiqun HU1, Yu SHEN2
Affiliations
  • 1School of Mechanical Engineering, Chongqing Key Laboratory of Green Design and Manufacturing of Intelligent Equipment, Chongqing Technology and Business University, Chongqing 400067, China
  • 2School of Electronic and Information Engineering,Lanzhou Jiaotong University, Lanzhou 730070, China
Published: 2026-01-25 doi: 10.3788/gzxb20265501.0110001
Outline
收藏切换

Infrared sensors capture the thermal radiation emitted from the surface of objects to create images. However, infrared images are prone to noise interference and are limited in capturing fine details and texture information. In contrast, visible sensors capture reflected light from the surface of objects, providing rich details and texture information that align with human visual perception. However, the quality of visible images can be unstable due to factors such as lighting conditions. Therefore, effectively fusing infrared and visible images to generate high-quality images is an important research direction.

Currently, image fusion methods for infrared and visible images can be broadly categorised into traditional fusion strategies and deep learning-based fusion strategies. Traditional methods struggle to effectively integrate the feature information from the two modalities, especially in complex fusion scenarios. Deep learning-based fusion methods, such as ResNet and DenseNet, can better handle multi-modal information, but they involve complex training processes and high computational overhead. To address these challenges, this paper proposes an infrared and visible image fusion method that combines CNN-Mamba feature extraction with an information selection mechanism. The aim is to overcome the shortcomings of traditional methods in complex scenarios while simplifying the training process of deep learning-based fusion strategies. In the image encoding phase, we introduce the Spatial Context Aware Module (SCAM) for the first time in the field of image fusion, which integrates contextual feature information and allocates attention weights adaptively. Then, we design the CNN-Mamba Feature Extract (CMFE) module, which integrates CNN and Mamba networks to perform deep feature extraction on both modalities of the image.In the fusion phase, considering the information differences between infrared and visible images in the same scene, we propose a novel adaptive weight selection fusion layer to retain more source information. Specifically, we calculate the normalised mutual information between the two modalities and set a threshold. When the normalised mutual information exceeds the threshold, the fusion weights are set to be equal to each other. When the mutual information is below the threshold, we assign higher weights to the modality with higher information entropy to preserve more source information. Finally, in the decoding phase, we utilise a decoder consisting of four CNN modules to progressively reduce the number of channels and reconstruct the fused image.

During the experimental phase, ablation studies validate the rationality of the proposed network structure and loss function design. When compared with seven advanced fusion methods, our method demonstrates excellent performance on the MSRS, Road-Scene, TNO, and M3FD public datasets, ranking first in at least three objective metrics. Moreover, the EN and CC values achieve the highest in all tests. This indicates that the proposed method effectively facilitates cross-modal information interaction and preserves the original information as much as possible. Additionally, pedestrian and vehicle detection experiments were conducted on the Road-Scene dataset, and the results show that our method achieved the optimal performance in both pedestrian and vehicle detection, further validating its practical application advantages.

In conclusion, the proposed method demonstrates significant advantages in both image fusion quality and subsequent applications.

Image fusion  /  Normalized mutual information  /  CMFE Module  /  Information Entropy  /  Adaptive weight selection fusion layer
仵晓聪, 冯鑫, 黄美娜, 赵彬, 胡开群, 沈瑜. 基于CNN-Mamba特征提取结合信息选择的红外与可见光图像融合方法. 光子学报, 2026 , 55 (1) : 144 -160 . DOI: 10.3788/gzxb20265501.0110001
Xiaocong WU, Xin FENG, Meina HUANG, Bin ZHAO, Kaiqun HU, Yu SHEN. A Method for Infrared and Visible Image Fusion Based on CNN-Mamba Feature Extraction Combined with Information Selection[J]. Acta Photonica Sinica, 2026 , 55 (1) : 144 -160 . DOI: 10.3788/gzxb20265501.0110001
Year 2026 volume 55 Issue 1
114
51
Cite this Article
BibTeX
Article Info
doi: 10.3788/gzxb20265501.0110001
  • Receive Date:2025-07-23
  • Online Date:2026-10-10
  • Published:2026-01-25
Article Data
Affiliations
History
  • Received:2025-07-23
  • Accepted:2025-09-22
Affiliations
    1School of Mechanical Engineering, Chongqing Key Laboratory of Green Design and Manufacturing of Intelligent Equipment, Chongqing Technology and Business University, Chongqing 400067, China
    2School of Electronic and Information Engineering,Lanzhou Jiaotong University, Lanzhou 730070, China
References
Share
https://castjournals.cast.org.cn/joweb/gzxb/EN/10.3788/gzxb20265501.0110001
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT