收藏切换
Infrared and Visible Image Fusion Based on Autoencoder Composed of CNN-transformer
收藏切换
PDF
Lin LI, Yongjian SHEN, Pengyu ZHANG, Hao YUAN, Chao WANG
Journal of Telemetry, Tracking and Command | 2024, 45(5) : 109 - 119
Less
收藏切换
Journal of Telemetry, Tracking and Command | 2024, 45(5): 109-119
Radar and Countermeasures
Infrared and Visible Image Fusion Based on Autoencoder Composed of CNN-transformer
Full
Lin LI, Yongjian SHEN, Pengyu ZHANG, Hao YUAN, Chao WANG
Affiliations
  • Beijing Research Institute of Telemetry, Beijing 100076, China
Published: 2024-09-15 doi: 10.12347/j.ycyk.20240326002
Outline
收藏切换

Image fusion model based on autoencoder network gets more attention because it does not need to design fusion rules manually. However, most autoencoder-based fusion networks use two-stream CNNs with the same structure as the encoder,which are unable to extract global features due to the local receptive field of convolutional operations and lack the ability to extract unique features from infrared and visible images. A novel autoencoder-based image fusion network which consist of encoder module, fusion module and decoder module is constructed in this paper. In the encoder module, the CNN and Transformer are combined to capture the local and global feature of the source images simultaneously. In addition, novel contrast and gradient enhancement feature extraction blocks are designed respectively for infrared and visible images to maintain the information specific to each source images. The feature images obtained by encoder module are concatenated by the fusion module and input to the decoder module to obtain the fused image. Experimental results on three datasets show that the proposed network can better preserve both the clear target and detailed information of infrared and visible images respectively, and outperforms some state-of-the-art methods in both subjective and objective evaluation. Meanwhile, the fused image obtained by the proposed network can acquire the highest mean average precision in the target detection which proves that image fusion is beneficial for downstream tasks.

Image fusion  /  Convolutional neural network  /  Transformer  /  Infrared image  /  Visible image
Lin LI, Yongjian SHEN, Pengyu ZHANG, Hao YUAN, Chao WANG. Infrared and Visible Image Fusion Based on Autoencoder Composed of CNN-transformer[J]. Journal of Telemetry, Tracking and Command, 2024 , 45 (5) : 109 -119 . DOI: 10.12347/j.ycyk.20240326002
Year 2024 volume 45 Issue 5
PDF
88
41
Cite this Article
BibTeX
Article Info
doi: 10.12347/j.ycyk.20240326002
  • Receive Date:2024-03-26
  • Online Date:2026-03-20
  • Published:2024-09-15
Article Data
Affiliations
History
  • Received:2024-03-26
  • Revised:2024-07-12
Affiliations
    Beijing Research Institute of Telemetry, Beijing 100076, China
References
Share
https://castjournals.cast.org.cn/joweb/ycyk/EN/10.12347/j.ycyk.20240326002
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数

Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)

Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT