收藏切换
Bi-temporal feature enhancement-guided multi-relational difference coupling for remote sensing change captioning
收藏切换
Jiexiao Yu, Yujie Fu, Jing Liu*
Journal of Image and Graphics | 2026, 31(9) : 3293 - 3307
Less
收藏切换
Journal of Image and Graphics | 2026, 31(9): 3293-3307
• The intelligent empowerment of large models in the aviation field •
Bi-temporal feature enhancement-guided multi-relational difference coupling for remote sensing change captioning
Full
Jiexiao Yu, Yujie Fu, Jing Liu*
Affiliations
  • School of Electrical and Information Engineering,Tianjin University,Tianjin 300072,China
Published: 2026-09-16 doi: 10.11834/jig.260171
Outline
收藏切换
Objective

Remote sensing change captioning (RSCC) aims to automatically generate natural language descriptions for changes that occur between remote sensing images acquired over the same geographic region. Unlike traditional change detection,which mainly focuses on determining whether changes occur and where they are located,RSCC further requires the model to understand the semantic category,spatial position,and transformation relationship of changed objects. As a result,RSCC becomes a challenging task that integrates visual change perception,bitemporal feature interaction,and language generation. Thus far,existing methods have made considerable progress by introducing attention mechanisms,Transformer-based structures,and generative modeling strategies. However,several problems remain. First,the deep features extracted by visual backbones usually contain a large amount of unchanged background information,while the truly changed regions occupy only a small portion of the image. As a result,key changed regions and discriminative semantic channels may not be sufficiently highlighted before difference modeling. Second,the relationship between bitemporal features is not limited to simple subtraction. The original pre- and post-change features,change magnitudes,multiplicative interactions,and semantic similarities may all contribute to change understanding. Directly relying on a single difference representation or simple feature concatenation may be insufficient to describe complex changes. Third,some recent methods have been shown to achieve better performance by increasing model complexity,but the balance between captioning performance and model size remains important for practical remote sensing applications. To address the abovementioned problems,this paper proposes an RSCC framework guided by bitemporal feature enhancement and multirelation difference coupling.

Method

The proposed framework follows an encoder-decoder structure. RemoteCLIP-RN50,which is a RemoteCLIP with a residual network-50 backbone,is adopted as the shared visual feature extractor to encode pre- and post-change remote sensing images. The two temporal images are processed by the same backbone to ensure that their visual features are represented in a unified semantic space. After feature extraction,a bitemporal spatial-channel enhancement (BSCE) strategy is introduced before difference modeling. The BSCE strategy then performs channel enhancement and spatial enhancement on the features of both temporal images. Channel enhancement uses global contextual information to recalibrate semantic channels,thereby allowing channels related to changed objects to obtain stronger responses. Spatial enhancement further emphasizes local regions with change potential and suppresses irrelevant background responses. In this way,the enhanced bitemporal features provide clearer and more discriminative inputs for subsequent difference modeling. On this basis,a multirelation difference coupling (MRDC) unit is constructed. Instead of using only simple subtraction,MRDC jointly models the original bitemporal features,absolute difference,multiplicative interaction,and cosine similarity. Here,absolute difference represents local change magnitude between the two temporal features. Multiplicative interaction captures coresponse and feature interaction between the two temporal images. Cosine similarity describes semantic consistency between corresponding spatial positions. These complementary relationships are concatenated and compressed through feature fusion layers to obtain a compact visual change representation. The fused feature map is then flattened into a visual memory sequence. Finally,a Transformer-based text decoder is employed to generate change captions autoregressively. During training,the model is optimized using cross-entropy loss under the teacher-forcing strategy. During inference,greedy search is adopted to generate the final change description.

Result

Experiments on the LEVIR-CC dataset reveal that the proposed method achieves 83.62% in BLEU-1,60.22% in BLEU-4,64.94% in ROUGE-L,and 128.58% in CIDEr,with BLEU-1 and CIDEr outperforming the comparison methods. Meanwhile,the proposed method contains 41.50 M parameters—a value that is lower than several classic and recent representative methods,demonstrating a favorable balance between performance and model complexity. Additional experiments on the DUBAI-CC dataset show that the proposed method achieves 63.75% in BLEU-1,34.14% in BLEU-4,56.62% in ROUGE-L,and 90.09% in CIDEr. The best performance is obtained in BLEU-4,ROUGE-L,and CIDEr,thus indicating its applicability across different datasets. Ablation studies demonstrate that both the bitemporal feature enhancement strategy and the MRDC unit improve change captioning performance. Further relation synergy experiments reveal that,after bitemporal feature enhancement,complete multirelation coupling achieves the best performance in BLEU-1,ROUGE-L,and CIDEr,thus indicating that feature enhancement helps improve the complementary representation of different difference relations.

Conclusion

The proposed method designs a targeted bitemporal change representation process for RSCC. By introducing BSCE before difference modeling and constructing MRDC to jointly represent multiple change relations,the method improves the discriminability of visual change features and provides more effective visual cues for language generation. Furthermore,experimental results on LEVIR-CC and DUBAI-CC demonstrate that the proposed method not only achieves a favorable balance between captioning performance and model complexity but also shows good applicability across different datasets. The ablation,relation decomposition,synergy,and visualization analyses further confirm that the performance gain is achieved from the cooperation between bitemporal feature enhancement and MRDC. Although the proposed method still has room for improvement in fine-grained word-level matching and complex scene description,it provides a practical and relatively simple framework for RSCC. Future works will explore stronger language decoding strategies,cross-dataset semantic alignment,multidataset joint training,and lightweight deployment to further improve the robustness and generalization ability of RSCC models.

remote sensing change captioning(RSCC)  /  bi-temporal remote sensing image  /  feature enhancement  /  multi-relation difference coupling  /  change description
于洁潇, 付雨杰, 刘婧. 双时相特征增强引导的多关系差异耦合遥感变化字幕生成. 中国图象图形学报, 2026 , 31 (9) : 3293 -3307 . DOI: 10.11834/jig.260171
Jiexiao Yu, Yujie Fu, Jing Liu. Bi-temporal feature enhancement-guided multi-relational difference coupling for remote sensing change captioning[J]. Journal of Image and Graphics, 2026 , 31 (9) : 3293 -3307 . DOI: 10.11834/jig.260171
  • National Natural Science Foundation of China(62371333);Fundamental Research Foundation of the National Key Laboratory of Automatic Target Recognition(WDZC20265290201)
Year 2026 volume 31 Issue 9
22
10
Cite this Article
BibTeX
Article Info
doi: 10.11834/jig.260171
  • Receive Date:2026-04-13
  • Online Date:2026-10-30
  • Published:2026-09-16
Article Data
Affiliations
History
  • Received:2026-04-13
  • Revised:2026-06-02
Funding
National Natural Science Foundation of China(62371333);Fundamental Research Foundation of the National Key Laboratory of Automatic Target Recognition(WDZC20265290201)
Affiliations
    School of Electrical and Information Engineering,Tianjin University,Tianjin 300072,China
References
Share
https://castjournals.cast.org.cn/joweb/zgtxtxxb/EN/10.11834/jig.260171
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT