Remote sensing change captioning (RSCC) aims to automatically generate natural language descriptions for changes that occur between remote sensing images acquired over the same geographic region. Unlike traditional change detection,which mainly focuses on determining whether changes occur and where they are located,RSCC further requires the model to understand the semantic category,spatial position,and transformation relationship of changed objects. As a result,RSCC becomes a challenging task that integrates visual change perception,bitemporal feature interaction,and language generation. Thus far,existing methods have made considerable progress by introducing attention mechanisms,Transformer-based structures,and generative modeling strategies. However,several problems remain. First,the deep features extracted by visual backbones usually contain a large amount of unchanged background information,while the truly changed regions occupy only a small portion of the image. As a result,key changed regions and discriminative semantic channels may not be sufficiently highlighted before difference modeling. Second,the relationship between bitemporal features is not limited to simple subtraction. The original pre- and post-change features,change magnitudes,multiplicative interactions,and semantic similarities may all contribute to change understanding. Directly relying on a single difference representation or simple feature concatenation may be insufficient to describe complex changes. Third,some recent methods have been shown to achieve better performance by increasing model complexity,but the balance between captioning performance and model size remains important for practical remote sensing applications. To address the abovementioned problems,this paper proposes an RSCC framework guided by bitemporal feature enhancement and multirelation difference coupling.
The proposed framework follows an encoder-decoder structure. RemoteCLIP-RN50,which is a RemoteCLIP with a residual network-50 backbone,is adopted as the shared visual feature extractor to encode pre- and post-change remote sensing images. The two temporal images are processed by the same backbone to ensure that their visual features are represented in a unified semantic space. After feature extraction,a bitemporal spatial-channel enhancement (BSCE) strategy is introduced before difference modeling. The BSCE strategy then performs channel enhancement and spatial enhancement on the features of both temporal images. Channel enhancement uses global contextual information to recalibrate semantic channels,thereby allowing channels related to changed objects to obtain stronger responses. Spatial enhancement further emphasizes local regions with change potential and suppresses irrelevant background responses. In this way,the enhanced bitemporal features provide clearer and more discriminative inputs for subsequent difference modeling. On this basis,a multirelation difference coupling (MRDC) unit is constructed. Instead of using only simple subtraction,MRDC jointly models the original bitemporal features,absolute difference,multiplicative interaction,and cosine similarity. Here,absolute difference represents local change magnitude between the two temporal features. Multiplicative interaction captures coresponse and feature interaction between the two temporal images. Cosine similarity describes semantic consistency between corresponding spatial positions. These complementary relationships are concatenated and compressed through feature fusion layers to obtain a compact visual change representation. The fused feature map is then flattened into a visual memory sequence. Finally,a Transformer-based text decoder is employed to generate change captions autoregressively. During training,the model is optimized using cross-entropy loss under the teacher-forcing strategy. During inference,greedy search is adopted to generate the final change description.
Experiments on the LEVIR-CC dataset reveal that the proposed method achieves 83.62% in BLEU-1,60.22% in BLEU-4,64.94% in ROUGE-L,and 128.58% in CIDEr,with BLEU-1 and CIDEr outperforming the comparison methods. Meanwhile,the proposed method contains 41.50 M parameters—a value that is lower than several classic and recent representative methods,demonstrating a favorable balance between performance and model complexity. Additional experiments on the DUBAI-CC dataset show that the proposed method achieves 63.75% in BLEU-1,34.14% in BLEU-4,56.62% in ROUGE-L,and 90.09% in CIDEr. The best performance is obtained in BLEU-4,ROUGE-L,and CIDEr,thus indicating its applicability across different datasets. Ablation studies demonstrate that both the bitemporal feature enhancement strategy and the MRDC unit improve change captioning performance. Further relation synergy experiments reveal that,after bitemporal feature enhancement,complete multirelation coupling achieves the best performance in BLEU-1,ROUGE-L,and CIDEr,thus indicating that feature enhancement helps improve the complementary representation of different difference relations.
The proposed method designs a targeted bitemporal change representation process for RSCC. By introducing BSCE before difference modeling and constructing MRDC to jointly represent multiple change relations,the method improves the discriminability of visual change features and provides more effective visual cues for language generation. Furthermore,experimental results on LEVIR-CC and DUBAI-CC demonstrate that the proposed method not only achieves a favorable balance between captioning performance and model complexity but also shows good applicability across different datasets. The ablation,relation decomposition,synergy,and visualization analyses further confirm that the performance gain is achieved from the cooperation between bitemporal feature enhancement and MRDC. Although the proposed method still has room for improvement in fine-grained word-level matching and complex scene description,it provides a practical and relatively simple framework for RSCC. Future works will explore stronger language decoding strategies,cross-dataset semantic alignment,multidataset joint training,and lightweight deployment to further improve the robustness and generalization ability of RSCC models.
| 科 Family | 属数 Number of genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) | 属 Genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) |
|---|---|---|---|---|---|---|
| 鹅膏菌科Amanitaceae | 2 | 11 | 5.26 | 鹅膏菌属 Amanita | 10 | 4.78 |
| 小菇科 Mycenaceae | 2 | 12 | 5.74 | 丝盖伞属 Inocybe | 5 | 2.39 |
| 多孔菌科 Polyporaceae | 8 | 14 | 6.70 | 蜡蘑属 Laccaria | 5 | 2.39 |
| 红菇科 Russulaceae | 3 | 23 | 11.00 | 小皮伞属 Marasmius | 6 | 2.87 |
| 小菇属 Mycena | 11 | 5.26 | ||||
| 光柄菇属 Pluteus | 5 | 2.39 | ||||
| 红菇属 Russula | 17 | 8.13 | ||||
| 栓菌属 Trametes | 5 | 2.39 |