Synthetic aperture radar (SAR) and optical remote sensing represent two complementary Earth observation modalities. SAR imagery enables all-weather,all-day observation due to its active imaging mechanism,while optical imagery provides higher spatial resolution and more intuitive visual details. Cross-modal translation between SAR and optical images not only compensates for missing data in either modality but also enhances downstream applications,such as image matching and semantic segmentation,through multi-modal information fusion. However,current methods encounter notable challenges. Approaches based on cycle-consistent generative adversarial networks (CycleGAN) predominantly emphasize macroscopic structural reconstruction and fail to exploit deep semantic correlations across source and target domains. In particular,traditional contrastive learning techniques are limited in remote sensing due to the high spatial autocorrelation of similar land-cover features,which blurs the distinction between positive and negative samples and causes the contrastive mechanism to underperform. This study addresses these challenges by proposing a semantic-guided contrastive learning method for SAR and optical image translation,which effectively mitigates feature homogenization in remote sensing image translation and significantly improves semantic fidelity and downstream task performance.
The proposed framework comprises three core components: 1) a semantic feature extraction module,2) a semantic-guided contrastive learning module,and 3) a joint optimization scheme combining cyclic generation and contrastive learning. Initially,this approach utilizes pre-trained SAR and optical semantic segmentation models (DeepLabV3) to extract pixel-level semantic features from real and reconstructed images. Subsequently,the semantic-guided contrastive learning module implements a class-consistency-based positive-negative sample selection strategy: In each query patch in the generated image,patches sharing the same semantic class in the real image serve as positive samples,while those from differing classes are treated as negatives. This strategy effectively mitigates the feature homogenization problem that challenges traditional contrastive learning in remote sensing contexts. Contrastive loss is computed within a shared feature space projected by a lightweight perceptron. Finally,the joint optimization framework integrates cyclic consistency losses at pixel and semantic levels with adversarial losses. The cyclic semantic segmentation loss enforces consistency in structure,texture,and semantics between generated and real images,while adversarial losses enhance image realism. The overall loss function balances these components via weighted hyperparameters.
Comprehensive experiments on two public datasets—WHU-OPT-SAR and DDHRNet—evaluate the proposed method against state-of-the-art approaches,including CycleGAN,contrastive unpaired translation,query-selected attention,and conditional diffusion,across image translation quality and downstream tasks. Regarding image translation,our method consistently achieves superior performance. On WHU-OPT-SAR,SAR-to-optical translation yielded a peak signal-to-noise ratio (PSNR) improvement of 11.9% and a mean absolute error (MAE) reduction of 31.1% over the second-best method; optical-to-SAR translation achieved gains of 3.8% in PSNR,5.6% in structural similarity index measure (SSIM),and a 5.2% reduction in MAE. On DDHRNet,our method sustained leading performance across diverse geographical contexts. In downstream tasks,the semantic segmentation and feature matching results confirm marked gains. The semantic segmentation pixel accuracy improved by 49.26% for optical image generation and 10.60% for SAR image generation on WHU-OPT-SAR,outperforming all baselines. In feature matching tasks,the inlier ratio (IR) and Euclidean distance-based inlier ratio improved by up to 1% and 0.49%,respectively. Qualitative comparisons reveal superior visual quality with enhanced detail preservation and grayscale consistency. Ablation studies demonstrate the necessity of each component: removal of the semantic-guided contrastive learning module caused a 10.75% drop in pixel accuracy for optical images,while omitting the cyclic semantic segmentation loss reduced PSNR by 7.0% in optical-to-SAR translation. These findings validate the critical role of our innovations in overcoming remote sensing translation challenges.
The proposed semantic-guided contrastive learning method for SAR and optical image translation effectively addresses the prevalent issue of feature homogenization in remote sensing image translation. Our method substantially enhances the semantic fidelity and downstream utility of generated images by integrating semantic segmentation guidance with contrastive learning within a cyclic generative framework,offering a novel solution for unsupervised SAR and optical image translation. Extensive experiments demonstrate that this approach provides a robust and effective solution for unpaired SAR-optical image translation with notable advantages in semantic consistency and cross-modal feature alignment. Our main contributions include: 1) identifying the feature homogenization problem inherent in traditional contrastive learning for remote sensing translation; 2) proposing a novel semantic-guided contrastive learning framework with category-consistent sample selection; and 3) developing a unified architecture combining cyclic generation and contrastive learning. Future work will explore self-supervised semantic guidance to improve applicability in scenarios lacking semantic annotations,thereby enhancing generalizability and robustness across diverse remote sensing applications. Our codes are available at https://www.scidb.cn/s/VVVBnu.
| 科 Family | 属数 Number of genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) | 属 Genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) |
|---|---|---|---|---|---|---|
| 鹅膏菌科Amanitaceae | 2 | 11 | 5.26 | 鹅膏菌属 Amanita | 10 | 4.78 |
| 小菇科 Mycenaceae | 2 | 12 | 5.74 | 丝盖伞属 Inocybe | 5 | 2.39 |
| 多孔菌科 Polyporaceae | 8 | 14 | 6.70 | 蜡蘑属 Laccaria | 5 | 2.39 |
| 红菇科 Russulaceae | 3 | 23 | 11.00 | 小皮伞属 Marasmius | 6 | 2.87 |
| 小菇属 Mycena | 11 | 5.26 | ||||
| 光柄菇属 Pluteus | 5 | 2.39 | ||||
| 红菇属 Russula | 17 | 8.13 | ||||
| 栓菌属 Trametes | 5 | 2.39 |