Medical image anomaly detection represents a fundamental challenge in clinical diagnostics，aiming to automatically identify and localize potential lesion regions without manual annotations. Current reconstruction-based anomaly detection methods predominantly rely on features extracted from models pretrained on natural image datasets such as ImageNet. While these approaches have demonstrated promising results，they face considerable limitations due to the substantial domain shifts between natural and medical images. These domain discrepancies lead to semantic misalignments during feature reconstruction，resulting in two primary issues： oversensitivity to normal anatomical variations and insufficient discriminative capability for detecting subtle or early-stage lesions. Such limitations substantially hinder the clinical applicability of existing methods，particularly in screening scenarios where early detection is crucial. To address these challenges and to overcome the dilemma where fine-tuning pretrained encoders often leads to pattern collapse while keeping them frozen limits domain adaptation，this study proposes a cross-domain self-supervised representation learning （CSRL） framework，specifically designed to enhance both the accuracy and robustness of unsupervised medical image anomaly detection through effective domain adaptation and multi-scale feature optimization.

The proposed CSRL framework operates through two carefully designed sequential learning stages，each targeting distinct aspects of the anomaly detection pipeline while maintaining architectural coherence. In the first stage，a domain-adaptive contrastive learning （DACL） network is constructed on the basis of the bootstrap your own latent architecture. This module employs an innovative dual-path online-target network framework to facilitate robust cross-domain transfer of pretrained encoders. The online and target networks process different strongly augmented views of the same medical image through random cropping，color jittering，and Gaussian blurring operations. The target network parameters are updated via an exponential moving average of the online network，with a momentum coefficient of 0.99 to ensure training stability. A symmetric negative cosine similarity loss function is employed to maximize feature consistency between the two views，effectively driving the encoder to learn augmentation-invariant and domain-adaptive representations. This sophisticated learning mechanism enables the model to capture essential medical image characteristics while discarding domain-specific artifacts from natural images，thereby establishing a solid foundation for subsequent anomaly detection tasks. In the second stage，we construct an end-to-end feature reconstruction network specifically optimized for anomaly scoring and precise localization. The encoder component is initialized with the adapted weights from the DACL module and remains completely frozen during this training phase to preserve the learned domain-invariant representations. The network extracts multi-scale deep feature maps at three different resolutions，which are subsequently processed by a specially designed bottleneck layer comprising three residual blocks with skip connections for effective feature fusion. A symmetric decoder architecture with transposed convolutional layers then reconstructs the features at corresponding scales，maintaining spatial integrity throughout the reconstruction process. To enhance the model’s focus on potential lesion regions further，we introduce a synergistic attention enhancement （SAE） module that innovatively integrates three complementary attention branches： coordinate attention for precise spatial localization，channel attention for feature recalibration，and spatial attention for regional emphasis. These attention mechanisms are dynamically aggregated through learnable weighting coefficients to refine feature responses in an adaptive manner. Additionally，a multi-scale anomaly fusion （MAF） module computes cosine distance-based anomaly maps at different scales and employs learnable weights combined with spatial attention to generate the final optimized anomaly map. The entire framework is optimized using a joint loss function that combines reconstruction error，fusion consistency，and sparsity constraints，ensuring comprehensive training supervision across multiple objectives.

Comprehensive experiments were conducted on three publicly available medical image datasets covering diverse modalities，i.e.，Asia Pacific Tele-Ophthalmology Society（APTOS），International Skin Imaging Collaboration（ISIC），and BR35H，to evaluate the proposed method rigorously under various clinical scenarios. The CSRL framework was compared with 10 state-of-the-art methods，including autoencoder-based models （AE，VAE），generative approaches （GANomaly，f-AnoGAN），memory-based models （PaDiM），and recent pretraining-based methods （STFPM，MKD，EDC，EA2D）. All experiments were performed using consistent evaluation protocols and five independent runs to ensure statistical significance. Quantitative results demonstrate that our method achieves optimal performance in image-level anomaly detection tasks. On the BR35H dataset，CSRL achieved an AUC of 99.90%； on the ISIC dataset，an AUC of 91.79%； and on the APTOS dataset，an AUC of 97.71%. Compared with the current state-of-the-art method，the best AUC of our method is improved by 0.35%，3.52%，and 0.02% on the APTOS，ISIC，and BR35H datasets，respectively. Extended ablation studies provided deeper insights into the individual contributions of each module. The DACL module alone improved AUC from 88.23% to 91.72% on the ISIC dataset，highlighting its crucial role in domain adaptation. The SAE module contributed considerably to precise localization，while the MAF module effectively integrated multi-scale information for comprehensive anomaly detection. When combined，the full model achieved 94.74% AUC in its best run，demonstrating the synergistic effect of all components. Qualitative visualization of anomaly localization maps revealed that CSRL produces highly concentrated and clinically interpretable heatmaps that align well with actual lesion contours，outperforming comparison methods in spatial consistency and anatomical plausibility. The generated activation maps show precise correspondence with pathological regions while maintaining minimal responses in healthy tissues，substantially improving clinical interpretability for medical practitioners.

This study presents a novel CSRL framework that effectively addresses the fundamental challenge of semantic discrepancy in transferring pretrained models to medical image anomaly detection. By systematically integrating domain-adaptive contrastive learning，synergistic attention enhancement，and intelligent multi-scale anomaly fusion，the framework achieves remarkable improvements in both discriminative capability and spatial precision of anomaly detection. Extensive experimental validation across multiple medical imaging modalities confirms the effectiveness and generalizability of the proposed approach，establishing new state-of-the-art performance while maintaining clinical interpretability. The framework provides a reliable and scalable solution for unsupervised anomaly detection in medical images，with immediate potential applications in early diagnosis and clinical screening programs. Future work will focus on three key directions： developing more efficient cross-domain adaptation strategies for few-shot and limited-data scenarios，optimizing model complexity and inference speed for practical clinical deployment，and enhancing sensitivity to subtle anomalies through advanced feature alignment mechanisms. These research directions aim to strengthen both the theoretical foundations and practical applicability of self-supervised learning in medical imaging，ultimately contributing to improved healthcare outcomes through more accessible and accurate diagnostic tools.