Latest ArticlesHyperspectral images (HSIs) capture reflectance values across hundreds of contiguous spectral bands at each spatial location,thus facilitating the precise discrimination of materials with subtle spectral differences by generating rich spectral-spatial information. This high spectral resolution makes HSIs particularly valuable for many application,such as environmental monitoring,land cover mapping,precision agriculture,and change detection (CD) applications. CD in multitemporal HSIs aims to identify meaningful surface alterations by comparing images of the same scene acquired at different times. Nowadays,accurate HSI-CD plays a critical role in ecological monitoring,urban expansion analysis,and disaster assessment. However,this task faces significant challenges,including high dimensionality,spectral redundancy,spatial heterogeneity,and the difficulty of effectively integrating spatial,spectral,and temporal features. Traditional methods like change vector analysis (CVA),principal component analysis (PCA),and multivariate alteration detection (MAD),rely on hand-crafted features and empirical thresholding. Despite being computationally efficient,these approaches are sensitive to noise,illumination variations,and atmospheric effects. Deep learning methods,particularly convolutional neural networks (CNNs),have demonstrated superior performance by automatically learning hierarchical features. While most CNN-based frameworks adopt dual-branch Siamese architectures,they still struggle to capture long-range dependencies and explicit temporal dynamics.
To address the abovementioned limitations,a novel time-difference-guided network (TDG-Net) is introduced in this paper. The proposed method employs a Siamese architecture with Vision Mamba as the feature extraction backbone to efficiently capture long-range spatial dependencies with linear computational complexity. Two core components are integrated: the time storage module (TSM) and the temporal difference guidance strategy. Bitemporal HSIs are first fed into the dual-branch Vision Mamba backbone to extract multilevel spatial-spectral features. Then,the TSM performs sequential temporal modeling on these features,using a simplified long short-term memory (sLSTM). To reduce computational cost,spectral compression via 1 × 1 convolution and spatial down-sampling via max pooling are applied before feeding features into the sLSTM. The sLSTM generates explicit temporal difference representations. The temporal difference guidance strategy then converts these low-resolution difference features into spatial attention weights through bilinear up-sampling and ReLU activation. These weights are then fed back to the dual-branch network via residual connections,adaptively emphasizing change-relevant regions and suppressing unchanged areas at each hierarchical level. Finally,to produce the binary change detection map,the enhanced multilevel features are fused and passed to the classification head. To mitigate severe class imbalance,Dice loss is employed instead of conventional cross-entropy loss,directly optimizing the overlap between predicted and ground-truth change regions. Compared with recent Mamba-based methods,the core distinction of TDG-Net lies in its explicit modeling and hierarchical guidance of temporal differences across multiple feature levels via a lightweight TSM and residual feedback,rather than relying primarily on high-level implicit fusion.
Comprehensive experiments are conducted on three widely used benchmark hyperspectral datasets: River,Farmland,and Hermiston. All experiments are implemented on an NVIDIA RTX 3090 GPU using TensorFlow-GPU 2.5.0. Performance is then evaluated using overall accuracy (OA) and Kappa coefficient,with results averaged over 10 independent runs. Ablation studies confirm the significant contributions of the TSM and temporal difference guidance strategy. Comparative experiments against state-of-the-art methods demonstrate that TDG-Net consistently outperforms all competitors. On the River dataset,TDG-Net achieves an OA of 96.54% and a Kappa of 78.51%. On the Hermiston dataset,it reaches an OA of 98.21% and a Kappa of 91.79%. On the Farmland dataset,it attains an OA of 95.87% and a Kappa of 90.11%. Additional analyses on model complexity show that TDG-Net maintains competitive parameter count,floating point operations (FLOPs),and inference time while achieving superior accuracy.
The proposed TDG-Net effectively addresses the key limitations of existing HIS CD methods by explicitly modeling temporal differences at multiple hierarchical levels and guiding feature learning with adaptive attention. Furthermore,by integrating a lightweight TSM based on simplified LSTM and a temporal difference guidance strategy within a Vision Mamba backbone,the method captures rich spatial-spectral-temporal dynamics while effectively suppressing pseudo-changes. The introduction of Dice loss further alleviates the severe class imbalance problem. Extensive experiments on three benchmark datasets demonstrate that TDG-Net achieves state-of-the-art performance in terms of quantitative metrics and visual quality,exhibiting strong robustness and generalization in complex scenarios. Additionally,compared with existing Mamba-based methods,such as SAVDGN and CDMamba,which primarily rely on implicit high-level fusion,TDG-Net explicitly models and hierarchically guides temporal differences across multiple feature levels through a lightweight TSM and residual feedback mechanism,thus achieving more fine-grained change representation.
The robust matching of multitemporal and multimodal remote sensing images is a fundamental prerequisite for image registration,change detection (CD),target localization,image fusion,and three-dimensional reconstruction in aerial and satellite remote sensing applications. However,stable correspondence estimation remains highly challenging when the matching process is simultaneously affected by significant spectral discrepancy,complex seasonal variation,repetitive textures,weakly textured regions,and large appearance inconsistency across acquisition times. In visible-to-near-infrared (NIR) remote sensing scenarios,the same geographic area may present markedly different reflectance patterns,local structures,and texture distributions under different seasons and spectral bands,making traditional hand-crafted descriptors and many existing deep matching methods unreliable when they depend mainly on local photometric similarity or low-level geometric consistency. To address these issues,this paper studies the problem from the joint perspectives of benchmark construction and semantically enhanced matching design. With the goal of establishing reliable geometric supervision under a unified evaluation protocol,we constructed a standardized benchmark based on the public seasonal contrast (SeCo) dataset. This is achieved by organizing cross-temporal visible and near-IR image pairs and introducing random homography perturbations. The resulting benchmark contains 15 000 image pairs,including 12 000 training pairs and 3 000 testing pairs,and provides a reproducible experimental setting for quantitative comparison under severe temporal and spectral changes. The proposed protocol also evaluates models using homography reprojection error and multithreshold matching homography accuracy (MHA),thus enabling a unified assessment of geometric precision and matching robustness. Building upon this benchmark,a semantically enhanced cross-modal feature matching framework is proposed on the basis of XoFTR —— a hierarchical coarse-to-fine architecture for cross-modal correspondence estimation. The proposed method injects dense semantic priors extracted by a frozen DINOv3 model into the coarse matching stage of XoFTR through a lightweight Semantic Boosting Module,thereby enabling joint modeling of high-level semantic consistency and geometric correspondence. For each modality branch,the DINOv3 features are aligned to the coarse feature resolution and fused with geometric tokens,allowing the model to emphasize semantically stable regions while preserving local geometric sensitivity. This design is important because the coarse stage determines the candidate regions explored by subsequent fine matching; here,later local refinement is unlikely to recover correct matches once the global correspondence field is biased by unstable appearance cues. The overall pipeline follows a progressive coarse-to-fine paradigm in which multiscale geometric features are first extracted from visible and near-infrared inputs,coarse tokens are enhanced by DINOv3-derived semantic priors,global correspondences are established through transformer-based context interaction,and local fine-level refinement is subsequently performed to improve correspondence precision. Furthermore,training is conducted in a single-stage in-domain optimization manner in which the DINOv3 encoder is kept frozen throughout optimization and the XoFTR backbone is jointly learned together with the semantic boosting components. This strategy allows the method to leverage semantic priors from large vision models in a lightweight and reproducible manner without introducing excessive trainable parameters or unstable large-model adaptation behavior. Extensive experiments under the unified SeCo evaluation protocol demonstrate that the proposed method outperforms the XoFTR baselines across all MHA thresholds. On the constructed benchmark,the proposed model achieves 33.03%,37.03%,and 39.17% in MHA@3,MHA@5,and MHA@7,respectively. These outperform the original pretrained XoFTR model by 12.91%,7.68%,and 4.62% and surpass the direct in-domain fine-tuning baseline by 14.64%,15.85%,and 16.61%,respectively. Qualitative comparisons further reveal that the proposed framework suppresses cross-region mismatches and yields denser correct correspondences in scenes with strong temporal appearance shifts and cross-modal ambiguity. These results indicate that simply adapting a geometric matching network to the target domain is insufficient for robust multitemporal multimodal remote sensing matching. In comparison,integrating high-level semantic priors from a vision foundation model can significantly enhance the stability of global correspondence establishment and improve final geometric estimation accuracy. The study reveals that high-level semantic representations learned by large-scale vision models can effectively alleviate the representation discrepancy caused by temporal evolution and spectral inconsistency. Furthermore,these can be incorporated into a hierarchical cross-modal matching framework in a computationally efficient and practically effective manner,thereby providing a lightweight and reproducible semantic injection route for large-model-empowered high-precision remote sensing image matching. The source code is publicly available at https://github.com/heng-shan/Dino_ft.
Remote sensing change captioning (RSCC) aims to automatically generate natural language descriptions for changes that occur between remote sensing images acquired over the same geographic region. Unlike traditional change detection,which mainly focuses on determining whether changes occur and where they are located,RSCC further requires the model to understand the semantic category,spatial position,and transformation relationship of changed objects. As a result,RSCC becomes a challenging task that integrates visual change perception,bitemporal feature interaction,and language generation. Thus far,existing methods have made considerable progress by introducing attention mechanisms,Transformer-based structures,and generative modeling strategies. However,several problems remain. First,the deep features extracted by visual backbones usually contain a large amount of unchanged background information,while the truly changed regions occupy only a small portion of the image. As a result,key changed regions and discriminative semantic channels may not be sufficiently highlighted before difference modeling. Second,the relationship between bitemporal features is not limited to simple subtraction. The original pre- and post-change features,change magnitudes,multiplicative interactions,and semantic similarities may all contribute to change understanding. Directly relying on a single difference representation or simple feature concatenation may be insufficient to describe complex changes. Third,some recent methods have been shown to achieve better performance by increasing model complexity,but the balance between captioning performance and model size remains important for practical remote sensing applications. To address the abovementioned problems,this paper proposes an RSCC framework guided by bitemporal feature enhancement and multirelation difference coupling.
The proposed framework follows an encoder-decoder structure. RemoteCLIP-RN50,which is a RemoteCLIP with a residual network-50 backbone,is adopted as the shared visual feature extractor to encode pre- and post-change remote sensing images. The two temporal images are processed by the same backbone to ensure that their visual features are represented in a unified semantic space. After feature extraction,a bitemporal spatial-channel enhancement (BSCE) strategy is introduced before difference modeling. The BSCE strategy then performs channel enhancement and spatial enhancement on the features of both temporal images. Channel enhancement uses global contextual information to recalibrate semantic channels,thereby allowing channels related to changed objects to obtain stronger responses. Spatial enhancement further emphasizes local regions with change potential and suppresses irrelevant background responses. In this way,the enhanced bitemporal features provide clearer and more discriminative inputs for subsequent difference modeling. On this basis,a multirelation difference coupling (MRDC) unit is constructed. Instead of using only simple subtraction,MRDC jointly models the original bitemporal features,absolute difference,multiplicative interaction,and cosine similarity. Here,absolute difference represents local change magnitude between the two temporal features. Multiplicative interaction captures coresponse and feature interaction between the two temporal images. Cosine similarity describes semantic consistency between corresponding spatial positions. These complementary relationships are concatenated and compressed through feature fusion layers to obtain a compact visual change representation. The fused feature map is then flattened into a visual memory sequence. Finally,a Transformer-based text decoder is employed to generate change captions autoregressively. During training,the model is optimized using cross-entropy loss under the teacher-forcing strategy. During inference,greedy search is adopted to generate the final change description.
Experiments on the LEVIR-CC dataset reveal that the proposed method achieves 83.62% in BLEU-1,60.22% in BLEU-4,64.94% in ROUGE-L,and 128.58% in CIDEr,with BLEU-1 and CIDEr outperforming the comparison methods. Meanwhile,the proposed method contains 41.50 M parameters—a value that is lower than several classic and recent representative methods,demonstrating a favorable balance between performance and model complexity. Additional experiments on the DUBAI-CC dataset show that the proposed method achieves 63.75% in BLEU-1,34.14% in BLEU-4,56.62% in ROUGE-L,and 90.09% in CIDEr. The best performance is obtained in BLEU-4,ROUGE-L,and CIDEr,thus indicating its applicability across different datasets. Ablation studies demonstrate that both the bitemporal feature enhancement strategy and the MRDC unit improve change captioning performance. Further relation synergy experiments reveal that,after bitemporal feature enhancement,complete multirelation coupling achieves the best performance in BLEU-1,ROUGE-L,and CIDEr,thus indicating that feature enhancement helps improve the complementary representation of different difference relations.
The proposed method designs a targeted bitemporal change representation process for RSCC. By introducing BSCE before difference modeling and constructing MRDC to jointly represent multiple change relations,the method improves the discriminability of visual change features and provides more effective visual cues for language generation. Furthermore,experimental results on LEVIR-CC and DUBAI-CC demonstrate that the proposed method not only achieves a favorable balance between captioning performance and model complexity but also shows good applicability across different datasets. The ablation,relation decomposition,synergy,and visualization analyses further confirm that the performance gain is achieved from the cooperation between bitemporal feature enhancement and MRDC. Although the proposed method still has room for improvement in fine-grained word-level matching and complex scene description,it provides a practical and relatively simple framework for RSCC. Future works will explore stronger language decoding strategies,cross-dataset semantic alignment,multidataset joint training,and lightweight deployment to further improve the robustness and generalization ability of RSCC models.
Aerial RGB-infrared (IR) object detection has received increasing attention in remote sensing because visible and IR images provide complementary information under complex imaging conditions. Visible images preserve rich texture,color,and structural details,whereas IR images are less sensitive to low illumination and can highlight thermal targets at night or in low-light scenes. However,the contribution of each modality changes with the imaging factors involved,such as illumination,exposure status,texture clarity,target-background contrast,background clutter,and artificial light interference. To date,existing methods have improved detection performance through cross-modal alignment,feature interaction,attention reweighting,and multiscale fusion. Nevertheless,most of them still focus on feature-level fusion and lack explicit modality reliability modeling. In other words,they are unable to sufficiently estimate how reliable each modality is under the current imaging condition,or how much each modality should contribute to detection. While some language-guided and condition-aware methods introduce semantic cues into multimodal detection,they usually rely on coarse scene descriptions,category-level prompts,or additional large-model branches during inference. These strategies are insufficient for characterizing modality quality attributes that are directly related to detection reliability. They may also increase the deployment burden on computation-constrained aerial platforms. To address these issues,this paper proposes a modality reliability modeling method for aerial RGB-IR object detection. This modeling method transfers modality quality perception from a vision-language model to the detector during training,thus enabling adaptive multimodal fusion without additional large-model inference cost.
The proposed method consists of structured modality quality description,semantic prior distillation,and reliability-aware adaptive fusion. First,a modality quality attribute description dataset is constructed for UAV-oriented aerial scenes,thus providing structured supervision for modality reliability learning. Instead of using only category labels or coarse scene tags,the annotation scheme explicitly describes key imaging factors that affect detection performance in both modalities. For the RGB modality,the attributes are illumination condition,exposure status,texture clarity,artificial light interference,and background clutter. For the IR modality,the attributes are target-background contrast,boundary clarity,and background cleanliness. Second,a vision-language model is used to encode the modality quality descriptions and generate semantic priors related to RGB and IR reliability. These priors are used only during training. Here,the detector is guided to learn detection-oriented reliability representations through the combination of semantic distillation with attribute supervision. Thus,modality quality perception is internalized into the visual detection network. Third,a global-local adaptive fusion mechanism is designed on the basis of learned reliability representations. Global scene reliability captures the overall effectiveness of RGB and IR cues under the current imaging condition,while local spatial reliability further adjusts the modality contribution at different spatial positions. In this way,the detector can dynamically fuse RGB and IR features in accordance with scene- and region-level reliability. Given that semantic priors are used only during training,the proposed framework does not require additional text prompts,language branches,or large-model participation during inference.
Experiments are conducted on DroneVehicle and VEDAI,two public aerial RGB-IR object detection datasets. On DroneVehicle,the proposed method achieves 79.7% mAP@0.5 and 53.7% mAP@0.5:0.95. On VEDAI,it achieves 67.1% mAP@0.5 and 29.8% mAP@0.5:0.95. The method also shows strong robustness in challenging scenarios,especially under nighttime,low-light,and complex interference conditions. Ablation studies further verify the effectiveness of modality quality attribute modeling,semantic prior distillation,and joint global-local modality reliability modeling. Additionally,the proposed method maintains good inference efficiency because no extra-large-model branch is introduced during testing.
This paper presents a modality reliability modeling method for aerial RGB-IR object detection. To encode modality quality descriptions and provide semantic supervision,the proposed method uses a vision-language model only during training. The detector then learns reliability-aware representations through semantic distillation and attribute supervision. By jointly modeling global scene reliability and local spatial reliability,the detector can adaptively adjust the contributions of visible and IR modalities under varying imaging conditions. Finally,experimental results on DroneVehicle and VEDAI demonstrate that the proposed method improves detection accuracy and robustness,especially in nighttime,low-light,and cluttered scenes.
Infrared video satellites serve as critical tools for detecting aerial moving targets,with infrared small target detection technology forming the essential foundation for this capability. The rapid advancement of deep learning has yielded numerous single-frame detection datasets and methodologies,enabling substantial progress in identifying spatially salient targets. However,moving aerial targets captured by infrared satellites typically exhibit low spatial salience and frequently occur in complex scenarios,rendering single-frame detection methods reliant on spatial information ineffective in such challenging conditions. This urgent challenge necessitates the development of multiframe infrared small and dim target detection techniques. A major bottleneck restricting the advancement and practical application of aerial moving target detection technology has been the lack of dedicated datasets for infrared video satellite-based detection,primarily due to the difficulties and high costs associated with data collection and annotation. To address this gap and promote technological development,we construct the first infrared video satellite aerial moving target detection dataset containing many real-world scenarios,called SatVideoIRSDT dataset,and organized the inaugural detection competition based on this dataset.
First,we collect 20 126 frames of real infrared video satellite data from Wuhan No.1 satellite featuring aerial moving targets and annotate 29 757 aerial targets. During the data annotation phase,we design a two-stage annotation method to address the challenge of extremely poor spatial salience of the targets that makes them difficult to distinguish. The first stage is data preannotation,where videos are initially cropped spatially into multiple locally enlarged subvideos. By repeatedly playing these subvideos,we determine the presence of targets in each and annotate their temporal and spatial range information. The second stage involves fine-grained data annotation based on the preannotation results,following a specific workflow: confirming target positions,drawing target boundaries,and filling to generate masks. In this manner,well-annotated real data can be obtained. Then,we integrate two simulated infrared aerial moving target datasets with authentic space-based backgrounds to enhance scenario diversity. The resulting dataset comprises 1 401 real scenarios,122 265 video frames,and 454 116 annotated targets,with mask labels distinguishing different target instances to support detection and tracking research. In the challenge based on this dataset,we establish two core tasks,i.e.,aerial moving target detection and tracking,to advance integrated detection-tracking technology. Corresponding to the two tasks,we design two special evaluation metric systems. Especially,the tracking performance metrics are proposed for the first time,including trajectory completeness rating and trajectory accuracy rating.
Analysis reveals the high complexity of our dataset: The real targets are extremely small and dim,with an average size of no more than 20 pixels and an average signal-to-noise ratio (SNR) of only 3.06. Among them,over 80% of the targets have an SNR below 2. The real videos contain rich dynamic variations of targets,including changes in motion state,grayscale distribution,and morphological size. The background also exhibits complex dynamic changes,including translation,rotation,stretching,and scaling in different directions,and local grayscale variations. Additionally,complex mutual radiation interference exists between targets and backgrounds. The diversity introduced by real-world scenarios and actual systems cannot be easily replicated by simulated data,which also causes our SatVideoIRSDT dataset more challenging than other public datasets. For the detection task,a state-of-the-art baseline algorithm achieved a recall of 0.578 2,precision of 0.495 8,and F1-score of 0.533 8 on this dataset. For the tracking task,a typical baseline algorithm yielded a trajectory completeness rating of only 0.241 9 and trajectory accuracy rating of 0.519 1.
The competition fully validated the practical value and high challenge of our SatVideoIRSDT dataset,providing crucial data support for research on infrared small and dim target detection technology. It also attracted substantial attention to the field,facilitating academic exchange and technological progress. Through this event,we collected numerous outstanding aerial moving target detection and tracking solutions,which have contributed valuable new insights to drive domain development. The dataset is available at https://github.com/TinaLRJ/DeepPro(or Science Data Bank: Infrared video satellite aerial moving target detection dataset).
Low-light environments in practical computer vision applications,such as security surveillance,autonomous driving,and intelligent urban monitoring,considerably degrade image quality and thus restrict the performance of high-level visual tasks. Images captured under poor illumination often exhibit insufficient brightness,heavy noise amplification,and severe loss of structural and textural information. These degradations not only harm perceptual quality but also compromise the reliability of downstream applications such as object tracking,object detection,and semantic segmentation. Existing methods have attempted to solve these problems by increasing image brightness,but they frequently suffer from inadequate overexposure suppression and heavy reliance on paired datasets collected under normal lighting conditions. Given that such datasets are expensive and difficult to obtain,the generalization ability of these supervised methods remains limited in real-world complex scenarios. Furthermore,some unsupervised methods attempt to reduce dependency on paired data,but they often fail to achieve noise suppression,exposure correction,and structure preservation simultaneously,leading to unsatisfactory performance in practice.
To address these challenges,this study proposes a novel zero-supervision low-light image enhancement framework that utilizes illumination components to guide reflectance estimation,thereby ensuring high-fidelity reconstruction. The framework is designed to operate without paired training data while simultaneously achieving effective noise suppression,overexposure control,and structure preservation. The process begins with a reversible brightness perturbation strategy combined with pseudoexposure generation,which synthesizes abundant pseudo low-light and pseudo well-lit samples without changing the intrinsic reflectance of the image and thus effectively simulates diverse illumination conditions and eliminates the dependence on paired supervision. A dual-domain collaborative attention denoising module is introduced to improve robustness further. This module operates in the spatial and channel domains: in the spatial domain,contextual dependencies are captured to distinguish true structures from random noise,while in the channel domain,correlations among different feature channels are modeled to suppress redundant information and emphasize critical features. By collaborating across these two domains,the module ensures effective denoising while preserving structural fidelity. For structural representation,multilayer residual convolutions and a structurally symmetric U-Net architecture work in concert to generate the illumination map. The symmetric encoder-decoder structure enables extraction of multiscale features,capturing global illumination and local detail,while the residual connections mitigate vanishing gradients and accelerate convergence. The U-Net is designed not only to generate illumination maps but also to extract illumination-related features that serve as explicit guidance for reflectance recovery. On this basis,a guided reflectance estimation method is introduced. Instead of directly treating reflectance as a residual,illumination priors are fused into the estimation process. This fusion enables the network to separate illumination variations effectively from intrinsic structures,restoring edges and textures with high fidelity while suppressing overexposed regions. The integration of illumination priors makes the restored images appear natural and structurally consistent,thereby addressing the shortcomings of previous enhancement methods.
Extensive comparative experiments were conducted on the LOL,DICM,and WildNight-8K datasets against nine representative state-of-the-art low-light image enhancement methods. On the LOL dataset,the proposed method outperformed Retinexformer and LightenDiffusion,achieving reductions of 0.62 and 0.92 in the natural image quality evaluator (NIQE) and decreases of 1.23 and 0.25 in the perceptual index (PI),respectively. On the DICM dataset,compared with Retinexformer and retinex-inspired unrolling with architecture search(RUAS),the proposed approach reduced NIQE by 0.11 and 0.81,and PI by 0.48 and 0.72,respectively. On the WildNight-8K dataset,relative to Retinexformer and EnlightenGAN,NIQE was further reduced by 1.32 and 1.87,while PI decreased by 0.33 and 5.98,respectively. These results demonstrate that the proposed method consistently delivers enhanced images with improved naturalness and perceptual quality across diverse low-light scenarios. In addition to image quality assessment,the enhanced images were further evaluated in a downstream animal detection task using YOLOv8. Experimental results show that the mean average precision (mAP@50–95) increased by 4.60% compared with the original low-light images,indicating that the proposed enhancement method not only improves visual quality but also remarkably boosts the reliability of downstream perception tasks in challenging low-light conditions.
This study introduces a zero-supervision low-light image enhancement framework that integrates illumination-guided reflectance estimation for high-fidelity image reconstruction. The key innovations include reversible brightness perturbation with pseudoexposure generation to remove paired dataset dependency,a dual-domain collaborative attention denoising module to reduce noise while preserving structural details,a residual U-Net backbone for effective multiscale representation,and an illumination-fused reflectance estimation mechanism to achieve structure-preserving restoration and overexposure suppression. Experimental results across multiple datasets and real-world applications demonstrate that the proposed method not only achieves state-of-the-art performance in quantitative and qualitative metrics but also remarkably improves downstream vision tasks,particularly object detection. These contributions indicate strong practical value and broad application potential. Looking forward,future work could explore extending this framework to dynamic video enhancement with temporal consistency,incorporating adaptive illumination priors derived from scene semantics,and optimizing lightweight deployment for edge devices such as autonomous vehicles and mobile cameras. With these directions,the proposed method could further strengthen its role as a reliable solution for low-light visual perception in safety-critical and real-time applications.
Low-light image enhancement (LLIE) aims to recover visually pleasing images from severely underexposed inputs by improving brightness,contrast,and structural clarity while preserving natural colors and fine details. This task is fundamentally important for numerous downstream vision applications,including video surveillance,autonomous driving,remote sensing,and medical imaging,where low illumination conditions frequently degrade visual quality and impair algorithmic reliability. In recent years,diffusion probabilistic models have demonstrated remarkable generative capacity and robustness,making them a promising paradigm for LLIE. However,directly applying diffusion models to low-light enhancement remains challenging due to several intrinsic limitations. First,the iterative denoising mechanism of diffusion models introduces substantial computational and memory overhead,hindering their deployment in real-time or resource-constrained scenarios. Second,most existing diffusion-based LLIE approaches operate in the spatial domain,which often struggles to recover high-frequency textures and fine structural details faithfully under severe illumination degradation,leading to oversmoothed results or visually disturbing artifacts. Third,the accumulation of prediction errors during reverse diffusion may cause color shifts and temporal inconsistencies across denoising steps,thereby reducing color fidelity and perceptual naturalness.
To address these challenges,this study proposes LLIE with diagonal frequency feature refinement and truncated sampling in conditional diffusion models. The core idea is to exploit jointly the complementary strengths of frequency-domain modeling and accelerated diffusion inference to achieve high-quality enhancement with considerably reduced computational cost. Specifically,instead of performing diffusion directly in the spatial domain,the proposed method conducts forward and reverse diffusion in the low-frequency subspace of the wavelet domain. This design choice is motivated by the observation that global illumination structure and coarse semantic content are predominantly concentrated in low-frequency components. By diffusing only low-frequency coefficients,the proposed framework effectively reduces spatial resolution and computational complexity while preserving the essential structural and illumination information required for faithful enhancement. However,operating in the low-frequency subspace inevitably leads to the loss of high-frequency details,particularly diagonal textures that are critical for visual sharpness. To compensate for this limitation,we introduce a dedicated high-frequency refinement module that explicitly models directional dependencies in the wavelet domain. In contrast to conventional approaches that treat high-frequency bands independently,the proposed module leverages horizontal and vertical high-frequency components to reconstruct missing diagonal information through a dual-path cross-directional attention mechanism. By enabling feature interaction across orthogonal directions,the refinement module effectively alleviates directional sparsity and enhances local texture fidelity,resulting in sharper edges and more natural fine details. To improve efficiency further,an efficient truncated sampling strategy is incorporated into the reverse diffusion process. Instead of executing the full sequence of denoising steps,the proposed approach adopts a step-interrupt mechanism that terminates diffusion at an earlier stage once sufficient reconstruction quality is achieved. This strategy considerably reduces inference time and memory consumption without introducing noticeable degradation in visual quality,making the method more suitable for practical applications. In addition,to mitigate color shifts and enhance stability during reverse diffusion,a contrast-aware correction module is integrated into the denoising stages. This module exploits contrast and color priors accumulated from previous diffusion steps to adjust brightness and chromatic consistency adaptively at each iteration. By explicitly enforcing contrast awareness throughout the sampling process,the proposed method effectively suppresses error accumulation and improves color fidelity in the final enhanced images.
To validate the effectiveness of the proposed method,we conducted extensive experiments on three publicly available paired low-light datasets,comparing the proposed method with several state-of-the-art LLIE methods. Subjective visual comparisons and objective quantitative metrics demonstrate that the proposed method consistently outperforms existing approaches. On the LOLv2-Real dataset,the proposed method achieves improvements over the second-best method by 8.46% in peak signal-to-noise ratio,1.22% in structural similarity index,a 10.19% reduction in learned perceptual image patch similarity(LPIPS),and a 0.38% reduction in naturalness image quality evaluator (NIQE),indicating improved perceptual quality and structural fidelity. To assess the cross-domain generalization ability,we further evaluated the model using the pretrained weights from LOLv2-Real datasets on four unpaired low-light datasets. The proposed method achieved lower NIQE scores than existing methods by 0.82 (LLIE via illumination map estimation),0.39 (multiexposure image fusion) and 0.86 (naturalness photo enhancement),demonstrating strong generalization ability without target-domain fine-tuning. Additionally,ablation studies confirm the contribution of each core module. Compared with spatial-domain diffusion baselines,the proposed method considerably reduced inference time from 7.617 s to 0.473 s per image,and memory usage decreased by 48.53%,highlighting the efficiency and practicality of the proposed design.
The proposed method integrates frequency-domain modeling with an efficient diffusion sampling strategy,introducing three key innovations: 1) performing the diffusion process in the low-frequency subspace of the wavelet domain considerably reduces computational complexity while preserving global structural and illumination information. 2) A dedicated high-frequency refinement module is developed to recover diagonal texture details by leveraging horizontal and vertical high-frequency information,thereby improving the sharpness and perceptual quality of the enhanced images. 3) The integration of an efficient truncated sampling strategy and a contrast-aware correction module improves sampling efficiency and color fidelity,effectively balancing enhancement quality and computational cost. Collectively,these contributions enable the proposed method to achieve superior enhancement performance with substantially reduced inference overhead,demonstrating strong potential for real-world low-light vision applications.
The relentless evolution of presentation attacks poses a threat to the security of facial recognition systems,and low-cost deepfake technology has caused widespread social security issues. With the continuous confrontation between forgery and anticounterfeiting technologies,the difference between real and forged data is becoming increasingly subtle,transient,and sparse. However,when the generated patches (or occluded patches) are mixed with the original image,global inconsistencies in lighting,geometric proportions,motion patterns,and other aspects are inevitable. This study proposes an adaptive face fraud detection model based on Transformer graph representation learning,which utilizes fraud clues with widespread global spatial inconsistency in fraud samples. This model aims to improve the accuracy and generalization ability of face fraud detection to address increasingly complex security challenges.
First,the model constructs the complementary advantages of graph neural networks (GNN) and Transformer layers by alternately stacking them. GNN shields irrelevant nodes and converts Transformer’s blind global attention into sparse,low rank,and structurally sensitive; Transformer expands the receptive field of GNN through long-range dependencies,making message transmission farther and more flexible. The combination of GNN and Transformer makes it easier for the model to capture globally inconsistent fraud clues. Second,given the strong dependence of GNN on graph topology,this model applies the dynamic K-nearest neighbor dense algorithm to improve the sampling method of neighboring nodes. The algorithm adaptively adjusts the connectivity (degree) of each node based on the local feature density distribution in the latent space. In sparse or ambiguous regions,nodes connect to more neighbors to gather broader context; in dense,discriminative regions,connections focus on the most relevant neighbors,thus dynamically constructing data-adaptive graph topologies that are inherently more resilient to noise and variations,providing a robust foundation for subsequent processing. Third,this model uses a dual attention mechanism to weight the nodes and edges in the graph separately. The weight of nodes reflects the degree of influence of image patches in different regions on image labels,and adding weights to nodes can enable the model to focus on image patches that carry more discriminative information. Meanwhile,the weight of edges reflects the degree of influence of neighboring nodes on the central node,and adding weights to edges can highlight the key pathways of information flow. This dual attention explicitly models the complex interplay between nodes and their relationships,allowing the model to focus on the most salient and informative features and connections,thereby dramatically boosting its relational learning capacity and sensitivity to subtle cues. Fourth,considering that static models fail against novel attacks,this model incorporates meta learning optimization strategies to enable the model to quickly adapt to new types of attacks or new scenarios. The core improvement of this strategy is a new combination of soft-label loss functions. In the inner-loop training stage,the model utilizes the sensitivity of the Kullback-Leibler (KL) divergence loss to adjust uncertain model parameters quickly,while in the outer-loop stage,it utilizes the strong fault tolerance of the soft-target cross-entropy loss function to ensure stable and robust feature representations across various data. The improved meta learning algorithm not only demonstrates increased stability during training,but also maintains good generalization performance on different datasets.
This study investigates the effectiveness of the proposed model on seven major,publicly available benchmarks,which belong to traditional face antispoofing or deepfake detection tasks. In the face antispoofing experiment,four classic datasets,i.e.,MSU-MFSD,Replay-Attack,OULU-NPU,and CASIA-FASD,were used for cross-dataset experiments. The average half total error rate (HTER) obtained from the experiment exceeded all comparison algorithms,indicating that the proposed model has good generalization performance in face antispoofing. In the deepfake detection experiment,FaceForensics++ (FF++) and Celeb-DF (v2) were used for intradataset experiments,achieving classification accuracies of 98.93% and 98.44%,respectively; Then,FF++ was used to conduct cross subset experiments within the dataset,and FF++,deepfake detection dataset),and Celeb-DF were used to conduct cross-dataset experiments. The above cross domain experimental results show that the accuracy and area under the curve of this model are higher than those of the latest model. Further ablation experiments confirmed that density-based dynamic K-nearest neighbors (DBD-KNN),meta learning optimization,TransGNN module,and FFN module in the model are all beneficial for improving model performance. The comparative experiment between K-nearest neighbors,and DBD-KNN confirms that even if the KNN algorithm uses the optimal K value,its performance remains inferior to DBD-KNN. The DBD-KNN algorithm leads by 5.55% in the HTER metric,mainly because of its ability to adapt to changes in data dynamically,thereby more effectively adapting to the characteristics of the dataset. In the loss function combination experiment of meta learning,it was confirmed that the best experimental results can be obtained when selecting the KL divergence loss function for the inner loop and the soft-target cross entropy loss function for the outer loop. After analyzing the complexity of the model,we found that although the proposed model has remarkable advantages in computational efficiency and parameter quantity,its detection accuracy remains at a high level. Finally,observing the evolution of the topological structure of the graph reveals that as the model training deepens,the connections between nodes are no longer limited to local areas,but begin to capture semantic features with distinctive characteristics throughout the entire image range. This result reveals the inherent logic of the model in making detection decisions.
The method proposed in this study can effectively capture and identify subtle fraud clues and global inconsistencies present in fraud samples,thereby improving the detection accuracy of the model. This method is particularly remarkable in enhancing the model’s generalization ability to different datasets and adaptability to new environments,ensuring that the model can achieve high recognition performance in diverse practical application scenarios. The dataset for this study has been archived in ScienceDB and can be accessed via https://doi.org/10.57760/sciencedb.j00240.00096.
Lip reading systems infer linguistic content by analyzing a speaker’s lip movements,relying entirely on visual modality compared with traditional speech recognition,thus facing greater uncertainty due to missing modalities and temporal variations. Although recent advances in lip motion modeling and semantic alignment have driven technological progress,existing methods predominantly focus on generic modeling while neglecting the critical impact of individual differences. In practical applications,considerable performance variations across speakers remain a major bottleneck for cross-scenario generalization,making speaker adaptation a core challenge in lip reading research. Current speaker adaptation approaches fall into two categories: 1) generalization strategies without target speaker data,such as constructing speaker-invariant representations or employing visual i-vectors for cross-speaker modeling,and 2) personalized strategies with limited target data,including fine-tuning,model pruning,parameter-efficient prompt learning,and contrastive learning. Recent studies have also explored vision-language collaborative mechanisms to enhance individual semantic and visual style perception. However,existing methods still inadequately address the explicit decoupling and utilization of speaker features under zero-resource conditions,particularly failing to distinguish the differential impacts of static appearance features and dynamic behavioral patterns on lip motion modeling. This study aims to tackle this zero-resource adaptation problem by proposing a novel speaker-adaptive lip reading method based on memory-guided prompting. The proposed method seeks to enhance the model’s ability to generalize across unseen speakers without accessing any speaker-specific adaptation data. To evaluate systematically the performance of lip reading models in speaker generalization scenarios,this study constructs a new benchmark dataset,LRS2-ID,on the basis of the public LRS2 dataset. LRS2-ID not only preserves the diversity of the original corpus but also introduces speaker identity-controlled annotations to support separate modeling and evaluation of “seen/unseen speakers”,providing a practical benchmark for assessing zero-shot adaptation capabilities.
In this study,we introduce an explicit memory prompting framework that leverages structured visual speaker prototypes as external guidance during training and inference. Specifically,the framework constructs two types of speaker-specific memory banks: a static visual pattern memory and a dynamic visual speech pattern memory. Static visual patterns capture facial appearance and are obtained by averaging face features over multiple representative frames. Dynamic speech patterns are derived from frame-level features in the frequency domain,emphasizing interframe variations while suppressing static components. Negative samples are introduced for contrastive learning,where shuffled and reversed sequences generate temporally incorrect samples,training the model to identify correct speaker-specific dynamic patterns. Additionally,a gradient reversal layer is incorporated at the frame level to suppress the model’s reliance on static appearance features during backpropagation,further enhancing dynamic pattern recognition. Static and dynamic patterns are clustered into prototype memory banks using K-means clustering,ensuring compactness and scalability. During inference,when encountering an unseen speaker,the model retrieves relevant prototypes from these memory banks based on similarity with the input sequence. The retrieved prototypes are then injected into the attention mechanism of a Conformer-based encoder as key and value components. A hierarchical injection strategy is adopted: Static prototypes are fused into shallow encoder layers to enhance appearance modeling,while dynamic prototypes are injected into deeper layers to guide temporal behavior modeling. This cross-layer prompting scheme allows the model to align and adapt to the unseen speaker’s patterns dynamically without modifying model parameters or requiring supervised adaptation.
Comprehensive experiments on the LRS2-ID dataset demonstrate the effectiveness of the proposed method. Results show that the method reduces the word error rate by over 4.3% absolutely,without requiring target speaker adaptation data. Ablation studies confirm the contributions of each component: Static prompting alone aids stable appearance modeling,particularly for speakers with consistent visual traits,while dynamic prompting captures personalized speech behavior and benefits speakers with diverse visual conditions. The fusion of both prompt types,combined with structured memory organization,consistently enhances model robustness across varying speaker profiles. Per-speaker performance analysis reveals consistent improvements across most speakers,including those with considerable intervideo appearance changes or high articulation variability. Visualization of representative samples highlights the complementary strengths of static and dynamic memory prompts,particularly in challenging scenarios involving illumination shifts,pose variations,or expressive speech.
This study presents a novel memory-based prompting framework for speaker-adaptive lip reading in zero-resource conditions. By explicitly modeling static and dynamic visual speaker patterns and integrating them hierarchically into the attention mechanism of a Conformer-based encoder,the proposed method achieves robust generalization to unseen speakers without requiring adaptation data. Experimental results on the LRS2-ID benchmark confirm the method’s effectiveness,stability,and practical value. This study advances structured prompting and memory-enhanced adaptation,paving the way for scalable and user-friendly lip reading systems capable of generalizing across diverse real-world users. Future work may explore finer-grained adaptive mechanisms and additional modalities to further enhance its performance.
Novel view synthesis from sparse views presents a critical and ill-posed challenge in 3D computer vision,primarily due to the severe underconstraint inherent in limited input imagery. Current state-of-the-art approaches can be broadly categorized into two streams: One relies on complex external prior models,such as monocular depth estimators or pretrained diffusion models,to compensate for the information deficit; the other,including methods such as few-shot Gaussian splatting (FSGS),operates within a more self-contained framework. However,a common limitation across these methods is their frequently inadequate reconstruction of detailed regions characterized by high-frequency geometry and thin structures,such as scene edges. Methods dependent on external priors can be misled by their generic nature,while self-contained strategies often lack targeted mechanisms in their densification and regularization processes to model these anisotropic areas effectively. To address these limitations,this study introduces a novel approach for sparse-view synthesis based on 3D Gaussian splatting,which is driven by the concept of edge clusters. The core idea is an “inside-out” optimization paradigm that deliberately avoids heavy external priors,instead seeking to mine and leverage structural information directly from the internal distribution of the 3D Gaussians. The objective is to achieve high-quality novel view synthesis with notably enhanced edge detail fidelity under challenging sparse-input conditions,exploring the potential of a fully self-contained and efficient framework.
The proposed framework is built upon the standard 3D Gaussian splatting pipeline,integrating three key components tailored for sparse view optimization. First,the concept of clustering analysis is introduced into the optimization process. Candidate points for potential edge regions are selected on the basis of their average spatial position gradient norm. These points are subsequently partitioned into clusters using the K-means algorithm. A multicriteria fusion algorithm is then proposed to identify “edge clusters” critical for representing scene structures robustly. This algorithm evaluates four distinct criteria per cluster: cluster-center gradient,average gradient magnitude,cluster spatial scale,and intercluster proximity. A voting mechanism is employed to finalize the edge cluster set,enhancing robustness against the failure of any single criterion. Second,a locally proximity-guided Gaussian unpooling strategy is designed to address under-representation within the identified edge clusters. In contrast to global proximity guidance,this strategy constructs a local proximity graph within each edge cluster. For Gaussians exhibiting insufficient local coverage,new Gaussians are spawned along the connections to their K-nearest neighbors within the cluster,specifically at the midpoints. This approach strategically reinforces the continuity and geometric integrity of edge contours. Third,a collaboratively supervised depth regularization mechanism is introduced to provide stronger geometric constraints,which is particularly beneficial for newly added Gaussians in underconstrained edge regions. This mechanism involves synthesizing pseudoviews for data augmentation and imposing a hybrid depth loss. The hybrid loss combines an absolute depth loss,which uses monocular depth estimates normalized via SfM point clouds to address scale ambiguity,and a relative depth loss based on the Pearson correlation coefficient to preserve depth structure consistency. The depth loss weight follows an exponential decay schedule. The overall loss function integrates standard photometric losses with the proposed absolute and relative depth losses.
Extensive experiments were conducted on the Mip-NeRF360,LLFF,Deep Blending,Tanks&Temples,and DTU datasets under standardized sparse-view settings. The proposed method was quantitatively compared with several state-of-the-art methods,including Mip-NeRF360,RegNeRF,SparseNeRF,vanilla 3DGS,and FSGS. Results demonstrate favorable performance of our approach. On the Mip-NeRF360 dataset at 1/8 resolution,our method achieved average PSNR,SSIM,and LPIPS values of 23.82 dB,0.753,and 0.201,respectively,showing improvements over the FSGS baseline. On the LLFF dataset at 1/8 resolution,it achieved 20.48 dB,0.662,and 0.268,indicating consistent gains. These trends were observed across most tested scenarios and at 1/4 resolution. Qualitatively,visual comparisons reveal that the proposed method successfully recovers fine edge details,such as bicycle spokes and plant stems,with notable clarity and continuity,whereas baseline methods often exhibit blurriness or fragmentation in these areas. The rendering speed remains high,exceeding 275 frames/s,confirming real-time capability. Ablation studies validate the complementary contributions of each proposed module,and further analysis confirms the robustness of the multicriteria clustering to its key parameters.
This study presents a novel edge cluster-driven approach for 3D Gaussian splatting that enhances sparse-view reconstruction quality through an internally structural-aware methodology. By leveraging clustering analysis for targeted edge cluster identification and augmentation,coupled with a collaboratively supervised regularization scheme,the method achieves measurable gains in quantitative metrics and improved visual fidelity for detail-rich regions. It provides a self-contained and efficient pathway for high-quality,real-time synthesis from sparse inputs.